From the Student Success edition of November 6, 2026
JMIR Medical Education: USC Floyd SoM crossover pilot finds curriculum-aligned custom GPT raises pathology SDL and discussion scores; delayed retention not significant
University of South Carolina Floyd School of Medicine; JMIR Medical Education · Higher education medical education, self-directed learning and AI tutoring · United States (South Carolina)
- Publisher
- JMIR Medical Education
- Original publication
- JMIR Medical Education, 6 November 2026
- Source retrieved
- 2026-11-07
What happened
A randomized crossover pilot (N=44 second-year medical students) at USC Floyd School of Medicine compared a curriculum-aligned custom GPT (Socratic, case-grounded) with unguided traditional preparation for two pathology small-group cases. Faculty-blinded SDL documentation scores rose from 9.4 to 10.3/12 (mean difference 0.9; p<.001; Cohen d=0.71) and small-group discussion scores from 13.7 to 14.9/16 (difference 1.2; p<.001; d=0.78). Delayed end-of-block retention did not reach significance (Wilcoxon p=.052). Student surveys reported higher reasoning clarity (77%) and conceptual understanding (70%); 62% of commenting GPT users reported greater preparation time. Authors note the control included some general-purpose AI use (38% of traditional survey respondents), single-site design, and no objective GPT usage logs.
Why it matters
Peer-reviewed HE evidence that curriculum-aligned AI tutoring can lift near-term faculty-assessed preparation and discussion performance without a clear delayed-retention win—useful for tutoring and medical-education procurement conversations.
Evidence and measured results
Crossover randomized pilot with faculty-blinded rubrics; CONSORT-EHEALTH reported; single institution; retention underpowered.
Limitations and uncertainty
Pilot N=44; delayed retention NS; time-on-task may confound benefits; traditional arm allowed some general AI; developers authored the tools; interrater reliability not assessed.
Put this evidence to work
Lighthouse Advisory interpretation, grounded in this source. Enriched 2026-11-07; this does not change the original publication date. Each role follows the same checklist so takeaways compare across sources.
Sales
Role takeaway
- Customer problem
- General-purpose AI prep is misaligned with case learning objectives and hard to evaluate against faculty SDL expectations.
- Stakeholders
- Curriculum dean
- Pathology course directors
- Teaching center
- Student affairs
- LCME readiness leads
- Discovery questions
- Which blocks already use standardized SDL forms?
- Do students have institutional ChatGPT accounts?
- What retention measures exist beyond two MCQs?
- Value hypothesis
- Interpretation: curriculum-aligned Socratic GPTs may improve near-term preparation quality versus unguided prep, without proven delayed-retention gains here.
- Potential engagement
- Single-block crossover or parallel pilot with blinded SDL/SGD rubrics and a multi-item retention test.
- Unsupported claims
- Guaranteed board-score gains
- Superiority to all general-purpose AI
- Zero time burden for students
Pre-sales engineering
Role takeaway
- Fit
- Case-specific custom GPTs with Socratic scaffolding on institutional ChatGPT, not open consumer bots.
- Architecture and integration
- SSO to institutional ChatGPT; frozen system prompts; links to open-access resources; LMS delivery of SDL forms.
- Prerequisites
- Faculty-authored case objectives
- Institutional ChatGPT access
- Blinded facilitator rubrics
- Constraints
- Single-site pilot evidence
- No objective usage logs in this study
- Some control students used general AI
- Security
- Use institutional accounts; avoid putting identifiable student performance data into prompts; document FERPA boundaries.
- Proof of value
- Replicate SDL/SGD effect sizes with blinding
- Pre-register multi-item retention
- Log time-on-task and message counts
Delivery
Role takeaway
- Work
- Build/freeze case GPTs, orient students, run blinded scoring, survey concerns, and report limitations.
- Dependencies
- Faculty facilitators
- IRB/exempt determination
- Institutional ChatGPT licensing
- Assessment items
- Ownership
- Course director with AI-in-medicine track support
- Skills and adoption
- Brief student orientation plus faculty prompt-review skills; avoid assuming ad-libitum use is effortless.
- Governance checkpoints
- Prompt freeze before data collection
- Blinded scoring audit
- Prespecified concern-domain review
- Acceptance criteria
- Documented paired SDL/SGD gains
- Retention analysis with adequate items
- Published time-burden and limitation notes
- Risks
- Pilot overgeneralization
- Time-on-task confounding
- Sequence/case effects
- Developer-evaluator overlap
Implementation considerations
Lighthouse Advisory interpretation across the operating dimensions a public-sector buyer must settle before this evidence becomes a design. Each note answers the question under its heading for this specific source.
Architecture and integration
What must connect, and where does the AI sit in the workflow?
Custom GPT configuration on institutional ChatGPT with frozen case prompts and open-access resources; not a free consumer chatbot free-for-all.
Governance
Who approves, reviews and stays accountable for outcomes?
Aligns AI scaffolding with LCME-oriented SDL forms and faculty learning objectives; IRB-exempt educational pilot; authors call for multisite registered trials.
Security and privacy
What data, permissions and controls need testing?
Students used institutional USC ChatGPT accounts; FERPA limited prior academic-record access for stratified randomization; hallucinations/privacy not formally evaluated.
Accessibility and workforce
Who is affected, and what skills or accommodations follow?
Exploratory post-hoc signal of larger retention difference among lower-performing students; faculty facilitation capacity still required for SGD scoring.
Procurement
What should contracts, pricing and exit terms secure?
Supports buying or building curriculum-aligned tutors with faculty-authored prompts rather than only licensing unguided consumer AI.
Operating model
Which teams own the service once it runs?
SDL prep plus small-group facilitation remains the operating model; AI is a scaffold, not a replacement for facilitators or NBME-style assessment.
Publication history
- 2026-11-06Student Success · Issue 604 resources
Stable resource ID: jmir-usc-pathology-curriculum-aligned-custom-gpt-crossover-20261106