{"resourceId":"jmir-usc-pathology-curriculum-aligned-custom-gpt-crossover-20261106","versions":[{"version":"external-7a9aa084fc01f1cfae46778c9f46f4376e4ee5547735b0c826e12c07eabf169d","resource":{"id":"jmir-usc-pathology-curriculum-aligned-custom-gpt-crossover-20261106","title":"JMIR Medical Education: USC Floyd SoM crossover pilot finds curriculum-aligned custom GPT raises pathology SDL and discussion scores; delayed retention not significant","organization":"University of South Carolina Floyd School of Medicine; JMIR Medical Education","sector":"Higher education medical education, self-directed learning and AI tutoring","geography":"United States (South Carolina)","publishedAt":"JMIR Medical Education, 6 November 2026","publicationDate":"2026-11-06","eventDate":null,"sourceName":"JMIR Medical Education","sourceLabel":"Peer-reviewed medical education journal","sourceUrl":"https://mededu.jmir.org/2026/1/e102908/","evidenceClass":"academic-research","outcomeClass":"mixed","topics":["knowledge-work","accessibility-workforce","operating-model"],"finding":"A randomized crossover pilot (N=44 second-year medical students) at USC Floyd School of Medicine compared a curriculum-aligned custom GPT (Socratic, case-grounded) with unguided traditional preparation for two pathology small-group cases. Faculty-blinded SDL documentation scores rose from 9.4 to 10.3/12 (mean difference 0.9; p<.001; Cohen d=0.71) and small-group discussion scores from 13.7 to 14.9/16 (difference 1.2; p<.001; d=0.78). Delayed end-of-block retention did not reach significance (Wilcoxon p=.052). Student surveys reported higher reasoning clarity (77%) and conceptual understanding (70%); 62% of commenting GPT users reported greater preparation time. Authors note the control included some general-purpose AI use (38% of traditional survey respondents), single-site design, and no objective GPT usage logs.","sledRelevance":"Peer-reviewed HE evidence that curriculum-aligned AI tutoring can lift near-term faculty-assessed preparation and discussion performance without a clear delayed-retention win—useful for tutoring and medical-education procurement conversations.","evidence":"Crossover randomized pilot with faculty-blinded rubrics; CONSORT-EHEALTH reported; single institution; retention underpowered.","architectureImplications":"Custom GPT configuration on institutional ChatGPT with frozen case prompts and open-access resources; not a free consumer chatbot free-for-all.","governanceImplications":"Aligns AI scaffolding with LCME-oriented SDL forms and faculty learning objectives; IRB-exempt educational pilot; authors call for multisite registered trials.","securityPrivacyImplications":"Students used institutional USC ChatGPT accounts; FERPA limited prior academic-record access for stratified randomization; hallucinations/privacy not formally evaluated.","caveats":"Pilot N=44; delayed retention NS; time-on-task may confound benefits; traditional arm allowed some general AI; developers authored the tools; interrater reliability not assessed.","streamIds":["student-success"],"roles":{"sales":"Sales framing: medical and health-professions programs want AI tutors that improve preparation quality without surrendering curriculum control. This peer-reviewed crossover pilot gives a bounded value story—higher faculty-assessed SDL and discussion scores with moderate-to-large effects—while honesty requires saying delayed retention was not significant and time burden rose. A credible engagement is a single-block curriculum-aligned GPT pilot with blinded rubrics, not a campus-wide guarantee of board-score gains.","engineering":"Engineering framing: fit is a case-grounded custom GPT on an institutional ChatGPT tenancy with Socratic prompts frozen for the study window. Prerequisites include faculty-authored learning objectives, SDL forms, and student SSO accounts. Constraints are single-site generalizability, lack of usage telemetry in this pilot, and control-arm leakage of general-purpose AI. Proof of value should replicate blinded SDL/SGD rubrics and a multi-item retention measure before claiming durable learning.","delivery":"Delivery framing: operating owners are pathology or course directors with an AI-in-medicine support track. Work includes prompt design, facilitator blinding, survey ops, and orientation to cut time burden. Proposed acceptance criteria: paired SDL/SGD gains with documented effect sizes and a pre-registered retention endpoint. Risks include over-claiming from a pilot, unmeasured time-on-task, and sequence effects by case."},"roleDetails":{"sales":{"customerProblem":"General-purpose AI prep is misaligned with case learning objectives and hard to evaluate against faculty SDL expectations.","stakeholders":["Curriculum dean","Pathology course directors","Teaching center","Student affairs","LCME readiness leads"],"discoveryQuestions":["Which blocks already use standardized SDL forms?","Do students have institutional ChatGPT accounts?","What retention measures exist beyond two MCQs?"],"valueHypothesis":"Interpretation: curriculum-aligned Socratic GPTs may improve near-term preparation quality versus unguided prep, without proven delayed-retention gains here.","potentialEngagement":"Single-block crossover or parallel pilot with blinded SDL/SGD rubrics and a multi-item retention test.","unsupportedClaims":["Guaranteed board-score gains","Superiority to all general-purpose AI","Zero time burden for students"]},"engineering":{"fit":"Case-specific custom GPTs with Socratic scaffolding on institutional ChatGPT, not open consumer bots.","architectureIntegration":"SSO to institutional ChatGPT; frozen system prompts; links to open-access resources; LMS delivery of SDL forms.","prerequisites":["Faculty-authored case objectives","Institutional ChatGPT access","Blinded facilitator rubrics"],"constraints":["Single-site pilot evidence","No objective usage logs in this study","Some control students used general AI"],"security":"Use institutional accounts; avoid putting identifiable student performance data into prompts; document FERPA boundaries.","proofOfValue":["Replicate SDL/SGD effect sizes with blinding","Pre-register multi-item retention","Log time-on-task and message counts"]},"delivery":{"work":"Build/freeze case GPTs, orient students, run blinded scoring, survey concerns, and report limitations.","dependencies":["Faculty facilitators","IRB/exempt determination","Institutional ChatGPT licensing","Assessment items"],"ownership":"Course director with AI-in-medicine track support","skillsAdoption":"Brief student orientation plus faculty prompt-review skills; avoid assuming ad-libitum use is effortless.","governanceCheckpoints":["Prompt freeze before data collection","Blinded scoring audit","Prespecified concern-domain review"],"acceptanceCriteria":["Documented paired SDL/SGD gains","Retention analysis with adequate items","Published time-burden and limitation notes"],"risks":["Pilot overgeneralization","Time-on-task confounding","Sequence/case effects","Developer-evaluator overlap"]}},"retrievedAt":"2026-11-07T04:18:20Z","enrichedAt":"2026-11-07T04:22:43Z","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Exploratory post-hoc signal of larger retention difference among lower-performing students; faculty facilitation capacity still required for SGD scoring.","procurementImplications":"Supports buying or building curriculum-aligned tutors with faculty-authored prompts rather than only licensing unguided consumer AI.","operatingModelImplications":"SDL prep plus small-group facilitation remains the operating model; AI is a scaffold, not a replacement for facilitators or NBME-style assessment.","peerReview":"peer-reviewed","sourceVerification":{"openedUrl":"https://mededu.jmir.org/2026/1/e102908/","referenceExcerpt":"access to a curriculum-aligned custom GPT was associated with significantly higher faculty-assessed SDL documentation and small group discussion performance","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}