{"resourceId":"google-learnlm-sierra-leone-gemini-guided-learning-rct-20260515","versions":[{"version":"external-a90b1dd6678e05b003b0bfb19163b2c7165efdeba4fa21f1d6573d3679ea4f54","resource":{"id":"google-learnlm-sierra-leone-gemini-guided-learning-rct-20260515","title":"Preregistered Sierra Leone RCT: teacher-led Gemini Guided Learning raises junior-secondary math +0.258 SD ITT, but stronger baseline students gain more","organization":"Google LearnLM Team & Fab AI","sector":"Vendor-affiliated education efficacy research (K-12 mathematics)","geography":"Sierra Leone (Port Loko District junior secondary schools; limited U.S. transfer)","publishedAt":"2026-5-15 (technical report header)","publicationDate":"2026-05-15","eventDate":"2025-12-05","sourceName":"Google DeepMind / LearnLM technical report","sourceLabel":"Vendor-affiliated preregistered classroom RCT technical report","sourceUrl":"https://storage.googleapis.com/deepmind-media/LearnLM/learnLM_sierraleone_may26.pdf","evidenceClass":"vendor-claim","outcomeClass":"mixed","topics":["knowledge-work","accessibility-workforce","operating-model","developers-agents"],"finding":"A preregistered two-arm classroom-clustered RCT (AEA RCT Registry AEARCTR-0016651) enrolled N=1763 students aged 13+ across 48 grades 7–8 mathematics classrooms in 12 government-supported junior secondary schools in Port Loko District, Sierra Leone. Teachers were trained identically; treatment classrooms used Gemini Guided Learning (Pro) in half of weekly math lessons for a requested 12 hours over eight weeks (intervention window 6 Oct–5 Dec 2025), with paired student device use and a four-step teacher-led structure. Intent-to-treat learning gains were +0.258 SD (p=0.029; 95% CI [0.027, 0.488]) on independently developed IRT-scaled assessments; treatment-on-the-treated at the 12-hour threshold was +0.380 SD. Conversational logs showed scaffolding-heavy Gemini behavior (76.4% scaffolding questions; 2.1% direct solutions) and 91.4% student effort toward understanding rather than answer-seeking. Heterogeneity: each additional SD of baseline math proficiency increased the treatment effect by +0.195 SD (p=0.002)—stronger students benefited more. Authors flag literacy/digital-literacy barriers and note the pattern can widen gaps. Why now: not archived; newly relevant after the 2026-10-07 k12 edition’s Utah district divergence on student Gemini access, showing that teacher-structured Guided Learning can move measured math outcomes while still requiring equity and transfer tests before U.S. district claims.","sledRelevance":"Relevant to K-12 math instructional design and student GenAI access debates: measured gains appear when AI is embedded in teacher-designed, paired, discussion-backed lessons—not as freeform homework chat. U.S. districts should not copy effect sizes without local pilots, given LMIC context, device ratios, and vendor authorship.","evidence":"Preregistered RCT with clustered randomization, independent Oxford MeasurEd assessments, ethics approvals, parental consent/student assent, automated de-identification of transcripts, and focus groups. Vendor (Google LearnLM) authorship with Fab AI collaboration and Google.org support is a conflict-of-interest limitation. Model versions changed mid-trial (Gemini 2.5 Pro then 3.0 Pro). Short eight-week dosage; no U.S. replication in this report.","architectureImplications":"Classroom pattern assumes shared devices (~2:1), teacher-authored starter prompts, Gemini Guided Learning mode rather than unconstrained chat, and logging/de-identification pipelines for quality review.","governanceImplications":"Interpretation: approve student GenAI for math only inside structured lesson protocols with teacher design and whole-class consolidation; require equity analysis because higher-baseline students gained more.","securityPrivacyImplications":"Authors report automated PII de-identification before transcript analysis; districts would still need local DPAs, age/consent rules, and limits on retaining student conversations.","caveats":"Vendor-affiliated report; Sierra Leone junior-secondary context differs from typical U.S. districts; mid-trial model upgrade; effect concentrated among stronger baseline students; focus-group positivity may not generalize; no long-term retention test beyond endline.","streamIds":["k12"],"roles":{"sales":"Sales framing: the credible story is teacher-led Guided Learning with measured +0.258 SD ITT in a preregistered RCT—not “Gemini raises scores everywhere.” Bound claims to structured classroom protocols, disclose vendor authorship, and insist on local pilots because stronger baseline students gained more and the setting is Sierra Leone junior secondary.","engineering":"Engineering framing: prefer pedagogically constrained Guided Learning modes, teacher-authored prompts, paired-device sessions, conversation logging with de-identification, and independent outcome measures. Do not equate unconstrained chat interfaces with this intervention.","delivery":"Delivery framing: replicate the four-step lesson wrapper (objectives, paired Guided Learning, class discussion, summary) with identical teacher training across arms in any pilot. Track dosage hours, baseline×treatment interactions, and literacy barriers before scaling."},"roleDetails":{"sales":{"customerProblem":"Districts hear vendor learning claims but lack preregistered classroom evidence tied to a specific pedagogical wrapper.","stakeholders":["Chief academic officer","Math curriculum leads","CIO","Research/evaluation office","Board"],"discoveryQuestions":["Will we run teacher-structured sessions or freeform chat?","How will we measure unassisted endline learning?","Do we have device ratios and teacher PD time?","How will we watch baseline×treatment equity gaps?"],"valueHypothesis":"A teacher-led Guided Learning pilot with independent assessment can test whether scaffolded GenAI moves math learning locally without copying Sierra Leone effect sizes.","potentialEngagement":"One-term clustered pilot in 8–12 math classrooms with identical teacher training, dosage logs, and independent pretest/posttest.","unsupportedClaims":["+0.258 SD will transfer to U.S. districts unchanged","Unguarded ChatGPT equals Guided Learning","Gap-closing for lowest performers (report shows opposite heterogeneity)"]},"engineering":{"fit":"Teacher-orchestrated Gemini Guided Learning inside math periods with shared devices and conversation analytics.","architectureIntegration":"Enable Guided Learning/Pro mode; provision shared tablets/desktops; capture de-identified transcripts; keep unconstrained answer-bots out of the pilot path.","prerequisites":["Teacher PD (5–6 hours in study)","Device access at ~2:1","Curriculum-aligned assessment partner","Consent/assent workflow"],"constraints":["Vendor-authored evidence","LMIC setting","Model version changes","Eight-week dosage"],"security":"Automate PII stripping on logs; minimize identifiable student data in prompts; restrict transcript access to evaluation staff.","proofOfValue":["IRT or equated pretest/posttest gain","Dosage hours vs outcomes","Baseline×treatment interaction","Share of scaffolding vs answer-seeking messages"]},"delivery":{"work":"Train teachers on the four-step wrapper, schedule two math periods/week, monitor uptake, run focus groups, and analyze equity heterogeneity.","dependencies":["Math department buy-in","Devices and connectivity","Independent assessment","Ethics/consent"],"ownership":"Math curriculum lead owns pedagogy; IT owns devices/accounts; evaluation office owns outcome analysis.","skillsAdoption":"Teachers must design starter prompts and lead consolidation discussions—not outsource lesson authorship to the model.","governanceCheckpoints":["Protocol registration or pre-analysis plan","Midpoint dosage check","Equity review of baseline interactions"],"acceptanceCriteria":["Documented ITT estimate with CI","Logged dosage distribution","Explicit decision on scale given equity pattern"],"risks":["Widening achievement gaps","Curriculum pacing disruption","Over-generalizing vendor RCT to U.S. policy"]}},"retrievedAt":"2026-10-08T22:03:20-05:00","enrichedAt":"2026-10-08T22:05:30-05:00","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Teachers reported literacy, typing, and foundational-knowledge barriers; audio-output requests suggest text-only Guided Learning may exclude some learners without additional supports.","procurementImplications":"If purchasing Guided Learning-capable tools, contract for classroom pedagogy support, logging/de-identification, and independent evaluation—not seats alone.","operatingModelImplications":"Positions GenAI as a teacher-designed instructional routine with peer pairing and whole-class consolidation, not an after-school homework chatbot.","sourceVerification":{"openedUrl":"https://storage.googleapis.com/deepmind-media/LearnLM/learnLM_sierraleone_may26.pdf","referenceExcerpt":"Guided Learning significantly improved student learning outcomes, with an intent-to-treat effect of +0.258 SD","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}