From the K–12 edition of October 8, 2026
Preregistered Sierra Leone RCT: teacher-led Gemini Guided Learning raises junior-secondary math +0.258 SD ITT, but stronger baseline students gain more
Google LearnLM Team & Fab AI · Vendor-affiliated education efficacy research (K-12 mathematics) · Sierra Leone (Port Loko District junior secondary schools; limited U.S. transfer)
- Publisher
- Google DeepMind / LearnLM technical report
- Original publication
- 2026-5-15 (technical report header)
- Source retrieved
- 2026-10-09
- Event date
- 2025-12-05
What happened
A preregistered two-arm classroom-clustered RCT (AEA RCT Registry AEARCTR-0016651) enrolled N=1763 students aged 13+ across 48 grades 7–8 mathematics classrooms in 12 government-supported junior secondary schools in Port Loko District, Sierra Leone. Teachers were trained identically; treatment classrooms used Gemini Guided Learning (Pro) in half of weekly math lessons for a requested 12 hours over eight weeks (intervention window 6 Oct–5 Dec 2025), with paired student device use and a four-step teacher-led structure. Intent-to-treat learning gains were +0.258 SD (p=0.029; 95% CI [0.027, 0.488]) on independently developed IRT-scaled assessments; treatment-on-the-treated at the 12-hour threshold was +0.380 SD. Conversational logs showed scaffolding-heavy Gemini behavior (76.4% scaffolding questions; 2.1% direct solutions) and 91.4% student effort toward understanding rather than answer-seeking. Heterogeneity: each additional SD of baseline math proficiency increased the treatment effect by +0.195 SD (p=0.002)—stronger students benefited more. Authors flag literacy/digital-literacy barriers and note the pattern can widen gaps. Why now: not archived; newly relevant after the 2026-10-07 k12 edition’s Utah district divergence on student Gemini access, showing that teacher-structured Guided Learning can move measured math outcomes while still requiring equity and transfer tests before U.S. district claims.
Why it matters
Relevant to K-12 math instructional design and student GenAI access debates: measured gains appear when AI is embedded in teacher-designed, paired, discussion-backed lessons—not as freeform homework chat. U.S. districts should not copy effect sizes without local pilots, given LMIC context, device ratios, and vendor authorship.
Evidence and measured results
Preregistered RCT with clustered randomization, independent Oxford MeasurEd assessments, ethics approvals, parental consent/student assent, automated de-identification of transcripts, and focus groups. Vendor (Google LearnLM) authorship with Fab AI collaboration and Google.org support is a conflict-of-interest limitation. Model versions changed mid-trial (Gemini 2.5 Pro then 3.0 Pro). Short eight-week dosage; no U.S. replication in this report.
Limitations and uncertainty
Vendor-affiliated report; Sierra Leone junior-secondary context differs from typical U.S. districts; mid-trial model upgrade; effect concentrated among stronger baseline students; focus-group positivity may not generalize; no long-term retention test beyond endline.
Put this evidence to work
Lighthouse Advisory interpretation, grounded in this source. Enriched 2026-10-08; this does not change the original publication date. Each role follows the same checklist so takeaways compare across sources.
Sales
Role takeaway
- Customer problem
- Districts hear vendor learning claims but lack preregistered classroom evidence tied to a specific pedagogical wrapper.
- Stakeholders
- Chief academic officer
- Math curriculum leads
- CIO
- Research/evaluation office
- Board
- Discovery questions
- Will we run teacher-structured sessions or freeform chat?
- How will we measure unassisted endline learning?
- Do we have device ratios and teacher PD time?
- How will we watch baseline×treatment equity gaps?
- Value hypothesis
- A teacher-led Guided Learning pilot with independent assessment can test whether scaffolded GenAI moves math learning locally without copying Sierra Leone effect sizes.
- Potential engagement
- One-term clustered pilot in 8–12 math classrooms with identical teacher training, dosage logs, and independent pretest/posttest.
- Unsupported claims
- +0.258 SD will transfer to U.S. districts unchanged
- Unguarded ChatGPT equals Guided Learning
- Gap-closing for lowest performers (report shows opposite heterogeneity)
Pre-sales engineering
Role takeaway
- Fit
- Teacher-orchestrated Gemini Guided Learning inside math periods with shared devices and conversation analytics.
- Architecture and integration
- Enable Guided Learning/Pro mode; provision shared tablets/desktops; capture de-identified transcripts; keep unconstrained answer-bots out of the pilot path.
- Prerequisites
- Teacher PD (5–6 hours in study)
- Device access at ~2:1
- Curriculum-aligned assessment partner
- Consent/assent workflow
- Constraints
- Vendor-authored evidence
- LMIC setting
- Model version changes
- Eight-week dosage
- Security
- Automate PII stripping on logs; minimize identifiable student data in prompts; restrict transcript access to evaluation staff.
- Proof of value
- IRT or equated pretest/posttest gain
- Dosage hours vs outcomes
- Baseline×treatment interaction
- Share of scaffolding vs answer-seeking messages
Delivery
Role takeaway
- Work
- Train teachers on the four-step wrapper, schedule two math periods/week, monitor uptake, run focus groups, and analyze equity heterogeneity.
- Dependencies
- Math department buy-in
- Devices and connectivity
- Independent assessment
- Ethics/consent
- Ownership
- Math curriculum lead owns pedagogy; IT owns devices/accounts; evaluation office owns outcome analysis.
- Skills and adoption
- Teachers must design starter prompts and lead consolidation discussions—not outsource lesson authorship to the model.
- Governance checkpoints
- Protocol registration or pre-analysis plan
- Midpoint dosage check
- Equity review of baseline interactions
- Acceptance criteria
- Documented ITT estimate with CI
- Logged dosage distribution
- Explicit decision on scale given equity pattern
- Risks
- Widening achievement gaps
- Curriculum pacing disruption
- Over-generalizing vendor RCT to U.S. policy
Implementation considerations
Lighthouse Advisory interpretation across the operating dimensions a public-sector buyer must settle before this evidence becomes a design. Each note answers the question under its heading for this specific source.
Architecture and integration
What must connect, and where does the AI sit in the workflow?
Classroom pattern assumes shared devices (~2:1), teacher-authored starter prompts, Gemini Guided Learning mode rather than unconstrained chat, and logging/de-identification pipelines for quality review.
Governance
Who approves, reviews and stays accountable for outcomes?
Approve student GenAI for math only inside structured lesson protocols with teacher design and whole-class consolidation; require equity analysis because higher-baseline students gained more.
Security and privacy
What data, permissions and controls need testing?
Authors report automated PII de-identification before transcript analysis; districts would still need local DPAs, age/consent rules, and limits on retaining student conversations.
Accessibility and workforce
Who is affected, and what skills or accommodations follow?
Teachers reported literacy, typing, and foundational-knowledge barriers; audio-output requests suggest text-only Guided Learning may exclude some learners without additional supports.
Procurement
What should contracts, pricing and exit terms secure?
If purchasing Guided Learning-capable tools, contract for classroom pedagogy support, logging/de-identification, and independent evaluation—not seats alone.
Operating model
Which teams own the service once it runs?
Positions GenAI as a teacher-designed instructional routine with peer pairing and whole-class consolidation, not an after-school homework chatbot.
Publication history
- 2026-10-08K–12 · Issue 324 resources
Stable resource ID: google-learnlm-sierra-leone-gemini-guided-learning-rct-20260515