From the K–12 edition of October 29, 2026
Google LearnLM team and Eedi: pre-registered RCT of 1,179 UK secondary students finds supervised AI tutoring matched expert human tutors in-lesson but neither beat standard remediation on Star Maths after 10 weeks
Google (LearnLM team) and Eedi · K-12 student-facing AI tutoring (secondary mathematics) · United Kingdom (10 state secondary schools)
- Publisher
- Google LearnLM technical report
- Original publication
- October 29, 2026 (technical report dated 2026-10-29; Eedi release same day)
- Source retrieved
- 2026-10-30
What happened
A pre-registered RCT (AEARCTR-0018079) across 10 UK state secondary schools and 58 classrooms assigned 1,179 students for a 10-week term to standard Eedi remediation, expert human tutoring, AI tutoring (Gemini with a pedagogical prompt), or AI tutoring with personalised learning-history context. Tutors reviewed every AI draft and approved 94.7% of 94,385 drafts unchanged; audits found maths errors in fewer than three per 1,000 AI messages, comparable to human tutors. On in-platform measures and a mid-trial quiz, both AI arms performed similarly to human tutors and better than standard remediation. On the externally scored Star Maths test, tutoring showed no overall advantage over standard remediation. Exploratory results favoured girls, while tutored boys trailed, and tutored students covered fewer study units.
Why it matters
Directly K-12 and new tonight: one of the larger school-based AI tutoring RCTs, published with subgroup and conversation-level detail. It separates assisted, in-the-moment performance from retained broad learning, which is the distinction the K-12 brief asks for.
Evidence and measured results
Pre-registered RCT, Bayesian analysis with 95% credible intervals; outcomes at three levels (in-platform, mid-trial quiz, Star Maths at baseline and endline; Star Maths analysis covers students who sat both tests). Each student kept one condition for the whole trial. Authored by the vendor teams that built the tutor.
Limitations and uncertainty
Vendor-authored, not peer reviewed. Ten weeks only; one all-boys school meant boys were 66.1% of the sample; gender findings are exploratory. Human-in-the-loop supervision of every message is costly and differs from the human-on-the-loop model Eedi is now testing. UK context limits transfer to U.S. districts.
Put this evidence to work
Lighthouse Advisory interpretation, grounded in this source. Enriched 2026-10-29; this does not change the original publication date. Each role follows the same checklist so takeaways compare across sources.
Sales
Role takeaway
- Customer problem
- Districts want scalable tutoring, but evidence that AI tutoring improves broad learning, not just in-session answers, is still thin.
- Stakeholders
- District maths and curriculum leaders
- Assessment and research office
- Teachers supervising sessions
- Families
- Tutoring vendors
- Discovery questions
- What broad outcome measure will a pilot use at baseline and endline?
- Will results be reported by gender and prior attainment?
- Who reviews AI messages, and how many students per reviewer?
- How long can the pilot run?
- Value hypothesis
- A constrained AI tutor may match human tutors on in-session remediation at lower marginal cost; broad learning gains are unproven.
- Potential engagement
- Evaluation design support for a one-term AI tutoring pilot with a standardized outcome and subgroup reporting.
- Unsupported claims
- That AI tutoring raises standardized test scores
- That AI tutoring can replace human tutors
- That results transfer unchanged to U.S. schools
Pre-sales engineering
Role takeaway
- Fit
- Constrained, misconception-aware AI tutor inside a maths practice platform.
- Architecture and integration
- Interpretation: pass the diagnosed misconception and current item to the model, keep a reviewer approval path, and log all conversations for audit.
- Prerequisites
- Diagnostic item bank with misconception tags
- Reviewer staffing during early use
- Conversation logging and audit process
- Constraints
- Student privacy law in the deploying jurisdiction
- Reviewer workload when supervising several students
- Vendor-controlled model updates
- Security
- Confirm retention and secondary use of student conversation data, and that safety audits cover tone and maths errors.
- Proof of value
- Audit a sample of AI messages for maths errors and tone
- Compare in-session gains with a broad standardized measure
- Check results by gender and prior attainment
Delivery
Role takeaway
- Work
- Run a teacher-present AI tutoring pilot with pre-registered measures at near-term and broad levels.
- Dependencies
- Baseline and endline assessment
- Reviewer or tutor staffing
- Teacher scheduling of weekly sessions
- Ownership
- District maths or assessment office with participating school leaders.
- Skills and adoption
- Reviewers need training on fast approval decisions; teachers need guidance for students who disengage or ask to stop.
- Governance checkpoints
- Pre-registration of measures
- Mid-term safety and engagement review
- Board briefing before scale-up
- Acceptance criteria
- Broad outcome measured at baseline and endline
- Subgroup results reported
- Error rate in audited AI messages tracked against human tutors
- Risks
- In-session gains without broad learning
- Uneven effects by gender
- Reduced curriculum coverage
Implementation considerations
Lighthouse Advisory interpretation across the operating dimensions a public-sector buyer must settle before this evidence becomes a design. Each note answers the question under its heading for this specific source.
Architecture and integration
What must connect, and where does the AI sit in the workflow?
The tutor is not an off-the-shelf chatbot: Gemini runs inside Eedi's constrained platform that supplies the diagnosed misconception and current question; adding personalised history did not clearly improve outcomes. Every message passed a human approval step.
Governance
Who approves, reviews and stays accountable for outcomes?
Districts judging AI tutors should ask for broad, externally scored outcomes and subgroup results, not only in-session accuracy. The authors themselves call for full-year trials before guiding policy.
Security and privacy
What data, permissions and controls need testing?
Limited relevance: the report discusses safety review of messages (no harmful content reached students) but this run did not inspect data-handling terms; conversation data from minors was analysed by the vendors.
Accessibility and workforce
Who is affected, and what skills or accommodations follow?
Tutors found AI drafts helpful and time-saving but supervising them less rewarding than tutoring directly; typed chat may favour confident readers and writers.
Procurement
What should contracts, pricing and exit terms secure?
Contracts for AI tutoring should require outcome and safety reporting similar to what this trial published; cost was not reported.
Operating model
Which teams own the service once it runs?
Human-in-the-loop approval of every message was used here; the vendor is moving to human-on-the-loop oversight in a 15,000-student year-long trial.
Publication history
- 2026-10-29K–12 · Issue 534 resources
Stable resource ID: google-eedi-expanded-uk-ai-tutoring-rct-1179-students-no-star-maths-gain-20261029