K-12Issue 534 records
K–12
Google and Eedi released a pre-registered trial of 1,179 UK secondary students: a supervised AI tutor matched expert human tutors during lessons, but after 10 weeks neither beat standard practice on a broad maths test, and boys gained less. A CEPR study of 26,811 Chinese students adds a warning that self-directed use of general AI raised homework scores while closed-book exam scores fell. Anne Arundel County will adopt its AI policy in December, after Maryland's Oct. 22 deadline.
- Evidence records
- 4
- Cross-source patterns
- 1
- Evidence classes
- 2 independent reporting1 vendor claim1 academic research
- Outcomes
- 2 mixed1 cautionary1 emerging
- Source freshness
- 3 new this fortnight1 older, newly relevant
- Research completed
- 2026-10-30
Choose a role to see its takeaway beside every record in the ledger.
Patterns across the evidence
Synthesis · Lighthouse Advisory interpretation
AI help shows up in the task but not yet on broad or closed-book tests
The Google and Eedi trial found supervised AI tutoring matched human tutors in-lesson but did not beat standard remediation on Star Maths after 10 weeks, and reporting on it stressed the flat result. The CEPR study found general-purpose AI raised homework scores while closed-book exam scores fell over months. Different tools and designs, but both show that task-level performance is a weak proxy for retained learning.
Operating questionFor any AI tool students use, what broad, unassisted measure will the district track at baseline and endline, and for how long?
Supporting evidenceGoogle (LearnLM team) and EediCentre for Economic Policy Research (Strömberg, Lei and Wu)Yahoo News (byline Greg Toppo)
Evidence ledger
Full record · every source keeps its link and limitations
Google LearnLM team and Eedi: pre-registered RCT of 1,179 UK secondary students finds supervised AI tutoring matched expert human tutors in-lesson but neither beat standard remediation on Star Maths after 10 weeks
A pre-registered RCT (AEARCTR-0018079) across 10 UK state secondary schools and 58 classrooms assigned 1,179 students for a 10-week term to standard Eedi remediation, expert human tutoring, AI tutoring (Gemini with a pedagogical prompt), or AI tutoring with personalised learning-history context. Tutors reviewed every AI draft and approved 94.7% of 94,385 drafts unchanged; audits found maths errors in fewer than three per 1,000 AI messages, comparable to human tutors. On in-platform measures and a mid-trial quiz, both AI arms performed similarly to human tutors and better than standard remediation. On the externally scored Star Maths test, tutoring showed no overall advantage over standard remediation. Exploratory results favoured girls, while tutored boys trailed, and tutored students covered fewer study units.
Why it matters, evidence and limitations
- Why it matters
- Directly K-12 and new tonight: one of the larger school-based AI tutoring RCTs, published with subgroup and conversation-level detail. It separates assisted, in-the-moment performance from retained broad learning, which is the distinction the K-12 brief asks for.
- Evidence and measured results
- Pre-registered RCT, Bayesian analysis with 95% credible intervals; outcomes at three levels (in-platform, mid-trial quiz, Star Maths at baseline and endline; Star Maths analysis covers students who sat both tests). Each student kept one condition for the whole trial. Authored by the vendor teams that built the tutor.
- Limitations and uncertainty
- Vendor-authored, not peer reviewed. Ten weeks only; one all-boys school meant boys were 66.1% of the sample; gender findings are exploratory. Human-in-the-loop supervision of every message is costly and differs from the human-on-the-loop model Eedi is now testing. UK context limits transfer to U.S. districts.
Yahoo News (Greg Toppo): five takeaways from the Eedi AI tutor study, including flat Star Maths results, weaker effects for boys, an overly Socratic AI and one in five students never replying
The article reports that the trial ran April through July with weekly 30-35 minute class sessions and a teacher in the room. Tutored students answered follow-up questions correctly 56-58% of the time versus 43% after standard remediation, but neither AI nor human tutoring moved Star Maths scores. It reports that the AI asked follow-up mastery questions more than 90% of the time versus about 60% for humans, with Eedi's Bibi Groot saying it was sometimes excessively Socratic. More than 80% of students engaged, but about one in five never replied when a tutor appeared. Groot said human review of every message is not scalable.
Why it matters, evidence and limitations
- Why it matters
- K-12 and current: independent reporting adds operational context to the same-day trial results, especially engagement and how vendors interpret null broad-test results.
- Evidence and measured results
- News article based largely on an interview with Eedi's chief impact officer and the released findings; no outside researcher is quoted on the methods.
- Limitations and uncertainty
- Relies heavily on the vendor's own spokesperson; overlaps with the technical report, so it is not independent confirmation of the results. Cost figures are not given.
CEPR DP21577: among 26,811 Chinese students in grades 7-12, self-directed generative AI use raised homework scores 18% but cut closed-book exam scores 20% within six months
Using 30 months of panel data on 26,811 students in grades 7-12 and a staggered difference-in-differences design, the authors find AI adoption raised homework scores 18% and cut completion time 30%, but lowered monthly closed-book exam scores 20% within six months. High-stakes entrance exam scores fell 18% and 24%, with the full penalty after about two years. Losses were largest in social science, and for junior, high-achieving and male students. About 80% of AI users showed homework-outsourcing behaviour; users who kept similar homework time had small losses.
Why it matters, evidence and limitations
- Why it matters
- Newly relevant tonight: published in June but not archived, and it is the clearest contrast to the Google and Eedi trial released today. That trial tested a constrained tutor for 10 weeks; this study tracks self-directed use of general-purpose tools for 30 months and finds that broad losses take months to appear.
- Evidence and measured results
- Observational administrative panel with staggered adoption; adoption timing from teacher-distributed student surveys. Not randomised. Abstract page inspected; full paper is paywalled.
- Limitations and uncertainty
- Not randomised; students chose when to adopt; self-reported adoption timing. Chinese exam-driven context limits transfer to U.S. districts. Discussion paper, not peer reviewed. Only the abstract was inspected.
Anne Arundel County (MD) board opens first reading of AI responsible-use Policy DM; adoption set for Dec. 16, missing Maryland's Oct. 22 district AI policy deadline
The board held the first reading of Policy DM on Oct. 21; it opens 30 days of public comment, with a second reading Dec. 2 and adoption Dec. 16. The draft says AI should support people, not replace their judgment, and covers oversight, privacy, equity, student use, employee training and tool vetting. Maryland SB 720 requires districts to adopt aligned AI policies, designate an AI coordinator, use a state rubric to evaluate tools and bars AI from final decisions on discipline or eligibility; the state deadline was Oct. 22. The article says Howard, Baltimore County and Baltimore City were also expected to miss it, and seven community sessions run Oct. 27 to Nov. 19.
Why it matters, evidence and limitations
- Why it matters
- K-12 and current: shows a state AI-policy mandate meeting local board procedure, with several large Maryland districts missing the deadline. Follows up on the Anne Arundel feedback sessions noted in the last edition.
- Evidence and measured results
- Secondary summary attributing facts to a district release, the Capital Gazette, CBS News Baltimore, Hoodline and the state law; marked as a developing story.
- Limitations and uncertainty
- Secondary aggregation, not original reporting; facts not independently verified against the policy text. Details of the state rubric were not inspected.
How to read this edition
Source findings, measured results and limitations come from the cited publications. Patterns, operating questions, role takeaways and implementation considerations are Lighthouse Advisory interpretation, stated as questions to validate locally rather than guaranteed outcomes. Vendor and operator claims are labeled as claims. Full research method.
- Independent reporting
- Independent reporting with attributable sources but without a formal evaluation design.
- Vendor claim
- A supplier-provided assertion that has not been upgraded to independent evidence.
- Academic research
- Research produced through an academic institution or peer-reviewed venue.