From the K–12 edition of September 20, 2026
EEF/NFER Maths-Whizz RCT: +1 month maths progress overall; +2 months for FSM pupils (g=0.08/0.14)
Education Endowment Foundation (evaluation by National Foundation for Educational Research) · K-12 primary mathematics intelligent tutoring evaluation · England, United Kingdom (63 primary schools; Years 2–5)
- Publisher
- Evaluation of Whizz Education: Maths-Whizz Intelligent Tutoring Programme (EEF PDF)
- Original publication
- June 2026 EEF evaluation report (NFER efficacy trial)
- Source retrieved
- 2026-09-21
- Event date
- 2025-07-01
What happened
In an interleaved RCT across 63 English primary schools with 10,369 pupils (Years 2–5), allocation to Maths-Whizz—an AI-enabled adaptive tutoring programme intended for 45–60 minutes per week alongside usual maths teaching—produced an average of one additional month of maths progress on Renaissance Star Maths (effect size 0.08, 95% CI 0.02–0.14; n=8,983; p=0.01) with a high EEF security rating. FSM-eligible pupils gained about two additional months (effect size 0.14, 95% CI 0.06–0.22; n=2,301; p=0.01), though that subgroup estimate carries lower security due to smaller n. Average delivery cost was about £21.51 per pupil per year for two year groups. Authors highlight personalised scaffolding and gap-filling as plausible mechanisms for stronger FSM effects.
Why it matters
Provides scarce high-security causal evidence that a purpose-built adaptive AI maths tutor can raise primary attainment—with larger gains for disadvantaged pupils—informing U.S. district tutoring procurement and equity claims while requiring UK-to-U.S. transferability caveats.
Evidence and measured results
Opened the EEF CloudFront evaluation PDF end-to-end via curl and pdftotext. Verified Table 1 key conclusions, Table 2 effect sizes/security/cost ratings, sample (63 schools; 10,369 baseline pupils), dosage guidance (45–60 minutes/week), and high-security language for the primary finding.
Limitations and uncertainty
UK primary efficacy trial (not U.S. K-12); vendor programme with school co-pay; FSM subgroup lower security; implementation fidelity and home access vary. Why now: first unarchived large independent AI-tutor RCT with equity-positive primary maths results, contemporaneous with U.S. homework-GenAI learning-cost and state AI-control debates.
Put this evidence to work
Lighthouse Advisory interpretation, grounded in this source. Enriched 2026-09-21; this does not change the original publication date. Each role follows the same checklist so takeaways compare across sources.
Sales
Role takeaway
- Customer problem
- Districts lack independent causal evidence that adaptive AI maths tutoring improves attainment—especially for economically disadvantaged students.
- Stakeholders
- Superintendents
- Chief academic officers
- Title I / equity leads
- CTOs
- Board research liaisons
- Discovery questions
- Can you schedule 45–60 minutes/week of adaptive maths practice without replacing core instruction?
- Will you accept UK RCT transferability with a local interim replication?
- How will you measure FSM/FRL subgroup effects on Star/MAP?
- What independent evaluation standard will the board require before scale?
- Value hypothesis
- Purpose-built adaptive AI tutoring can deliver roughly one month of maths progress overall and larger equity gains when dosage and interim assessments are measurable—without promising U.S. causal equivalence.
- Potential engagement
- One-year efficacy-style pilot: PD, dosage telemetry, and pre-registered interim maths analysis with published caveats.
- Unsupported claims
- Guaranteed U.S. causal gains identical to EEF results
- That general-purpose chatbots produce the same equity effects
- Cost savings that replace human tutoring entirely
Pre-sales engineering
Role takeaway
- Fit
- Fit for districts with device access, interim maths assessments, and capacity to schedule supplemental adaptive practice; poor fit where AI use is unmanaged consumer chat.
- Architecture and integration
- Student IdP → adaptive tutor tenancy → dosage warehouse → roster join to interim maths assessments; teacher resource library optional.
- Prerequisites
- Devices and Wi-Fi for concurrent classroom sessions
- Interim maths assessment extracts
- Teacher PD time for facilitation sessions
- Data-sharing agreement for dosage–score joins
- Constraints
- UK evidence base may not transfer to U.S. curricula
- FSM subgroup estimate has lower security
- Home access inequality can affect dosage
- Security
- Keep pupil identifiers and usage logs in enterprise tenancy; minimize PII in vendor exports; align with FERPA/state student-privacy statutes.
- Proof of value
- Reproduce dosage distributions vs 45–60 minute target
- Replicate overall and FRL/FSM interim effect directions
- Document high-security vs lower-security subgroup caveats
- Show cost-per-pupil within district budget envelope
Delivery
Role takeaway
- Work
- Onboard year groups, train teachers on engagement facilitation, monitor weekly dosage, administer endpoint interim maths, and brief leadership on months-of-progress and equity contrasts.
- Dependencies
- Assessment vendor accounts
- School timetable slots
- Vendor implementation support
- Privacy review clearance
- Ownership
- Academic/teaching-and-learning lead owns outcomes; CTO owns platform and data joins; equity lead owns subgroup reporting.
- Skills and adoption
- Teachers need short PD on facilitating engagement and interpreting dashboards; avoid treating the tutor as unsupervised homework replacement.
- Governance checkpoints
- Pre-registration of primary outcome and subgroups
- Midyear dosage fidelity review
- Endpoint limitations briefing before scale decision
- Acceptance criteria
- Target share of pupils hitting weekly dosage threshold
- Published interim effect estimate with CI and caveats
- FRL/FSM subgroup table delivered to board
- Go/no-go memo signed by CAO and CTO
- Risks
- Attrition and test-day absence bias
- Over-claiming UK results as U.S. proof
- Dosage collapse when home access is weak
Implementation considerations
Lighthouse Advisory interpretation across the operating dimensions a public-sector buyer must settle before this evidence becomes a design. Each note answers the question under its heading for this specific source.
Architecture and integration
What must connect, and where does the AI sit in the workflow?
Treat adaptive tutoring as a scheduled supplemental CAL workload (SSO, device density, ~0.5 Mbps per concurrent pupil) with dosage telemetry joinable to interim maths assessments—not as unmanaged general-purpose chat.
Governance
Who approves, reviews and stays accountable for outcomes?
Pre-register Star-like interim outcomes and FSM/equity subgroup analyses; publish security/limitations slides matching independent evaluator standards before scaling licenses.
Security and privacy
What data, permissions and controls need testing?
Pupil identifiers, usage logs, and assessment joins require DPA/minimisation; UK trial used UPNs and school-held data—U.S. districts must map to FERPA and state student-privacy rules before similar evaluation designs.
Accessibility and workforce
Who is affected, and what skills or accommodations follow?
Stronger FSM effects imply adaptive tutors may help close maths gaps if schools ensure device access and scheduled practice for disadvantaged pupils; PD must keep teachers facilitating engagement rather than offloading instruction.
Procurement
What should contracts, pricing and exit terms secure?
Prefer independent efficacy evidence and per-pupil cost transparency; require dosage reporting, interim assessment joins, and equity subgroup analyses in contracts before multi-year commitments.
Operating model
Which teams own the service once it runs?
Operate as supplemental weekly tutoring blocks with fidelity monitoring—not as optional homework AI—so dosage, not license counts, drives the operating model.
Publication history
- 2026-09-20K–12 · Issue 154 resources
Stable resource ID: eef-nfer-maths-whizz-rct-2026