From the K–12 edition of October 26, 2026
Education Sciences reanalysis of a Nigerian secondary-school RCT: six weeks of teacher-guided Copilot tutoring raised overall assessment scores 0.31 SD, but AI's own contribution cannot be isolated
Universidad Peruana de Ciencias Aplicadas, Universidad Autónoma del Perú and co-authors; reanalysis of World Bank trial data (De Simone et al.) · K-12 (upper secondary) generative AI tutoring: English learning, AI knowledge and digital equity · Benin City, Edo State, Nigeria (nine public secondary schools)
- Publisher
- Education Sciences (MDPI)
- Original publication
- Published 23 October 2026 (Education Sciences 16(11), 1780; received 15 Sept, accepted 20 Oct 2026)
- Source retrieved
- 2026-10-27
What happened
Using the World Bank's public replication package, the authors re-estimated a trial in which 1,328 interested first-year upper-secondary students were randomized; 657 were offered twelve 90-minute after-school sessions over six weeks (June-July 2024) combining Microsoft Copilot, curriculum-aligned English activities, pair work and teacher guidance, while 671 received regular instruction. Intention-to-treat effects were 0.310 SD on the overall assessment (95% CI 0.176-0.444; n=654), 0.238 SD in English, 0.309 SD in AI knowledge and 0.206 SD on the regular third-term school English exam; digital skills (0.139 SD) was imprecise. Effects were larger for female students and higher prior achievers. The digital-access interaction was negative but imprecise (-0.182 SD). Assessments were observed for 64.2% of treated versus 50.2% of control students; inverse-probability weighting gave 0.298 SD, and excluding one large school lowered the estimate to 0.173 SD.
Why it matters
Why now: published this week and not in the archive, it is an independent reanalysis that complements archived cautionary evidence (homework-outsourcing learning penalty, sparse Khanmigo engagement). It supports a structured, teacher-guided package rather than open access. Transferability to U.S. districts is limited: Nigerian urban schools, volunteer students, after-school setting, older Copilot version and short follow-up.
Evidence and measured results
Randomized trial, secondary analysis; OLS with school fixed effects, baseline adjustment and HC3 errors; IPW, 2,000-permutation inference and leave-one-school-out checks. The authors corrected a coding error in the original internet-access variable. Limitations: differential attrition, no AI-free tutoring arm, no persistence data, no cost data, no multiple-comparison adjustment.
Limitations and uncertainty
Effect is for the complete package (extra time, curriculum, pair work, teacher guidance); AI's independent contribution is not identified. Endline observed for only 759 of 1,328 students with differential attrition. Interested volunteers, 79% female sample, six-week horizon, urban schools with labs. Secondary analysis by authors not involved in the original trial; the original World Bank working paper was not opened here.
Put this evidence to work
Lighthouse Advisory interpretation, grounded in this source. Enriched 2026-10-26; this does not change the original publication date. Each role follows the same checklist so takeaways compare across sources.
Sales
Role takeaway
- Customer problem
- Districts want evidence that generative AI tutoring improves learning, while archived studies warn that unstructured use can harm it.
- Stakeholders
- Curriculum and instruction leaders
- Teachers who would facilitate sessions
- Research and assessment office
- Technology department
- Families of participating students
- Discovery questions
- Would a pilot include an AI-free tutoring comparison?
- What curriculum-aligned materials and prompts exist?
- Who facilitates sessions, and with what training?
- How will attendance and attrition be tracked?
- Which independent assessment will measure outcomes?
- Value hypothesis
- A structured, teacher-guided AI tutoring pilot with independent measurement could show whether gains like those in Nigeria transfer locally; this is a hypothesis.
- Potential engagement
- A bounded pilot design engagement: package design, comparison-arm setup and evaluation plan for one subject and grade.
- Unsupported claims
- That AI alone caused the 0.31 SD gain
- That gains persist beyond six weeks
- Cost-effectiveness
- That results transfer directly to U.S. classrooms
Pre-sales engineering
Role takeaway
- Fit
- Supervised, curriculum-scripted AI tutoring on school-managed devices with session logging.
- Architecture and integration
- Interpretation: deploy an approved model with pinned versions, preloaded prompts and logs linked to attendance records.
- Prerequisites
- Approved AI tool with student accounts
- Lab or device access
- Curriculum-aligned prompt library
- Logging and attendance capture
- Constraints
- Connectivity and power reliability
- Age and consent rules for student accounts
- Model version changes
- Security
- Minimize student data in prompts; document data storage and jurisdiction; log incidents.
- Proof of value
- Run an AI-free tutoring comparison arm
- Track attrition by arm
- Measure outcomes on an independent test, not only program-aligned items
Delivery
Role takeaway
- Work
- Build and run a structured tutoring package with facilitation, materials, attendance tracking and independent evaluation.
- Dependencies
- Teacher facilitator capacity
- Device and network readiness
- Evaluation partner
- Ownership
- Curriculum and instruction office with the research office.
- Skills and adoption
- Teachers need training in facilitation, verification of AI output and spotting dependency.
- Governance checkpoints
- Privacy review of the AI tool
- Mid-pilot attendance and attrition review
- Independent evaluation sign-off
- Acceptance criteria
- Attendance target set and tracked
- Attrition balanced across arms within a set margin
- Independent endline assessment completed
- Risks
- Widening achievement gaps
- Differential attrition biasing results
- Model behavior changes mid-pilot
Implementation considerations
Lighthouse Advisory interpretation across the operating dimensions a public-sector buyer must settle before this evidence becomes a design. Each note answers the question under its heading for this specific source.
Architecture and integration
What must connect, and where does the AI sit in the workflow?
Delivered in school computer labs with routers and mobile signal; power and internet disruptions occurred but bandwidth was not measured. The authors argue low-connectivity designs (offline content, small payloads) need separate testing.
Governance
Who approves, reviews and stays accountable for outcomes?
The authors recommend defining curriculum goals and tutor behavior guidelines before choosing a model, logging incidents and model versions, and negotiating data protection and continuity if licenses or models change.
Security and privacy
What data, permissions and controls need testing?
Student accounts on a commercial model raise age, consent, storage and jurisdiction questions; the article notes model behavior can change without notice. No privacy incidents are reported.
Accessibility and workforce
Who is affected, and what skills or accommodations follow?
Larger effects for higher prior achievers raise gap-widening risk; the digital-access interaction is imprecise. Teacher supervision time is not free, the authors note.
Procurement
What should contracts, pricing and exit terms secure?
The authors caution that pilots may rely on free licenses or sunk costs and that total cost must include electricity, routers, equipment, moderation, data protection and support.
Operating model
Which teams own the service once it runs?
Teachers acted as facilitators rather than delivering a second lesson; pair work and supervision were core to the package.
Publication history
- 2026-10-26K–12 · Issue 505 resources
Stable resource ID: mdpi-nigeria-teacher-guided-copilot-tutoring-rct-reanalysis-0-31sd-digital-equity-20261023