From the K–12 edition of September 28, 2026
AEFP Live Handbook: K–12 AI evidence thin; assisted gains often fail independent proficiency tests
Association for Education Finance and Policy (AEFP) Live Handbook · Academic research synthesis · United States focus with international evidence (Stanford SCALE / research network authors)
- Publisher
- AEFP Live Handbook
- Original publication
- June 2026 (last updated)
- Source retrieved
- 2026-09-29
What happened
Cristina Barnard, Lily Fesler, and Susanna Loeb’s June 2026 AEFP Live Handbook chapter synthesizes that schools adopt AI faster than researchers can evaluate it: a recent review found only about 20 high-quality causal studies amid 800+ papers, with especially thin U.S. K–12 student-facing causal evidence. Key patterns include assisted performance that does not consistently become independent proficiency; design choices (guided reasoning vs answer generation) shaping learning; and educator-facing time savings whose value depends on redirecting attention to high-value human interactions. They cite RAND 2025 survey rates (~54% of secondary students and ~53% of core teachers using AI) to underscore the adoption–evidence gap.
Why it matters
Independent academic frame for distinguishing retained learning from assisted output—central to the K12 stream brief—and for challenging sales claims built on in-tool metrics.
Evidence and measured results
Peer-oriented research handbook chapter with cited RCTs/reviews; synthesis rather than a new primary trial.
Limitations and uncertainty
June 2026 update—older than last k12 run but not previously archived; why-now is direct contradiction of access-as-learning narratives after recent tutor engagement failures; many cited studies outside U.S. K–12.
Put this evidence to work
Lighthouse Advisory interpretation, grounded in this source. Enriched 2026-09-28; this does not change the original publication date. Each role follows the same checklist so takeaways compare across sources.
Sales
Role takeaway
- Customer problem
- Districts face pressure to adopt AI while rigorous U.S. K–12 causal evidence—and unaided mastery proof—remains thin.
- Stakeholders
- research/evaluation leads
- curriculum
- superintendents
- boards
- edtech vendors
- Discovery questions
- Do pilots include closed-book or unaided post-tests after AI practice?
- Does the tool generate answers or require student reasoning/hints?
- What share of cited evidence is U.S. K–12 causal versus postsecondary/international?
- If teachers save prep time, where is that time reallocated?
- Value hypothesis
- If buyers require unaided transfer metrics and reasoning-first designs, they can avoid procuring answer engines that inflate assisted performance without learning.
- Potential engagement
- Evidence-review workshop mapping a proposed AI pilot to AEFP key findings and drafting unaided outcome measures.
- Unsupported claims
- That AI adoption itself improves durable learning
- That assisted homework gains equal independent mastery
- That U.S. K–12 student-facing causal evidence is already robust
Pre-sales engineering
Role takeaway
- Fit
- Cross-cutting evaluation frame for any student- or teacher-facing AI pilot in K–12.
- Architecture and integration
- Instrument AI-on practice separately from AI-off assessments; log hint vs answer modes; export teacher time-use changes.
- Prerequisites
- Clear learning outcomes
- Ability to disable AI for assessment
- Baseline and delayed measures
- Privacy-compliant logging
- Constraints
- Causal U.S. student-facing evidence scarce
- Tools change faster than trials
- Short-term task metrics dominate literature
- Security
- Evaluation data still needs FERPA-aligned handling; companion-style social uses raise additional developmental risks noted as research gaps.
- Proof of value
- Unaided post-test non-inferiority vs control
- Hint-acceptance and reasoning-turn telemetry
- Teacher time reallocation log to instructional interactions
Delivery
Role takeaway
- Work
- Design AI pilot with assisted vs unaided measures, reasoning-first configuration, and end-of-pilot evidence review against handbook criteria.
- Dependencies
- assessment warehouse
- feature flags for AI modes
- IRB/research partnership if publishing
- teacher PD
- Ownership
- Evaluation/curriculum co-own outcomes; IT owns telemetry; principals own instructional time use.
- Skills and adoption
- Train teachers to treat AI as scaffold for struggle and to score unaided performance separately.
- Governance checkpoints
- Pre-pilot evidence review
- Mid-pilot unaided check
- Scale/no-scale cabinet decision
- Acceptance criteria
- Unaided assessment fielded
- Answer-generation mode off or tightly limited
- Documented time-reallocation plan for any teacher AI
- Risks
- Vendors report only in-tool gains
- Students cognitive-offload on general chatbots
- Overgeneralizing international RCTs to local U.S. contexts
Implementation considerations
Lighthouse Advisory interpretation across the operating dimensions a public-sector buyer must settle before this evidence becomes a design. Each note answers the question under its heading for this specific source.
Architecture and integration
What must connect, and where does the AI sit in the workflow?
Prefer tutoring-style scaffolds, unaided assessment hooks, and telemetry that separates help-seeking from answer delivery.
Governance
Who approves, reviews and stays accountable for outcomes?
Pilot governance should require durable-learning measures before scale; treat product-specific studies as perishable.
Security and privacy
What data, permissions and controls need testing?
Companion/social AI uses flagged as understudied wellbeing risks; still require standard student-data protections.
Accessibility and workforce
Who is affected, and what skills or accommodations follow?
Equity risks if high-quality tools and implementation supports concentrate in better-resourced districts; teacher PD capacity is uneven.
Procurement
What should contracts, pricing and exit terms secure?
Write RFPs around unaided outcomes and reasoning-first design, not feature checklists alone.
Operating model
Which teams own the service once it runs?
Evaluation cadence must keep pace with model changes; system-level AI scheduling/coordination uses remain largely unevaluated.
Publication history
- 2026-09-28K–12 · Issue 235 resources
Stable resource ID: aefp-live-handbook-ai-k12-202606