{"resourceId":"aefp-live-handbook-ai-k12-202606","versions":[{"version":"external-c66c97250de9b52cf451128562cec3b4f7e0408ec4524c1e95077bb2b8a30db8","resource":{"id":"aefp-live-handbook-ai-k12-202606","title":"AEFP Live Handbook: K–12 AI evidence thin; assisted gains often fail independent proficiency tests","organization":"Association for Education Finance and Policy (AEFP) Live Handbook","sector":"Academic research synthesis","geography":"United States focus with international evidence (Stanford SCALE / research network authors)","publishedAt":"June 2026 (last updated)","publicationDate":"2026-06-01","eventDate":null,"sourceName":"AEFP Live Handbook","sourceLabel":"Artificial Intelligence in K–12 Schools (Barnard, Fesler, Loeb)","sourceUrl":"https://livehandbook.org/k-12-education/miscellaneous/k-12-education/school-resources/artificial-intelligence-in-k%E2%80%9312-schools/","evidenceClass":"academic-research","outcomeClass":"cautionary","topics":["knowledge-work","operating-model","accessibility-workforce"],"finding":"Cristina Barnard, Lily Fesler, and Susanna Loeb’s June 2026 AEFP Live Handbook chapter synthesizes that schools adopt AI faster than researchers can evaluate it: a recent review found only about 20 high-quality causal studies amid 800+ papers, with especially thin U.S. K–12 student-facing causal evidence. Key patterns include assisted performance that does not consistently become independent proficiency; design choices (guided reasoning vs answer generation) shaping learning; and educator-facing time savings whose value depends on redirecting attention to high-value human interactions. They cite RAND 2025 survey rates (~54% of secondary students and ~53% of core teachers using AI) to underscore the adoption–evidence gap.","sledRelevance":"Independent academic frame for distinguishing retained learning from assisted output—central to the K12 stream brief—and for challenging sales claims built on in-tool metrics.","evidence":"Peer-oriented research handbook chapter with cited RCTs/reviews; synthesis rather than a new primary trial.","architectureImplications":"Prefer tutoring-style scaffolds, unaided assessment hooks, and telemetry that separates help-seeking from answer delivery.","governanceImplications":"Pilot governance should require durable-learning measures before scale; treat product-specific studies as perishable.","securityPrivacyImplications":"Companion/social AI uses flagged as understudied wellbeing risks; still require standard student-data protections.","caveats":"June 2026 update—older than last k12 run but not previously archived; why-now is direct contradiction of access-as-learning narratives after recent tutor engagement failures; many cited studies outside U.S. K–12.","streamIds":["k12"],"roles":{"sales":"Sales framing: AEFP’s June 2026 Live Handbook chapter by Barnard, Fesler, and Loeb warns that schools are adopting AI faster than researchers can evaluate it—especially for U.S. student-facing causal evidence. Their synthesis finds assisted task gains often fail to become independent proficiency, and design (hints/reasoning vs answer generation) drives whether tools help or harm. Do not sell license counts as learning; insist on unaided transfer measures and tutoring-style guardrails. Interpretation of a research synthesis; not a product trial.","engineering":"Engineering framing: Treat 'performance with AI on' as a misleading success metric. The handbook highlights experiments where general-purpose chatbots raised practice scores but hurt closed-book exams, while tutoring-style hint systems avoided that decline without always beating no-AI baselines. Prefer architectures that force explanation, revision, and struggle; capture unaided assessments; and separate educator-facing time-savings tools from student answer machines. Interpretation for evaluation design, limited by thin U.S. K–12 causal literature.","delivery":"Delivery framing: Build pilots that measure durable learning without the tool, not only in-tool accuracy. Use the handbook’s experience lens—personalized instruction, complex problem-solving, discussion, and sustained educator relationships—as explicit go/no-go criteria before scale. Redirect any teacher time saved into human instructional interactions rather than claiming hour reductions alone as success. Interpretation for SLED evaluation plans; the evidence base remains early and often international or postsecondary."},"roleDetails":{"sales":{"customerProblem":"Districts face pressure to adopt AI while rigorous U.S. K–12 causal evidence—and unaided mastery proof—remains thin.","stakeholders":["research/evaluation leads","curriculum","superintendents","boards","edtech vendors"],"discoveryQuestions":["Do pilots include closed-book or unaided post-tests after AI practice?","Does the tool generate answers or require student reasoning/hints?","What share of cited evidence is U.S. K–12 causal versus postsecondary/international?","If teachers save prep time, where is that time reallocated?"],"valueHypothesis":"If buyers require unaided transfer metrics and reasoning-first designs, they can avoid procuring answer engines that inflate assisted performance without learning.","potentialEngagement":"Evidence-review workshop mapping a proposed AI pilot to AEFP key findings and drafting unaided outcome measures.","unsupportedClaims":["That AI adoption itself improves durable learning","That assisted homework gains equal independent mastery","That U.S. K–12 student-facing causal evidence is already robust"]},"engineering":{"fit":"Cross-cutting evaluation frame for any student- or teacher-facing AI pilot in K–12.","architectureIntegration":"Instrument AI-on practice separately from AI-off assessments; log hint vs answer modes; export teacher time-use changes.","prerequisites":["Clear learning outcomes","Ability to disable AI for assessment","Baseline and delayed measures","Privacy-compliant logging"],"constraints":["Causal U.S. student-facing evidence scarce","Tools change faster than trials","Short-term task metrics dominate literature"],"security":"Evaluation data still needs FERPA-aligned handling; companion-style social uses raise additional developmental risks noted as research gaps.","proofOfValue":["Unaided post-test non-inferiority vs control","Hint-acceptance and reasoning-turn telemetry","Teacher time reallocation log to instructional interactions"]},"delivery":{"work":"Design AI pilot with assisted vs unaided measures, reasoning-first configuration, and end-of-pilot evidence review against handbook criteria.","dependencies":["assessment warehouse","feature flags for AI modes","IRB/research partnership if publishing","teacher PD"],"ownership":"Evaluation/curriculum co-own outcomes; IT owns telemetry; principals own instructional time use.","skillsAdoption":"Train teachers to treat AI as scaffold for struggle and to score unaided performance separately.","governanceCheckpoints":["Pre-pilot evidence review","Mid-pilot unaided check","Scale/no-scale cabinet decision"],"acceptanceCriteria":["Unaided assessment fielded","Answer-generation mode off or tightly limited","Documented time-reallocation plan for any teacher AI"],"risks":["Vendors report only in-tool gains","Students cognitive-offload on general chatbots","Overgeneralizing international RCTs to local U.S. contexts"]}},"retrievedAt":"2026-09-28T22:07:10-05:00","enrichedAt":"2026-09-28T22:10:30-05:00","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Equity risks if high-quality tools and implementation supports concentrate in better-resourced districts; teacher PD capacity is uneven.","procurementImplications":"Write RFPs around unaided outcomes and reasoning-first design, not feature checklists alone.","operatingModelImplications":"Evaluation cadence must keep pace with model changes; system-level AI scheduling/coordination uses remain largely unevaluated.","sourceVerification":{"openedUrl":"https://livehandbook.org/k-12-education/miscellaneous/k-12-education/school-resources/artificial-intelligence-in-k%E2%80%9312-schools/","referenceExcerpt":"AI tools often improve performance during use, but these gains do not consistently reflect independent proficiency.","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}