From the K–12 edition of September 17, 2026
Secondary-school RCT: note-taking beats LLM-alone for reading retention despite student preference for AI
Cambridge University Press & Assessment / Microsoft Research (Computers & Education) · Academic research / secondary education · England, United Kingdom (transferable caution for U.S. K-12)
- Publisher
- Computers & Education (Kreijkes et al.)
- Original publication
- 2026 (Computers & Education 243); available online 24 November 2025
- Source retrieved
- 2026-09-18
What happened
Why now as Florida and other systems still permit instructor-authorized AI tutoring/feedback: a pre-registered England secondary RCT (n=405, ages 14–15) finds note-taking alone and LLM+notes significantly outperform LLM-alone on reading comprehension and three-day retention, even though most students prefer the LLM and rate it more helpful—showing preference and cognitive-load reduction are not reliable proxies for independent learning.
Why it matters
U.S. districts designing allowed AI reading supports should require effortful encoding activities and off-tool assessments; transferability is limited by England sample and short retention window but directionally relevant to cognitive-offloading concerns.
Evidence and measured results
Kreijkes et al., Computers & Education 243 (2026) 105514, DOI 10.1016/j.compedu.2025.105514. Pre-registered RCT with within- and between-participant elements in English secondary schools; 405 students studied two passages; outcomes literal retention, comprehension, free recall after three days; mixed-effects models; qualitative preference/archetype analyses. Abstract and results state significant positive effects of notes and LLM+notes vs LLM alone; preference mismatch documented.
Limitations and uncertainty
England sample; three-day retention; Microsoft Research co-authorship; journal volume dated 2026 with online availability Nov 2025—older than last SLED run but newly relevant to current U.S. permitted-use debates; not a U.S. district field trial.
Put this evidence to work
Lighthouse Advisory interpretation, grounded in this source. Enriched 2026-09-18; this does not change the original publication date. Each role follows the same checklist so takeaways compare across sources.
Sales
Role takeaway
- Customer problem
- Districts authorizing AI reading supports lack evidence that student preference translates into retained learning.
- Stakeholders
- Curriculum directors
- Secondary principals
- Instructional coaches
- Assessment leads
- Discovery questions
- Will pilots measure closed-book retention after several days?
- Are students required to take notes while using the LLM?
- What prompting behaviors will you disallow?
- How will you separate preference surveys from mastery evidence?
- Value hypothesis
- A bounded reading-support pilot that mandates notes-plus-LLM and off-tool tests can demonstrate responsible use without claiming LLM-alone superiority.
- Potential engagement
- 6–8 week secondary reading pilot design: conditions, measures, teacher protocols, and go/no-go criteria tied to retention—not NPS.
- Unsupported claims
- That LLM use improves long-term learning in this study
- Direct U.S. causal generalization without local replication
- Guaranteed grade gains from chatbot access
Pre-sales engineering
Role takeaway
- Fit
- Fit for supervised secondary reading tools that can enforce note capture and disable answer-only modes; poor fit for unsupervised homework chatbots.
- Architecture and integration
- Session design: passage → constrained LLM help → mandatory notes → delayed closed-book quiz; analytics on prompt archetypes.
- Prerequisites
- Teacher-supervised sessions
- Assessment items for delayed tests
- Logging of prompt types
- Constraints
- England adolescent sample
- Short retention interval
- May not match proprietary U.S. tutor pedagogies
- Security
- Keep passage and student responses in district-controlled environments; minimize retention of prompts beyond evaluation needs.
- Proof of value
- Off-tool retention not worse than notes baseline
- Reduced pure answer-seeking prompt share
- Teacher-confirmed note artifacts per session
Delivery
Role takeaway
- Work
- Co-design pilot protocols, train teachers on note requirements, run delayed assessments, and report preference vs learning separately.
- Dependencies
- English/literacy department
- Assessment office
- Scheduling for two-session design
- Ownership
- Secondary curriculum lead owns instructional protocol; research/assessment owns measurement.
- Skills and adoption
- Teachers need PD on prompting archetypes and on resisting student pressure to drop notes because AI 'feels easier.'
- Governance checkpoints
- Pre-registration of measures
- Mid-pilot fidelity check on note requirements
- Results review before scaling
- Acceptance criteria
- Delayed comprehension/retention analyzed by condition
- Preference data reported separately from learning
- Written decision on scale/stop/modify
- Risks
- Students abandon notes when unsupervised
- Over-generalizing UK results to U.S. elementary
- Vendor dashboards that optimize engagement over retention
Implementation considerations
Lighthouse Advisory interpretation across the operating dimensions a public-sector buyer must settle before this evidence becomes a design. Each note answers the question under its heading for this specific source.
Architecture and integration
What must connect, and where does the AI sit in the workflow?
If deploying LLM reading aids, default workflows should pair AI explanation with mandatory note-taking or other generative learning activities and block answer-only prompting patterns identified as weak archetypes.
Governance
Who approves, reviews and stays accountable for outcomes?
Allowed-use policies should distinguish scaffolding from substitution; student satisfaction metrics cannot alone greenlight expansion.
Security and privacy
What data, permissions and controls need testing?
School-managed LLM sessions still need age-appropriate logging and prompt controls; study itself is learning-focused rather than privacy-focused.
Accessibility and workforce
Who is affected, and what skills or accommodations follow?
LLM accessibility benefits (simpler language) can trade off against memory encoding; teachers need strategies that preserve access without removing generative learning effort.
Procurement
What should contracts, pricing and exit terms secure?
Require vendors to support notes-plus-AI workflows and independent delayed assessments in pilots; reject preference-only success metrics.
Operating model
Which teams own the service once it runs?
Shift from access rollout KPIs to instructional-condition KPIs (AI-alone vs AI+notes) before expanding secondary reading AI.
Publication history
- 2026-09-17K–12 · Issue 124 resources
Stable resource ID: kreijkes-llm-notes-secondary-rct-2026