Education · Issue 06 ·
Student Success
Three new archive sources cover a U.S. course-chatbot null trial, selective student AI use in Australia and Irish teaching-and-learning controls. None establishes durable learning, retention or savings. The September 9 survey is recent evidence; the trial and framework are explicit backfill. One pattern separates the educational case for learner memory from data-control approval. Advising impact, disability-specific outcomes and total cost remain gaps.
- Evidence records
- 3
- Cross-source patterns
- 1
- Evidence classes
- 2 academic research1 standards or public-body guidance
- Outcomes
- 1 cautionary1 mixed1 emerging
- Source freshness
- 1 older, newly relevant1 new this fortnight1 undated
- Research completed
- 2026-09-12
Choose a role to see its takeaway beside every record in the ledger.
Synthesis · Lighthouse Advisory interpretation
Patterns across the evidence
Evaluate learner memory through educational and data-control reviews
The U.S. trial identifies absent cross-session memory as a limitation, without testing whether adding it improves learning. The Irish framework supplies data-control considerations for such an addition. Together they support separate evidence gates for educational value and acceptable data handling; passing either gate does not establish the other.
Operating questionWhat controlled learning comparison and independently verified retention, access and deletion tests would justify storing learner history?
Supporting evidenceAndrew Thoeni and Luke K. Fryer; University of North Florida and University of Hong KongHigher Education Authority; James O’Sullivan, Colin Lowry, Ross Woods and Tim Conlon
Full record · every source keeps its link and limitations
Evidence ledger
U.S. course-chatbot trial finds no significant measured benefit; design limits matter
A course-grounded chatbot produced no significant treatment effects on interest, self-efficacy, eBook engagement or test achievement.
Why it matters, evidence and limitations
- Why it matters
- Direct U.S. public-university evidence for a bounded course-assistance decision; it cannot determine institution-wide retention value.
- Evidence and measured results
- Methods report 454 students (231 control, 223 treatment), randomized within three marketing sections. The 16-week course included 11 active treatment weeks. Baseline preceded access; comparison used Quizlet participation assignments. Table 3 achievement interaction p=.744, d=.015. End-of-course assessment was non-comprehensive, not delayed retention.
- Limitations and uncertainty
- One instructor/course; participation incentives shifted between tools; study-habit substitution was unmeasured. Table 2 uses doubled group counts; regression degrees of freedom require clarification before replication. No causal evidence that adding memory would improve learning.
Australian survey identifies selective AI use without establishing learning effects
Higher self-reported GPA was associated with less enthusiastic AI engagement; qualitative accounts describe selective, verification-intensive use. This is not evidence that avoiding AI improves grades.
Why it matters, evidence and limitations
- Why it matters
- Useful for U.S. college discovery about student choice and the meaning of adoption metrics, with substantial institutional and cultural transfer limits.
- Evidence and measured results
- Voluntary early-2025 survey: 484 responses, 469 valid GPA bands. Cross-sectional Spearman analyses, demographic sensitivity checks and author-led qualitative analysis. Table 2 links GPA to perceived effective learning negatively (rho=-.249) and concern about independent thinking positively (rho=.252); both p<.001. No learning intervention or experimental baseline.
- Limitations and uncertainty
- Self-reported achievement, self-selection, unknown response rate and one site. Composite reliability was weak; item-level checks help but do not establish causality. Successful-student identity is an interpretive label, not a validated construct. A 2025 snapshot, not current adoption prevalence.
Irish framework connects learning oversight, equitable access and data control
Guidance links teaching use to human accountability, equitable access, student data rights and ongoing review; it reports no measured implementation benefits.
Why it matters, evidence and limitations
- Why it matters
- A teaching-specific checklist for U.S. colleges, not a statement of U.S. legal duties or a validated intervention.
- Evidence and measured results
- Version 1.0, December 2025. Operational sections address accessible participation, academic oversight, vendor data terms, deletion and external-model training consent. No evaluation baseline, outcome sample or causal estimate is supplied.
- Limitations and uncertainty
- Normative Irish framework, not audit or outcome research. Controls require resources and enforcement; possession of a policy cannot establish compliance, equity or educational effectiveness.
How to read this edition
Source findings, measured results and limitations come from the cited publications. Patterns, operating questions, role takeaways and implementation considerations are Lighthouse Advisory interpretation, stated as questions to validate locally rather than guaranteed outcomes. Vendor and operator claims are labeled as claims. Full research method.
- Academic research
- Research produced through an academic institution or peer-reviewed venue.
- Standards or public-body guidance
- Normative or advisory guidance from a standards body or public institution.