Lighthouse AdvisorySLED AI Adoption Intelligence
← Back to results

From the Student Success edition of September 6, 2026

Academic researchMixedUndated source

Chile trial separates adoption from learning: tutor-use guidance improves final-exam performance

Sebastian Gallegos, Universidad Adolfo Ibáñez; IZA@LISER · Higher education teaching · Chile; selective university

Publisher
Guidance Over Adoption: Experimental Evidence on AI-Assisted Learning
Original publication
March 2026; exact publication day unknown
Source retrieved
2026-09-07
Read original source

What happened

Randomized encouragement increased tool adoption without detectable midterm improvement; separate tutor-use guidance improved final-exam outcomes. Table 3 reports a 0.218 SD intention-to-treat grade gain.

Why it matters

A testable design for U.S. college tutoring support, with limited transfer from one selective Chilean econometrics course.

Evidence and measured results

Two independently randomized interventions across seven sections in August–December 2025; analytic n=303 and n=289. Controls retained assistant access. Table 3 gives SE=.100 and adjusted p=.049 for standardized grades; the comparison mean is zero by normalization.

Limitations and uncertainty

Working paper; one course, partial participation, self-reported usage and peer spillovers. No delayed learning measure. Abstract rounds differently from Table 3; use the table estimate, not a stronger universal claim.

Put this evidence to work

Lighthouse Advisory interpretation, grounded in this source. Enriched 2026-09-07; this does not change the original publication date. Labels below come from the analysis itself.

Sales

Role takeaway

Teaching leaders and student-success teams may have broad AI access without a clear learning benefit. Include course faculty, tutors, accessibility staff and procurement. Ask how students use the tool, whether support prompts encourage independent reasoning and how exams assess that reasoning. A credible value hypothesis is that explicit guidance improves the quality of study. Offer one coordinated course pilot with an agreed comparison and an evaluation plan. The trial motivates testing a guidance package, not purchasing a particular platform. Do not promise its effect size locally, infer durable retention, equate increased usage with learning or propose guaranteed reductions in tutoring staff.

Pre-sales engineering

Role takeaway

Fit a course assistant to approved instructional materials and an existing learning platform. Version content and guidance, keep source references available and provide an instructor escalation path. Prerequisites include a maintained content set, shared assessment rubric and an approved provider. Check answer quality, inappropriate answer disclosure and response behavior after model updates. Minimize student data and verify cloud processing terms; on-premises or hybrid designs require separate feasibility work.

Proposed proof of value
compare guidance with access alone using independently scored reasoning tasks and delayed reassessment, while recording participation and spillovers. The evidence concerns a teaching workflow; autonomous agents and student-information-system writes are unnecessary to this validation.

Delivery

Role takeaway

Course coordination should own the pilot calendar, faculty should approve prompts and teaching assistants should handle escalation. Prepare accessible guidance, orient students to verification, and preserve existing support channels. Dependencies include content approval, privacy review, assessment capacity and a process for correcting misleading responses. Proposed acceptance criteria include delivery of all planned guidance, complete reporting of participation and missing outcomes, independently scored learning comparisons and no unresolved critical privacy or accessibility defects. These are proposed gates rather than trial findings. Review delayed performance and support workload before expansion. Risks include peer contamination, self-selection in reporting and overgeneralizing from one course.

Implementation considerations

Lighthouse Advisory interpretation across the operating dimensions a public-sector buyer must settle before this evidence becomes a design. Each note answers the question under its heading for this specific source.

Architecture and integration

What must connect, and where does the AI sit in the workflow?

The study used a course-specific GPT through ChatGPT and existing course messaging. Interpretation: use approved course content and versioned tutoring instructions; the paper's 'trained' terminology does not establish fine-tuning. No on-premises comparison or agent benchmark is supplied.

Governance

Who approves, reviews and stays accountable for outcomes?

Evaluate guidance as an instructional package and preserve human escalation; separate access and usage from attainment.

Security and privacy

What data, permissions and controls need testing?

Review provider handling of prompts and course materials, prohibit sensitive records in examples, and collect only consented evaluation data. No product security certification follows from exam results.

Accessibility and workforce

Who is affected, and what skills or accommodations follow?

Co-design guidance with accessibility staff and tutors, supply equivalent alternatives, and budget faculty review and student support time.

Procurement

What should contracts, pricing and exit terms secure?

Pilot within approved tooling before additional licensing; require portability of materials, model-change notification and evidence of accessible access.

Operating model

Which teams own the service once it runs?

Course coordination owns guidance; faculty own content accuracy and assessment; support staff handle unresolved student questions.

What changed

New to the searched canonical archive; no repeated source or prior completed student-success run was found. Included as evidence backfill, not asserted to be a new event on the edition date.

Publication history

  1. 2026-09-06Student Success · Issue 013 resources
Read preserved resource versions (JSON)

Stable resource ID: chile-gpt-uai-guidance-rct-2026