Lighthouse AdvisorySLED AI Adoption Intelligence

Higher Education & ResearchIssue 243 records

Student Success

For 2026-09-30, student-success evidence shifts from yesterday’s equity-of-depth and homework-integrity analytics toward whether constraint-first GenAI “thinking partners” actually build higher-order learning—and how early LMS signals should be used when integrity risk appears. A University of Tennessee DBR pilot in undergraduate research methods (arXiv 29 Sep 2026) finds semester-long ChatGPT partnership raised every statistical-learning self-assessment dimension and eight of nine AI-literacy dimensions while Watson-Glaser critical-thinking percentiles did not move, with students mostly treating the model as a validator. A companion graduate-methods cycle (arXiv 29 Sep 2026) shows AI-literacy self-reports rising while critical thinking, metacognition, and knowledge stay flat, warning that fluency may be confidence and that sparse interaction logs undercut design claims. Against both, Akçapınar’s programming-course study (arXiv 29 Sep 2026) shows week-1–8 LMS traces can classify AI-assisted exam-risk labels at 73.1% accuracy (n=52)—explicitly for academic guidance, not misconduct findings. Operating takeaway: measure reasoning and log enactment before celebrating literacy gains, and keep early integrity analytics in outreach queues with humans in the loop.

Evidence records
3
Cross-source patterns
2
Evidence classes
3 academic research
Outcomes
1 mixed1 cautionary1 emerging
Source freshness
3 new this fortnight
Research completed
2026-10-01

Choose a role to see its takeaway beside every record in the ledger.

Patterns across the evidence

Synthesis · Lighthouse Advisory interpretation

2 patterns, each supported by at least two sources

  1. Constraint-first GenAI partners can lift AI-literacy self-reports without moving critical thinking or knowledge

    The undergrad thinking-partner pilot moved AASCDM/MAILS but not WGCTA percentiles, with validation as the modal LLM relationship. The graduate Fluency Without Evidence cycle likewise raised MAILS while critical thinking, metacognition, and knowledge stayed flat, and sparse logs suggested confidence may outrun capacity. Together they warn student-success programs not to equate literacy surveys or chatbot fluency with durable higher-order learning.

    Operating questionFor each GenAI teaching-partner pilot, which independent reasoning/knowledge measures and interaction-enactment metrics sit beside AI-literacy surveys?

    Supporting evidenceUniversity of Tennessee, Knoxville & The Ohio State University (Zahra, He, Bowers, Wang)University of Tennessee at Knoxville (Zahra, Wang, Harper, He)

  2. Thin behavioral and self-report signals need human judgment before high-stakes student decisions

    Fluency Without Evidence shows self-report literacy gains and incomplete interaction records are weak bases for claiming higher-order learning. Akçapınar shows early LMS engagement can predict integrity-risk labels yet must trigger guidance rather than misconduct findings because labels are proxies with material error. The pair pushes campuses to treat fluency scores and early-risk analytics as decision-support for humans, not automatic proof.

    Operating questionWhere do literacy dashboards and early integrity-risk scores stop—and where does required human review or outreach begin?

    Supporting evidenceUniversity of Tennessee at Knoxville (Zahra, Wang, Harper, He)Hacettepe University (Gökhan Akçapınar)

Evidence ledger

Full record · every source keeps its link and limitations

3 records
  1. Academic researchMixedNew this fortnight

    Constraint-first GenAI thinking-partner pilot: domain and AI-literacy gains, no WGCTA critical-thinking move

    Design-based research pilot in a Spring 2025 undergraduate research methods and statistics course (N=14) used ChatGPT as a constrained thinking partner. Pre–post tests showed gains on every AASCDM statistical-learning dimension and on eight of nine MAILS AI-literacy dimensions (e.g., Apply AI M=36.64→53.00, t(13)=5.82, p<.001), while WGCTA critical-thinking subscale percentiles did not change significantly (all p>.05). Qualitative themes: students positioned the LLM as answer generator, validator, or co-thinker; engagement was often procedural; modal relationship was validation rather than dialogue. Authors offer design principles for the next DBR iteration; AASCDM showed posttest ceiling effects (α fell .73→.51).

    University of Tennessee, Knoxville & The Ohio State University (Zahra, He, Bowers, Wang)United States (University of Tennessee, Knoxville undergraduate course)Submitted to arXiv 29 September 2026 (cs.HC)

    Why it matters, evidence and limitations
    Why it matters
    Direct HE evidence that semester-long GenAI-assisted instruction can move domain self-assessment and AI literacy without shifting standardized critical thinking—the outcome student-success leaders often claim GenAI will protect or improve. Campuses deploying thinking-partner tutors should measure WGCTA-like reasoning and interaction quality, not only satisfaction or literacy surveys.
    Evidence and measured results
    Mixed-methods DBR: paired AASCDM/MAILS t-tests, Wilcoxon on WGCTA percentiles, instructor notes, artifacts, and student–AI logs. Single-course pilot N=14; author-developed AASCDM with posttest ceiling; preprint.
    Limitations and uncertainty
    N=14 single course; DBR pilot not powered for population inference; AASCDM ceiling; WGCTA stable may reflect low sensitivity or true null; preprint; ChatGPT-specific.
    Academic preprint (cs.HC) Role takeaways & implementation
  2. Academic researchCautionaryNew this fortnight

    Fluency without evidence: constraint-first AI partner raises AI literacy self-reports, not CT/metacognition/knowledge

    First DBR cycle of a constraint-first AI teaching partner in two graduate research-methods courses (demographics N=20; analytic samples ~18–19). MAILS total rose pretest M=204.00 to posttest M=228.72 (t(17)=3.97, Holm p=.009, dz=.94), concentrating in understanding AI, while critical thinking disposition, metacognitive awareness, and objectively scored knowledge items did not change. Interaction records showed brief exchanges, uneven constraint-first enactment, and missing records—leading authors to argue self-report fluency may reflect confidence, not capacity, and that interaction records are a more defensible evaluation basis.

    University of Tennessee at Knoxville (Zahra, Wang, Harper, He)United States (large public research university; two graduate research methods courses)Submitted to arXiv 29 September 2026 (cs.HC)

    Why it matters, evidence and limitations
    Why it matters
    Strong scrutiny signal for HE student-success GenAI evaluations: self-reported AI literacy can rise while critical thinking, metacognition, and knowledge stay flat, and sparse interaction logs undercut claims that constraint-first designs were actually used. Demands telemetry-backed evaluation before scaling graduate or undergraduate AI teaching partners.
    Evidence and measured results
    DBR with pre/post MAILS, adapted CT/metacognition scales, Course B knowledge items, and de-identified conversation transcripts. Adapted scales (CT pretest α=.64); incomplete logs; preprint.
    Limitations and uncertainty
    Two courses; adapted CT/metacognition instruments; incomplete interaction records; graduate/professional sample; sensitivity limits on null CT findings; preprint.
    Academic preprint (cs.HC) Role takeaways & implementation
  3. Academic researchEmergingNew this fortnight

    LMS traces from weeks 1–8 predict AI-assisted exam-cheating risk at 73.1% accuracy—for guidance, not discipline

    Among 52 first-year undergraduates in an Introduction to Programming course, 23 (44.2%) were labeled high-risk for AI-assisted cheating based on suspicious final-exam log behaviors (copy, focus-loss, right-click) in a proctored face-to-face exam. Using five features from 27 early-semester LMS/video candidates, Logistic Regression under LOOCV with fold-specific preprocessing achieved Accuracy 73.1% (Balanced Accuracy 0.718, AUC 0.760). Most stable predictors—course-module views, assignment submissions, and video-access days—were lower in the high-risk group. Authors frame predictions for timely academic guidance, not misconduct findings; labels are behavioral proxies, not verified AI use.

    Hacettepe University (Gökhan Akçapınar)Türkiye (public university; first-year Computer Education and Instructional Technology undergraduates)Submitted to arXiv 29 September 2026 (cs.CY); to be presented at ICETC 2026

    Why it matters, evidence and limitations
    Why it matters
    Offers HE student-success teams an early-warning pattern complementary to post-hoc homework detectors: thin early engagement may foreshadow integrity-risk behaviors, but only as outreach triggers with human judgment. Reinforces that analytics must stay outside automatic discipline pipelines.
    Evidence and measured results
    LOOCV classification on n=52 with exam-log-derived risk labels and early LMS/video features; feature selection via Mann-Whitney U per fold. Small single-course sample; proxy labels; preprint/conference paper.
    Limitations and uncertainty
    n=52; single course/country; high-risk label from exam-log proxies not confirmed AI use; 73.1% accuracy leaves substantial error; LOOCV optimism risk; preprint.
    Academic preprint (cs.CY) Role takeaways & implementation

How to read this edition

Source findings, measured results and limitations come from the cited publications. Patterns, operating questions, role takeaways and implementation considerations are Lighthouse Advisory interpretation, stated as questions to validate locally rather than guaranteed outcomes. Vendor and operator claims are labeled as claims. Full research method.

Academic research
Research produced through an academic institution or peer-reviewed venue.