{"resourceId":"maier-metacognitive-feedback-offloading-20260917","versions":[{"version":"external-1564d7fdec9f9da695f0dea4c4cb1772306ece8f1bf15f99ab8c0b12b93e167b","resource":{"id":"maier-metacognitive-feedback-offloading-20260917","title":"Preregistered experiment: metacognitive feedback cuts LLM answer-offloading and lifts unaided test odds","organization":"LMU Munich & Munich Center for Machine Learning (Maier, Schwabe, Schneider, Feuerriegel)","sector":"Higher education learning sciences / HCI","geography":"United Kingdom Prolific adults (transferable to HE tutoring design; fraction-arithmetic task)","publishedAt":"2026-09-17","publicationDate":"2026-09-17","eventDate":null,"sourceName":"arXiv","sourceLabel":"Preregistered HCI/education experiment preprint on LLM offloading design","sourceUrl":"https://arxiv.org/pdf/2609.20143","evidenceClass":"academic-research","outcomeClass":"mixed","topics":["knowledge-work","developers-agents","operating-model","accessibility-workforce"],"finding":"In a preregistered online experiment (N=704) practicing fraction arithmetic with an LLM assistant that gave complete answers only on request, metacognitive feedback on learners’ own AI-use patterns reduced answer offloading (OR=0.47) and improved subsequent unaided test performance (OR=1.51). An effort-based point reward that paid less for complete-answer requests did not significantly change offloading or unaided scores. AI access alone did not significantly harm unaided performance versus a no-AI control under this request-friction design.","sledRelevance":"Actionable HE tutoring-design signal for U.S. SLED buyers: pair AI assistants with in-product metacognitive feedback that mirrors answer-seeking, rather than relying on points/gamification or access bans alone—directly addressing cognitive-offloading risks raised in recent student-success coverage.","evidence":"2×2-plus-control preregistered mixed-effects logistic regressions; UK Prolific adults; IRB at LMU; arXiv:2609.20143v1 dated 17 Sep 2026; outcomes are answer-offloading share and unaided six-item test.","architectureImplications":"Log assistance-tier per item (guided vs intermediate vs complete answer); insert post-item metacognitive feedback (process mirror + practice implication + monitoring cue); default to guidance-first replies that require explicit complete-answer requests.","governanceImplications":"Treat offloading analytics as pedagogical telemetry with student notice; evaluate tutors on unaided retention checks, not practice-time accuracy alone; avoid claiming seat access equals learning gain.","securityPrivacyImplications":"Keep chat and feedback logs under institutional tenancy with short retention; do not train external models on student problem-solving transcripts without explicit agreement.","caveats":"Single-session fraction task with adult Prolific sample, not a semester campus course; assistant withheld unsolicited full answers; delayed retention untested; transfer to U.S. credit-bearing courses requires local pilots.","streamIds":["student-success"],"roles":{"sales":"For teaching-and-learning and student-success buyers worried that GenAI seats drive deskilling, Maier et al.’s September 2026 preregistered experiment (N=704) shows metacognitive feedback on learners’ own LLM requests cut answer-offloading odds nearly in half (OR=0.47) and raised unaided test odds (OR=1.51), while an effort-based reward did not. Sell tutors that make complete-answer requests deliberate and surface use-implications—not gamified point schemes or unrestricted ChatGPT seats as learning insurance. Discovery should ask how the product mirrors student offloading and what unaided assessments will verify skill.","engineering":"Engineering fit is an LLM tutor that classifies assistance tiers, requires explicit complete-answer requests, and injects brief metacognitive feedback between items using the learner’s chat history. Architecture should log prompt tiers for IR evaluation and keep the final assessment AI-off. Prerequisites include an institutional model tenancy and item bank with unambiguous correctness; constraints include evidence from a short online fraction task rather than semester LMS courses. Security needs transcript retention limits and no vendor training on student chats.","delivery":"Delivery is a course-embedded pilot: enable guidance-first tutoring with metacognitive feedback screens, disable AI on a delayed or end-of-unit unaided quiz, and report offloading rates alongside scores. Teaching faculty own learning design; academic technology owns logging; IR or educational research validates effects. Acceptance criteria include documented assistance-tier taxonomy, unaided assessment, and subgroup review. Risks include over-generalizing Prolific adults to enrolled undergraduates and mistaking lower chat volume for learning."},"roleDetails":{"sales":{"customerProblem":"Campuses fear GenAI practice raises homework scores while eroding unaided skill, yet bans and points systems are blunt.","stakeholders":["teaching & learning","faculty","academic technology","student success","IR/assessment"],"discoveryQuestions":["Does the tutor require explicit complete-answer requests or auto-solve by default?","Can we surface metacognitive feedback based on each student’s own AI-use pattern?","What unaided assessment will define deskilling risk for the pilot course?"],"valueHypothesis":"Metacognitive feedback embedded in AI practice can reduce answer-offloading and protect unaided performance without banning assistance.","potentialEngagement":"4–6 week unit pilot with assistance-tier logging, metacognitive feedback on, and an AI-off mastery check.","unsupportedClaims":["Guaranteed semester GPA or retention lifts","That effort-based rewards alone will fix deskilling","That Prolific adult effects equal gateway-course outcomes"]},"engineering":{"fit":"Fit for homework tutors where chat can be instrumented per item and assessments can turn AI off.","architectureIntegration":"Guidance-first LLM; assistance-tier classifier; post-item metacognitive feedback generator; export offloading and score metrics to IR.","prerequisites":["Institutional LLM tenancy","Item bank with verifiable answers","Consent/notice for interaction logging"],"constraints":["Evidence from short online fraction practice","UK adult sample","Immediate unaided test only"],"security":"Institutional control of chats; minimize PII; prohibit secondary vendor training on student prompts.","proofOfValue":["Lower complete-answer request rate vs AI-only","Unaided test odds/score contrast with feedback on","No reliance on reward points as primary lever"]},"delivery":{"work":"Configure guidance-first tutor with metacognitive feedback; run unit practice; administer unaided check; debrief faculty on offloading patterns.","dependencies":["Faculty content scope","Analytics export","Assessment window without AI"],"ownership":"Course instructor owns pedagogy; academic technology owns platform; IR/research owns evaluation.","skillsAdoption":"Brief students that complete answers remove practice; coach instrumental help-seeking prompts.","governanceCheckpoints":["Pre-pilot privacy/integrity review","Midpoint offloading audit","Post-unit go/no-go on scale"],"acceptanceCriteria":["Assistance-tier taxonomy operational","Unaided assessment completed and reported","Public claims limited to measured OR effects"],"risks":["Students circumventing friction outside the platform","Feedback wording fatigue","Equity gaps if low-confidence students still over-offload"]}},"retrievedAt":"2026-09-26T03:15:10Z","enrichedAt":"2026-09-26T03:21:39Z","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Metacognitive feedback may help students who would otherwise over-delegate; ensure feedback UI is screen-reader accessible and that opting into guided help remains available for students who need scaffolds.","procurementImplications":"Require vendors to expose assistance-tier analytics and metacognitive feedback features; evaluate on unaided outcomes, not seat utilization alone.","operatingModelImplications":"Operating model pairs AI practice with deliberate unaided demonstration and T&L ownership of feedback copy—not license expansion alone.","sourceVerification":{"openedUrl":"https://arxiv.org/pdf/2609.20143","referenceExcerpt":"Metacognitive feedback reduced answer offloading (OR = 0.47) and improved test performance (OR = 1.51).","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}