{"resourceId":"jin-helpcoach-targeted-helpseeking-20260924","versions":[{"version":"external-4e1dd32b95511caa6de258d7b5650dd4a8ecc30780c39f7ae16e80075f77b431","resource":{"id":"jin-helpcoach-targeted-helpseeking-20260924","title":"HelpCoach: in-situ GenAI help-seeking scaffold raises question specificity and week-later retention","organization":"University of Michigan / KAIST","sector":"Higher education / academic research","geography":"United States (college web-programming lab; transferable with limits)","publishedAt":"2026-09-24","publicationDate":"2026-09-24","eventDate":null,"sourceName":"arXiv","sourceLabel":"Preprint (arXiv:2609.28918)","sourceUrl":"https://arxiv.org/pdf/2609.28918","evidenceClass":"academic-research","outcomeClass":"effective","topics":["knowledge-work","developers-agents","operating-model","accessibility-workforce"],"finding":"In a between-subjects study (N=40), HelpCoach—an add-on that monitors question specificity and prompts templated revision of knowledge component and scaffold type—increased spontaneous specific questions (Task2 65.4% vs 33.9%, p<.001, d=1.262) and week-later knowledge retention (p=.005, d=1.100) versus pre-task help-seeking training alone. Immediate post-test gains and MSLQ self-regulation scores did not differ significantly; targeted chatbot-response share did not mediate retention.","sledRelevance":"Gives HE tutoring and CS/IS programs an operable alternative to ambient ChatGPT: instrument help-seeking metacognition in the chat UI and measure unaided retention, not chat volume.","evidence":"Lab RCT-style between-subjects design with pre/post/retention knowledge tests; specificity classifier on initial drafts before revision; open interaction logs. Single web-programming domain.","architectureImplications":"Add specificity classifier + revision template + scoped knowledge/scaffold suggestions before send; fade prompts when students already ask specifically; tune chatbot system prompts for coverage/precision alignment.","governanceImplications":"Require evaluation of spontaneous help-seeking behavior and delayed unaided tests, not only satisfaction or assignment completion with AI present.","securityPrivacyImplications":"Code snapshots and chat logs used for scoped recommendations are education records—keep in approved tenancy with retention limits and no vendor training on identifiable sessions.","caveats":"Programming-lab transfer limits; chatbot still often over-helped; retention sample smaller; self-report SRL nulls.","streamIds":["student-success"],"roles":{"sales":"For teaching-and-learning and CS/IS program buyers, HelpCoach shows that forcing students to name the knowledge gap and scaffold type before the model answers improves spontaneous questioning and week-later retention versus one-shot help-seeking training. Sell discovery around in-situ metacognitive controls and delayed unaided tests—not ambient tutor seats. Bounded value is a course pilot that instruments question specificity and retention, not a guaranteed GPA lift from any chatbot.","engineering":"Engineering takeaway: place a pre-send specificity gate (knowledge component + scaffold type) with adaptive fade, scoped option lists from code/chat state, and chatbot prompts constrained to requested scope. Prerequisites include a knowledge-component inventory per task and near-real-time classifier latency. Constraints: evidence is web-programming lab N=40. Security: treat code and chats as education records. Proof of value: raise spontaneous specificity and replicate week-later retention locally before campus rollout.","delivery":"Delivery work: pick one programming or quantitative course, author knowledge/scaffold inventories, deploy HelpCoach-like add-on beside an approved tutor, and run baseline vs scaffold arms with a one-week unaided retention quiz. Dependencies: faculty SMEs, academic technology, IR/ethics. Ownership: course lead with teaching-and-learning. Skills: TA training on scaffold types. Governance: syllabus AI policy and data retention. Acceptance: documented specificity and retention lifts; risks include over-help still leaking answers and weak transfer outside coding."},"roleDetails":{"sales":{"customerProblem":"Students use GenAI for answers, then forget; campuses buy tutors without help-seeking controls.","stakeholders":["teaching & learning","CS/IS chairs","academic technology","student success"],"discoveryQuestions":["Do you measure question specificity in student–AI chats?","Do you run unaided retention tests after AI-supported practice?","What fades help prompts as students improve?"],"valueHypothesis":"Bounded value: pilot in-situ help-seeking scaffolds and measure week-later retention before scaling ambient tutors.","potentialEngagement":"One-course HelpCoach-style pilot with specificity analytics and retention quiz—not a grade guarantee.","unsupportedClaims":["Guaranteed GPA or retention gains campus-wide","Any chatbot equals HelpCoach effects","Immediate test gains always accompany retention gains"]},"engineering":{"fit":"Fits as chat-UI middleware ahead of approved course tutors/LMS assistants.","architectureIntegration":"Draft capture → specificity classifier → revision template/scoped options → constrained tutor response → analytics.","prerequisites":["Task knowledge-component list","Scaffold typology","Approved LLM tenancy"],"constraints":["Web-programming evidence base","Lab N=40","Classifier false positives interrupt"],"security":"Encrypt chats/code snapshots; disable vendor training; minimize retention.","proofOfValue":["Spontaneous specificity lift","Week-later unaided retention lift","Reduced full-solution over-help rate"]},"delivery":{"work":"Instrument chat, author inventories, randomize or match sections, score retention quiz.","dependencies":["Faculty SME","Academic technology","Assessment support"],"ownership":"Course instructor with teaching-and-learning co-owner.","skillsAdoption":"Train TAs on scaffold types and when to skip guidance.","governanceCheckpoints":["Syllabus AI disclosure","Data-retention review","IRB if research-framed"],"acceptanceCriteria":["Documented specificity improvement on initial drafts","Documented week-later retention comparison"],"risks":["Expertise-reversal annoyance","Chatbot still dumps answers","Transfer failure outside programming"]}},"retrievedAt":"2026-09-29T03:18:41Z","enrichedAt":"2026-09-29T03:21:30Z","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Templated options can aid students who struggle to name help needs, but poor option coverage may frustrate advanced or disability-related clarification asks—preserve skip paths and text alternatives.","procurementImplications":"Prefer tutors/vendors exposing pre-send help-seeking controls and retention-test hooks over seat licenses alone; require evaluation against training-only baselines.","operatingModelImplications":"Move KPIs from message count to specificity rate and delayed unaided mastery; staff time shifts to authoring knowledge inventories and reviewing over-help logs.","sourceVerification":{"responseId":"curl-arxiv-2609-28918","openedUrl":"https://arxiv.org/pdf/2609.28918","referenceExcerpt":"HelpCoach led to more specific questions during chatbot interactions and greater knowledge retention than pre-task help-seeking training alone.","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}