From the Student Success edition of September 28, 2026
HelpCoach: in-situ GenAI help-seeking scaffold raises question specificity and week-later retention
University of Michigan / KAIST · Higher education / academic research · United States (college web-programming lab; transferable with limits)
- Publisher
- arXiv
- Original publication
- 2026-09-24
- Source retrieved
- 2026-09-29
What happened
In a between-subjects study (N=40), HelpCoach—an add-on that monitors question specificity and prompts templated revision of knowledge component and scaffold type—increased spontaneous specific questions (Task2 65.4% vs 33.9%, p<.001, d=1.262) and week-later knowledge retention (p=.005, d=1.100) versus pre-task help-seeking training alone. Immediate post-test gains and MSLQ self-regulation scores did not differ significantly; targeted chatbot-response share did not mediate retention.
Why it matters
Gives HE tutoring and CS/IS programs an operable alternative to ambient ChatGPT: instrument help-seeking metacognition in the chat UI and measure unaided retention, not chat volume.
Evidence and measured results
Lab RCT-style between-subjects design with pre/post/retention knowledge tests; specificity classifier on initial drafts before revision; open interaction logs. Single web-programming domain.
Limitations and uncertainty
Programming-lab transfer limits; chatbot still often over-helped; retention sample smaller; self-report SRL nulls.
Put this evidence to work
Lighthouse Advisory interpretation, grounded in this source. Enriched 2026-09-29; this does not change the original publication date. Each role follows the same checklist so takeaways compare across sources.
Sales
Role takeaway
- Customer problem
- Students use GenAI for answers, then forget; campuses buy tutors without help-seeking controls.
- Stakeholders
- teaching & learning
- CS/IS chairs
- academic technology
- student success
- Discovery questions
- Do you measure question specificity in student–AI chats?
- Do you run unaided retention tests after AI-supported practice?
- What fades help prompts as students improve?
- Value hypothesis
- Bounded value: pilot in-situ help-seeking scaffolds and measure week-later retention before scaling ambient tutors.
- Potential engagement
- One-course HelpCoach-style pilot with specificity analytics and retention quiz—not a grade guarantee.
- Unsupported claims
- Guaranteed GPA or retention gains campus-wide
- Any chatbot equals HelpCoach effects
- Immediate test gains always accompany retention gains
Pre-sales engineering
Role takeaway
- Fit
- Fits as chat-UI middleware ahead of approved course tutors/LMS assistants.
- Architecture and integration
- Draft capture → specificity classifier → revision template/scoped options → constrained tutor response → analytics.
- Prerequisites
- Task knowledge-component list
- Scaffold typology
- Approved LLM tenancy
- Constraints
- Web-programming evidence base
- Lab N=40
- Classifier false positives interrupt
- Security
- Encrypt chats/code snapshots; disable vendor training; minimize retention.
- Proof of value
- Spontaneous specificity lift
- Week-later unaided retention lift
- Reduced full-solution over-help rate
Delivery
Role takeaway
- Work
- Instrument chat, author inventories, randomize or match sections, score retention quiz.
- Dependencies
- Faculty SME
- Academic technology
- Assessment support
- Ownership
- Course instructor with teaching-and-learning co-owner.
- Skills and adoption
- Train TAs on scaffold types and when to skip guidance.
- Governance checkpoints
- Syllabus AI disclosure
- Data-retention review
- IRB if research-framed
- Acceptance criteria
- Documented specificity improvement on initial drafts
- Documented week-later retention comparison
- Risks
- Expertise-reversal annoyance
- Chatbot still dumps answers
- Transfer failure outside programming
Implementation considerations
Lighthouse Advisory interpretation across the operating dimensions a public-sector buyer must settle before this evidence becomes a design. Each note answers the question under its heading for this specific source.
Architecture and integration
What must connect, and where does the AI sit in the workflow?
Add specificity classifier + revision template + scoped knowledge/scaffold suggestions before send; fade prompts when students already ask specifically; tune chatbot system prompts for coverage/precision alignment.
Governance
Who approves, reviews and stays accountable for outcomes?
Require evaluation of spontaneous help-seeking behavior and delayed unaided tests, not only satisfaction or assignment completion with AI present.
Security and privacy
What data, permissions and controls need testing?
Code snapshots and chat logs used for scoped recommendations are education records—keep in approved tenancy with retention limits and no vendor training on identifiable sessions.
Accessibility and workforce
Who is affected, and what skills or accommodations follow?
Templated options can aid students who struggle to name help needs, but poor option coverage may frustrate advanced or disability-related clarification asks—preserve skip paths and text alternatives.
Procurement
What should contracts, pricing and exit terms secure?
Prefer tutors/vendors exposing pre-send help-seeking controls and retention-test hooks over seat licenses alone; require evaluation against training-only baselines.
Operating model
Which teams own the service once it runs?
Move KPIs from message count to specificity rate and delayed unaided mastery; staff time shifts to authoring knowledge inventories and reviewing over-help logs.
Publication history
- 2026-09-28Student Success · Issue 223 resources
Stable resource ID: jin-helpcoach-targeted-helpseeking-20260924