{"resourceId":"eduguard-safe-rag-programming-tutor-2026","versions":[{"version":"external-ccbaf4be76e07c0a0ce9cd7c8a5509a787322a6d4bf8016fee375570b49b57d4","resource":{"id":"eduguard-safe-rag-programming-tutor-2026","title":"EduGuard: safe RAG programming tutor cuts hallucination/leakage; small pilot reduces overreliance","organization":"Independent academic authors / arXiv cs.CY","sector":"Higher education CS education / GenAI tutoring safety","geography":"International HE (English and Bangla-English code-mixed queries; CS50-style validation)","publishedAt":"arXiv:2607.15738v2 [cs.CY] 4 Sep 2026","publicationDate":"2026-09-04","eventDate":null,"sourceName":"arXiv preprint PDF","sourceLabel":"Hossain, Shayoni, Mridha & Shin (2026) — EduGuard safe RAG tutor","sourceUrl":"https://arxiv.org/pdf/2607.15738","evidenceClass":"academic-research","outcomeClass":"emerging","topics":["developers-agents","data-security","governance-procurement"],"finding":"EduGuard combines query understanding, instructor-approved retrieval, pedagogical strategy selection, rubric-aware generation, claim-level NLI verification, and overreliance control. On BILearn-CS (600 instructor-authored queries) it reports best correctness (90.1%), grounding (89.4%), rubric alignment (90.8%), with lowest hallucination (4.9%) and direct-answer leakage (9.8%) versus strong baselines. A counterbalanced 10-undergraduate pilot raised immediate post-test accuracy from 68.4% to 81.2% and cut overreliance from 38.0% to 17.0% relative to GPT-4o-mini Tutor. Authors state the pilot is underpowered for long-term retention/grades.","sledRelevance":"Fresh September 2026 preprint addressing integrity and overreliance—Pass B scrutiny priorities after prior language-equity integrity work. Why now: operationalizes safety controls campuses need as GenAI coding tutors scale.","evidence":"Source findings: benchmark comparisons with CIs/tests; CS50-forum validation; N=10 pilot. Limitations: underpowered pilot; constructed benchmark dominance; remaining code-reasoning failures; not U.S. persistence study.","architectureImplications":"Interpretation: separate verifier and leakage policy from the generator; ground answers in instructor corpora.","governanceImplications":"Interpretation: treat direct-answer leakage as an integrity KPI alongside plagiarism detection skepticism.","securityPrivacyImplications":"Interpretation: adversarial prompt testing and solution-leakage controls are mandatory for assignment-bearing courses.","caveats":"Do not treat benchmark leadership as proof of semester learning or retention. Pilot N=10.","streamIds":["student-success"],"roles":{"sales":"Sales framing: present EduGuard (arXiv Sep 2026) as architecture evidence—hybrid RAG, rubric-aware generation, separate NLI verification, overreliance control—showing best-in-benchmark correctness/grounding and lower hallucination (4.9%) and direct-answer leakage (9.8%), plus a 10-student pilot lifting post-test accuracy 68.4%→81.2% and cutting overreliance 38%→17% vs GPT-4o-mini Tutor. Do not sell as proven persistence technology.","engineering":"Engineering framing: implement explicit control pipeline—retrieve, strategize, generate, verify, overreliance-gate—not prompt-only tutors. Prerequisites include approved corpora and separate verifier. Constraints: underpowered pilot. Proof of value: leakage/hallucination KPIs plus learning items, not chat satisfaction alone.","delivery":"Delivery framing: delivery is safety operations for tutoring, not a chatbot soft-launch. Ownership with the course team plus integrity. Acceptance needs leakage KPIs. Risks include equating benchmark scores with course success."},"roleDetails":{"sales":{"customerProblem":"Programming courses need GenAI tutors that reduce hallucination and solution leakage while still helping novices learn.","stakeholders":["CS departments","Academic integrity office","Teaching & learning","CISOs/IT"],"discoveryQuestions":["Will responses be grounded in instructor-approved materials?","How do you measure leakage and overreliance?","Is verification architecturally separate from generation?","What is the human TA escalation path?"],"valueHypothesis":"Safe RAG tutoring with claim verification and overreliance controls can cut leakage/hallucination and reduce overreliance versus stronger unconstrained tutor baselines in programming education—though student outcome evidence remains a small pilot.","potentialEngagement":"Course-scoped safe-tutor pilot on one intro CS course with leakage KPIs and pre/post learning checks.","unsupportedClaims":["That benchmark wins equal semester grade gains","That N=10 pilot proves retention","Vendor claims of zero hallucination"]},"engineering":{"fit":"Fits intro programming courses needing constrained help rather than code-completion engines.","architectureIntegration":"Instructor-approved corpus + hybrid retrieval; pedagogical strategy selector; generator LLM; separate verifier; leakage/overreliance policy layer; TA review queue.","prerequisites":["Course corpus curation","Rubric/assistance levels","Logging of leakage events","Eval harness (BILearn-style or local)"],"constraints":["Primary evidence is benchmark + N=10 pilot","South Asian bilingual context may differ","Code-reasoning failures remain"],"security":"Prevent assignment solution exfiltration; isolate student code; no consumer training; audit adversarial direct-answer prompts.","proofOfValue":["Track hallucination and leakage rates in production","Compare overreliance surveys pre/post","Ablate verifier vs self-check"]},"delivery":{"work":"Curate course materials, set assistance rubrics, deploy constrained tutor, train TAs on escalation, monitor leakage dashboards.","dependencies":["Faculty corpus time","Infra for RAG+verifier","Integrity office alignment","Student onboarding"],"ownership":"CS course instructor with academic integrity and teaching-center partners.","skillsAdoption":"TAs need to interpret verifier flags; students need help-seeking norms that avoid answer-seeking.","governanceCheckpoints":["Pre-term adversarial prompt red-team","Weekly leakage review","End-of-module learning check"],"acceptanceCriteria":["Leakage rate below agreed threshold","Hallucination monitoring live","Documented TA escalation for failed verifies"],"risks":["Verifier false confidence","Students jailbreaking to full solutions","Scaling before larger outcome trials"]}},"retrievedAt":"2026-09-20T03:08:33Z","enrichedAt":"2026-09-20T03:15:04Z","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Interpretation: bilingual/code-mixed support is a plus for language equity, but disability accessibility of the UI is not evaluated.","procurementImplications":"Interpretation: require leakage/hallucination SLAs and instructor-corpus control in tutoring procurements.","operatingModelImplications":"Interpretation: operating model keeps humans for escalation and rubric policy; AI is constrained help, not autograder-oracles.","sourceVerification":{"responseId":"curl-arxiv-2607.15738-2026-09-19","openedUrl":"https://arxiv.org/pdf/2607.15738","referenceExcerpt":"lowest hallucination (4.9%) and direct-answer leakage (9.8%). In the","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}