{"resourceId":"lethbridge-coursechat-onprem-rag-tutor-20261001","versions":[{"version":"external-6b418884b46da06fc6084f6a7276ec0eceb910edb1052b2b80cc12e448955bc9","resource":{"id":"lethbridge-coursechat-onprem-rag-tutor-20261001","title":"Lethbridge CourseChat: on-premises multi-course RAG tutor keeps an 8B model after larger models failed a classroom speed gate","organization":"Dhillon School of Business, University of Lethbridge (Shapiro, Lindemann)","sector":"Higher education business courses / campus-hosted GenAI tutoring infrastructure","geography":"Canada (University of Lethbridge, Alberta)","publishedAt":"Submitted to arXiv 1 October 2026 (cs.AI)","publicationDate":"2026-10-01","eventDate":null,"sourceName":"arXiv","sourceLabel":"Academic preprint (deployment engineering report)","sourceUrl":"https://arxiv.org/pdf/2610.02510","evidenceClass":"academic-research","outcomeClass":"emerging","topics":["infrastructure","data-security","developers-agents","operating-model"],"finding":"CourseChat serves six isolated course offerings (keyed by CRN) from twin on-premises DGX Spark-class hosts running FastAPI, Qdrant, local embeddings and reranking, and Ollama, behind a campus IIS gateway for Moodle embedding. Several larger models failed a speed gate (median latency no more than 30% above the 8B baseline); a 12B and a 7B model passed, but the team kept llama3.1:8b pending a demonstrated overall improvement. In a five-course replay, all 400 scope checks passed, each course hit the 90% follow-up target, and median per-turn latency ran 3.24–6.19 s; 93 no-match turns remained. 435 prebuilt quiz questions across 65 modules separate practice from live generation. The authors state no learning gains are established.","sledRelevance":"Campuses weighing a course-grounded tutor that keeps textbooks and student dialogue on institutional hardware get a concrete reference architecture and its limits, a useful counterpoint to vendor-hosted tutors after the UMD RCT showed unstructured access can backfire.","evidence":"Engineering evaluations: two model bake-off rounds with fixed retrieval, a source-fidelity comparison, frozen 20-dialogue/80-turn fixtures per course, quiz audits; serial server-local latency; no load study, no faculty ratings, no learning-outcome study.","architectureImplications":"Per-course isolated vector collections, replicated local stacks per node, retrieval-topic resolution for follow-ups, bounded evidence selection, and prebuilt practice banks; model choice is a joint decision with retrieval, serving compatibility, and latency.","governanceImplications":"Promotion rules for model changes (speed gate, scope preservation, reviewed improvement) are a reusable change-control pattern; faculty must own corpus content and question review.","securityPrivacyImplications":"On-premises hosting keeps copyrighted textbooks and student dialogue off external APIs; strict course isolation prevents cross-course leakage; gateway acceptance for every course slug was still open.","caveats":"Single business school; engineering regressions are not a representative sample of student use; latency measurements serial; peak-load capacity untested; faculty ratings pending; no learning outcomes.","streamIds":["student-success"],"roles":{"sales":"The framing point is control: a business school built a course-grounded tutor that keeps textbooks and student chats on campus hardware. This interests CIOs and deans worried about data leaving the institution or tutors answering from the wrong course. Position a bounded architecture-and-evaluation engagement. Be clear that the paper reports engineering results, not learning gains, and that the school already licenses Copilot and Gemini, so added value must be shown.","engineering":"For engineering, the point is that quality lives in retrieval and scoping, not in a bigger model. CourseChat isolates course collections, replicates full local stacks on two GB10-class nodes, and gates any model swap on latency within 30% of baseline plus reviewed improvement. Plan load testing early, since only serial latency was measured, and verify gateway streaming and every course route. Expect cold-load delays for larger models and keep prebuilt quiz banks to decouple practice from generation.","delivery":"Delivery should run this as a joint service between IT and faculty. Budget faculty time for preparing materials and reviewing questions each term, add a student flagging loop that feeds corrections and regression tests, and hold a change-control review before any model swap. Before claiming educational value, plan paired faculty review on held-out dialogues and a comparison against the general AI tools students already use."},"roleDetails":{"sales":{"customerProblem":"Institutions want course-grounded AI tutoring without sending textbooks or student dialogue to external providers.","stakeholders":["CIO / research computing","Business school dean","Faculty course leads","Privacy officer","Procurement"],"discoveryQuestions":["Which courses have licensed materials that cannot leave campus?","What concurrency do you expect at peak class times?","How will this add value beyond existing Copilot/Gemini licenses?"],"valueHypothesis":"An on-premises, course-isolated RAG tutor can provide grounded help with institutional data control, if latency and faculty workload are managed.","potentialEngagement":"Architecture review and load-tested pilot for 2–3 courses with faculty-rated answer quality.","unsupportedClaims":["Learning gains","Classroom-scale capacity","That a larger model would improve tutoring"]},"engineering":{"fit":"Course-grounded tutoring for a defined set of courses with institution-owned materials.","architectureIntegration":"Campus gateway (TLS, routing) → replicated nodes with FastAPI, vector DB, embeddings/reranker, local LLM → Moodle embed.","prerequisites":["On-premises GPU appliances with unified memory","Course corpus ingestion pipeline","Gateway SSE support"],"constraints":["Serial-only latency evidence","Cold-load delays for large models","Per-node artifact replication"],"security":"Keep data local, enforce per-CRN collection isolation, and verify every public route before acceptance.","proofOfValue":["Concurrent browser-to-gateway latency under representative load","False-refusal and source-error rates on held-out dialogues","Faculty usefulness ratings"]},"delivery":{"work":"Ingest course materials, build and review quiz banks, run regression fixtures, and operate change control for model updates.","dependencies":["Faculty content review time","Local MLOps staff","Gateway configuration"],"ownership":"Central or school IT for the platform; faculty for course content.","skillsAdoption":"Faculty learn corpus curation and question review; students are guided to attempt problems before seeking hints.","governanceCheckpoints":["Model promotion rule met before swap","Course isolation test each term","Privacy review of logging"],"acceptanceCriteria":["Median latency within agreed target at expected concurrency","Defined follow-up topic-hit rate per course","Faculty-rated accuracy threshold on held-out set"],"risks":["Faculty workload underestimated","Capacity shortfall at peak","No measured learning benefit"]}},"retrievedAt":"2026-10-05T03:17:35Z","enrichedAt":"2026-10-05T03:21:28Z","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Requires local MLOps skills and faculty time to prepare materials, review questions, and fix errors between offerings; authors propose tracking that workload.","procurementImplications":"On-premises appliance purchases should be scoped against measured concurrency and latency under load, not memory alone; compare against already-licensed Copilot/Gemini access.","operatingModelImplications":"Run as a joint IT and faculty service with change control for model swaps, course-corpus refresh each term, and a student flagging loop feeding regression cases.","sourceVerification":{"openedUrl":"https://arxiv.org/pdf/2610.02510","referenceExcerpt":"They do not establish learning gains: faculty ratings, peak-load capacity, and complete public-gateway acceptance remain separate evaluation needs.","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}