{"resourceId":"cal-poly-pomona-billy-chat-wrong-registration-alert-20261103","versions":[{"version":"external-ac9b4b98e138eec81a38607630cdda2ed707e37e4e4f3b5ceae1880147562aed","resource":{"id":"cal-poly-pomona-billy-chat-wrong-registration-alert-20261103","title":"Cal Poly Pomona's Billy Chat texting bot sent an incorrect spring 2027 registration alert after a copied campaign, following the departure of a key bot manager","organization":"Cal Poly Pomona (California State University), Student Success / Billy Chat team","sector":"Higher education enrollment communications operations","geography":"United States (California)","publishedAt":"3 November 2026 (student newspaper); incident on 20 October 2026","publicationDate":"2026-11-03","eventDate":"2026-10-20","sourceName":"The Poly Post","sourceLabel":"Student-newspaper reporting with named administrator quotes","sourceUrl":"https://thepolypost.com/news/2026/11/03/billy-chat-sends-students-wrong-registration-alert/","evidenceClass":"independent-reporting","outcomeClass":"cautionary","topics":["operating-model","accessibility-workforce","data-security"],"finding":"On 20 October Billy Chat, Cal Poly Pomona's Mainstay-based texting bot, told students their spring 2027 registration window had opened although general registration opened on 22 October; a correction followed 30–60 minutes later. The associate vice president for student success said a staff member copied a 2025 campaign that included all registration date options, a human error after the departure of a co-founder who managed the bot; it is now run by two people. She met the vendor on prevention and plans better training and safeguards. Every student receives alerts unless they opt out, and each has a data feed including enrollment status and the tone they use with the bot. The administrator says the bot is AI but not a large language model and relates higher hold-clearing rates to it.","sledRelevance":"An operations lesson for any campus running automated, personalised outreach: campaign configuration, change control and staffing continuity are the failure points, not the model.","evidence":"Student journalism with named administrator and student quotes; no incident report, affected-student count or hold-clearance data was published.","architectureImplications":"Vendor platform with planned campaigns on a calendar, per-student data feeds and a 2,000 Q&A knowledge base; campaign templates reused across years.","governanceImplications":"No described pre-send review or testing of reused campaigns; opt-out default for all students.","securityPrivacyImplications":"Per-student feeds include enrollment status and conversational tone; students voiced uncertainty about who controls the bot.","caveats":"Single reported incident; not an LLM failure; outcome claims about hold clearance are unquantified administrator statements; student-success boundary noted, included for its operational-control lesson.","streamIds":["campus-operations"],"roles":{"sales":"Sales framing: the risk in campus automation is often configuration and staffing, not model accuracy. Cal Poly Pomona's wrong registration text came from a copied 2025 campaign after a key operator left. Interpretation: a bounded operational-resilience review of automated outreach (runbooks, pre-send checks, cross-training) is a credible offer. The story does not support claims about the bot's retention impact or about LLM hallucination.","engineering":"Engineering framing: look for controls in the messaging platform itself: campaign cloning safeguards, date-parameter validation, test sends to seeded audiences, approval workflows and audit logs. Per-student data feeds, including inferred tone, expand the data footprint and need documented purposes. Confirm what the vendor can enforce versus what relies on staff procedure.","delivery":"Delivery framing: the fix is operational ownership. Document campaign runbooks, require a second reviewer for reused campaigns, cross-train at least two operators, and define an incident and correction process with timing targets. Track incidents per term. Students also asked who runs the bot, so publish an accountable owner and an opt-out explanation."},"roleDetails":{"sales":{"customerProblem":"Automated student outreach depends on a few operators and reused campaigns without pre-send checks.","stakeholders":["AVP Student Success","Registrar","Communications","Messaging platform vendor"],"discoveryQuestions":["Who can launch campaigns and is there a second reviewer?","How many people can operate the platform today?","How are reused campaigns validated before sending?"],"valueHypothesis":"Change-control and cross-training may reduce misfires in automated outreach; the source reports no incident rates.","potentialEngagement":"An operational-resilience review of automated student messaging with runbooks and pre-send controls.","unsupportedClaims":["Retention or hold-clearance gains attributable to the bot","That this was an LLM failure","Incident frequency across CSU"]},"engineering":{"fit":"Campuses running vendor texting or chatbot platforms with personalised campaigns.","architectureIntegration":"Campaign calendar, per-student data feeds from SIS and a Q&A knowledge base in a vendor platform.","prerequisites":["Platform admin access","Campaign templates inventory","SIS data feed documentation"],"constraints":["Vendor feature limits on validation","Opt-out default","FERPA obligations"],"security":"Limit per-student feed fields to documented purposes and review access to conversation data.","proofOfValue":["Seeded test-audience sends before launch","Approval workflow enabled for reused campaigns","Audit log review of campaign changes"]},"delivery":{"work":"Write runbooks, add two-person review, cross-train operators and define correction procedures.","dependencies":["Vendor cooperation","Registrar calendar data","Staff time"],"ownership":"Student Success leadership with communications and registrar.","skillsAdoption":"Cross-train at least two operators on campaign build and validation.","governanceCheckpoints":["Pre-send review of reused campaigns","Post-incident review with vendor","Termly incident reporting"],"acceptanceCriteria":["Proposed: at least two trained operators","Proposed: corrections issued within an hour","Proposed: zero unreviewed reused campaigns"],"risks":["Key-person dependency","Student distrust of automated messages","Data-feed scope creep"]}},"retrievedAt":"2026-11-07T21:48:40-06:00","enrichedAt":"2026-11-07T22:00:00-06:00","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Shows key-person dependency: loss of one experienced operator preceded the error.","procurementImplications":"Vendor-hosted campaigns place prevention partly on vendor features such as validation and staging.","operatingModelImplications":"Automated outreach needs documented runbooks, cross-trained operators and pre-send checks for reused campaigns.","sourceVerification":{"openedUrl":"https://thepolypost.com/news/2026/11/03/billy-chat-sends-students-wrong-registration-alert/","referenceExcerpt":"the error occurred when a staff member copied a previous registration messaging campaign.","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}