{"resourceId":"pitt-cic-athletics-survey-analysis-agent-2026","versions":[{"version":"external-3441e917919b2050b14cbba004c80756e5b86ac384f86c5cca2873978c653324","resource":{"id":"pitt-cic-athletics-survey-analysis-agent-2026","title":"Pitt CIC builds GenAI Survey Analysis Agent so Athletics can query fan feedback in plain English","organization":"University of Pittsburgh Cloud Innovation Center / Pitt Athletics","sector":"College athletics fan experience / athletic-department analytics operations","geography":"United States (NCAA Division I; University of Pittsburgh)","publishedAt":"February 25, 2026","publicationDate":"2026-02-25","eventDate":null,"sourceName":"Pitt Digital","sourceLabel":"University success story — athletics GenAI survey analytics implementation","sourceUrl":"https://www.digital.pitt.edu/news/success-stories/how-pitt-cic-helping-pitt-athletics-turn-fan-feedback-better-game-day","evidenceClass":"independent-reporting","outcomeClass":"emerging","topics":["knowledge-work","developers-agents","infrastructure","operating-model"],"finding":"Pitt CIC student developers, working with Pitt Athletics, built an open-source GenAI Survey Analysis Agent that lets staff ask natural-language questions over fan survey data. The system embeds survey text, applies similarity thresholds and keyword filters before LLM calls, uses RAG/semantic search via Amazon Bedrock, and returns summaries with citations tied to original response IDs. Associate AD Richard Turnquist describes replacing slow report/dashboard pull cycles. Supporting GitHub docs specify Titan embeddings in S3 Vectors, Claude Sonnet 4.5, and a serverless AWS stack. No independent accuracy or time-savings study is published in the success story.","sledRelevance":"Newly relevant non-archived DI athletic-department GenAI implementation focused on fan-ops qualitative data—complements archived Equipe chat and Morgan academic chatbot with an open AWS RAG pattern and explicit citation design.","evidence":"Pitt Digital article (2026-02-25 datetime): Turnquist quote on slow answers; Farrell similarity-threshold quote; Shelke citation/response-ID language; Bedrock integration; GitHub README stack and ~0.15 threshold testing.","architectureImplications":"Interpretation: treat athletics survey GenAI as a RAG pipeline over approved survey corpora with Cognito auth, async jobs, and mandatory citation payloads—not unconstrained chat over SIS/medical stores.","governanceImplications":"Interpretation: define which staff may query fan PII-adjacent comments; require human review before public or executive decisions based on synthesized themes.","securityPrivacyImplications":"Interpretation: fan survey text can include personal details—apply least privilege, retention limits, and ban secondary model training on uploaded CSVs without review.","caveats":"Success-story framing; no published hallucination audit; earlier 2026 date with why-now as first archive hit for this pattern; fan-ops scope not student-athlete performance AI.","streamIds":["college-athletics"],"roles":{"sales":"Open with athletics data/analytics leads stuck waiting on survey dashboards while game-day experience decisions move weekly—Pitt’s CIC story is a DI department using a cited GenAI survey agent, not a guaranteed ROI case. Discover current survey vendors, who pulls reports, and whether open-ended comments are used. Bound value to faster qualitative triage with evidence links. Do not sell measured prep-time cuts this article does not quantify.","engineering":"Fit is serverless RAG over athletics survey CSVs: embeddings, similarity cutoffs, Bedrock LLM synthesis, and downloadable citation sets. Integrate with existing AWS survey pipelines; keep Cognito-gated; exclude medical/eligibility stores. Prerequisites: cleaned survey schema, auth groups, and a quality eval set. Constraints: success-story evidence; embedding threshold chosen in preliminary tests. Security: encrypt survey objects; log queries; no training on fan text by default.","delivery":"Delivery work is a bounded pilot: deploy open-source stack or equivalent, upload one season’s surveys, train staff on citation checks, and define acceptance on answer audit samples. Dependencies: athletics analytics owner, cloud account, and survey export rights. Ownership: athletics data/analytics owns use; IT/CIC-like builders own platform. Skills: staff verify cited response IDs before acting. Governance: written ban on automated public statements from agent output. Risks: hallucinated themes, over-trust, and scope creep into student-athlete records."},"roleDetails":{"sales":{"customerProblem":"Athletic departments collect dense fan survey text but wait on analysts/dashboards before game-day experience decisions.","stakeholders":["Associate AD data/analytics","Fan experience / ticket ops","Athletic communications","IT/cloud","Privacy office"],"discoveryQuestions":["Which survey platform and export grain do you use?","Who today answers ad-hoc theme questions?","What decisions are timed to next home event?","Would Cognito-style staff auth meet campus policy?"],"valueHypothesis":"If staff can query surveys in natural language with cited source responses, qualitative turnaround may shrink—without claiming audited accuracy.","potentialEngagement":"Three-to-four-week pilot: stand up RAG survey agent on one sport’s surveys, run citation audit, and brief fan-experience leadership.","unsupportedClaims":["Guaranteed time savings percentage","Zero-hallucination summaries","That the tool covers student-athlete academic/medical data"]},"engineering":{"fit":"Conversational RAG analytics over athletic-department fan survey corpora with mandatory citations.","architectureIntegration":"CSV/survey ingest to object storage; Titan (or equiv.) embeddings to vector store; Bedrock/Claude agent with similarity filters; async job API; React UI behind SSO/Cognito.","prerequisites":["Authoritative survey export","Staff identity groups","Human review SOP for published claims"],"constraints":["Preliminary 0.15 similarity cutoff not externally validated","Success-story evidence base","AWS cost variability with query volume"],"security":"Least-privilege IAM; encrypt survey text at rest; query/audit logs; prohibit secondary training on uploads unless contracted.","proofOfValue":["Blind audit of n cited answers vs source text","Median time-to-first-insight vs prior dashboard path","Staff trust rating after two home events"]},"delivery":{"work":"Pilot deployment, data upload validation, staff training, and citation-audit checklist.","dependencies":["Cloud landing zone","Survey vendor export","Analytics staff time"],"ownership":"Athletics analytics owns product use; IT owns infrastructure; privacy reviews data classes.","skillsAdoption":"Train users to open cited response IDs before acting on themes.","governanceCheckpoints":["Pre-prod privacy review","Sample answer audit gate","Quarterly access review"],"acceptanceCriteria":["Citations present on every answer","One sport’s surveys queryable","Documented exclusion of medical/SIS data"],"risks":["Hallucinated themes","PII leakage in open-ended answers","Shadow use beyond fan surveys"]}},"retrievedAt":"2026-09-24T03:23:40Z","enrichedAt":"2026-09-24T03:28:32Z","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Interpretation: open-source CIC pattern may help mid-major analytics shops without large BI teams—if they can staff citation review.","procurementImplications":"Interpretation: prefer tools that return source response IDs and allow self-host/VPC deployment over black-box survey chat.","operatingModelImplications":"Interpretation: keep GenAI as an analyst accelerator under athletics data ownership, not a replacement for fan-experience judgment.","sourceVerification":{"openedUrl":"https://www.digital.pitt.edu/news/success-stories/how-pitt-cic-helping-pitt-athletics-turn-fan-feedback-better-game-day","referenceExcerpt":"We used minimum similarity thresholds and keyword queries to filter results","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}