From the College Athletics edition of September 23, 2026
Pitt CIC builds GenAI Survey Analysis Agent so Athletics can query fan feedback in plain English
University of Pittsburgh Cloud Innovation Center / Pitt Athletics · College athletics fan experience / athletic-department analytics operations · United States (NCAA Division I; University of Pittsburgh)
- Publisher
- Pitt Digital
- Original publication
- February 25, 2026
- Source retrieved
- 2026-09-24
What happened
Pitt CIC student developers, working with Pitt Athletics, built an open-source GenAI Survey Analysis Agent that lets staff ask natural-language questions over fan survey data. The system embeds survey text, applies similarity thresholds and keyword filters before LLM calls, uses RAG/semantic search via Amazon Bedrock, and returns summaries with citations tied to original response IDs. Associate AD Richard Turnquist describes replacing slow report/dashboard pull cycles. Supporting GitHub docs specify Titan embeddings in S3 Vectors, Claude Sonnet 4.5, and a serverless AWS stack. No independent accuracy or time-savings study is published in the success story.
Why it matters
Newly relevant non-archived DI athletic-department GenAI implementation focused on fan-ops qualitative data—complements archived Equipe chat and Morgan academic chatbot with an open AWS RAG pattern and explicit citation design.
Evidence and measured results
Pitt Digital article (2026-02-25 datetime): Turnquist quote on slow answers; Farrell similarity-threshold quote; Shelke citation/response-ID language; Bedrock integration; GitHub README stack and ~0.15 threshold testing.
Limitations and uncertainty
Success-story framing; no published hallucination audit; earlier 2026 date with why-now as first archive hit for this pattern; fan-ops scope not student-athlete performance AI.
Put this evidence to work
Lighthouse Advisory interpretation, grounded in this source. Enriched 2026-09-24; this does not change the original publication date. Each role follows the same checklist so takeaways compare across sources.
Sales
Role takeaway
- Customer problem
- Athletic departments collect dense fan survey text but wait on analysts/dashboards before game-day experience decisions.
- Stakeholders
- Associate AD data/analytics
- Fan experience / ticket ops
- Athletic communications
- IT/cloud
- Privacy office
- Discovery questions
- Which survey platform and export grain do you use?
- Who today answers ad-hoc theme questions?
- What decisions are timed to next home event?
- Would Cognito-style staff auth meet campus policy?
- Value hypothesis
- If staff can query surveys in natural language with cited source responses, qualitative turnaround may shrink—without claiming audited accuracy.
- Potential engagement
- Three-to-four-week pilot: stand up RAG survey agent on one sport’s surveys, run citation audit, and brief fan-experience leadership.
- Unsupported claims
- Guaranteed time savings percentage
- Zero-hallucination summaries
- That the tool covers student-athlete academic/medical data
Pre-sales engineering
Role takeaway
- Fit
- Conversational RAG analytics over athletic-department fan survey corpora with mandatory citations.
- Architecture and integration
- CSV/survey ingest to object storage; Titan (or equiv.) embeddings to vector store; Bedrock/Claude agent with similarity filters; async job API; React UI behind SSO/Cognito.
- Prerequisites
- Authoritative survey export
- Staff identity groups
- Human review SOP for published claims
- Constraints
- Preliminary 0.15 similarity cutoff not externally validated
- Success-story evidence base
- AWS cost variability with query volume
- Security
- Least-privilege IAM; encrypt survey text at rest; query/audit logs; prohibit secondary training on uploads unless contracted.
- Proof of value
- Blind audit of n cited answers vs source text
- Median time-to-first-insight vs prior dashboard path
- Staff trust rating after two home events
Delivery
Role takeaway
- Work
- Pilot deployment, data upload validation, staff training, and citation-audit checklist.
- Dependencies
- Cloud landing zone
- Survey vendor export
- Analytics staff time
- Ownership
- Athletics analytics owns product use; IT owns infrastructure; privacy reviews data classes.
- Skills and adoption
- Train users to open cited response IDs before acting on themes.
- Governance checkpoints
- Pre-prod privacy review
- Sample answer audit gate
- Quarterly access review
- Acceptance criteria
- Citations present on every answer
- One sport’s surveys queryable
- Documented exclusion of medical/SIS data
- Risks
- Hallucinated themes
- PII leakage in open-ended answers
- Shadow use beyond fan surveys
Implementation considerations
Lighthouse Advisory interpretation across the operating dimensions a public-sector buyer must settle before this evidence becomes a design. Each note answers the question under its heading for this specific source.
Architecture and integration
What must connect, and where does the AI sit in the workflow?
Treat athletics survey GenAI as a RAG pipeline over approved survey corpora with Cognito auth, async jobs, and mandatory citation payloads—not unconstrained chat over SIS/medical stores.
Governance
Who approves, reviews and stays accountable for outcomes?
Define which staff may query fan PII-adjacent comments; require human review before public or executive decisions based on synthesized themes.
Security and privacy
What data, permissions and controls need testing?
Fan survey text can include personal details—apply least privilege, retention limits, and ban secondary model training on uploaded CSVs without review.
Accessibility and workforce
Who is affected, and what skills or accommodations follow?
Open-source CIC pattern may help mid-major analytics shops without large BI teams—if they can staff citation review.
Procurement
What should contracts, pricing and exit terms secure?
Prefer tools that return source response IDs and allow self-host/VPC deployment over black-box survey chat.
Operating model
Which teams own the service once it runs?
Keep GenAI as an analyst accelerator under athletics data ownership, not a replacement for fan-experience judgment.
Publication history
- 2026-09-23College Athletics · Issue 174 resources
Stable resource ID: pitt-cic-athletics-survey-analysis-agent-2026