From the Emergency Services edition of September 28, 2026
Prepared/Axon case study: St. John's County 911 AI QA from ~2% sampling to 100% coverage
Prepared (Axon 911 Intelligence) featuring St. John's County 911, Florida · Vendor case study / emergency communications QA · United States (St. John's County / St. Augustine, Florida)
- Publisher
- Prepared911.com
- Original publication
- 2026-08-31
- Source retrieved
- 2026-09-29
- Event date
- 2026-08-31
What happened
Prepared's August 31, 2026 case study reports St. John's County 911 (St. Augustine, FL) moved from manually reviewing about 2% of ~101,000 annual 911 calls (20–30 minutes per review) to Automated QA + Training from Axon 911 Intelligence by Prepared. Quoted floor manager Debra Smith: the center went from QAing very few calls to doing 100%. In the past 90 days the center QA'd 4,928 calls including 100% of failing calls (score <90%); manual review time is down 90%, with a weekend example of ~120 failing calls in ~3 hours versus ~30 hours manually. Auto and manual scores are said to align within 3%. Managers keep AQA up during shifts to catch issues in progress; coaching happens within days. Humans still review failures and own coaching.
Why it matters
Direct PSAP workforce-AI signal with quantified QA coverage and review-time claims for SLTT emergency communications. Why now: not in emergency-services archive; published 2026-08-31 and newly relevant as 911 AI QA spreads beside non-emergency answering coverage after 2026-09-27's wildfire/triage edition.
Evidence and measured results
Opened Prepared case-study HTML via curl. Date August 31, 2026; 2%→100% QA framing; 101,000 911 calls; 4,928 calls/90 days; 90% manual-time reduction; within-3% score alignment; HITL review of <90% scores taken from page.
Limitations and uncertainty
Vendor marketing case study with single-site self-reported metrics; no independent audit of 90% time savings or 3% score agreement; no outcome data on call quality or patient/public safety.
Put this evidence to work
Lighthouse Advisory interpretation, grounded in this source. Enriched 2026-09-28; this does not change the original publication date. Each role follows the same checklist so takeaways compare across sources.
Sales
Role takeaway
- Customer problem
- PSAPs cannot manually QA more than a tiny sample of 911 calls without burning out staff.
- Stakeholders
- ECC directors
- QA/floor managers
- Telecommunicators
- HR/labor
- Discovery questions
- What % of 911 calls are QA'd today?
- Who owns coaching?
- How are scores used in evals?
- What SOP scorecard exists?
- Value hypothesis
- Automated QA may expand coverage and cut manual review hours if humans still coach fails—not if scores become unreviewed punishment.
- Potential engagement
- 90-day AQA pilot with agreement and coaching-latency KPIs.
- Unsupported claims
- Guaranteed 90% time savings
- Better patient outcomes proven
- AI replaces QA supervisors
Pre-sales engineering
Role takeaway
- Fit
- Automated 911 call scoring and fail surfacing against agency QA SOPs.
- Architecture and integration
- Recorder/NG911 audio → AI score/transcript → fail queue → supervisor review/coaching log.
- Prerequisites
- Recording access
- Digitized SOP scorecard
- Fail threshold
- Identity/access controls
- Constraints
- Vendor self-reported metrics
- Local protocol mismatch
- Dialect/noise robustness
- Security
- Limit vendor access to 911 audio/PII; set retention and subprocessors in the DPA.
- Proof of value
- Auto–manual score agreement
- Manual QA hours
- Time-to-coaching after fail
Delivery
Role takeaway
- Work
- Configure scorecards, train supervisors on fail queues, and stand up dispute/coaching SOPs.
- Dependencies
- ECC ops
- Recording/IT
- HR evaluation policy
- Ownership
- Floor/QA manager owns coaching; ECC director owns policy on score use.
- Skills and adoption
- Train coaches to treat AI scores as prompts for listening, not verdicts.
- Governance checkpoints
- Monthly agreement audit
- Dispute log review
- Eval-policy check
- Acceptance criteria
- Human review of all fails
- Documented dispute path
- Local KPI dashboard
- Risks
- Eval misuse
- Fail-queue overload
- Opaque model changes
Implementation considerations
Lighthouse Advisory interpretation across the operating dimensions a public-sector buyer must settle before this evidence becomes a design. Each note answers the question under its heading for this specific source.
Architecture and integration
What must connect, and where does the AI sit in the workflow?
Treat automated QA as a scoring/transcription layer alongside recording systems with supervisor queues for fails—not as live call-taking or CAD replacement.
Governance
Who approves, reviews and stays accountable for outcomes?
Define fail thresholds, dispute processes, and how AI scores feed employee evaluations; separate vendor case-study metrics from agency-audited KPIs.
Security and privacy
What data, permissions and controls need testing?
Full-call AI scoring implies broad access to 911 audio/transcripts; require data-processing agreements, retention limits, and access controls for vendor platforms.
Accessibility and workforce
Who is affected, and what skills or accommodations follow?
Positions AI as burnout relief for collateral-duty QA staff while keeping supervisors as coaches; train telecommunicators on dispute/feedback workflows.
Procurement
What should contracts, pricing and exit terms secure?
Evaluate AQA against local SOPs, scorecard alignment studies, and staffing models—not headline 100% coverage alone.
Operating model
Which teams own the service once it runs?
Keep humans reviewing fails and conducting coaching; use AI to prioritize which calls enter the human QA queue.
Publication history
- 2026-09-28Emergency Services · Issue 234 resources
Stable resource ID: prepared-st-johns-911-aqa-20260831