{"resourceId":"logidebrief-nashville-911-qa-ijcai-2025","versions":[{"version":"external-a54993ec38c17659946878691f2d3fe42ab3624f6ecd5c17e49d0cd6414bf97e","resource":{"id":"logidebrief-nashville-911-qa-ijcai-2025","title":"LogiDebrief STL+LLM system reports 1,701 Nashville 9-1-1 QA reviews and ~312 hours saved","organization":"Vanderbilt University / Metro Nashville Department of Emergency Communications","sector":"911 / ECC quality assurance","geography":"Nashville, Tennessee, United States (academic + municipal ECC partnership)","publishedAt":"2025","publicationDate":"2025-05-06","eventDate":null,"sourceName":"arXiv / IJCAI 2025 proceedings","sourceLabel":"Academic research with municipal deployment claims","sourceUrl":"https://arxiv.org/html/2505.03985v1","evidenceClass":"academic-research","outcomeClass":"emerging","topics":["knowledge-work","developers-agents","operating-model","governance-procurement"],"finding":"LogiDebrief combines Signal-Temporal Logic specifications with LLM checks to automate post-call 9-1-1 debriefing. Authors report deployment at Metro Nashville DEC assisting 1,701 real-world call reviews, an estimated 311.85 hours saved, and QA coverage rising from a prior human baseline into the mid-80% range (text cites boost to 85.05%). Why now: after the 2026-09-18 Long Beach CommsCoach QA edition, this is the first archived measured U.S. ECC post-call AI coverage/hours package distinct from Sim911 training simulation.","sledRelevance":"Provides quantified post-call QA automation evidence for U.S. ECCs seeking coverage gains without placing AI on live emergency answering—aligned to SAFECOM caution on life-safety AI.","evidence":"Abstract and deployment claims: 1,701 calls; 311.85 hours; coverage to 85.05%; hybrid STL+LLM method; limitations note rare-incident coverage gaps and possible inflated false-positive dynamics because professional operators err rarely.","architectureImplications":"Formal specs + modular LLM predicates for protocol checks; QA report generation after calls; keep offline from live call-taking path.","governanceImplications":"Expert-authored STL specs and QA officer review remain necessary; automated scores should not become unreviewed discipline evidence without policy.","securityPrivacyImplications":"Processes recorded 9-1-1 audio/transcripts—needs CJIS-aligned storage, access audit, and retention limits similar to other ECC QA AI.","caveats":"IJCAI/arXiv 2025 vintage (pre-Sept 2026); metrics are author-reported from partner deployment; dataset may under-represent rare critical incidents; not a clinical outcome trial.","streamIds":["emergency-services"],"roles":{"sales":"Position as post-call QA coverage tooling for ECCs drowning in manual review backlogs—not live triage AI. Lead with Nashville’s reported hours and coverage lift, then immediately bound claims: author-reported partnership metrics, not a multi-site RCT. Offer a QA-policy and sample-audit engagement before any production scoring used in personnel actions.","engineering":"Fit is offline STL-specified protocol checking with LLM evaluators producing QA forms. Integrate with logging recorder/CAD exports after call completion; never in the live answer path. Prerequisites: protocol formalization workshops, audio/transcript pipelines, QA officer workflow. Constraints: rare-event sparsity, LLM context brittleness the paper itself documents for naive RAG. Security: CJIS, least-privilege QA access, immutable audit of score changes. Proof: parallel human–AI agreement sampling and time-per-review before/after.","delivery":"Work includes protocol-to-STL encoding, pilot on historical calls, QA officer calibration, and coverage KPI dashboard. Ownership: ECC QA manager with IT. Skills: QA staff learn to contest AI flags. Governance: ban automated discipline without human review. Acceptance: ≥target agreement on stratified sample; documented hours returned to coaching. Risks: over-trust in F1 on imbalanced labels; missing rare protocols."},"roleDetails":{"sales":{"customerProblem":"ECC QA cannot manually review enough 9-1-1 calls, delaying coaching while live AI answering remains politically and safety-constrained.","stakeholders":["ECC directors","QA supervisors","Training coordinators","City risk/privacy"],"discoveryQuestions":["What % of calls are reviewed today?","Minutes per manual debrief?","Will AI scores enter HR files?","CJIS path for transcripts?"],"valueHypothesis":"If STL+LLM QA raises review coverage and cuts minutes per debrief under human confirmation, ECCs may fund post-call AI before touching live triage.","potentialEngagement":"Protocol formalization + 4-week parallel QA pilot with agreement audit.","unsupportedClaims":["Guaranteed clinical outcome improvement","Nationwide identical 311-hour savings","Safe replacement of QA officers","Live 911 answering"]},"engineering":{"fit":"Post-call automated debriefing via STL specs and LLM compliance checks generating QA forms.","architectureIntegration":"Batch/stream after call recording; export to QA workflow; hard isolation from live ACD/911 answer path.","prerequisites":["Protocol manuals","Audio/transcript access","Expert STL authors","QA UI"],"constraints":["Rare-incident undercoverage","LLM context limits on long calls","Label imbalance inflating some scores"],"security":"CJIS-aligned storage, access logging, retention limits, no model training on other customers without contract.","proofOfValue":["Human–AI agreement on stratified sample","Minutes per review before/after","Coverage % of total calls","Appeal/override rate"]},"delivery":{"work":"Encode protocols, calibrate on historical calls, train QA to override, publish coverage dashboard.","dependencies":["Recorder/CAD vendors","Legal/privacy","Union/HR policy if scores affect personnel"],"ownership":"MNDEC-style QA leadership with research/IT partners.","skillsAdoption":"QA officers practice contesting AI flags with audio evidence.","governanceCheckpoints":["HR use policy","Weekly override review","Rare-protocol gap register"],"acceptanceCriteria":["Coverage target met without live-path integration","Documented override workflow","Agreement threshold on audit sample"],"risks":["HR misuse of scores","Silent failure on rare events","Vendor lock-in on specs"]}},"retrievedAt":"2026-09-19T22:11:19.661643-05:00","enrichedAt":"2026-09-19T22:15:56-05:00","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Frees QA time for coaching but requires staff literacy to challenge model errors; training burden shifts toward AI-assisted review skills.","procurementImplications":"Buy as QA assist with measurable coverage/time SLAs and explicit ban on unsupervised disciplinary use.","operatingModelImplications":"Moves ECC AI effort into after-action learning loops rather than live triage—lower life-safety blast radius if governed.","sourceVerification":{"openedUrl":"https://arxiv.org/html/2505.03985v1","referenceExcerpt":"it has assisted in debriefing 1,701 real-world calls, saving 311.85 hours of active engagement","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}