{"resourceId":"explainx-nola-carbyne-audit-gaps-20260810","versions":[{"version":"external-b402a23a3ae61014d9304352f01e59fd38e8ff9e0eeba6f7c69674ce0a5a5e3d","resource":{"id":"explainx-nola-carbyne-audit-gaps-20260810","title":"explainx: New Orleans Carbyne 911 AI narrow but unaudited—zero FP/FN claim remains unverified","organization":"explainx.ai analysis of OPCD / Carbyne (Axon) deployment","sector":"911 AI governance / independent scrutiny","geography":"New Orleans, Louisiana, United States (national transferability discussion)","publishedAt":"2026-08-10","publicationDate":"2026-08-10","eventDate":"2026-08-06","sourceName":"explainx.ai","sourceLabel":"Independent technical analysis","sourceUrl":"https://www.explainx.ai/blog/new-orleans-ai-911-dispatch-first-us-city-rollout-august-2026","evidenceClass":"independent-reporting","outcomeClass":"cautionary","topics":["governance-procurement","data-security","operating-model"],"finding":"An Aug 10 2026 explainx deep dive argues New Orleans has run Carbyne AI on live 911 traffic since 2023 for crash-call deduplication (with ~200 m matching and human escalation), confirmed publicly around Aug 6 2026 after limited prior disclosure, and itemizes what remains unconfirmed—including any independent third-party audit of the oft-repeated zero false-positive / zero false-negative claim from the initial fully monitored window. Why now: pairs with Gambit's scope interview to give SLED buyers a scrutiny checklist after non-emergency AVA and CommsCoach themes already in the archive.","sledRelevance":"Independent scrutiny template for U.S. SLED PSAPs evaluating vendor accuracy claims on live emergency call AI.","evidence":"Opened explainx HTML dated Aug 10, 2026. TL;DR table lists unconfirmed independent audit of zero FP/FN, full incident-level error rate since 2023, and contract terms beyond a reported ~$600K/year figure. Describes classifier-style architecture, 100% human monitoring at launch, and escalation on uncertainty.","architectureImplications":"Treat vendor accuracy claims as pilot-window observations unless continuous sampling and third-party audit are contractually required; separate classifier screening from any generative agent stack.","governanceImplications":"Mandate public disclosure timelines, independent error audits, and change-control before expanding beyond crash duplicates; document monitoring intensity after the initial 100% review period.","securityPrivacyImplications":"Axon/Carbyne ownership and multi-year call handling raise supply-chain and data-use questions; require written bans on reuse of 911 audio for unrelated model training.","caveats":"Secondary analysis synthesizing press and official statements; not a formal government audit; some dollar and staffing figures are reported, not independently verified here.","streamIds":["emergency-services"],"roles":{"sales":"Use this as the objection-handling brief when a vendor cites New Orleans 'zero errors.' Acknowledge the narrow crash-duplicate design as defensible in concept, then insist the zero FP/FN figure is an initial monitored-window claim without a published independent audit. Pitch an evaluation engagement that buys audit rights, caller disclosure, and continuous sampling—not a story that the city already proved permanent accuracy for live 911 AI.","engineering":"Fit scrutiny to classifier-based duplicate screening with human escalation—not generative dispatch. Architecture should log every automated close, retain audio for audit, and expose confidence/uncertainty transfers. Prerequisites: sampling plan, third-party review clause, and metrics dashboard. Constraints: post-pilot monitoring may drop to spot checks. Security: contractually bar training reuse across customers. Proof of value: independent sample of FP/FN after the honeymoon monitoring period.","delivery":"Work is a governance package: disclosure timeline, audit RFP, monthly error sampling, and expansion freeze gates. Dependencies: legal, PSAP QA, and vendor data exports. Ownership: PSAP director with city auditor or external evaluator. Skills adoption: teach leaders to distinguish pilot-window zero defects from steady-state rates. Governance checkpoints: public confirmation date, audit kickoff, and board review of any new use case. Acceptance: published independent sample within 180 days; no expansion until then. Risks: reputation harm if undisclosed live use continues."},"roleDetails":{"sales":{"customerProblem":"Agencies need to separate narrow, defensible 911 AI designs from unaudited accuracy marketing.","stakeholders":["PSAP directors","City auditors","Procurement","Civil liberties advocates","Vendor account teams"],"discoveryQuestions":["Was the zero-error window 100% monitored?","Who can independently sample closes?","Is live use disclosed to callers?","What monitoring remains after month three?"],"valueHypothesis":"If independent sampling confirms low misclose rates under narrow rules, councils may approve continued use without broader automation.","potentialEngagement":"Audit-rights contract annex plus 180-day independent sampling study.","unsupportedClaims":["Permanent zero FP/FN","Proof that AI can safely answer medical 911","Transfer of New Orleans rates to other cities without local audit"]},"engineering":{"fit":"Duplicate-call classifier with mandatory uncertainty escalation and auditable close logs.","architectureIntegration":"Immutable close/transfer logs tied to recordings; metrics API for external auditors.","prerequisites":["Exportable event logs","Retained audio","Sampling protocol","Third-party NDA/access"],"constraints":["Vendor black-box models","Post-pilot spot-check drift","Staffing for review"],"security":"Ban cross-customer training on PSAP audio; encrypt recordings at rest.","proofOfValue":["Independent FP/FN sample after monitored launch","Disclosure completeness check","Escalation latency distribution"]},"delivery":{"work":"Stand up disclosure, retain logs, commission sample audit, brief board on results.","dependencies":["Budget for auditor","Vendor cooperation","Legal review","QA staffing"],"ownership":"PSAP director with city auditor co-sponsor.","skillsAdoption":"Train executives to read pilot vs steady-state accuracy correctly.","governanceCheckpoints":["Kickoff audit","Midpoint sample","Expansion vote"],"acceptanceCriteria":["Independent sample published","No new use cases until sample accepted","Caller disclosure live"],"risks":["Vendor refusal of data access","Public backlash to delayed disclosure","Silent monitoring reduction"]}},"retrievedAt":"2026-09-20T22:09:51-05:00","enrichedAt":"2026-09-20T22:09:51-05:00","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Highlights staffing shortfalls as the motive for automation while warning that audit labor is still required; accessibility of disclosure materials for the public is part of trust.","procurementImplications":"Negotiate audit rights, data-export SLAs, and training-use bans up front; treat ~$600K/year class spend as needing performance schedules.","operatingModelImplications":"Encourages a two-phase operating model: 100% review at launch, then statistically valid sampling—never marketing the pilot window as permanent accuracy.","sourceVerification":{"responseId":"curl-explainx-nola-20260810","openedUrl":"https://www.explainx.ai/blog/new-orleans-ai-911-dispatch-first-us-city-rollout-august-2026","referenceExcerpt":"Independent, third-party audit of the \"zero false positive / zero false negative\" claim","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}