{"resourceId":"ucsd-chatcpr-ohca-911-dispatcher-benchmark-20260518","versions":[{"version":"external-b25d50b1de6e2a0d6cda923c6959286ceed84c2cf1da20fbafa2918b91800be7","resource":{"id":"ucsd-chatcpr-ohca-911-dispatcher-benchmark-20260518","title":"UC San Diego: ChatCPR open-source AI beats 911 dispatchers on CPR guideline checklists in proof-of-concept","organization":"UC San Diego Qualcomm Institute / University of Pittsburgh / Johns Hopkins et al. (JAMA Internal Medicine study)","sector":"Academic EMS / OHCA telecommunicator-CPR research","geography":"United States (multi-institution; tested on public U.S. 911 call recordings)","publishedAt":"2026-05-18","publicationDate":"2026-05-18","eventDate":"2026-05-18","sourceName":"UC San Diego Today","sourceLabel":"UC San Diego news release on ChatCPR JAMA Internal Medicine proof-of-concept versus 911 dispatchers","sourceUrl":"https://today.ucsd.edu/story/ai-powered-cpr-coach-outperforms-911-dispatchers-in-guiding-bystander-resuscitation","evidenceClass":"academic-research","outcomeClass":"mixed","topics":["knowledge-work","developers-agents","accessibility-workforce","operating-model"],"finding":"May 18, 2026 UC San Diego Today summarizes a JAMA Internal Medicine proof-of-concept: ChatCPR, an open-source AI CPR instructor (built on Llama 3.3 70B Instruct Turbo per related coverage; grounded in T-CPR materials), scored 100% on simulated OHCA checklist criteria and, on 12 real de-identified 911 CPR calls, 100% minimally viable and ~99% maximally effective criteria versus human dispatchers at 85% and 63% (about +15 and +36 percentage points). Authors stress support-not-replace framing, need for real-world trials/safeguards/legal clarity on AI-enabled CPR protections, and open release for adaptation. Not a prospective live-dispatch outcome study.","sledRelevance":"Measured OHCA instructional-AI evidence for 911/EMS alongside today’s Lake County EMD QA procurement—shows life-safety coaching AI advancing while still pre-operational. Why now: not archived; newly relevant as PSAPs scale assistive AI and need checklist-grade T-CPR evidence with explicit real-world-validation caveats after 2026-09-28.","evidence":"Opened UCSD Today HTML via curl. Date May 18, 2026; JAMA Internal Medicine citation; 100%/~99% vs ~85%/63% dispatcher comparisons; 12 real 911 calls; support-not-replace and real-world-trial caveats; open-source release statements from page.","architectureImplications":"Position as dispatcher-assist or training/sim tool with human telecommunicator control of DA-CPR—not autonomous caller-facing replacement without prospective trials.","governanceImplications":"Require prospective field trials, Good Samaritan/liability clarity for AI-guided CPR, and medical-director approval before any live prompt use.","securityPrivacyImplications":"Retrospective use of real 911 call recordings needs lawful de-identification regimes; live assist would process highly sensitive audio—minimize retention.","caveats":"Proof-of-concept on 12 public calls and simulations; checklist adherence ≠ survival benefit; institutional news release summarizing the journal article.","streamIds":["emergency-services"],"roles":{"sales":"Frame ChatCPR’s checklist wins versus dispatchers on 12 real 911 CPR calls (+~15 pp basic / +~36 pp advanced) as promising open OHCA instructional evidence—not a live PSAP product. Discovery should cover current T-CPR QA, medical-director appetite for prospective trials, and liability questions for AI-guided CPR. Bounded engagement: simulation lab comparison under medical direction using local protocols. Unsupported claims include survival improvement, readiness to replace dispatchers, or that JAMA proof-of-concept equals field authorization.","engineering":"Fit is guideline-concordant CPR instruction agent for training or dispatcher assist. Architecture needs low-latency dialogue control, protocol versioning, barge-in/human override, and logging of prompts vs spoken guidance. Prerequisites: T-CPR protocol digitization, audio/ASR path, and IRB/data rules if using real calls. Constraints: n=12 real calls; no prospective outcomes here. Proof of value is prospective time-to-compressions and protocol adherence with HITL—not retrospective checklist scores alone.","delivery":"Delivery starts in training and simulation, then carefully gated assistive prompts only if medical direction approves. Dependencies include the EMS medical director, ECC training staff, legal/risk counsel, and research partners for prospective design. Ownership: medical director for clinical content; ECC for live workflow. Acceptance criteria require documented human control of caller speech, a prospective trial protocol, and completed liability review. Risks include bystander confusion, dispatcher deskilling, and overclaiming from small-n checklist scores."},"roleDetails":{"sales":{"customerProblem":"Telecommunicator-CPR quality is inconsistent under stress while OHCA survival remains low.","stakeholders":["EMS medical directors","ECC training leads","Telecommunicators","Legal/risk"],"discoveryQuestions":["Current T-CPR audit scores?","Sim lab available?","Liability stance on AI CPR prompts?","Appetite for open-source evaluation?"],"valueHypothesis":"Open CPR agents may standardize high-quality prompts if prospectively validated—checklist deltas on 12 calls are not field proof.","potentialEngagement":"Medical-director-led simulation comparison before any live assist pilot.","unsupportedClaims":["Survival benefit proven","Replaces 911 dispatchers","Ready for unsupervised caller use"]},"engineering":{"fit":"Protocol-grounded CPR instruction agent for training or HITL dispatcher assist.","architectureIntegration":"Call/sim audio → ASR/dialogue agent → prompt cards to telecommunicator or trainee → human-spoken guidance.","prerequisites":["Digitized T-CPR protocol","Override/barge-in controls","Logging","Data-use approval"],"constraints":["Small real-call n","Retrospective design","No live outcome data"],"security":"Strict handling of 911 training audio; disable vendor secondary training use.","proofOfValue":["Prospective protocol adherence","Time-to-first compression","Dispatcher accept/edit rate"]},"delivery":{"work":"Stand up sim evaluation, train users, and draft prospective HITL protocol with legal review.","dependencies":["Medical direction","ECC training","Counsel/risk","Research collaborators"],"ownership":"EMS medical director owns clinical content; ECC training owns rollout.","skillsAdoption":"Teach dispatchers to use agent prompts as checklists they still voice and adapt.","governanceCheckpoints":["IRB/data gate","Sim go/no-go","Pre-live liability review"],"acceptanceCriteria":["Human always speaks to caller","Prospective trial registered","Failover without agent"],"risks":["Bystander confusion","Deskilling","Overgeneralizing from n=12"]}},"retrievedAt":"2026-09-30T22:07:36-05:00","enrichedAt":"2026-09-30T22:07:36-05:00","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"May raise the floor of T-CPR instruction quality if integrated as prompts for multitasking dispatchers; training must prevent automation complacency.","procurementImplications":"Prefer open evaluation materials over closed T-CPR copilots; budget prospective HITL trials rather than buying on checklist deltas alone.","operatingModelImplications":"Keep telecommunicators responsible for DA-CPR delivery; use ChatCPR-like agents for standardized prompts and training QA.","sourceVerification":{"responseId":"curl-ucsd-chatcpr-20260930T220736","openedUrl":"https://today.ucsd.edu/story/ai-powered-cpr-coach-outperforms-911-dispatchers-in-guiding-bystander-resuscitation","referenceExcerpt":"ChatCPR won every head-to-head comparison with human dispatchers","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}