From the Emergency Services edition of October 20, 2026
Scoping review: AI for EMS call-taking and dispatch lacks external validation, and two randomized trials show no consistent benefit
Yonsei University College of Medicine · Emergency medical dispatch research · International (18 studies, 13 countries)
- Publisher
- MDPI Healthcare
- Original publication
- 2026-09-24
- Source retrieved
- 2026-10-21
What happened
A PRISMA-ScR scoping review in MDPI Healthcare (vol. 14, issue 19, article 3176; Crossref publication date Sept 24, 2026) by Yonsei University authors searched six databases for January 2021 to August 2026 and included 18 studies from 13 countries on AI in EMS call-taking and dispatch; 11 focused on patient recognition and classification, five on dispatch support. Eleven were retrospective model-development studies with internal validation only, and none of these conducted independent external validation or calibration assessment; all 11 were rated high risk of bias under PROBAST+AI. The two randomized trials showed no consistent improvement: in Denmark, ML cardiac-arrest alerts had higher sensitivity (85.0% vs 77.5%) but PPV of 17.8% vs 55.8% and did not improve dispatcher recognition (93.1% vs 90.5%, p=0.15); Sweden's MADLAD showed modestly better low-acuity prioritization (68.3% vs 62.5%) with high risk of bias. The authors call for prospective multicenter studies, local recalibration, drift monitoring, dispatcher override authority and auditable logs. Why now: most recent synthesis of dispatch AI evidence, directly relevant to tonight's Korean guide recommending real-time 119 call-audio AI.
Why it matters
911 and EMS dispatch leaders weighing AI call-audio or triage tools can use it as a checklist of what evidence vendors should show: external validation, calibration and operational outcomes.
Evidence and measured results
Peer-reviewed PRISMA-ScR scoping review with risk-of-bias assessment.
Limitations and uncertainty
No registered protocol; English search terms only; conference papers and preprints excluded, so fast-moving technical work is missed; risk-of-bias assessment by one reviewer and added during peer review; funded by a Korean fire/EMS technology program; MADLAD trial already archived separately; published about four weeks before this edition; opened via WebFetch because box curl returned 403.
Put this evidence to work
Lighthouse Advisory interpretation, grounded in this source. Enriched 2026-10-20; this does not change the original publication date. Each role follows the same checklist so takeaways compare across sources.
Sales
Role takeaway
- Customer problem
- Dispatch centers face vendor claims for AI triage and recognition tools without evidence that they work locally.
- Stakeholders
- 911 or EMS dispatch director
- EMS medical director
- Quality assurance manager
- Procurement officer
- Discovery questions
- Has the vendor validated on data from other centers?
- What false-alert rate can your dispatchers absorb?
- Which outcomes would prove value to you?
- Value hypothesis
- An evidence-based evaluation plan could avoid buying tools whose accuracy does not translate into better dispatch outcomes.
- Potential engagement
- Independent evaluation design and silent-mode validation for a proposed dispatch AI tool.
- Unsupported claims
- Improved survival from dispatch AI
- Generalizable model accuracy
- Reduced dispatcher workload
Pre-sales engineering
Role takeaway
- Fit
- Evidence base for evaluating call-audio, transcript and structured-data AI in EMS dispatch.
- Architecture and integration
- ASR and NLP on live call audio feeding CAD-integrated alerts, with logging of outputs and overrides.
- Prerequisites
- Local labeled call data
- CAD integration
- Outcome linkage to EMS records
- Constraints
- Accent, dialect and multilingual speech degrade models
- Different protocols across centers
- Latency limits in time-critical calls
- Security
- Govern secondary use of call audio, de-identify for development, and audit vendor access.
- Proof of value
- External validation on local calls
- Calibration plot and PPV at chosen threshold
- Silent-mode comparison with dispatcher decisions
Delivery
Role takeaway
- Work
- Stage dispatch AI from silent validation to controlled rollout with monitoring.
- Dependencies
- Vendor
- CAD team
- Medical direction
- Data governance
- Ownership
- Dispatch center director with EMS medical director.
- Skills and adoption
- Dispatchers trained on interpreting alerts and resisting automation bias.
- Governance checkpoints
- Local validation sign-off
- Override and output logging live before rollout
- Scheduled drift and recalibration review
- Acceptance criteria
- Local sensitivity and PPV meet thresholds agreed before go-live
- Every override logged and reviewed monthly
- No increase in under-triage over six months
- Risks
- Alert fatigue from low PPV
- Performance drift after protocol changes
- Language-related misses
Implementation considerations
Lighthouse Advisory interpretation across the operating dimensions a public-sector buyer must settle before this evidence becomes a design. Each note answers the question under its heading for this specific source.
Architecture and integration
What must connect, and where does the AI sit in the workflow?
Systems combine ASR, NLP and structured CAD data; must integrate with CAD without adding latency and with fallback for low-confidence or incomplete audio.
Governance
Who approves, reviews and stays accountable for outcomes?
Dispatchers should keep override authority; AI outputs, overrides and final decisions should be logged and auditable; models need drift monitoring and recalibration.
Security and privacy
What data, permissions and controls need testing?
Emergency-call audio contains health, location and bystander data requiring lawful-use, de-identification, retention and third-party access rules.
Accessibility and workforce
Who is affected, and what skills or accommodations follow?
Models may underperform for non-native speakers and distressed callers; dispatchers need training on automation bias.
Procurement
What should contracts, pricing and exit terms secure?
Contracts should require external validation evidence, calibration data, logging and monitoring obligations.
Operating model
Which teams own the service once it runs?
AI as decision support with dispatcher final authority and fallback procedures.
Publication history
- 2026-10-20Emergency Services · Issue 446 resources
Stable resource ID: mdpi-healthcare-scoping-review-ai-ems-dispatch-no-external-validation-20260924