From the Emergency Services edition of November 10, 2026
Scoping review of AI in EMS call-taking and dispatch: no independent external validation in 11 model-development studies, and two trials without consistent benefit
Yonsei University and Yonsei University Health System (published in Healthcare, MDPI) · EMS call-taking and dispatch research · International (18 studies from 13 countries and one multinational setting); authors in Republic of Korea
- Publisher
- Healthcare (Basel) via PubMed Central
- Original publication
- 2026-09-24
- Source retrieved
- 2026-11-11
What happened
A PRISMA-ScR scoping review searched six databases for studies from January 2021 to August 2026 and included 18 studies from 13 countries and one multinational setting. Eleven studies (61.1%) addressed patient recognition and classification and five (27.8%) dispatch support. Eleven were retrospective model development with internal validation, and only two were randomized trials. None of the 11 model-development studies did independent external validation or calibration assessment, and the two trials (one at high risk of bias, one with some concerns) showed no consistent improvement across dispatch outcomes. The authors conclude evidence is insufficient to support widespread implementation and call for prospective real-world studies, local validation and monitoring for drift.
Why it matters
Independent, peer-reviewed context for how much weight to give vendor and pilot claims about AI triage and dispatch support. Newly relevant now because the archive already holds several live-deployment and simulation items (Seattle, Seoul, Wellington) and this review supplies the evidence-quality frame. It is global, not U.S.-specific, so transferability is limited.
Evidence and measured results
Scoping review in Healthcare (Basel) 2026;14(19):3176, received 14 Jul 2026, accepted 21 Sep 2026, published 24 Sep 2026. Descriptive synthesis, no meta-analysis. Findings as stated in the opened article's abstract and discussion.
Limitations and uncertainty
Scoping review, descriptive and not a meta-analysis; no pooled effect sizes. Studies are heterogeneous and mostly retrospective, from outside the U.S. in many cases. Search window ended August 2026. The review is not an evaluation of any specific product.
Put this evidence to work
Lighthouse Advisory interpretation, grounded in this source. Enriched 2026-11-11; this does not change the original publication date. Each role follows the same checklist so takeaways compare across sources.
Sales
Role takeaway
- Customer problem
- Agencies are pitched AI triage and dispatch tools whose reported accuracy rarely comes from independent or prospective testing.
- Stakeholders
- 911 center director
- EMS medical director
- Fire/EMS chief
- Quality assurance manager
- Procurement and risk officers
- Discovery questions
- What validation evidence would you require before using AI prompts on live calls?
- Who would review performance across languages and accents?
- What outcome, not model score, would define success?
- Value hypothesis
- A validation-and-monitoring framework may lower adoption risk for AI call-taking tools; the review supports the need but not any specific benefit.
- Potential engagement
- Bounded evidence review and pilot design: local validation plan, shadow-mode run and monitoring dashboard specification.
- Unsupported claims
- That AI improves dispatch outcomes
- That any specific product has been externally validated
- That results from these 18 studies apply to a given U.S. center
Pre-sales engineering
Role takeaway
- Fit
- Directly relevant to designing proofs of value for call-taking and dispatch AI.
- Architecture and integration
- Shadow-mode inference alongside CAD and call-taking with logging of model output, dispatcher action and outcome for later calibration checks.
- Prerequisites
- Historical call and outcome data for local validation
- Data-sharing and privacy approvals
- Baseline performance metrics
- Constraints
- Evidence is mostly retrospective and non-U.S.
- Performance may drift as protocols and terminology change
- Subgroup performance is often unreported
- Security
- Not assessed by the review; call audio and clinical data need agency-level privacy review.
- Proof of value
- External-style validation on a held-out local dataset
- Calibration check and subgroup analysis
- Shadow-mode comparison with dispatcher decisions before any live prompt
Delivery
Role takeaway
- Work
- Sequence the project as local validation, shadow-mode run, limited live pilot, then monitoring and recalibration.
- Dependencies
- Local data access
- Medical director protocol alignment
- Analytics staff for monitoring
- Ownership
- Dispatch center director with the EMS medical director; QA lead owns monitoring.
- Skills and adoption
- Dispatcher training on advisory prompts and override documentation; analyst training on drift monitoring.
- Governance checkpoints
- Go/no-go after local validation
- Review after shadow-mode run
- Periodic drift and equity review
- Acceptance criteria
- Proposed: documented local validation with calibration and subgroup results
- Proposed: shadow-mode agreement and miss analysis reviewed by medical director
- Proposed: monthly drift report with named owner
- Risks
- Model degradation over time
- Over-reliance on prompts
- Unrepresentative training data
- Metrics focused on model scores rather than outcomes
Implementation considerations
Lighthouse Advisory interpretation across the operating dimensions a public-sector buyer must settle before this evidence becomes a design. Each note answers the question under its heading for this specific source.
Architecture and integration
What must connect, and where does the AI sit in the workflow?
Predictive performance is not clinical or operational utility; designs should allow local recalibration and drift monitoring, because the authors note performance can degrade as terminology, protocols and case mix change.
Governance
Who approves, reviews and stays accountable for outcomes?
The authors call for prospective and real-world implementation studies before integration, local validation and, where needed, recalibration for language, protocols, prevalence and resources. Interpretation: make external validation and ongoing monitoring a procurement and go-live gate.
Security and privacy
What data, permissions and controls need testing?
Not the focus of the review. Interpretation: call audio and text used for model development raise data-governance requirements that this review does not assess.
Accessibility and workforce
Who is affected, and what skills or accommodations follow?
The authors note language, documentation practices and resources differ across systems, so local validation matters; dispatcher workload effects are not quantified in the review.
Procurement
What should contracts, pricing and exit terms secure?
Require evidence of external or prospective validation, local recalibration support and drift monitoring in solicitations.
Operating model
Which teams own the service once it runs?
The review concludes AI should be integrated only after prospective and real-world implementation studies establish safety and utility; ongoing monitoring is an operating responsibility.
Publication history
- 2026-11-10Emergency Services · Issue 654 resources
Stable resource ID: healthcare-mdpi-ai-ems-call-taking-dispatch-scoping-review-20260924