From the Emergency Services edition of November 10, 2026
Korean simulation: a Pre-KTAS-trained ChatGPT matched paramedics on triage accuracy (0.81 vs 0.80), but only on text scenarios with no field validation
Fire Science and Engineering journal (summarized by Paramedic Papers) · Prehospital triage / EMS decision support · Republic of Korea
- Publisher
- Paramedic Papers
- Original publication
- 2026-11-03
- Source retrieved
- 2026-11-11
What happened
Paramedic Papers summarizes a Korean simulation study of 100 prehospital triage scenarios (Pre-KTAS scale), each run 30 times. A Pre-KTAS-trained ChatGPT matched 10 active-duty 119 paramedics on accuracy (0.81 vs 0.80), with the lowest under-triage rate (6.1%) and highest sensitivity (0.95), while paramedics had the highest specificity (0.96) and PPV (0.93). An untrained LLM scored 0.69, significantly lower (p<.001). The summary lists caveats: simulated text scenarios, a Korean triage scale, repetition that may inflate apparent precision, and no field validation, and labels the work hypothesis-generating.
Why it matters
A fresh data point that grounding an LLM in the local triage guideline changes results, and that comparable accuracy to paramedics in text simulation says little about live calls. It is relevant to dispatch and prehospital decision support evaluation, with strong transferability limits from Korean protocols and simulated text.
Evidence and measured results
Summary page (added 3 November 2026) reports design, participants and main findings of a simulation-based comparative study in Fire Science and Engineering. The underlying journal page could not be opened (DOI redirect returned HTTP 500), so numbers are as reported by the summary and unverified against the paper.
Limitations and uncertainty
Secondary summary of a paper the agent could not open; the page itself notes simulated text scenarios, a Korean triage scale, repeated runs and no field validation. Paramedic comparison group is 10 people. Peer-review status of the journal paper was not confirmed by the agent.
Put this evidence to work
Lighthouse Advisory interpretation, grounded in this source. Enriched 2026-11-11; this does not change the original publication date. Each role follows the same checklist so takeaways compare across sources.
Sales
Role takeaway
- Customer problem
- Agencies want to know whether LLMs can safely support triage decisions, but published evidence is early.
- Stakeholders
- EMS medical director
- Dispatch center director
- Clinical informatics lead
- Risk and legal counsel
- Discovery questions
- Which triage protocol would the tool need to follow, and who maintains it?
- What under-triage rate is clinically acceptable?
- What field testing would you require before live use?
- Value hypothesis
- Protocol-grounded LLM decision support may merit a bounded simulation and shadow-mode evaluation; this simulation does not show live benefit.
- Potential engagement
- Bounded simulation study design using the agency's own protocol and de-identified cases, ending in a go/no-go on shadow-mode testing.
- Unsupported claims
- That LLMs triage as well as paramedics in live calls
- That Korean Pre-KTAS results apply to U.S. protocols
- That the paper's numbers are verified, since the primary page was inaccessible
Pre-sales engineering
Role takeaway
- Fit
- Exploratory only; relevant as a design input for evaluation methodology.
- Architecture and integration
- Protocol-grounded LLM (guideline provided as training or retrieval context) behind a decision-support interface, with version-controlled protocol content and full logging.
- Prerequisites
- Machine-readable local triage protocol
- De-identified scenario set with reference answers
- Evaluation harness with repeated runs
- Constraints
- Text-only simulation
- Hosted-model data privacy limits
- Protocol differences across jurisdictions
- Security
- Not addressed; keep prehospital case data out of unapproved hosted models and review vendor data terms.
- Proof of value
- Compare trained and untrained configurations on local scenarios
- Report under-triage separately from accuracy
- Shadow-mode test on real cases before live prompts
Delivery
Role takeaway
- Work
- Run a simulation phase on local scenarios, then shadow mode, with protocol-content ownership and under-triage monitoring.
- Dependencies
- Medical director approval of protocol content
- Scenario dataset with expert reference labels
- Privacy review of data flows
- Ownership
- EMS medical director for protocol and clinical safety; informatics lead for the evaluation harness.
- Skills and adoption
- Training for evaluators on repeated-run variability and under-triage review.
- Governance checkpoints
- Approval of scenario set
- Go/no-go for shadow mode
- Safety review of under-triage cases
- Acceptance criteria
- Proposed: under-triage rate at or below a threshold set by the medical director
- Proposed: results reproducible across repeated runs
- Proposed: no live prompts until shadow-mode results are reviewed
- Risks
- Simulation results not generalizing
- Under-triage harm
- Protocol drift between model context and current guideline
- Unverified source numbers
Implementation considerations
Lighthouse Advisory interpretation across the operating dimensions a public-sector buyer must settle before this evidence becomes a design. Each note answers the question under its heading for this specific source.
Architecture and integration
What must connect, and where does the AI sit in the workflow?
Guideline grounding or training material appears to matter more than model choice alone; any deployment would need version control of protocol content and a way to test updates before release.
Governance
Who approves, reviews and stays accountable for outcomes?
Because results are simulated and hypothesis-generating, governance should require prospective field testing and defined under-triage tolerances before any live use.
Security and privacy
What data, permissions and controls need testing?
Not addressed in the summary. Interpretation: sending prehospital case data to a hosted LLM would need privacy review.
Accessibility and workforce
Who is affected, and what skills or accommodations follow?
Compares an LLM against 10 active-duty paramedics; the summary does not address workforce effects. Interpretation: positioned as support, not replacement, until field data exist.
Procurement
What should contracts, pricing and exit terms secure?
Require vendors to demonstrate performance on the buyer's own protocol and to disclose how protocol content is supplied to the model.
Operating model
Which teams own the service once it runs?
A protocol-owner role and controlled update process would be needed to keep the model's guidance aligned with current guidelines.
Publication history
- 2026-11-10Emergency Services · Issue 654 resources
Stable resource ID: paramedic-papers-korean-prektas-chatgpt-triage-simulation-20261103