From the Emergency Services edition of October 20, 2026
General-purpose LLMs misread prehospital ECGs for cath-lab activation; monitor algorithm outperformed all three
LSU Health Sciences Center-Shreveport · Emergency medical services research · Shreveport, Louisiana, USA
- Publisher
- Heart & Lung: The Journal of Cardiopulmonary and Acute Care
- Original publication
- 2026-09-15
- Source retrieved
- 2026-10-21
What happened
A retrospective single-center study in Heart & Lung (published online Sept 15, 2026; LSU Health Sciences Center-Shreveport) tested three general-purpose multimodal LLMs (ChatGPT-5, Claude Sonnet 4, Gemini 2.5 Pro) on 615 prehospital 12-lead ECG images from 270 Shreveport EMS encounters (2019-2024) with suspected acute MI, against cardiology STEMI-pathway activation as the reference (101 activations, 232 positive ECGs). Gemini had 95.3% sensitivity but 9.4% specificity; ChatGPT 68.1%/42.3% and Claude 67.2%/46.5%. The monitor's built-in Glasgow algorithm was more balanced at 67.7% sensitivity and 64.2% specificity, with the highest accuracy (65.5% versus 41.8-54.3%); LLM PPV ranged 38.9-43.2%. Authors conclude general-purpose LLMs are not appropriate for cath-lab activation decisions and should be limited to education and documentation, and stress the results do not generalize to dedicated AI-ECG models. Why now: tonight's Jeju result credits a purpose-built ECG AI; this study shows why buyers must distinguish dedicated ECG tools from general chatbots that paramedics can reach on a phone.
Why it matters
EMS agencies need policy on clinicians using consumer chatbots for ECG reads; this U.S. study gives medical directors evidence to prohibit it while evaluating purpose-built tools separately.
Evidence and measured results
Peer-reviewed retrospective diagnostic-accuracy study with open dataset on Figshare.
Limitations and uncertainty
Single-center, retrospective, zero-shot prompt, static cropped images without clinical context; encounter-level labels applied to individual ECGs; reference standard is activation decisions, not angiographic occlusion for non-activated patients; no repeat runs to measure variability; published five weeks before this edition; opened via WebFetch because box curl returned 403, so publisher excerpt checks may fail.
Put this evidence to work
Lighthouse Advisory interpretation, grounded in this source. Enriched 2026-10-20; this does not change the original publication date. Each role follows the same checklist so takeaways compare across sources.
Sales
Role takeaway
- Customer problem
- EMS leaders lack evidence to judge AI ECG claims and to govern informal chatbot use by clinicians.
- Stakeholders
- EMS medical director
- Cardiology STEMI coordinator
- EMS quality manager
- Agency privacy officer
- Discovery questions
- Do any staff use chatbots for clinical questions?
- How do you validate ECG decision tools today?
- What is your false-activation rate?
- Value hypothesis
- A clear AI ECG policy and evaluation method could prevent unsafe tool use and focus pilots on validated products.
- Potential engagement
- Clinical AI use policy and ECG-tool evaluation framework for an EMS agency.
- Unsupported claims
- Performance of dedicated AI-ECG products
- Any benefit of LLMs for ECG triage
- Results under few-shot or context-rich prompting
Pre-sales engineering
Role takeaway
- Fit
- Benchmark evidence against using general multimodal LLMs for prehospital ECG triage.
- Architecture and integration
- Not an integration target; informs evaluation protocols for monitor-integrated or dedicated ECG AI.
- Prerequisites
- Local ECG archive
- Reference outcomes from cath-lab logs
- Monitor algorithm outputs for comparison
- Constraints
- Single center
- Static images only
- Zero-shot prompting
- Security
- Block upload of patient ECGs to consumer AI services; log any sanctioned AI use.
- Proof of value
- Specificity and PPV versus monitor algorithm on local ECGs
- Output consistency across repeated runs
- Performance on noisy field tracings
Delivery
Role takeaway
- Work
- Adopt and enforce a clinical AI use policy covering ECG interpretation.
- Dependencies
- Medical direction
- Training staff
- IT and privacy office
- Ownership
- EMS medical director.
- Skills and adoption
- Paramedics learn why general chatbots are unsuitable for ECG decisions and which tools are approved.
- Governance checkpoints
- Policy approval by medical director
- Training completion
- Periodic audit for unsanctioned use
- Acceptance criteria
- Written policy published to all clinicians
- 100% of clinicians complete training within one quarter
- Any new ECG AI pilot shows local validation before approval
- Risks
- Clinician use of consumer chatbots
- Overgeneralizing results to dedicated tools
- Privacy breaches from image uploads
Implementation considerations
Lighthouse Advisory interpretation across the operating dimensions a public-sector buyer must settle before this evidence becomes a design. Each note answers the question under its heading for this specific source.
Architecture and integration
What must connect, and where does the AI sit in the workflow?
General multimodal LLMs read cropped ECG images without waveform data or context; dedicated ECG models use signal-specific training.
Governance
Who approves, reviews and stays accountable for outcomes?
Supports explicit EMS policy banning general-purpose LLMs for diagnostic or activation decisions and requiring validated, regulated tools.
Security and privacy
What data, permissions and controls need testing?
Uploading patient ECG images to consumer AI services raises privacy concerns beyond accuracy.
Accessibility and workforce
Who is affected, and what skills or accommodations follow?
Authors note clinicians without access to dedicated tools may be tempted to use LLMs; training is the main lever.
Procurement
What should contracts, pricing and exit terms secure?
Procurements should require task-specific validation metrics and exclude general LLMs for ECG decisions.
Operating model
Which teams own the service once it runs?
Cardiology activation remains a clinician decision; general AI limited to education and documentation.
Publication history
- 2026-10-20Emergency Services · Issue 446 resources
Stable resource ID: heart-lung-general-llms-unreliable-prehospital-ecg-cath-lab-activation-20260915