From the Emergency Services edition of September 27, 2026
npj Digital Medicine: stigmatizing ED-use language harms LLM triage prioritization of deteriorating patients
University of Texas Southwestern Medical Center (Emergency Medicine & Bioinformatics) · Academic emergency medicine / clinical AI evaluation · United States (two academic emergency departments; open-weight LLM experiment)
- Publisher
- npj Digital Medicine
- Original publication
- 2026-09-19
- Source retrieved
- 2026-09-28
- Event date
- 2026-09-19
What happened
Published September 19, 2026, this outcome-grounded experiment at two academic EDs tested whether inserting demographic, social, or stigma-related attributes changes open-weight LLM (Gemma, Qwen, DeepSeek) Emergency Severity Index–style prioritization on matched pairs (one patient deteriorating within 6 hours, one not). Across 221,556 comparisons and 18 conditions, the stigmatizing frequent-ED-use formulation produced significant harmful reprioritization in all six model–dataset cells (up to 9.4%), exceeding its neutral counterpart within pairs in every cell (+1.4 to +7.1 points). Psychiatric history also produced significant shifts (up to 9.5%). Race, language, and insurance showed no consistent harmful shifts. Authors conclude input selection/formulation are safety-critical design choices for LLM triage.
Why it matters
Direct caution for any EMS/ED/dispatch LLM that consumes free-text notes or call narratives containing stigma cues. Why now: published 2026-09-19, absent from the archive, and newly relevant after the 2026-09-26 JHIR/CEEM clinical LLM pattern.
Evidence and measured results
Opened npj Digital Medicine HTML via curl. Publication date 19 September 2026; 221,556 comparisons; frequent-ED-use harmful reprioritization up to 9.4%; psychiatric history shifts; non-consistent race/language/insurance effects; safety-critical formulation conclusion from abstract/page.
Limitations and uncertainty
Open-weight models in a controlled ED-pair experiment—not a live EMS dispatch deployment; full PDF may require institutional access beyond the open abstract HTML.
Put this evidence to work
Lighthouse Advisory interpretation, grounded in this source. Enriched 2026-09-27; this does not change the original publication date. Each role follows the same checklist so takeaways compare across sources.
Sales
Role takeaway
- Customer problem
- Triage LLM pilots may silently deprioritize deteriorating patients when notes contain stigmatizing frequent-utilizer language.
- Stakeholders
- EMS/ED medical directors
- PSAP/QI leaders
- Clinical informatics
- Equity officers
- Discovery questions
- What free text reaches the model?
- How are frequent utilizers labeled?
- Who signs fairness gates?
- Is the LLM advisory only?
- Value hypothesis
- Formulation-safety controls may reduce harmful LLM reprioritization if audited before wider triage use—not a claim of bias-free automation.
- Potential engagement
- Stigma-prompt audit on local templates with published harm metrics.
- Unsupported claims
- Race-null result means overall fairness
- Safe for autonomous dispatch
- All models behave identically
Pre-sales engineering
Role takeaway
- Fit
- Input-safety control layer for advisory ED/EMS triage LLMs consuming narratives.
- Architecture and integration
- Note/CAD text → stigma redaction/structuring → LLM acuity suggestion → clinician final ESI.
- Prerequisites
- Template inventory
- Outcome-labeled pairs
- Multi-model eval harness
- Constraints
- Open-weight lab study
- Not live dispatch
- Limited model set
- Security
- Minimize sensitive behavioral history sent to models; log redactions.
- Proof of value
- Local harmful-shift rate after controls
- Clinician override rate
- No silent acuity writes to CAD
Delivery
Role takeaway
- Work
- Implement redaction rules, run stigma cells, and wire clinician confirmation.
- Dependencies
- EHR/CAD text feeds
- Clinical governance
- Model hosting
- Ownership
- ED/EMS medical director owns clinical acceptance; informatics owns pipelines.
- Skills and adoption
- Train staff that stigma phrasing is a model hazard, not just tone.
- Governance checkpoints
- Pre-pilot stigma audit
- Quarterly re-audit
- Incident review if harm suspected
- Acceptance criteria
- Harmful shift below threshold
- Advisory-only mode
- Documented overrides
- Risks
- Narrative leakage
- Overgeneralizing lab results
- Equity optics without fixes
Implementation considerations
Lighthouse Advisory interpretation across the operating dimensions a public-sector buyer must settle before this evidence becomes a design. Each note answers the question under its heading for this specific source.
Architecture and integration
What must connect, and where does the AI sit in the workflow?
Strip or neutralize stigma-laden free text before any triage LLM; prefer structured ESI features over narrative stigma cues; keep clinicians final.
Governance
Who approves, reviews and stays accountable for outcomes?
Require stigma/fairness audits on input formulations—not only demographic counters—before advisory triage LLM pilots.
Security and privacy
What data, permissions and controls need testing?
Narrative notes may encode sensitive behavioral/social history; minimize what is sent to models and log redactions.
Accessibility and workforce
Who is affected, and what skills or accommodations follow?
Clinicians/dispatchers must learn which narrative phrases are unsafe model inputs; stigma language in charts can become an equity failure mode.
Procurement
What should contracts, pricing and exit terms secure?
Write RFPs requiring formulation-safety tests (frequent-utilizer and psych-history prompts) with published harm rates—not accuracy alone.
Operating model
Which teams own the service once it runs?
Keep LLM triage advisory; ban stigma-laden auto-summaries from altering acuity without clinician override.
Publication history
- 2026-09-27Emergency Services · Issue 224 resources
Stable resource ID: npj-llm-ed-triage-stigma-reprioritization-20260919