{"resourceId":"npj-llm-ed-triage-stigma-reprioritization-20260919","versions":[{"version":"external-0c622a177be709bdebf9f64d3c0dc01324e7dfeaa6257bbbc4c2c45dfebad203","resource":{"id":"npj-llm-ed-triage-stigma-reprioritization-20260919","title":"npj Digital Medicine: stigmatizing ED-use language harms LLM triage prioritization of deteriorating patients","organization":"University of Texas Southwestern Medical Center (Emergency Medicine & Bioinformatics)","sector":"Academic emergency medicine / clinical AI evaluation","geography":"United States (two academic emergency departments; open-weight LLM experiment)","publishedAt":"2026-09-19","publicationDate":"2026-09-19","eventDate":"2026-09-19","sourceName":"npj Digital Medicine","sourceLabel":"Open-access npj Digital Medicine article on stigmatizing inputs and LLM emergency triage","sourceUrl":"https://www.nature.com/articles/s41746-026-03270-5","evidenceClass":"academic-research","outcomeClass":"cautionary","topics":["knowledge-work","developers-agents","data-security","governance-procurement","accessibility-workforce"],"finding":"Published September 19, 2026, this outcome-grounded experiment at two academic EDs tested whether inserting demographic, social, or stigma-related attributes changes open-weight LLM (Gemma, Qwen, DeepSeek) Emergency Severity Index–style prioritization on matched pairs (one patient deteriorating within 6 hours, one not). Across 221,556 comparisons and 18 conditions, the stigmatizing frequent-ED-use formulation produced significant harmful reprioritization in all six model–dataset cells (up to 9.4%), exceeding its neutral counterpart within pairs in every cell (+1.4 to +7.1 points). Psychiatric history also produced significant shifts (up to 9.5%). Race, language, and insurance showed no consistent harmful shifts. Authors conclude input selection/formulation are safety-critical design choices for LLM triage.","sledRelevance":"Direct caution for any EMS/ED/dispatch LLM that consumes free-text notes or call narratives containing stigma cues. Why now: published 2026-09-19, absent from the archive, and newly relevant after the 2026-09-26 JHIR/CEEM clinical LLM pattern.","evidence":"Opened npj Digital Medicine HTML via curl. Publication date 19 September 2026; 221,556 comparisons; frequent-ED-use harmful reprioritization up to 9.4%; psychiatric history shifts; non-consistent race/language/insurance effects; safety-critical formulation conclusion from abstract/page.","architectureImplications":"Strip or neutralize stigma-laden free text before any triage LLM; prefer structured ESI features over narrative stigma cues; keep clinicians final.","governanceImplications":"Require stigma/fairness audits on input formulations—not only demographic counters—before advisory triage LLM pilots.","securityPrivacyImplications":"Narrative notes may encode sensitive behavioral/social history; minimize what is sent to models and log redactions.","caveats":"Open-weight models in a controlled ED-pair experiment—not a live EMS dispatch deployment; full PDF may require institutional access beyond the open abstract HTML.","streamIds":["emergency-services"],"roles":{"sales":"Use this paper to reframe EMS/ED triage LLM deals around input-formulation safety—not just acuity accuracy. Discovery should ask how frequent-utilizer and psych-history language appears in CAD/EHR text fed to models. Bounded engagement: stigma-prompt audit on the agency’s real note templates before any pilot expansion. Unsupported claims include demographic fairness from this study alone or safe autonomous triage.","engineering":"Fit is a controlled finding that stigma phrasing in prompts/notes can reprioritize deteriorating patients downward in open-weight triage LLMs. Architecture should redact or structure away frequent-ED-use and similar stigma strings, evaluate multiple models, and keep human final acuity. Prerequisites: access to note templates, ESI labels, and paired outcome labels. Constraints: lab setting; three model families. Proof of value is a local re-audit showing reduced harmful shifts after input controls.","delivery":"Delivery includes prompt/template hygiene, stigma test harnesses, and clinician override UX. Dependencies: ED/EMS informatics, privacy office, and model vendors. Ownership: clinical AI governance for go-live gates. Acceptance: documented stigma-cell tests with harm rates below an agreed threshold and no silent acuity changes. Risks: residual narrative leakage and assuming race-null results imply overall fairness."},"roleDetails":{"sales":{"customerProblem":"Triage LLM pilots may silently deprioritize deteriorating patients when notes contain stigmatizing frequent-utilizer language.","stakeholders":["EMS/ED medical directors","PSAP/QI leaders","Clinical informatics","Equity officers"],"discoveryQuestions":["What free text reaches the model?","How are frequent utilizers labeled?","Who signs fairness gates?","Is the LLM advisory only?"],"valueHypothesis":"Formulation-safety controls may reduce harmful LLM reprioritization if audited before wider triage use—not a claim of bias-free automation.","potentialEngagement":"Stigma-prompt audit on local templates with published harm metrics.","unsupportedClaims":["Race-null result means overall fairness","Safe for autonomous dispatch","All models behave identically"]},"engineering":{"fit":"Input-safety control layer for advisory ED/EMS triage LLMs consuming narratives.","architectureIntegration":"Note/CAD text → stigma redaction/structuring → LLM acuity suggestion → clinician final ESI.","prerequisites":["Template inventory","Outcome-labeled pairs","Multi-model eval harness"],"constraints":["Open-weight lab study","Not live dispatch","Limited model set"],"security":"Minimize sensitive behavioral history sent to models; log redactions.","proofOfValue":["Local harmful-shift rate after controls","Clinician override rate","No silent acuity writes to CAD"]},"delivery":{"work":"Implement redaction rules, run stigma cells, and wire clinician confirmation.","dependencies":["EHR/CAD text feeds","Clinical governance","Model hosting"],"ownership":"ED/EMS medical director owns clinical acceptance; informatics owns pipelines.","skillsAdoption":"Train staff that stigma phrasing is a model hazard, not just tone.","governanceCheckpoints":["Pre-pilot stigma audit","Quarterly re-audit","Incident review if harm suspected"],"acceptanceCriteria":["Harmful shift below threshold","Advisory-only mode","Documented overrides"],"risks":["Narrative leakage","Overgeneralizing lab results","Equity optics without fixes"]}},"retrievedAt":"2026-09-27T22:06:12-05:00","enrichedAt":"2026-09-27T22:06:12-05:00","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Clinicians/dispatchers must learn which narrative phrases are unsafe model inputs; stigma language in charts can become an equity failure mode.","procurementImplications":"Write RFPs requiring formulation-safety tests (frequent-utilizer and psych-history prompts) with published harm rates—not accuracy alone.","operatingModelImplications":"Keep LLM triage advisory; ban stigma-laden auto-summaries from altering acuity without clinician override.","sourceVerification":{"responseId":"curl-npj-stigma-20260927T220612","openedUrl":"https://www.nature.com/articles/s41746-026-03270-5","referenceExcerpt":"Stigmatizing formulations of clinical information can bias LLM triage prioritization against deteriorating patients.","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}