{"resourceId":"arxiv-llm-disaster-triage-multagent-audit-capacity-20260807","versions":[{"version":"external-0c622a177be709bdebf9f64d3c0dc01324e7dfeaa6257bbbc4c2c45dfebad203","resource":{"id":"arxiv-llm-disaster-triage-multagent-audit-capacity-20260807","title":"arXiv simulation: multi-agent LLM disaster triage does not cut bias; overloaded auditors miss ~44% of biased outcomes","organization":"Institute for Future Technologies (Paul-Peter Arslan)","sector":"Independent/preprint AI safety research for disaster triage agents","geography":"Synthetic disaster-triage simulation (GPT-4o-mini; not a field deployment)","publishedAt":"2026-08-07","publicationDate":"2026-08-07","eventDate":"2026-08-07","sourceName":"arXiv","sourceLabel":"arXiv HTML preprint on multi-agent LLM resource allocation under audit capacity constraints","sourceUrl":"https://arxiv.org/html/2608.06949v1","evidenceClass":"independent-research","outcomeClass":"cautionary","topics":["developers-agents","governance-procurement","operating-model","accessibility-workforce"],"finding":"This August 7, 2026 preprint studies whether splitting disaster-triage resource allocation across a nine-agent LLM pipeline (assessors, allocators, independent auditors) reduces demographic bias versus a single agent. Using a synthetic twin-pair simulator (192 episodes; 2,304 resolved pairs; GPT-4o-mini), bias rates did not differ (6.9% vs 6.1%, p=0.498). However, 30.0% of biased outcomes left no trace; that silent share rose to 43.8% when the auditor was limited to one review per tick and fell to 18.4% when not overloaded. Coverage—not auditor judgment—collapsed under load (100.0% → 65.6%, p<0.001). Risk-prioritized audit queues recovered coverage (65.6% → 91.7%, p=0.028). Limitations: one model, modest samples, not peer-reviewed, no adversarial replication.","sledRelevance":"Operating constraint for any EMS/disaster multi-agent oversight design: independent audit fails when review capacity is the bottleneck. Why now: absent from archive; newly relevant beside today’s stigma-triage finding and yesterday’s multi-agent EM scoping gap.","evidence":"Opened arXiv HTML v1 via curl. Submitted 7 Aug 2026; bias rate non-difference; 30%/43.8%/18.4% silent-bias figures; coverage collapse; risk-queue recovery; stated limitations taken from page.","architectureImplications":"Budget auditor throughput explicitly; implement risk-ordered review queues; do not equate multi-agent topology with fairness.","governanceImplications":"Publish audit coverage SLOs and silent-bias monitors before any multi-agent triage pilot claims oversight.","securityPrivacyImplications":"Minimize unnecessary demographic attributes passed between agents (policy P4 in the study) to reduce leakage and bias pathways.","caveats":"Synthetic simulator on one model; not a live EMS/EM system; preprint without peer review or adversarial replication.","streamIds":["emergency-services"],"roles":{"sales":"Reframe multi-agent disaster/EMS triage offerings around audit capacity—not the mere presence of reviewer agents. Discovery should quantify expected case rate versus review slots and whether risk-based queues exist. Bounded engagement: overload stress test with silent-bias and coverage metrics. Unsupported claims include multi-agent pipelines eliminating bias or self-audit sufficiency.","engineering":"Fit is a caution that role-differentiated LLM pipelines did not reduce twin-pair bias and lost audit coverage under load. Architecture should separate auditors, meter reviews/tick, strip unneeded demographics between stages, and prefer risk-ordered queues. Prerequisites: trace logging, capacity metrics, twin-pair eval harness. Constraints: GPT-4o-mini only; synthetic cases. Proof of value is coverage and catch-rate under surge, not topology diagrams.","delivery":"Implement audit SLOs, queue policies, and dashboards for silent bias. Dependencies: model hosting, incident surge plans, and equity reviewers. Ownership: EM/EMS clinical governance for acceptance; platform team for queueing. Acceptance: coverage ≥ agreed floor under load tests; documented risk-queue behavior. Risks: false assurance from unused auditors and unreplicated preprint results."},"roleDetails":{"sales":{"customerProblem":"Agencies may believe multi-agent review removes triage bias while overloaded auditors silently miss biased denials.","stakeholders":["EM/EMS medical directors","AI platform owners","Equity/oversight offices","EOC tech leads"],"discoveryQuestions":["Reviews available per surge hour?","FIFO or risk queue?","What is logged when audit is skipped?","Single-agent baseline bias rate?"],"valueHypothesis":"Capacity-aware audit design may catch more biased allocations than adding agents alone—if coverage is measured under load.","potentialEngagement":"Audit-capacity stress test with twin-pair silent-bias metrics.","unsupportedClaims":["Multi-agent removes bias","Self-audit is enough","Results generalize to all models"]},"engineering":{"fit":"Oversight/queueing controls for multi-agent LLM disaster or EMS resource-allocation pilots.","architectureIntegration":"Assess → allocate → independent audit with metered capacity and risk-ordered queue; minimize demographic fan-out.","prerequisites":["Traceable decision logs","Capacity counters","Twin-pair eval set"],"constraints":["One model in study","Synthetic data","Preprint status"],"security":"Stop passing protected attributes to stages that do not need them.","proofOfValue":["Coverage under load","Silent-bias rate","Catch rate with risk queue"]},"delivery":{"work":"Configure auditor capacity, risk queues, and silent-bias monitors before pilot scale-up.","dependencies":["Model API limits","Surge staffing plan","Governance reviewers"],"ownership":"Clinical/EM governance owns go-live; platform owns queue metrics.","skillsAdoption":"Train reviewers to use risk queues rather than rubber-stamp FIFO.","governanceCheckpoints":["Load-test signoff","Weekly silent-bias report","Model-change retest"],"acceptanceCriteria":["Coverage floor met","Risk queue enabled","No production use without audit SLO"],"risks":["False assurance from unused auditors","Unreplicated findings","Demographic leakage between agents"]}},"retrievedAt":"2026-09-27T22:06:16-05:00","enrichedAt":"2026-09-27T22:06:16-05:00","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Human/automated auditors need staffing models that preserve coverage under surge; overload creates invisible equity failures.","procurementImplications":"Require vendors to demonstrate audit-capacity stress tests and queueing policy—not only single-agent fairness scores.","operatingModelImplications":"Treat oversight as a scarce resource; prioritize high-risk denials/protected-attribute rationales for review first.","sourceVerification":{"responseId":"curl-arxiv-2608.06949-20260927T220616","openedUrl":"https://arxiv.org/html/2608.06949v1","referenceExcerpt":"30.0% of biased outcomes go entirely undetected, rising to 43.8% when the auditor is overloaded.","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}