Public Safety & Emergency ServicesIssue 634 records
Emergency Services
Four signals on testing emergency AI against cases it has not seen. A Freiburg randomized trial reported at the AHA symposium found arrival-time alerting of cardiac-arrest volunteers beat a 600 m radius (58% vs 31% arriving before EMS) but matched a 2,500 m radius, and measured no patient outcomes. An arXiv flood-depth model scored 0.39 m error on sampled events but R2 of 0.03 on eleven held-out storms. A second preprint found an LLM weather assistant without access to NWS alerts gave false reassurance in 37.7% of alert cases, and a disclosure instruction did not help. Texas A&M announced a crisis-post classifier that has not been used in a real disaster.
- Evidence records
- 4
- Cross-source patterns
- 1
- Evidence classes
- 3 academic research1 independent reporting
- Outcomes
- 2 cautionary1 mixed1 emerging
- Source freshness
- 4 new this fortnight
- Research completed
- 2026-11-09
Choose a role to see its takeaway beside every record in the ledger.
Patterns across the evidence
Synthesis · Lighthouse Advisory interpretation
Results hold on the cases tested; the open question is unseen ones
Two preprints show how performance depends on what was held out. The flood-depth model reached 0.39 m error on sampled events but R2 of 0.03 on eleven held-out events, while the LLM weather pipeline had 0% false reassurance only when it could see official alerts and 37.7% without them. Interpretation: both findings argue for testing on unseen events and with the production data access conditions before relying on an AI output.
Operating questionBefore adopting an AI output for emergency decisions, have we tested it on events and data-access conditions it was not tuned on?
Supporting evidenceWen Zhou, Shaowen Wang, Upmanu Lall, Venkatesh Merwade and Dingqi Ye (arXiv preprint authors; affiliations not shown on the abstract page)Independent author (Fadurudeen), arXiv preprint
Evidence ledger
Full record · every source keeps its link and limitations
Freiburg randomized trial: alerting cardiac-arrest volunteers by predicted arrival time beat a 600 m radius but not a 2,500 m radius
Healio reports a prospective randomized trial in Freiburg (January to July 2024) that sent 1,482 volunteers to 823 suspected cardiac arrests across three arms. A "smart alerting" algorithm that selects volunteers by estimated travel time and screens out those predicted to arrive after the ambulance was compared with fixed 600 m and 2,500 m radius alerting. Volunteers arrived before EMS 58% of the time with smart alerts, 57% with 2,500 m and 31% with 600 m. Smart alerting beat 600 m (risk difference 27 percentage points, 95% CI 19-34, P < .001) but did not differ significantly from 2,500 m. AED arrival before the ambulance was 27% (smart), 28% (2,500 m) and 11% (600 m) among volunteers reaching the scene. The system rejected 31% of responders as likely to arrive later than the ambulance, which the researchers say may reduce futile dispatches.
Why it matters, evidence and limitations
- Why it matters
- A transferable dispatch-layer question for U.S. EMS agencies running smartphone responder apps: whether predicted-arrival selection adds value over a simply larger alert radius. The algorithm is travel-time routing rather than generative AI, so relevance to AI programs is partial.
- Evidence and measured results
- Opened the Healio article via HTTPS curl (User-Agent SLED-research-bot/1.0) and saved the text; quoted figures come from the conference presentation as reported by Healio. The abstract itself was not opened.
- Limitations and uncertainty
- Conference presentation reported by a trade outlet, not a peer-reviewed paper; only system performance was measured, not patient survival; randomization was by day of week rather than emergency type; German setting; no significant advantage over the 2,500 m radius.
Preprint: flood peak-depth model from 5,689 USGS high-water marks reaches 0.39 m error 6 hours ahead, but shows no skill on eleven held-out events
The authors assembled 5,689 USGS high-water-mark surveys from 39 flood events (2011-2025) and trained a cross-modal fusion network on terrain, geospatial foundation-model embeddings and antecedent hourly meteorological forcing. It predicts peak depth at surveyed marks with a mean absolute error of 0.39 m six hours before the peak, against 0.93 m for differencing the measured peak water level with the same terrain model. Forcing accounts for 77% of the attributed error reduction. On eleven held-out events, the model still separates flooded from rain-free conditions but no longer resolves depth at the marks (R2 = 0.03). The authors state the skill is confined to the events and regions sampled and transfer to unseen storms remains the open problem.
Why it matters, evidence and limitations
- Why it matters
- Emergency managers evaluating AI flood-depth products for pre-peak decisions should ask vendors for performance on storms not seen in training. This is a clear example of in-sample skill that does not carry over.
- Evidence and measured results
- Opened the arXiv abstract page via HTTPS curl (User-Agent SLED-research-bot/1.0) and saved the text. The full paper PDF and methods tables were not opened, so findings rest on the abstract.
- Limitations and uncertainty
- Preprint, not peer reviewed; findings drawn from the abstract only; high-water marks are post-event surveys, so labels are sparse; no operational deployment reported.
Preprint: LLM weather guidance without alert access gave false reassurance in 37.7% of 69 real NWS alert cases; alert access removed it
The paper replays 117 frozen cases built from real National Weather Service alerts (69 alert cases, 32 no-alert controls, 16 non-U.S. locations) through five conditions of a Claude-based preparedness pipeline. One rater hand-labeled 342 alert-case responses. False reassurance, meaning calm guidance while a warning is active, was 0.0% for the production pipeline (0/66) and for the same alerts under a neutral prompt (0/69), but 37.7% (26/69) for a blind neutral prompt and 44.9% (31/69) when the blind prompt also told the model to disclose its blind spot. Disclosure instructions did not help (p = 0.40). With a neutral prompt the model raised some alarm on 87.5% of no-alert controls versus 0% under the production prompt. The author also reports NWS test products ingested as real alerts and a token budget failing on complex cases.
Why it matters, evidence and limitations
- Why it matters
- Agencies and vendors adding LLM assistants to public-facing preparedness or alert communications need guaranteed access to authoritative alert feeds and tested prompts; a disclaimer alone did not prevent unsafe reassurance.
- Evidence and measured results
- Opened the arXiv abstract page via HTTPS curl (User-Agent SLED-research-bot/1.0) and saved the text. The full paper was not opened; dataset and code links were not followed.
- Limitations and uncertainty
- Preprint by one author with one rater; the evaluated pipeline is the author’s own Claude-based system; a seven-category scheme was written after an initial binary pass, which the author discloses; U.S. NWS alerts only; no field deployment.
Texas A&M announces CrisisSense-LLM, a disaster social-media classifier still in development and not yet deployed in an active disaster
A university press release describes an NSF-funded large language model that filters and categorizes social media posts during crises such as hurricanes into disaster-related, weather-related and humanitarian information, and further by event type, informativeness and humanitarian information type. It was tested on a benchmark of human-annotated posts from multiple disaster events. The release states the system is under development and has not been deployed during an active disaster, and the team plans an interactive AI agent for emergency professionals. The related paper (CrisisSense-LLM, DOI 10.1016/j.aei.2026.104836, May 26, 2026) was not opened, and the release gives no accuracy figures.
Why it matters, evidence and limitations
- Why it matters
- Shows research interest in LLM triage of crisis information flows for emergency operations centers; it is a signal of direction, not evidence of operational benefit.
- Evidence and measured results
- Opened the Texas A&M Stories release via HTTPS curl (User-Agent SLED-research-bot/1.0) and saved the text. The journal article was not opened.
- Limitations and uncertainty
- University press release with no performance metrics; benchmark evaluation only; not deployed; paper not opened; social media data bias toward connected, English-speaking users is not discussed.
How to read this edition
Source findings, measured results and limitations come from the cited publications. Patterns, operating questions, role takeaways and implementation considerations are Lighthouse Advisory interpretation, stated as questions to validate locally rather than guaranteed outcomes. Vendor and operator claims are labeled as claims. Full research method.
- Academic research
- Research produced through an academic institution or peer-reviewed venue.
- Independent reporting
- Independent reporting with attributable sources but without a formal evaluation design.