{"resourceId":"arxiv-flood-peak-depth-high-water-marks-heldout-events-20261103","versions":[{"version":"external-8db0d339a73a855ad4b5ec19033169012c222f429dbb39eebaea0a9b0eac4d23","resource":{"id":"arxiv-flood-peak-depth-high-water-marks-heldout-events-20261103","title":"Preprint: flood peak-depth model from 5,689 USGS high-water marks reaches 0.39 m error 6 hours ahead, but shows no skill on eleven held-out events","organization":"Wen Zhou, Shaowen Wang, Upmanu Lall, Venkatesh Merwade and Dingqi Ye (arXiv preprint authors; affiliations not shown on the abstract page)","sector":"Flood forecasting / emergency management","geography":"United States (24 states and Puerto Rico)","publishedAt":"arXiv:2611.05220, submitted Nov. 3, 2026","publicationDate":"2026-11-03","eventDate":null,"sourceName":"arXiv","sourceLabel":"Fusing Earth observation embeddings and spaceborne terrain with meteorological forcing for flood peak depth prediction from high-water marks","sourceUrl":"https://arxiv.org/abs/2611.05220","evidenceClass":"academic-research","outcomeClass":"cautionary","topics":["data-security","infrastructure"],"finding":"The authors assembled 5,689 USGS high-water-mark surveys from 39 flood events (2011-2025) and trained a cross-modal fusion network on terrain, geospatial foundation-model embeddings and antecedent hourly meteorological forcing. It predicts peak depth at surveyed marks with a mean absolute error of 0.39 m six hours before the peak, against 0.93 m for differencing the measured peak water level with the same terrain model. Forcing accounts for 77% of the attributed error reduction. On eleven held-out events, the model still separates flooded from rain-free conditions but no longer resolves depth at the marks (R2 = 0.03). The authors state the skill is confined to the events and regions sampled and transfer to unseen storms remains the open problem.","sledRelevance":"Emergency managers evaluating AI flood-depth products for pre-peak decisions should ask vendors for performance on storms not seen in training. This is a clear example of in-sample skill that does not carry over.","evidence":"Opened the arXiv abstract page via HTTPS curl (User-Agent SLED-research-bot/1.0) and saved the text. The full paper PDF and methods tables were not opened, so findings rest on the abstract.","architectureImplications":"Inputs include spaceborne terrain, Earth-observation embeddings and 144 hours of hourly forcing; operational use needs reliable data feeds and retraining with local observed depths.","governanceImplications":"Decision rules should treat depth estimates as unvalidated for new events until event-level holdout testing is reported.","securityPrivacyImplications":"Uses public and open data; no personal data described.","caveats":"Preprint, not peer reviewed; findings drawn from the abstract only; high-water marks are post-event surveys, so labels are sparse; no operational deployment reported.","streamIds":["emergency-services"],"roles":{"sales":"Sales framing: this preprint is a useful caution for flood-AI conversations. The model estimated depth with a 0.39 m error six hours ahead on its sampled events, yet on eleven held-out storms it could tell flooded from dry but not how deep. Offer a validation-design service that tests any flood product on unseen storms. Do not claim the approach is ready for operations, that it predicts depth in new events, or that it saves response time; none of that was shown.","engineering":"Engineering view: the model fuses terrain, foundation-model embeddings and 144 hours of meteorological forcing, with forcing contributing 77% of the error reduction. Teams should replay historical local storms as a holdout, compare against simple baselines and log which inputs were available at the stated lead time. The abstract does not describe latency, deployment or uncertainty outputs. A flooded-versus-dry classifier may still be useful where depth is not needed.","delivery":"Delivery plan: set an evaluation protocol before adopting any flood-depth AI, using held-out events from the agency’s own history and a named decision, such as road closures, that the output will inform. Dependencies are emergency management GIS staff, hydrologists and the data providers. Acceptance should be event-level error thresholds, reported separately for flooded detection and depth. Risks include over-trusting depth numbers and unmaintained input feeds."},"roleDetails":{"sales":{"customerProblem":"Emergency managers need credible pre-peak flood depth estimates but vendor claims often lack unseen-event testing.","stakeholders":["Emergency management director","State or county GIS lead","Hydrologist","Public works director"],"discoveryQuestions":["Which decisions depend on forecast depth versus flooded or not?","Has any flood tool been tested on storms outside its training set?","What observed depth data do you hold from past events?"],"valueHypothesis":"A held-out-event validation step could prevent misplaced trust in depth products, as interpretation.","potentialEngagement":"Short evaluation design and holdout replay for a candidate flood-AI product.","unsupportedClaims":["That the model works on new storms","That it shortens response times","That it is operationally deployed"]},"engineering":{"fit":"Relevant to teams assessing ML flood-depth or inundation products.","architectureIntegration":"Fusion network consuming terrain, embeddings and lead-time-masked hourly forcing to output depth at points.","prerequisites":["Local observed depth data","Reliable meteorological feeds","Terrain and embedding data"],"constraints":["R2 of 0.03 on held-out events","Preprint status","Sparse post-event labels"],"security":"Open data inputs; secure model artefacts and feeds against tampering.","proofOfValue":["Holdout on unseen local events","Comparison with simple terrain and forcing baselines","Separate scoring of flooded detection and depth"]},"delivery":{"work":"Define and run an event-level validation protocol for flood-depth AI before operational adoption.","dependencies":["Historical event data","Hydrology expertise","Data feed owners"],"ownership":"Emergency management planning lead with GIS and hydrology support.","skillsAdoption":"Operators need training to read depth versus flooded outputs and their uncertainty.","governanceCheckpoints":["Validation plan approved","Holdout results reviewed","Go or no-go decision documented"],"acceptanceCriteria":["Depth error threshold met on held-out events","Flooded detection accuracy reported separately","Feed uptime meets agreed level"],"risks":["Over-trusting depth outputs","Feed failures","Training-set mismatch with local terrain"]}},"retrievedAt":"2026-11-09T03:59:00Z","enrichedAt":"2026-11-09T04:01:55Z","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Not addressed by the source.","procurementImplications":"Require held-out-event validation, not random splits within the same storms, in any flood-AI evaluation criteria.","operatingModelImplications":"Forecasts support pre-peak planning; humans must interpret flooded versus depth skill separately.","peerReview":"preprint","sourceVerification":{"openedUrl":"https://arxiv.org/abs/2611.05220","referenceExcerpt":"the model still separates flooded from rain-free conditions but no longer resolves depth at the marks (R2 = 0.03)","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}