{"resourceId":"sim911-nashville-training-preprint-2024","versions":[{"version":"external-d446aed4926e5ef2375ba820e216af5a5f65ffc81c693e1dc7d759193fa9bda3","resource":{"id":"sim911-nashville-training-preprint-2024","title":"Sim911 training deployment reports favorable user feedback while retaining hallucination limits","organization":"Vanderbilt University and Metro Nashville Department of Emergency Communications","sector":"911 dispatcher training","geography":"Nashville, Tennessee, United States","publishedAt":"December 26, 2024, inspected preprint version 3","publicationDate":"2024-12-26","eventDate":null,"sourceName":"Vanderbilt University and Metro Nashville Department of Emergency Communications","sourceLabel":"Original research preprint with methods, tables and appendices","sourceUrl":"https://arxiv.org/html/2412.16844v3","evidenceClass":"academic-research","outcomeClass":"mixed","topics":["knowledge-work","developers-agents","infrastructure","data-security","governance-procurement","accessibility-workforce","operating-model"],"finding":"Sim911 supports supervised dispatcher practice, but reported satisfaction and simulation quality do not establish operational competence.","sledRelevance":"Interpretation: Useful for emergency medical, fire and severe-weather call training; the study also includes routine policing scenarios, without separate emergency-services effect estimates.","evidence":"The study uses 2,641 historical-call configurations for component comparisons and reports 228 completed deployment simulations. Nine of ten respondents rated it at least comparable to human-led training; respondents included two trainees, two dispatchers and six training officers. Tables assess linguistic, location and scenario proxies, not retained skills or patient outcomes. Logged simulation time is not a controlled net labor-saving estimate.","architectureImplications":"Interpretation: Separate training from emergency call routing; maintain versioned local protocols, scenario data and evaluator components.","governanceImplications":"Interpretation: Require instructor review and separate authorization for any expansion beyond simulation.","securityPrivacyImplications":"Interpretation: Audit de-identification and external API payloads; derived scenarios can still expose sensitive source details.","caveats":"Single-center developer/operator study with a small convenience survey. Hallucinations remain. The accessible version is a 2024 preprint; the 2025 proceedings PDF could not be retrieved.","streamIds":["emergency-services"],"roles":{"sales":"Interpretation — Engage the training manager, dispatch director, medical-dispatch lead and workforce development staff about practice capacity and rare-call exposure. Ask how trainees are currently assessed, which scenarios are difficult to stage and how much instructor time is actually constrained. Offer a bounded curriculum and evaluation pilot focused on medical, fire and disaster calls. The value hypothesis is more supervised practice opportunities without proportionate role-playing demand. Do not convert favorable user ratings into proven competence or assume logged simulation hours equal net savings. Include scenario maintenance and coaching costs. Applicability depends on local protocol fit and staff willingness to review imperfect generated calls.","engineering":"Interpretation — Create an isolated training environment with approved scenario inputs and no permission to route calls or modify operational CAD. Prerequisites include current local protocols, validated locations, a privacy-reviewed corpus and a pinned model configuration. Challenge retrieval and evaluator components with misleading instructions embedded in call text, invalid addresses and contradictory incident details. Measure undetected errors independently rather than relying solely on the system's own evaluator. Compare cloud APIs with local options for data handling and service continuity; no hosting choice is validated by this study. A useful proof of value pairs instructor scoring with delayed trainee assessment, covering missing data, language variation and service failure.","delivery":"Interpretation — A training officer should own rollout with a dispatch quality specialist, data steward and IT support. Select a limited emergency scenario set, cleanse source material, test instructor controls and train users to stop and flag implausible output. Dependencies include review time and a maintained protocol inventory. Approve corpus access and evaluation design before the pilot, then review results before expanding. Proposed acceptance includes zero connections to live dispatch, logged instructor interventions, successful shutdown and recovery, and prespecified delayed-skill and workload comparisons with existing training. Monitor stereotype complaints and scenario drift. Proposed criteria are not study findings; substitution for experienced coaching requires separate evidence."},"retrievedAt":"2026-09-12T03:01:09Z","enrichedAt":"2026-09-12T03:04:47Z","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Interpretation: Involve language-access and disability advisers in checking simulated caller profiles; proxy tag matching cannot establish equitable real-world service.","procurementImplications":"Interpretation: Require independently scored skill retention, instructor workload accounting and local protocol support before promising training savings.","operatingModelImplications":"Interpretation: Training management owns curriculum; clinical/dispatch specialists validate scenarios; IT manages service dependencies and data boundaries.","updateExplanation":"Previously absent from all 217 archive records. Adds historical deployed training evidence, distinct from the already archived DispatchMAS emergency dialogue simulation.","sourceVerification":{"openedUrl":"https://arxiv.org/html/2412.16844v3","referenceExcerpt":"the simulation sometimes generates inaccurate or fabricated information.","promptVersion":"sled-research-v3.1","model":null,"basis":"agent-reported inspection"}}}]}