{"resourceId":"digital-times-seoul-119-ai-performance-72-pilot-hold-20261105","versions":[{"version":"external-18b6be30910069ba304be7082741f2b99c5ff34e1443c4e315fbbe57f800396e","resource":{"id":"digital-times-seoul-119-ai-performance-72-pilot-hold-20261105","title":"Seoul council audit: 5.078 billion-won 119 AI system scored 72/100 and documents put formal pilot call-taking on hold","organization":"Seoul Metropolitan Fire & Disaster Headquarters / Seoul Metropolitan Council","sector":"Fire / 119 emergency call-taking","geography":"Seoul, Republic of Korea","publishedAt":"2026-11-05","publicationDate":"2026-11-05","eventDate":"2026-11-05","sourceName":"Digital Times (디지털타임스)","sourceLabel":"Independent reporting on Seoul city council administrative audit","sourceUrl":"https://www.dt.co.kr/article/12093418","evidenceClass":"independent-reporting","outcomeClass":"cautionary","topics":["governance-procurement","operating-model","data-security"],"finding":"At the Nov. 5, 2026 Seoul council administrative audit, councilmember Kim Jun-sung said the Fire & Disaster Headquarters' 119 AI project—about 5.078 billion won across two phases, past completion inspection and residual payment—received a comprehensive evaluation score of 72/100, and the performance measurement report listed formal pilot operation as fully postponed. Fire headquarters replied that functions other than AI directly accepting/handling 119 calls are operating and that 'full postpone' applies to some functions; the headquarters chief said he would verify facts and consider external review. Council also questioned duplicate maintenance staffing budgets and low field use of other advanced gear (quadruped robot, drone repairs).","sledRelevance":"Directly relevant to U.S. fire/EMS/911 AI call-taking pilots: shows life-safety AI can clear vendor payment yet fail an internal scorecard for live call acceptance, and that legislative oversight will interrogate module-level go-live. Transferability: Korean 119 and council audit process differ from U.S. PSAPs, but the procurement-versus-performance pattern travels.","evidence":"Digital Times report dated 2026-11-05 15:01 KST summarizing the Urban Safety Construction Committee audit; quotes Kim that the official report states full pilot postpone; HQ claims partial operation excluding AI direct call acceptance.","architectureImplications":"Architect emergency-call AI as separable modules with independent authorization for answer/triage automation versus assistive STT/classification.","governanceImplications":"Require council- or board-visible scorecards and payment holds tied to module-level go-live, not merely construction completion.","securityPrivacyImplications":"Article focuses on performance and budget; still implies sensitive 119 audio in AI systems needing clear operational status disclosure to the public.","caveats":"Single newspaper account of an audit dispute; full evaluation protocol and raw metrics are not published in the article; HQ contests the scope of 'postpone.' International transferability limits apply.","streamIds":["emergency-services"],"roles":{"sales":"Sales framing: Seoul's 5.078 billion-won 119 AI build shows procurement risk when a system scores 72/100 and documents put formal pilot call-taking on hold while the fire headquarters still says other functions run. For U.S. PSAPs and fire EMS buyers comparing callbot pilots, the bounded value is helping councils or 911 boards demand the actual performance report, pass/fail criteria and which modules are live versus deferred before final payment. This article does not prove U.S. products would score similarly, guarantee savings, or show patient outcome harm from the Seoul system.","engineering":"Engineering framing: treat the Seoul AI 119 stack as modular—direct AI call acceptance/handling can be gated separately from STT, classification assist or dispatch support. Replicate that separation with feature flags, dual-run logging and a documented go/no-go against a published scorecard before enabling live callbot answer. Integration must preserve human answer paths for 911/119 and record when AI declines or transfers. Transferability is limited: Korean 119 mixes fire/EMS/rescue under one center, and scoring methods are not fully published in the article.","delivery":"Delivery framing: for a PSAP or fire department pilot, define acceptance as (1) independent or council-visible performance evaluation against agreed metrics, (2) explicit list of deferred versus live modules, and (3) maintenance staffing that does not double-pay vendor-resident support. Schedule an administrative or board review checkpoint before residual payment and before expanding AI beyond non-emergency or assistive modes. Risk includes political backlash if paid systems remain 'running' while reports say pilot is postponed."},"roleDetails":{"sales":{"customerProblem":"Elected officials and 911/fire leaders need assurance that a multimillion-dollar emergency call AI meets published performance gates before live call-taking expands.","stakeholders":["City council public-safety committee","Fire/EMS chief and 911 coordinator","Procurement/IT","PSAP QA supervisors","Resident callers"],"discoveryQuestions":["What is the written go/no-go scorecard and who signed the 72/100 evaluation?","Which modules are live versus deferred under 'pilot postponed'?","Was residual payment made after the failing evaluation?","How are maintenance headcount and vendor-resident support contracted?"],"valueHypothesis":"A short governance review that maps contract milestones to an independent scorecard can prevent paying for live call-taking capability that is not cleared for pilot.","potentialEngagement":"Bounded advisory for a city or regional 911 board: contract-to-performance gap analysis using public audit documents and vendor acceptance tests.","unsupportedClaims":["That Seoul's 72/100 score applies to any named U.S. vendor","Guaranteed reduction in wait times or lives saved","That all AI modules are unsafe or all are running safely"]},"engineering":{"fit":"Relevant to PSAP callbot and AI assist architectures that separate answer/triage automation from human-supervised STT and classification.","architectureIntegration":"Keep AI call acceptance behind a hard switch with human 911 answer fallback; log model version, decision and transfer reason for every AI-touched call.","prerequisites":["Published acceptance metrics and baseline human performance","Dual recording and QA sampling of AI and human paths","Clear module inventory (answer, classify, dispatch support)"],"constraints":["Korean 119 operational mix differs from many U.S. fire/EMS-only or multi-agency PSAPs","Article does not publish the full test protocol or error taxonomy"],"security":"Emergency-call audio and location are sensitive; any AI path needs retention, access control and breach reporting aligned to local 911 policy.","proofOfValue":["Reproduce vendor scorecard on local accent/street-name test set","Measure abandoned-call and misroute rates with AI answer off vs gated on","Confirm deferred modules cannot be toggled live without change control"]},"delivery":{"work":"Stand up a pilot governance package: scorecard, module matrix, payment hold points and council reporting cadence before enabling AI answer.","dependencies":["Access to evaluation report and contract SOW","PSAP telephony integration owner","Legal/privacy review for call recording to AI"],"ownership":"911 coordinator or fire communications chief owns go-live; procurement owns payment gates; council oversees public reporting.","skillsAdoption":"Supervisors need QA rubrics for AI-handled calls; call-takers need scripts for AI handoff and failure modes.","governanceCheckpoints":["Pre-payment review of evaluation report","Module-level authorization before AI answer","Quarterly public metrics if any AI path is live"],"acceptanceCriteria":["Written mapping of score to allowed modules","No residual payment without documented exception for deferred scope","Human answer SLA unchanged when AI is off"],"risks":["Political dispute over whether system is 'running'","Caller harm if deferred answer mode is quietly enabled","Duplicate maintenance spend without added coverage"]}},"retrievedAt":"2026-11-06T04:00:55Z","enrichedAt":"2026-11-06T04:05:00Z","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"If AI answer is deferred, workforce plans must keep human staffing for surge; disclosure gaps can erode caller trust, including for accent/disability equity concerns raised in other 911 AI cases.","procurementImplications":"Completion inspection and residual payment before a passing pilot score create clawback and acceptance-test lessons for U.S. cooperative purchases.","operatingModelImplications":"Operate with an explicit deferred-module register and public status so 'system running' cannot mean unsupervised AI call-taking.","peerReview":"unknown","sourceVerification":{"openedUrl":"https://www.dt.co.kr/article/12093418","referenceExcerpt":"종합평가에서는 100점 만점에 72점을 받았다는 지적이다.","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}