{"resourceId":"utah-ai-sandbox-liability-insurance-evaluators-nextgov-20261016","versions":[{"version":"external-a0ec5dbb76b2f7ba003e701b3e56bdf3d4e0f38c131d20dc8bea050dd459692c","resource":{"id":"utah-ai-sandbox-liability-insurance-evaluators-nextgov-20261016","title":"Nextgov/FCW (Oct 16): Utah's AI policy office says its sandbox still lacks a liability framework, requires pilot companies to carry liability insurance, and plans a vendor-liability bill as health pilots expand","organization":"Utah Office of Artificial Intelligence Policy (Department of Commerce), with the Division of Professional Licensing; reported by Nextgov/FCW","sector":"State regulatory sandbox and oversight of AI in licensed health care","geography":"Utah","publishedAt":"2026-10-16","publicationDate":"2026-10-16","eventDate":"2026-10-15","sourceName":"Nextgov/FCW","sourceLabel":"Nextgov/FCW, 'Utah AI sandbox launches new health care pilots — with question marks' by Christian Robles, Oct 16, 2026","sourceUrl":"https://www.nextgov.com/artificial-intelligence/2026/10/utah-ai-sandbox-launches-new-health-care-pilots-question-marks/416655/","evidenceClass":"independent-reporting","outcomeClass":"emerging","topics":["governance-procurement","operating-model","data-security"],"finding":"Nextgov reports remarks by Utah OAIP director Zach Boyd at the ADAPT conference on Thursday, Oct 15, plus an interview. Boyd says Utah has evidentiary standards, is working on governance guidance, and is drafting a liability bill he hopes to unveil this month or next during the interim session, starting a 'multi-year process.' Regulators read the sandbox law as giving no authority to change liability rules, so OAIP requires Doctronic and later participants to carry special liability insurance and not waive liability. Boyd calls insurance less sustainable long term and wants vendors held directly liable, expecting physicians would otherwise bear too much. The Utah Medical Licensing Board, which sought suspension of the Doctronic pilot in April, now meets weekly with regulators; the 190-drug refill list dropped Butalbital and Flecainide. Future pilots must contract with one of six independent evaluators, and an evaluator CEO expects assessments to be repeated as models change. Utah's public pilot register, opened as a cross-check, says Doctronic is still in Phase 1, with a licensed practitioner authorizing every request.","sledRelevance":"New, current evidence on a question state regulators and agency counsel now face: who carries liability when a state-authorized AI system acts. Utah is the most cited state sandbox, and tonight's material adds the stopgap (mandatory insurance, no liability waivers), a planned statute, and a frank admission that its technical expectations are immature. Relevance to non-health state agencies is in the oversight model (phased human review, monthly agreement-rate reporting, statutory audit, independent evaluation), not the clinical use cases.","evidence":"Opened the Nextgov article over HTTPS and read it in full (dateline Oct 16, 2026 3:50 PM ET). For cross-checking, opened Utah's AI Authorized Pilots register at commerce.utah.gov. The register confirms Doctronic remains in Phase 1, with every request authorized by a licensed practitioner, and that a 2026 amendment tightened the Phase 2 threshold and removed two medications. It lists the Doctronic term as Oct 24, 2025 to Oct 24, 2026 and new agreements (August AI refills, Nolla Health topical acne prescriptions, Expect Fitness physical therapy plans) with phased human review. It also states monthly reporting of AI-clinician agreement rates, a statutory audit duty, and an annual report to the Business and Labor Interim Committee before Nov 30.","architectureImplications":"No architecture details are given. The register's controls imply integration requirements for any AI that acts in a regulated workflow: identity and prescription verification against pharmacy records, hard stop rules that route to clinicians, adverse-event reporting within 24 hours, and data feeds to a regulator dashboard and monthly reports.","governanceImplications":"Utah separates regulatory relief (enforcement forbearance on named code sections) from liability, which the sandbox cannot change; it is filling the gap with insurance and contract terms until a statute exists. Phase advancement requires written approval from the office, and objections from professional boards are published with the office's response. Boyd's remark that half of the health AI expectations document 'is probably wrong' signals that standards are provisional.","securityPrivacyImplications":"Register terms ban selling patient data and advertising against it and require third-party adversarial testing for Doctronic and HIPAA-compliant systems for some pilots. The article does not report security testing results or incidents.","caveats":"Statements are a regulator's conference remarks and interview, not published outcomes; no agreement rates, adverse-event counts or evaluator results are reported. The liability bill has not been released. The pilot launches and the evaluator network were archived Oct 5; tonight's new material is the liability approach, the bill timeline, the licensing-board working group and the drug-list change. The 190-drug figure comes from Nextgov citing OAIP and does not appear on the register page. The register page carries no visible update date. Health care sits partly outside this stream; the oversight model is the transferable element.","streamIds":["state-government"],"roles":{"sales":"Interpretation: the opening is liability and oversight design, not AI capability. Utah's AI office admits it has no liability framework yet and is relying on mandatory insurance while it drafts a vendor-liability bill. Offer state regulators and agency counsel a bounded review of how their AI agreements allocate liability, insurance, evaluation and phase gates. Do not claim the Utah sandbox has demonstrated safety or savings: the article reports no outcome data, and Doctronic remains in Phase 1 with every request authorized by a licensed practitioner.","engineering":"Interpretation: fit is agencies or regulators that authorize AI to take actions in licensed or rights-affecting workflows. Utah's register spells out the integration controls: identity and record verification before action, hard stop rules routing cases to humans, 24-hour adverse-event reporting and monthly AI-human agreement rates. Proof of value should demonstrate that these signals can be captured and reported reliably during a human-reviewed phase. Do not rely on autonomous performance claims, because Utah has not approved any Doctronic Phase 2 and its own expectations document is described as immature.","delivery":"Interpretation: delivery is standing up an oversight operation, owned by the authorizing regulator or agency, around each AI pilot. Work includes phase-gate criteria, reviewer staffing for the first several hundred cases, adverse-event and agreement-rate reporting, an independent evaluator contract and published objections and responses. Acceptance criteria should be proposed, measurable thresholds agreed before any phase advances. Risks include unresolved liability, evaluator independence and cost, possible federal FDA jurisdiction, and reviewer burden shifting liability onto clinicians or caseworkers."},"roleDetails":{"sales":{"customerProblem":"States authorizing or buying AI that acts on residents lack a settled way to allocate liability, evaluate claims and decide when human review can be reduced.","stakeholders":["State AI policy or innovation office","Agency general counsel and attorney general's office","Professional licensing boards","Risk management and insurance office","Legislative interim committees"],"discoveryQuestions":["Who is liable today if an AI system your agency authorizes or buys harms a resident?","Do your AI agreements require insurance and prohibit liability waivers?","What evidence must a pilot produce before human review is reduced?","Who evaluates vendor claims independently, and who pays for it?"],"valueHypothesis":"A structured review of AI agreements and phase gates may help a state reduce unallocated liability and make advancement decisions on reported evidence rather than vendor assurances; benefit size is unknown.","potentialEngagement":"Fixed-scope review of one to three AI pilot or procurement agreements against Utah-style controls (insurance, waiver prohibition, phase gates, agreement-rate reporting, independent evaluation), with a gap memo for counsel.","unsupportedClaims":["That Utah's sandbox has proven AI prescribing or refills safe","Any cost savings or access gains from the pilots","That insurance requirements resolve liability","That independent evaluators have produced results"]},"engineering":{"fit":"Agencies or regulators where AI would take or recommend actions in licensed, benefit or rights-affecting workflows and must remain auditable.","architectureIntegration":"Build verification against systems of record before any AI action, rule-based stop conditions that route to humans, logging of AI and reviewer decisions for agreement-rate reporting, and an adverse-event feed to the overseeing office.","prerequisites":["Defined list of permitted actions and exclusions","Access to authoritative records for verification","Reviewer staffing for initial full-review phase","Agreed reporting schema with the regulator"],"constraints":["Sandbox relief covers named code sections only, not liability","Possible federal FDA premarket requirements for clinical software","Utah statute caps demonstration terms at 12 months with up to two extensions","State-specific licensing rules limit transferability"],"security":"Require third-party adversarial testing before launch, ban data sale and advertising use, keep HIPAA-grade controls where health data is involved, and verify identity before any action.","proofOfValue":["Measure AI-reviewer agreement rates during full human review before proposing any reduction","Demonstrate stop rules route every flagged case to a human in testing","Show adverse events can be reported within 24 hours end to end","Have an independent evaluator re-run assessment after any model change"]},"delivery":{"work":"Design and run the oversight lifecycle for an AI pilot: agreement controls, phase-gate criteria, reviewer operations, reporting, independent evaluation, and public posting of objections and responses.","dependencies":["Authorizing statute or policy","Counsel decision on liability and insurance terms","Independent evaluator contract","Licensing board or program owner engagement"],"ownership":"The authorizing regulator or agency AI office owns oversight; the vendor or provider owns operation within the agreed protocol; the relevant board or program office is consulted on each phase.","skillsAdoption":"Train reviewers on stop rules and on recording disagreement with the AI; train regulator staff to interpret agreement-rate and adverse-event data.","governanceCheckpoints":["Written approval before each phase advance","Monthly numeric report review","Annual legislative report","Re-evaluation after model or population change"],"acceptanceCriteria":["Proposed: agreed agreement-rate threshold met over a defined case count before review is reduced","Proposed: 100% of flagged cases routed to humans in audit samples","Proposed: adverse events reported within 24 hours in all sampled cases"],"risks":["Liability remains unallocated until statute passes","Evaluator independence or funding questioned","Federal jurisdiction overrides state relief","Reviewer workload and liability concentrated on clinicians"]}},"retrievedAt":"2026-10-16T22:20:13-05:00","enrichedAt":"2026-10-16T22:27:17-05:00","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Clinician review workload is built into each phase, for example review of the first 250 refills before they reach patients and then the next 1,000 in some agreements, so staffing for oversight is a real cost. The article raises concern that physicians supervising AI may carry disproportionate liability.","procurementImplications":"Interpretation: states that contract for or authorize AI acting on residents may need insurance requirements, prohibitions on liability waivers and independent evaluation clauses in agreements, as Utah does, until statutes allocate vendor liability.","operatingModelImplications":"The regulator runs a learning loop of phased approvals, monthly numeric reporting, weekly engagement with the objecting licensing board and an annual legislative report. That loop requires dedicated regulator capacity beyond the original agreement.","sourceVerification":{"openedUrl":"https://www.nextgov.com/artificial-intelligence/2026/10/utah-ai-sandbox-launches-new-health-care-pilots-question-marks/416655/","referenceExcerpt":"OAIP has required Doctronic and AI pilot companies that have engaged with the sandbox since then to carry special liability insurance and not waive liability.","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}