{"resourceId":"arxiv-triage-score-counterfactual-202606","versions":[{"version":"external-0bff7af54df0cb44783b499d8946d671367f8f9ec28396dcee8c51244fe498c0","resource":{"id":"arxiv-triage-score-counterfactual-202606","title":"arXiv: Triage Score — a counterfactual risk assessment instrument (Greiner/Imai et al.)","organization":"arXiv (stat.AP preprint)","sector":"Courts / criminal justice methodology","geography":"United States (methodological; pretrial RCT illustration)","publishedAt":"June 19, 2026 (submitted)","publicationDate":"2026-06-19","eventDate":null,"sourceName":"arXiv:2606.21050","sourceLabel":"Academic preprint","sourceUrl":"https://arxiv.org/abs/2606.21050","evidenceClass":"academic-research","outcomeClass":"emerging","topics":["knowledge-work","developers-agents","governance-procurement","operating-model"],"finding":"Imai, Shin, Greiner, and Halen (submitted 2026-06-19) propose triage scores based on additive counterfactual utilities, with conventional risk scores as a special case that ignore outcomes under intervention. They argue effective decision-making requires comparing potential outcomes across alternative decisions and illustrate with their randomized controlled trial evaluating a pretrial risk score, finding triage scores can yield substantively distinct policy-evaluation and learning results. Newly relevant on 2026-09-18 because Norfolk's Harvard A2J plea pilot is described as implementing this triage-score approach with overlapping investigators.","sledRelevance":"Provides the methodological backbone SLED courts/prosecutors should demand when vendors or labs pitch 'better than risk scores' decision AI—clarify counterfactuals, utilities, and evaluation design.","evidence":"Opened arXiv abstract and HTML full text via curl; confirmed authors, submit date, triage vs risk definition, and pretrial RCT illustration.","architectureImplications":"Model outcomes under alternative actions; expose utility parameters; keep recommendation UI separate from estimation; version configs.","governanceImplications":"Utility weights are policy choices needing sign-off; preprint status requires independent validation before operational reliance.","securityPrivacyImplications":"Experimental/administrative justice data used for fitting need research governance and access controls.","caveats":"Preprint; illustration is pretrial PSA RCT, not the Norfolk plea field study; does not report Norfolk/Philadelphia results.","streamIds":["public-safety"],"roles":{"sales":"Sales framing: the triage-score paper supplies the methodological why behind prosecutor interest in alternatives to risk scores—decision makers need estimated effects of intervening, not only baseline risk. Use it to shape evaluation RFPs and pilot metrics, not as proof a commercial product works in a named county. Value hypothesis is better-aligned policy learning under explicit utilities; unsupported claims include any guarantee about Norfolk/Philadelphia outcomes, which this preprint does not measure.","engineering":"Engineering framing: implement triage as an estimator of counterfactual utilities with risk scores as a nested special case; expose assumptions, uncertainty, and sensitivity to utility weights. Architecture should separate prediction of outcomes under alternative actions from the UI recommendation layer. Prerequisites: identifiable treatment variation or experimental design (paper illustrates with authors' pretrial RCT). Constraints: utilities encode ethics/policy choices that must be governed, not hidden in model weights.","delivery":"Delivery framing: translate the paper into an evaluation workbench—define actions (e.g., plea offer), outcomes, utilities, and learning protocol—before production scoring. Dependencies: statistician/methods partner, prosecutor domain experts for utility elicitation, and IRB/privacy review. Ownership shared between research lead and justice-agency product owner. Acceptance: documented utility rationale, validation against held-out or experimental data, and advisory-only deployment. Risks: misspecified utilities and overconfidence transferring pretrial RCT lessons to pleas."},"roleDetails":{"sales":{"customerProblem":"Conventional risk scores omit what happens if the agency intervenes, misaligning tools with plea and release decisions that change outcomes.","stakeholders":["Prosecutor leadership","Pretrial services","Court administration","Academic methods partners","Defense stakeholders"],"discoveryQuestions":["Which decisions and counterfactual outcomes must the score support?","Who sets and reviews utility weights?","What experimental or quasi-experimental data exist?","How will results be compared to existing risk tools?"],"valueHypothesis":"Bounded value is clearer policy evaluation when tools estimate effects of actions—not automatic superiority over current practice.","potentialEngagement":"Methods workshop + utility elicitation sprint to decide whether a triage-score pilot is feasible with available data.","unsupportedClaims":["Norfolk plea pilot results","Guaranteed fairness or accuracy gains","That pretrial RCT findings transfer unchanged to pleas"]},"engineering":{"fit":"Statistical decision-support methodology for justice pilots; requires strong evaluation design rather than off-the-shelf risk API swap.","architectureIntegration":"Model outcomes under alternative actions; configuration layer for utilities; reporting compares triage vs risk-score policies; human decision remains outside the model.","prerequisites":["Defined action set and outcomes","Credible identification strategy or RCT","Governance for utility parameters"],"constraints":["Preprint status","Illustration uses pretrial PSA RCT, not plea bargaining field results","Utility choices are normative"],"security":"Research data minimization; restrict who can alter utility parameters; version all scoring configs.","proofOfValue":["Replication of paper's policy-evaluation contrast on local holdout/experimental data","Sensitivity analysis across utility weights","Documented assumption checks"]},"delivery":{"work":"Elicit utilities, prepare analysis pipelines, validate triage vs risk policies, and package advisory outputs for practitioners.","dependencies":["Agency data access","Methods authors or equivalent statisticians","Legal review of research use"],"ownership":"Research PI for methods integrity; agency lead for operational interpretation limits.","skillsAdoption":"Train staff that triage outputs depend on stated utilities and counterfactual assumptions.","governanceCheckpoints":["Utility sign-off","Pre-analysis plan","External methods review before operational advice"],"acceptanceCriteria":["Written utility rationale","Comparative evaluation report","Advisory-only label on any pilot UI"],"risks":["Silent utility encoding of bias","Over-transfer from pretrial to plea context","Treating preprint as deployed product evidence"]}},"retrievedAt":"2026-09-19T03:15:14Z","enrichedAt":"2026-09-19T03:16:30Z","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Methods literacy for non-statistician prosecutors is required or agencies will misread triage outputs as simple risk labels.","procurementImplications":"RFPs for decision AI should require counterfactual evaluation plans and utility documentation, not accuracy-on-risk alone.","operatingModelImplications":"Use triage methodology to structure pilots; do not equate paper publication with production readiness.","sourceVerification":{"openedUrl":"https://arxiv.org/abs/2606.21050","referenceExcerpt":"Unlike risk scores, triage scores can incorporate counterfactual outcomes under alternative decisions","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}