{"resourceId":"umkc-llm-parole-transcript-rearrest-prediction-20261006","versions":[{"version":"external-e265b23ac0d4851e22952b5940a4d5536ec782a3a338c52b03e1ffeea9e97b00","resource":{"id":"umkc-llm-parole-transcript-rearrest-prediction-20261006","title":"UMKC and UC Davis researchers find GPT-5 reading 141 parole-hearing transcripts beat a statistical rearrest model, but its stated reasons did not match what drove its predictions","organization":"University of Missouri-Kansas City School of Law and University of California, Davis","sector":"Corrections / parole risk assessment research","geography":"United States (study data; jurisdiction not stated in the release)","publishedAt":"Oct 6, 2026","publicationDate":"2026-10-06","eventDate":null,"sourceName":"UMKC Today","sourceLabel":"UMKC news release on Copus and Laqueur's paper 'Computational Clinical Judgment: Predicting Risk with Large Language Models'","sourceUrl":"https://www.umkc.edu/news/posts/2026/october/can-ai-predict-who-will-be-rearrested.html","evidenceClass":"academic-research","outcomeClass":"mixed","topics":["knowledge-work","governance-procurement","accessibility-workforce"],"finding":"A UMKC release published Oct. 6, 2026 describes a paper by UMKC law professor Ryan Copus and UC Davis researcher Hannah Laqueur, 'Computational Clinical Judgment: Predicting Risk with Large Language Models.' They prompted six LLMs to assess rearrest risk from 141 parole hearing transcripts and compared them with a statistical model trained on more than 4,000 cases and 91 administrative variables. GPT-5 outperformed the statistical model on standard accuracy measures, added predictive information the model lacked, and was closer to the actual outcome in 83 of 141 cases (59%); a second model's predictions correlated 0.95 with GPT-5's. Much of the gain came where the statistical model overestimated risk. But GPT-5's written explanations emphasized remorse and insight while its predictions more closely tracked criminal history and offense severity. Copus says the explanations 'are not yet ready to be trusted' and that there are 'too many unknowns' for high-stakes deployment.","sledRelevance":"Corrections and parole boards are being offered AI summarization and risk tools; this study shows potential accuracy gains from unstructured hearing records alongside a concrete explainability failure that matters for due process.","evidence":"University news release summarizing a faculty paper; the paper itself returned HTTP 403 to automated retrieval, so methods, accuracy metrics, outcome definition and jurisdiction were not inspected. Small sample of 141 transcripts.","architectureImplications":"Using LLMs on hearing transcripts requires transcript capture, secure model hosting and comparison against the incumbent actuarial tool, not replacement of it.","governanceImplications":"Explanations that do not reflect the drivers of a prediction cannot serve as reasons in a parole decision; any use would need validation, disclosure and a bar on sole reliance.","securityPrivacyImplications":"Parole transcripts contain sensitive personal, victim and health information; sending them to external model APIs raises data-handling questions the release does not address.","caveats":"Press release only; paper not opened (403); 141 transcripts; rearrest is a proxy outcome; the release names a second model as 'Claude Fable 5', reported as written; no subgroup fairness results reported in the release.","streamIds":["public-safety"],"roles":{"sales":"Sales framing: this is research, not a product, but parole boards and corrections agencies will hear LLM risk pitches citing results like these. Discovery should establish what risk instrument is used today, whether hearing transcripts are digitized and who would validate any AI comparison. A bounded engagement is an evaluation design for a research-only benchmark; do not claim that LLMs predict recidivism better or that their explanations are usable as reasons.","engineering":"Engineering framing: the study pairs unstructured transcripts with an actuarial baseline. Engineers would need secure transcript ingestion, a private or contractually protected model, and logging of prompts and outputs for replication. The key validation is whether stated explanations match the factors that actually move predictions, which the authors found they did not. Prompt sensitivity and subgroup error rates must be measured before any operational trial.","delivery":"Delivery framing: any agency exploring this would run a closed retrospective study, not a live pilot. Ownership sits with the parole board's research unit with legal counsel and an external evaluator. Governance checkpoints include a no-decision-use rule, subgroup analysis and publication of methods. Risks include automation bias at hearings, unreliable explanations entering decision records, and transcript data leaving agency control."},"roleDetails":{"sales":{"customerProblem":"Parole decision-makers want better risk information than structured actuarial tools provide, without losing explainable reasons.","stakeholders":["Parole board members","Corrections research units","Agency counsel","Defense and reentry advocates","Victim services"],"discoveryQuestions":["Which risk instrument does the board use now?","Are hearing transcripts digitized and retained?","Who would validate an LLM comparison locally?"],"valueHypothesis":"Interpretation: a retrospective benchmark could show whether transcript context reduces risk overestimation locally, without affecting live decisions.","potentialEngagement":"Design of a closed retrospective evaluation comparing LLM outputs with the current instrument on historical cases.","unsupportedClaims":["LLMs are ready for parole decisions","LLM explanations reflect their reasoning","Results transfer to other jurisdictions"]},"engineering":{"fit":"Limited to research and benchmarking on transcript-rich parole records.","architectureIntegration":"Secure transcript store feeding a privately hosted or contractually restricted LLM, scored against the existing actuarial model.","prerequisites":["Digitized hearing transcripts","Outcome data for follow-up","Approved research protocol"],"constraints":["Sample of 141 transcripts","Jurisdiction not stated in the release","Prompt sensitivity unresolved"],"security":"Keep transcripts in agency-controlled environments with no vendor training on the data and full prompt logging.","proofOfValue":["Replicate accuracy on a held-out local sample","Test whether explanations match prediction drivers","Report error rates by race, sex and age"]},"delivery":{"work":"Run a retrospective study of LLM risk outputs on historical transcripts under research governance.","dependencies":["Data-sharing approval","External evaluator","Outcome data access"],"ownership":"Parole board research unit with agency counsel.","skillsAdoption":"Researchers need LLM evaluation skills; board members need briefing on automation bias.","governanceCheckpoints":["Research-only use rule","Subgroup fairness review","Publication of methods and limits"],"acceptanceCriteria":["Proposed: no outputs shown to decision-makers during the study","Proposed: explanation-faithfulness test reported","Proposed: subgroup error rates published"],"risks":["Automation bias","Untrustworthy explanations in records","Data leaving agency control"]}},"retrievedAt":"2026-10-07T02:58:43Z","enrichedAt":"2026-10-07T03:02:11Z","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Parole analysts and board members would need training to treat LLM outputs as one input and to recognize unreliable narrative explanations.","procurementImplications":"Vendors pitching LLM risk tools should be asked for local validation, subgroup error rates and evidence that explanations match model behavior.","operatingModelImplications":"The realistic near-term role is research or second-opinion benchmarking under board oversight, not decision support at hearings.","sourceVerification":{"openedUrl":"https://www.umkc.edu/news/posts/2026/october/can-ai-predict-who-will-be-rearrested.html","referenceExcerpt":"In its written explanations, GPT-5 emphasized factors such as remorse and insight.","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}