{"resourceId":"uconn-social-work-ai-symposium-bias-two-tier-care-20261110","versions":[{"version":"external-493c06d7d143d5691f2845be8a3df377cf57d3c971daeb17dbd685b7f81a358e","resource":{"id":"uconn-social-work-ai-symposium-bias-two-tier-care-20261110","title":"UConn social work AI symposium: researchers warn off-the-shelf models would have worsened racial bias in LA homeless housing assessments; Maine advocate cites two-tier chatbot risk for Medicaid","organization":"UConn School of Social Work","sector":"Social services and behavioral health","geography":"Connecticut and United States","publishedAt":"2026-11-10","publicationDate":"2026-11-10","eventDate":"2026-11-06","sourceName":"UConn Today","sourceLabel":"University news report of symposium","sourceUrl":"https://today.uconn.edu/2026/11/socialwork_ai_symposium/","evidenceClass":"independent-reporting","outcomeClass":"cautionary","topics":["governance-procurement","accessibility-workforce","data-security"],"finding":"At a Nov. 6 symposium, UNC researcher Eric Rice said machine learning showed Los Angeles homeless housing assessments were more accurate for white people, fair-ML redesign addressed it, and an off-the-shelf model on a decade of administrative data would have worsened the bias; the revised tool has assessed over 19,000 people. Former Maine legislator Charlotte Warren said advocates resisted a government-use exception that could steer MaineCare and Medicare patients to chatbots while privately insured people see human therapists.","sledRelevance":"Human services agencies prioritizing scarce housing or behavioral health resources with algorithms, and state Medicaid programs considering AI behavioral health tools, face the bias and two-tier access risks raised here.","evidence":"UConn Today (Nov. 10, 2026) reports panel remarks by named researchers, practitioners and policy experts, with attendance figures and the 19,000-assessment count.","architectureImplications":"Bias audits of administrative training data and fairness-aware model design are prerequisites before prioritization models go live.","governanceImplications":"Calls for practitioner and community involvement in model design, written client consent and licensing-board enforcement for AI in behavioral health.","securityPrivacyImplications":"Confidentiality of client data in behavioral health AI was a stated research-panel theme; no specific controls described.","caveats":"Symposium coverage by the host university; findings are summarized from talks rather than published papers; the LA work predates this event and is reported without methods detail.","streamIds":["health-human-services"],"roles":{"sales":"The buyer concern here is equity and trust, not efficiency. Agencies using prioritization tools for housing or behavioral health need evidence that models do not worsen disparities, and Medicaid programs need to show AI does not create a lesser tier of care. Offer bias review and co-design support. Do not imply that AI therapy for Medicaid members is acceptable to these stakeholders.","engineering":"The key technical lesson is that historical administrative data can encode bias, so an off-the-shelf model can amplify it. Any prioritization model needs subgroup accuracy testing, fairness-aware methods and drift monitoring. For behavioral health tools, design for clinician oversight and documented consent, and check state laws such as Maine's emerging rules before scoping.","delivery":"Delivery should include community and practitioner co-design sessions, subgroup performance reports before launch and on a schedule, and a clear escalation path when bias is found. For behavioral health, confirm consent workflows and licensing rules. Assign an equity owner alongside the technical owner so bias findings lead to action."},"roleDetails":{"sales":{"customerProblem":"Human services and behavioral health agencies risk worsening disparities or creating unequal care when they adopt AI without bias review.","stakeholders":["Human services agency directors","Continuum of Care and housing authority leads","State Medicaid behavioral health directors"],"discoveryQuestions":["Have your prioritization tools been tested for accuracy by race and other subgroups?","Would any AI tool change whether Medicaid members see a human clinician?","How are practitioners and residents involved in design?"],"valueHypothesis":"Bias review and co-design before deployment can lower the risk of inequitable outcomes and public backlash.","potentialEngagement":"Subgroup accuracy and fairness review of one existing assessment or prioritization tool.","unsupportedClaims":["AI therapy is suitable for Medicaid members","The LA method transfers without local validation","Bias can be fully removed"]},"engineering":{"fit":"Assurance and fairness review for prioritization and behavioral health AI, not a product deployment.","architectureIntegration":"Historical administrative data feeding assessment models, with subgroup accuracy testing and fairness-aware retraining before use in prioritization.","prerequisites":["Access to historical assessment and outcome data","Demographic fields for subgroup analysis","Practitioner and community advisory group"],"constraints":["State behavioral health AI laws (for example Maine) may require consent and clinician oversight","Small subgroup samples limit statistical tests","Model drift requires ongoing maintenance"],"security":"Protect sensitive behavioral health and housing data during analysis, with de-identification where possible and strict access logging.","proofOfValue":["Accuracy gap across racial subgroups before and after redesign","Practitioner acceptance in co-design reviews","Drift checks over successive quarters"]},"delivery":{"work":"Audit existing tools by subgroup, convene co-design sessions, redesign or recalibrate, and publish a monitoring schedule.","dependencies":["Data access agreements","Research partner or analytics team","Community advisory participation"],"ownership":"Agency director or Continuum of Care lead, with an analytics lead owning model monitoring.","skillsAdoption":"Social workers need enough data literacy to question model outputs and the data behind them.","governanceCheckpoints":["Bias review before launch","Scheduled drift and subgroup reviews","Consent and licensing compliance check for behavioral health uses"],"acceptanceCriteria":["Subgroup accuracy gaps reported before launch (proposed)","Co-design sessions held with practitioners and residents (proposed)","Written consent captured where AI is used in care (proposed)"],"risks":["Encoding historical discrimination into prioritization","Two-tier care for Medicaid members","Model drift going unnoticed"]}},"retrievedAt":"2026-11-10T22:19:40-06:00","enrichedAt":"2026-11-10T22:22:56-06:00","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Practitioners want a voice in AI policy; staff are asking agencies to take a stand on unregulated AI therapy use.","procurementImplications":"Agencies may need contract terms requiring subgroup performance reporting and consent support from vendors.","operatingModelImplications":"Keeps professional judgment and accountability with practitioners while AI informs practice.","peerReview":"unknown","sourceVerification":{"openedUrl":"https://today.uconn.edu/2026/11/socialwork_ai_symposium/","referenceExcerpt":"Applying an off-the-shelf machine learning algorithm to a decade of administrative data would have made the racial bias worse.","promptVersion":"sled-research-v4.0 HHS nightly brief","model":null,"basis":"agent-reported inspection"}}}]}