{"resourceId":"centre-for-homelessness-impact-guide-evaluating-ai-homelessness-tools-20261028","versions":[{"version":"external-013a630e6cbf37f977ad1bb31388543c1426d0daeec03f3b9e9886dc2955d9d7","resource":{"id":"centre-for-homelessness-impact-guide-evaluating-ai-homelessness-tools-20261028","title":"Centre for Homelessness Impact guide tells councils to judge AI homelessness tools on housing outcomes, not usage or hours saved","organization":"Centre for Homelessness Impact (funded by UK Cabinet Office Evaluation Task Force)","sector":"Homelessness prevention and housing advice services","geography":"United Kingdom","publishedAt":"2026-10-28","publicationDate":"2026-10-28","eventDate":null,"sourceName":"Centre for Homelessness Impact","sourceLabel":"Practical guide (publication page)","sourceUrl":"https://www.homelessnessimpact.org/publication/evaluating-ai-homelessness-interventions-a-practical-guide-for-local-government","evidenceClass":"standards-guidance","outcomeClass":"cautionary","topics":["governance-procurement","data-security","operating-model"],"finding":"The Centre for Homelessness Impact published (Oct 28) a guide, funded by the UK Cabinet Office Evaluation Task Force, on evaluating AI tools in homelessness prevention. It tells councils to ask first whether AI adds value over existing guidance, phone lines and webpages; to build in the strongest evaluation design they can, noting that digital tools make randomisation easier; to measure outcomes such as homelessness duties and sustained tenancies, not just usage, satisfaction or hours saved, and to expect a lag; to monitor safety incidents, escalations and accuracy; to count full costs including integration and upkeep; to keep a change log because tools change mid-evaluation; to treat chatbot logs as sensitive data with human escalation for high-risk cases, a Data Protection Impact Assessment and an Algorithmic Transparency Recording Standard record; and to share null and negative results. It uses an ongoing trial of one AI housing advice tool as a worked example; findings will be published separately.","sledRelevance":"U.S. human services and homelessness agencies piloting chatbots and AI tools mostly report usage, satisfaction and time saved. This guide gives a ready evaluation standard for asking whether housing outcomes actually improve. Transfer to U.S. programs is an interpretation.","evidence":"Recommendations from an evidence center; explicitly presents no findings.","architectureImplications":"Tools need monitoring data from the start: usage and reach, safety incidents and escalations, accuracy checks, and a version change log.","governanceImplications":"Recommends a Theory of Change, safety assurance before launch, impact assessment, transparency records, and publishing null and negative results.","securityPrivacyImplications":"Chatbot logs from people at risk of homelessness should be treated as sensitive data, with human escalation for high-risk cases.","caveats":"Guidance, not evidence of effectiveness. UK statutory framework (homelessness duties, DPIA, ATRS) differs from U.S. requirements. Only the publication landing page summary was inspected; the full guide is behind a link shortener and was not read.","streamIds":["health-human-services"],"roles":{"sales":"Sales framing: this is a credibility tool for agencies, not a product. Human services leaders being sold AI chatbots need a way to judge them on outcomes. Offer an evaluation design for an existing or planned AI service tool, with baselines and outcome measures agreed before launch. Do not suggest the guide endorses any tool or shows AI works; it presents no findings.","engineering":"Engineering framing: build evaluation into the system. Capture usage and reach, safety incidents, escalations and accuracy checks from day one, version every prompt and model change, and support randomised or staggered rollout where possible. Treat chatbot logs as sensitive records with defined retention and access, and route high-risk conversations to people.","delivery":"Delivery framing: the owner is the program running the service, with an evaluation partner. Write a Theory of Change, agree outcome measures and follow-up periods, keep a change log, and plan to publish results even if null. Proposed criteria include outcome measures beyond satisfaction and a documented decision to scale or stop. These are proposals drawn from the guide, not results."},"roleDetails":{"sales":{"customerProblem":"Agencies lack a clear way to decide whether AI service tools improve outcomes enough to scale or stop them.","stakeholders":["Human services or housing director","Program evaluation or performance lead","Privacy officer","Procurement lead"],"discoveryQuestions":["What outcome, beyond usage, would justify keeping this AI tool?","Is there a baseline and a comparison group?","Who reviews safety incidents and escalations?"],"valueHypothesis":"Bounded hypothesis: designing evaluation in from the start lets agencies make scale-or-stop decisions on outcomes rather than vendor usage metrics.","potentialEngagement":"An evaluation plan and monitoring data specification for one AI resident-facing tool.","unsupportedClaims":["That AI homelessness tools are effective","That the guide endorses a product","That UK requirements apply directly in the U.S."]},"engineering":{"fit":"Applies to any AI advice, chatbot or triage tool in human services.","architectureIntegration":"Telemetry for usage, escalations and accuracy sampling; version control of prompts and models; feature flags for randomised rollout.","prerequisites":["Defined outcome measures","Data access agreements for outcome follow-up","Logging infrastructure"],"constraints":["UK frameworks (DPIA, ATRS) need U.S. equivalents","Outcome data may sit in other systems such as HMIS","Only the summary page was inspected"],"security":"Treat conversation logs as sensitive, restrict access, set retention, and escalate high-risk cases to staff.","proofOfValue":["Randomised or staggered rollout design approved","Accuracy checks run on a sample each month","Outcome follow-up data linkage tested"]},"delivery":{"work":"Write the Theory of Change, set baselines, build monitoring, run the evaluation and publish results.","dependencies":["Program staff","Evaluation partner","Data owners for outcome data"],"ownership":"The program operating the AI tool, with an internal or external evaluation lead.","skillsAdoption":"Program teams need basic evaluation design skills and a habit of logging tool changes.","governanceCheckpoints":["Safety assurance before launch","Impact or privacy assessment","Scale-or-stop review on outcome data"],"acceptanceCriteria":["Outcome measures beyond usage defined before launch (proposed)","Change log maintained for every tool change (proposed)","Results published including null findings (proposed)"],"risks":["Short pilots measuring only satisfaction","Tool changes invalidating comparisons","Outcome data hard to link"]}},"retrievedAt":"2026-10-31T22:12:47-05:00","enrichedAt":"2026-10-31T22:24:00-05:00","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Asks agencies to compare the AI tool with existing phone lines and webpages, which matters for people with limited digital access; staff time should be counted as a cost.","procurementImplications":"Commissioning, scaling or stopping decisions should rest on outcome evidence, and full costs (integration, upkeep) belong in the business case.","operatingModelImplications":"Budget for longer-term follow-up and expect the tool to change during evaluation.","sourceVerification":{"openedUrl":"https://www.homelessnessimpact.org/publication/evaluating-ai-homelessness-interventions-a-practical-guide-for-local-government","referenceExcerpt":"This is a practical guide, not an evaluation report, and does not present findings.","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}