{"resourceId":"sf-ai-rollout-grand-jury-response-2025","versions":[{"version":"external-01c3792934278c4d7fa79e0b012a71b1fbd88c3bb1e951c134bb252e242241d2","resource":{"id":"sf-ai-rollout-grand-jury-response-2025","title":"San Francisco's audit response separates AI rollout from unfinished benefit measurement","organization":"City and County of San Francisco","sector":"Local government administration","geography":"San Francisco, California, United States","publishedAt":"Letter dated August 11, 2025; exact web publication date unknown","publicationDate":null,"eventDate":"2025-08-11","sourceName":"San Francisco Mayor and departmental responses to the Civil Grand Jury","sourceLabel":"Official audit response with operator claims; not an independent efficacy evaluation","sourceUrl":"https://media.api.sf.gov/documents/AI_Consolidated_Responses.pdf","evidenceClass":"government-audit","outcomeClass":"mixed","topics":["knowledge-work","infrastructure","governance-procurement","data-security","accessibility-workforce","operating-model"],"finding":"The city reported broad assistant access while its benefit-evaluation framework remained prospective in this historical response.","sledRelevance":"New-to-archive historical context for municipal renewal and accountability decisions. Subsequent completion of commitments is unverified.","evidence":"The letter reports Copilot Chat availability for 30,000 employees following an assistant pilot involving over 2,000 staff. It claims time savings without a quantified estimate, baseline, control group or evaluation sample. In R1.4, DT says an evaluation framework will be developed and departments will judge effectiveness. The city accepts fragmentation concerns but disputes the jury's characterization of committee expertise.","architectureImplications":"Interpretation: Verify each department's permissions and records interfaces before expanding shared assistants. The response mentions a Snowflake data platform; it does not establish secure integration or compare hosting models.","governanceImplications":"Interpretation: Separate central platform authorization from departmental acceptance of service results.","securityPrivacyImplications":"Interpretation: Test retrieval boundaries using synthetic records and require evidence of retention behavior before live data access.","caveats":"An August 2025 management response, not an audit finding of savings or a current compliance verdict. Broad access is not active adoption. No current service-quality or accessibility measurement is established.","streamIds":["local-government"],"roles":{"sales":"Interpretation: Discuss administrative workload with the department director, CIO, finance lead and frontline staff. Ask which tasks consume time, what happens after a draft is generated, and who can authorize a renewal. The credible value hypothesis is reduced net handling effort with maintained service quality. A bounded engagement could measure one internal correspondence workflow and produce an evidence-based renewal decision. San Francisco's reported availability does not justify multiplying assumed savings across a customer workforce. Discover existing licenses and unused capacity before proposing expansion. Smaller municipalities may need shared evaluation support; do not infer that this large-city operating structure or budget is suitable for them.","engineering":"Interpretation: Start with draft assistance where a reviewer can compare outputs against approved source material. Map identity, retrieval, document storage and final records capture; clarify whether the proposed workflow needs internal data access at all. Prerequisites include representative tasks, a quality rubric and access to ordinary-work timings. Test denied-document retrieval, misleading source text, unsupported details and interrupted transfers using synthetic or approved records. Compare end-to-end effort with the existing process, including corrections. Keep agent write permissions disabled until a separate test supports them. The historical response supplies no evidence that a particular cloud, on-premises or hybrid design meets the customer's requirements.","delivery":"Interpretation: Assign the service manager responsibility for the pilot outcome and IT responsibility for configuration and support. Establish a baseline, train reviewers, record non-use and correction effort, and schedule finance and records-management checkpoints. Dependencies include reviewer time, representative tasks and a working manual fallback. Proposed acceptance criteria are a locally agreed reduction in net task time, no deterioration in independently scored record quality, and successful permission and retention tests. These are future criteria, not reported city results. Include accessible training and a feedback route for reluctant users. Budget recurring assurance work before expanding, and pause if the workflow generates unsupported facts in official records."},"retrievedAt":"2026-09-11T03:02:04Z","enrichedAt":"2026-09-11T03:04:59Z","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Interpretation: Include disabled staff and nonusers in testing; attendance or license access should not substitute for usable workflows.","procurementImplications":"Interpretation: Link renewal to local net effort, quality and recurring support costs.","operatingModelImplications":"Interpretation: Name departmental benefit owners alongside central IT support and finance approval.","updateExplanation":"New to the full 182-resource archive checked across offsets 0 and 100. No substantive source update or post-last-run event is claimed.","sourceVerification":{"openedUrl":"https://media.api.sf.gov/documents/AI_Consolidated_Responses.pdf","referenceExcerpt":"Individual departments will ultimately be responsible for determining the effectiveness of AI technology.","promptVersion":"sled-research-v3.1","model":null,"basis":"agent-reported inspection"}}}]}