San Francisco's audit response separates AI rollout from unfinished benefit measurement

Discuss administrative workload with the department director, CIO, finance lead and frontline staff. Ask which tasks consume time, what happens after a draft is generated, and who can authorize a renewal. The credible value hypothesis is reduced net handling effort with maintained service quality. A bounded engagement could measure one internal correspondence workflow and produce an evidence-based renewal decision. San Francisco's reported availability does not justify multiplying assumed savings across a customer workforce. Discover existing licenses and unused capacity before proposing expansion. Smaller municipalities may need shared evaluation support; do not infer that this large-city operating structure or budget is suitable for them.

Start with draft assistance where a reviewer can compare outputs against approved source material. Map identity, retrieval, document storage and final records capture; clarify whether the proposed workflow needs internal data access at all. Prerequisites include representative tasks, a quality rubric and access to ordinary-work timings. Test denied-document retrieval, misleading source text, unsupported details and interrupted transfers using synthetic or approved records. Compare end-to-end effort with the existing process, including corrections. Keep agent write permissions disabled until a separate test supports them. The historical response supplies no evidence that a particular cloud, on-premises or hybrid design meets the customer's requirements.

Assign the service manager responsibility for the pilot outcome and IT responsibility for configuration and support. Establish a baseline, train reviewers, record non-use and correction effort, and schedule finance and records-management checkpoints. Dependencies include reviewer time, representative tasks and a working manual fallback. Proposed acceptance criteria are a locally agreed reduction in net task time, no deterioration in independently scored record quality, and successful permission and retention tests. These are future criteria, not reported city results. Include accessible training and a feedback route for reluctant users. Budget recurring assurance work before expanding, and pause if the workflow generates unsupported facts in official records.