Lighthouse AdvisorySLED AI Adoption Intelligence

Public Sector & Government · Issue 05 ·

Local Government

Three new-to-archive sources connect San Francisco's historical AI rollout and unfinished benefit-measurement commitments, Portland's 2026 controls for embedded and internally developed AI, and independent research on procurement-checklist limits. Two patterns support funded review expertise and review beyond the initial purchase. No verified post-last-run development or demonstrated service savings is claimed. Material gaps include inaccessible Brent, Montgomery County and NSW reports, unverified later SF implementation, and limited small-government, utility and accessibility outcome evidence.

Evidence records
3
Cross-source patterns
2
Evidence classes
1 government audit1 standards or public-body guidance1 academic research
Outcomes
1 mixed1 emerging1 cautionary
Source freshness
2 undated1 older, newly relevant
Research completed
2026-09-11

Choose a role to see its takeaway beside every record in the ledger.

Synthesis · Lighthouse Advisory interpretation

Patterns across the evidence

2 patterns, each supported by at least two sources

  1. Give review owners the expertise and capacity to act

    San Francisco's response locates effectiveness judgments within departments, while the academic critique explains why generalist checklists can be insufficient. Together they support testing whether assigned owners can obtain and challenge evidence. Neither proves that a particular staffing structure improves outcomes.

    Operating questionDoes each service owner have funded expert support and the authority to resolve an inadequate evaluation before renewal or expansion?

    Supporting evidenceCity and County of San FranciscoTom Zick, Mason Kortz, David Eaves and Finale Doshi-Velez; Harvard University and University College London

  2. Keep AI review connected to changes after purchase

    Portland expressly addresses internally built AI and features added to existing products; the academic critique identifies why purchase-centered checks can miss these paths. This supports testing inventory and change-review coverage, without implying Portland's policy has eliminated the gaps.

    Operating questionWill an internal build or supplier feature change trigger review before it affects city data or resident services?

    Supporting evidenceCity of Portland, Bureau of Technology ServicesTom Zick, Mason Kortz, David Eaves and Finale Doshi-Velez; Harvard University and University College London

Full record · every source keeps its link and limitations

Evidence ledger

3 records
  1. Government auditMixedUndated source

    San Francisco's audit response separates AI rollout from unfinished benefit measurement

    The city reported broad assistant access while its benefit-evaluation framework remained prospective in this historical response.

    City and County of San FranciscoSan Francisco, California, United StatesLetter dated August 11, 2025; exact web publication date unknown

    Why it matters, evidence and limitations
    Why it matters
    New-to-archive historical context for municipal renewal and accountability decisions. Subsequent completion of commitments is unverified.
    Evidence and measured results
    The letter reports Copilot Chat availability for 30,000 employees following an assistant pilot involving over 2,000 staff. It claims time savings without a quantified estimate, baseline, control group or evaluation sample. In R1.4, DT says an evaluation framework will be developed and departments will judge effectiveness. The city accepts fragmentation concerns but disputes the jury's characterization of committee expertise.
    Limitations and uncertainty
    An August 2025 management response, not an audit finding of savings or a current compliance verdict. Broad access is not active adoption. No current service-quality or accessibility measurement is established.
    Official audit response with operator claims; not an independent efficacy evaluation Role takeaways & implementation
  2. Standards or public-body guidanceEmergingUndated source

    Portland extends AI review to embedded features and internally developed systems

    The rule covers internal development and AI added to existing systems, beyond the initial purchase.

    City of Portland, Bureau of Technology ServicesPortland, Oregon, United StatesExact page publication date unknown; rule effective March 6, 2026

    Why it matters, evidence and limitations
    Why it matters
    A new-to-archive U.S. municipal control design addressing purchasing blind spots. Its requirements apply to Portland; adoption elsewhere requires local tailoring.
    Evidence and measured results
    BTS reviews business cases and risk. Equivalent documentation is required for in-house AI, and bureaus must notify BTS when existing systems gain AI. Vendor model training with city data requires written authorization. Contracts require verification rights proportionate to risk. Public-facing use requires disclosure and language access. No compliance sample, error baseline or outcome evaluation is provided.
    Limitations and uncertainty
    Policy existence does not prove enforcement. Supplemental public guidance was inspected; detailed employee usage guidance requires intranet login and was not accessed.
    Official administrative rule; implementation effectiveness not evaluated Role takeaways & implementation
  3. Academic researchCautionaryNewly relevant · Apr 2024

    Procurement checklist research identifies expertise and review-coverage gaps

    The paper argues that checklists need expert interpretation and can miss low-cost, embedded and in-house AI.

    Tom Zick, Mason Kortz, David Eaves and Finale Doshi-Velez; Harvard University and University College LondonCanada, Brazil and Singapore discussions; qualified transfer to U.S. municipalitiesApril 23, 2024, arXiv version 1

    Why it matters, evidence and limitations
    Why it matters
    New-to-archive historical scrutiny helps test whether municipal review processes reach actual systems. International examples are not U.S. legal requirements.
    Evidence and measured results
    The authors inspect two checklist frameworks, discuss practice with government officials, and describe a Harvard lab red-teaming exercise. Examples expose narrow supplier answers, purchasing thresholds and hidden components. The paper provides no representative interview sample size, causal effectiveness estimate or measured reduction in harm.
    Limitations and uncertainty
    Historical qualitative analysis; illustrative cases cannot estimate prevalence. Policy references describe 2024 circumstances and are not asserted as current law. Version 2 was inaccessible; version 1 was fully inspected.
    Academic qualitative analysis and red-teaming exercise; historical preprint Role takeaways & implementation

How to read this edition

Source findings, measured results and limitations come from the cited publications. Patterns, operating questions, role takeaways and implementation considerations are Lighthouse Advisory interpretation, stated as questions to validate locally rather than guaranteed outcomes. Vendor and operator claims are labeled as claims. Full research method.

Government audit
An oversight review of performance, controls, or operations.
Standards or public-body guidance
Normative or advisory guidance from a standards body or public institution.
Academic research
Research produced through an academic institution or peer-reviewed venue.