From the SLED-wide archive edition of August 30, 2026
Cross-country review finds experimentation widespread but monitoring and evaluation weak
Organisation for Economic Co-operation and Development · Public administration · International; official guidance from 14 countries
- Publisher
- Generative AI experimentation in government: Learning from emerging guidelines
- Original publication
- July 20, 2026
- Source retrieved
- Not recorded in the historical archive
What happened
OECD reviewed official experimentation guidance across 14 countries, academic literature, and case studies. It found rapid decentralized uptake, fragmented guidance, and few governments systematically measuring performance, impact, or compliance.
Why it matters
The gap mirrors SLED conditions: employees experiment before governance is mature, smaller organizations face inconsistent rules, and pilots are often counted without demonstrating service, learning, workforce, or compliance outcomes.
Evidence and measured results
The working paper synthesizes documented practices rather than testing one intervention. It proposes evaluation across five dimensions—performance, public value, feasibility, usability, and risk management—and identifies structured experimentation as a bridge between principles and scaled operation.
Limitations and uncertainty
This is comparative guidance and synthesis, not causal outcome evidence. Official guidelines may differ from actual agency practice, and the review's international scope means legal and administrative assumptions do not transfer uniformly to U.S. SLED organizations.
Put this evidence to work
Lighthouse Advisory interpretation, grounded in this source as summarized in the preserved archive. Enriched 2026-09-05; this does not change the original publication date. Labels below come from the analysis itself.
Sales
Role takeaway
Problem and stakeholders: Innovation teams, CIOs, procurement, risk leaders, and program managers may count pilots without knowing whether they improve public service.
- Discovery
- Does each experiment have an owner, baseline, public-value hypothesis, stop rule, and route to operational funding?
- Value hypothesis
- Shared experiment discipline could make results comparable and prevent unsupported scaling.
- Potential engagement
- Review the pilot portfolio and apply an evidence charter to a small set of varied workflows.
- Evidence boundary
- OECD synthesizes guidance and cases across 14 countries rather than evaluating one controlled intervention. Its five dimensions provide useful questions; they do not prove a sandbox, central team, or governance model will improve outcomes under U.S. SLED conditions.
Pre-sales engineering
Role takeaway
- Fit
- Use governed experimentation for bounded knowledge-work, copilot, or agent trials before granting production authority.
- Architecture
- Reuse segregated sandboxes, approved data paths, versioned prompts and models, evaluation harnesses, and monitoring.
- Prerequisites
- Defined use cases, data classification, baseline tasks, and agreed performance, public-value, feasibility, usability, and risk measures.
- Constraints
- International administrative assumptions require local translation, and sandbox results may not transfer to production integrations.
- Security
- Limit sensitive data, tool permissions, and network access to approved scope.
- Proposed validation
- Run comparable tasks with and without assistance, record quality, latency, cost, accessibility, and failures, then demonstrate a controlled production handoff or an evidence-based decision to stop.
Delivery
Role takeaway
Work and dependencies: Inventory experiments and introduce a charter, evaluation record, and graduation decision through existing portfolio processes.
- Ownership
- Programs own hypotheses and public-service outcomes; central technology, procurement, privacy, and security teams supply reusable controls and assurance.
- Skills and adoption
- Train pilot leads to measure baselines and involve affected users in usability and accessibility review.
- Governance checkpoints
- Approve data and scope before testing, inspect evidence before integration, and confirm funding and support before production.
- Proposed acceptance
- Reviewed pilots have results across the five OECD dimensions, limitations, and reasoned continue, change, or stop decisions.
- Risks
- Fragmented guidance, weak measurement, and unfunded operations can survive a technically successful sandbox; international guidance does not resolve local staffing or legal constraints.
Implementation considerations
Lighthouse Advisory interpretation across the operating dimensions a public-sector buyer must settle before this evidence becomes a design. Each note answers the question under its heading for this specific source.
Architecture and integration
What must connect, and where does the AI sit in the workflow?
Provide segregated sandboxes, approved data paths, reusable evaluation harnesses, model and prompt logging, and a governed route from experiment to production. Instrument quality, cost, latency, accessibility, and risk from the beginning rather than after a pilot is declared successful.
Governance
Who approves, reviews and stays accountable for outcomes?
Use a common experiment charter with an accountable owner, hypothesis, baseline, success and stop criteria, affected-user review, and evidence package. Allow local experimentation within shared guardrails while central teams provide templates, expertise, procurement, and assurance services.
Security and privacy
What data, permissions and controls need testing?
Match experiment environments to data sensitivity; prohibit uncontrolled sensitive-data use; document model and vendor handling; and perform privacy, security, bias, and misuse testing before expanding access or authority.
The preserved archive analysis covered architecture, governance and security. Not assessed for this record: accessibility and workforce, procurement, operating model.
Publication history
- 2026-08-30SLED-wide archive · Issue 035 resources
Stable resource ID: oecd-government-experimentation