Issue 03 · Evidence briefing
Measured digital-service performance, student-use evidence, procurement memory, and security controls for public-sector AI.
A decision-oriented read of what public institutions tried, what the evidence supports, and what leaders should design for next. Vendor claims are treated as claims, not outcomes.
Edition intelligence
Synthesis
GOV.UK Chat improved measured answer accuracy while remaining restricted to published government guidance and declining personal-case work. That combination—authoritative content, an in-scope answer threshold, source links, and human handoff—is a stronger public-service pattern than an unconstrained general assistant.
GAO found that four mature federal buyers did not systematically collect acquisition lessons, while OECD found weak and fragmented monitoring across government experimentation. SLED consortia and shared-service organizations can reduce repeated procurement mistakes by treating evaluation results, contract clauses, and operating lessons as managed institutional assets.
RAND's nationally representative youth survey found rising homework use alongside growing concern that AI harms critical thinking. The useful policy distinction is therefore cognitive augmentation versus cognitive offloading, supported by consistent course and school rules rather than a blanket approved-tool label.
RAND's vulnerability analysis places the greatest structural exposure in training data and user-facing inference components such as context windows and retrieval pipelines. Those risks are often probabilistic and only partly patchable, so procurement and operations need compensating controls, provenance, monitoring, and repeated adversarial evaluation across model updates.
Full record
Showing 5 of 5 records · All
Updated August 30, 2026
Two GOV.UK Chat pilots involved more than 10,000 users and roughly 26,000 questions about taxes, benefits, visas, and other public services. The system was constrained to published GOV.UK guidance and did not access personal records or perform case-specific transactions.
Two GOV.UK Chat pilots involved more than 10,000 users and roughly 26,000 questions about taxes, benefits, visas, and other public services. The system was constrained to published GOV.UK guidance and did not access personal records or perform case-specific transactions.
Government evaluators report accuracy increasing from an early 76% benchmark to 90% across topics, an 88% answer rate for in-scope questions, and 73% usefulness and 64% satisfaction in an app follow-up survey. All 508 recorded jailbreak attempts were blocked. The evidence combines expert and automated scoring, usability tests, surveys, conversation reviews, and journey analysis, but the government evaluated its own service.
This is one of the larger measured public tests of a citizen-facing government assistant and provides practical evidence for state portals, municipal service navigation, and education-administration help desks.
Use authoritative retrieval, source links, scope detection, clarification for ambiguous questions, and a clean handoff to authenticated human support. The implementation used Amazon Bedrock and Anthropic models behind a model-changeable service layer, illustrating how a public interface can remain stable while models are re-evaluated and replaced.
Publish the assistant's purpose and limits, define a strict answer-quality rubric, track accuracy by topic, and treat latency, refusals, escalation, and user trust as production service measures—not secondary UX concerns.
Keep personal case data outside the public assistant until identity, authorization, data minimization, logging, and human escalation are designed. Re-test jailbreak and content safeguards whenever models, retrieval, or streaming behavior changes.
The findings are operator-reported, not independently audited; accuracy scoring reflects GOV.UK's rubric and sampled conversations; satisfaction survey samples were much smaller than total participation; and a 90% score still leaves material room for incomplete or incorrect answers in consequential domains.
GAO reviewed 13 AI acquisitions and 44 contracts or agreements across Defense, Homeland Security, GSA, and Veterans Affairs. None of the four agencies had policies requiring systematic collection of lessons learned for government-wide reuse.
GAO reviewed 13 AI acquisitions and 44 contracts or agreements across Defense, Homeland Security, GSA, and Veterans Affairs. None of the four agencies had policies requiring systematic collection of lessons learned for government-wide reuse.
The performance audit found agencies were missing opportunities to capture practices such as data-rights terms and testing requirements or to avoid recurring mistakes. GAO made four policy recommendations, one to each agency, and all concurred. The sample was intentionally varied but nongeneralizable.
States, localities, districts, and public universities face the same fast-changing market with less contracting capacity. A shared acquisition memory can prevent every entity from rediscovering data-rights, testing, competition, monitoring, and exit problems independently.
Procurement artifacts should identify model, infrastructure, data, integration, testing, monitoring, version-change, and portability responsibilities across the full AI stack rather than buying 'AI' as an undifferentiated capability.
Require an acquisition closeout and renewal record covering outcome evidence, failures, contract clauses, vendor performance, data rights, testing, cost behavior, model changes, and exit experience; publish reusable lessons through a statewide, systemwide, or consortium repository.
Contracts should preserve audit and testing rights, define handling of government and constituent data, require change notification and ongoing performance monitoring, and establish deletion, portability, incident response, and termination obligations.
The audit covers four federal agencies and a nongeneralizable sample selected partly for maturity and impact; it evaluates acquisition practice rather than the performance of the acquired AI systems and does not establish which contract approach yields the best return.
A nationally representative RAND survey of 1,214 U.S. youth ages 12–29 found that reported homework use of AI increased substantially during 2025 while students remained uncertain about school rules and the effect on their own learning.
A nationally representative RAND survey of 1,214 U.S. youth ages 12–29 found that reported homework use of AI increased substantially during 2025 while students remained uncertain about school rules and the effect on their own learning.
Reported AI homework use rose from 48% in May 2025 to 62% in December 2025. Sixty-seven percent agreed that greater AI use for schoolwork would harm critical-thinking skills, more than 10 points higher than ten months earlier. Older students more often reported teacher-dependent rules and concern about being accused of cheating.
District and institution policy must respond to ordinary student behavior, not a hypothetical future. The evidence also supports designing assessment around cognitive work rather than relying primarily on detection or inconsistent teacher-by-teacher rules.
Learning platforms and managed AI workspaces should make allowed modes visible at the assignment level, preserve process evidence where appropriate, and support AI-free as well as AI-augmented work rather than assuming one configuration fits every learning objective.
Create consistent schoolwide or institution-wide categories for cognitive augmentation, permitted assistance, disclosure, and independent work; involve students in policy design; and align assignments and appeals with those categories.
Avoid turning concern about misuse into pervasive surveillance or automated accusations. Minimize collection of student prompts and drafts, restrict access, set retention limits, and keep detector scores out of consequential decisions without corroborating evidence and due process.
The results are self-reported perceptions and behavior, not direct measures of learning loss or causal effects. The 12–29 age range spans substantially different educational settings, and concern about critical thinking does not demonstrate that harm occurred.
OECD reviewed official experimentation guidance across 14 countries, academic literature, and case studies. It found rapid decentralized uptake, fragmented guidance, and few governments systematically measuring performance, impact, or compliance.
OECD reviewed official experimentation guidance across 14 countries, academic literature, and case studies. It found rapid decentralized uptake, fragmented guidance, and few governments systematically measuring performance, impact, or compliance.
The working paper synthesizes documented practices rather than testing one intervention. It proposes evaluation across five dimensions—performance, public value, feasibility, usability, and risk management—and identifies structured experimentation as a bridge between principles and scaled operation.
The gap mirrors SLED conditions: employees experiment before governance is mature, smaller organizations face inconsistent rules, and pilots are often counted without demonstrating service, learning, workforce, or compliance outcomes.
Provide segregated sandboxes, approved data paths, reusable evaluation harnesses, model and prompt logging, and a governed route from experiment to production. Instrument quality, cost, latency, accessibility, and risk from the beginning rather than after a pilot is declared successful.
Use a common experiment charter with an accountable owner, hypothesis, baseline, success and stop criteria, affected-user review, and evidence package. Allow local experimentation within shared guardrails while central teams provide templates, expertise, procurement, and assurance services.
Match experiment environments to data sensitivity; prohibit uncontrolled sensitive-data use; document model and vendor handling; and perform privacy, security, bias, and misuse testing before expanding access or authority.
This is comparative guidance and synthesis, not causal outcome evidence. Official guidelines may differ from actual agency practice, and the review's international scope means legal and administrative assumptions do not transfer uniformly to U.S. SLED organizations.
RAND decomposed generative AI architectures from training data through deployment interfaces and identified 31 vulnerability classes. Its highest aggregate risks clustered around training data and user-facing inference boundaries, including context windows and retrieval-augmented generation pipelines.
RAND decomposed generative AI architectures from training data through deployment interfaces and identified 31 vulnerability classes. Its highest aggregate risks clustered around training data and user-facing inference boundaries, including context windows and retrieval-augmented generation pipelines.
The researchers combined literature review, public incident and attack sources monitored from August 2025 through March 2026, architectural decomposition, and structured threat and impact metrics. They conclude that some weaknesses persist across model versions and can be reduced but not eliminated through conventional patching.
SLED security teams need to integrate AI into vulnerability management without pretending probabilistic model behavior maps neatly to conventional CVEs or patch cycles. The framework gives architects and buyers a component-level way to assign controls and residual risk.
Threat-model training and fine-tuning data, provenance, embeddings, context, retrieval, prompts, output interfaces, and any connected tools separately. Add input validation, retrieval isolation, provenance checks, least privilege, output controls, anomaly detection, and stochastic adversarial testing as compensating controls.
Require component-level risk assessments and residual-risk acceptance; connect AI findings to existing vulnerability, change, incident, and supplier-management processes; and re-evaluate after model, data, retrieval, or tool changes.
Prioritize dataset provenance and controls at context and retrieval boundaries, where poisoned or injected content can affect confidentiality and integrity. Logging and monitoring must detect probabilistic exploitation and resource-exhaustion patterns, not only deterministic signatures.
The taxonomy combines real-world and theoretical attack evidence and scores vulnerability classes rather than product-specific defects. It excludes bias harms, attacks that merely use AI, and external infrastructure or supply-chain vulnerabilities, and should be treated as an expandable baseline rather than a complete standard.
How to read this briefing
This edition prioritizes primary government material, public audits, independent research, and relevant public-sector association guidance available for theAugust 30, 2026 run. Every surfaced item remains in the All view and keeps its original source.
Vendor, operator, and association claims are attributed and are not upgraded to independent evidence. Caveats identify self-reporting, bounded pilots, contested findings, and missing outcome measures.