{"resourceId":"defra-ask-the-toolkit-atrs-claude-sonnet-4-6-missed-grounding-thresholds-20261110","versions":[{"version":"external-cae271e231a4798197f30637bee0329e6e0fe56438c3fb268b7490b9aaaa8660","resource":{"id":"defra-ask-the-toolkit-atrs-claude-sonnet-4-6-missed-grounding-thresholds-20261110","title":"UK Defra transparency record: Claude Sonnet 4.6 guidance assistant met 4 of 6 pre-launch pass marks, missing rule-quoting and grounding thresholds","organization":"Department for Environment, Food and Rural Affairs (Defra), UK","sector":"National government (UK)","geography":"United Kingdom","publishedAt":"Published October 27, 2026 (page metadata modified November 9, 2026)","publicationDate":"2026-10-27","eventDate":"2026-10-27","sourceName":"GOV.UK Algorithmic Transparency Records","sourceLabel":"Government algorithmic transparency record","sourceUrl":"https://www.gov.uk/algorithmic-transparency-records/defra-ask-the-toolkit","evidenceClass":"government-evaluation","outcomeClass":"mixed","topics":["knowledge-work","governance-procurement","data-security","operating-model"],"finding":"Defra's transparency record for Ask the toolkit, a guidance assistant on Claude Sonnet 4.6 via Amazon Bedrock in London, reports that before launch it met four of six pass marks but missed the two requiring every answer to pass: quoting the expected rule (16 to 19 of 22) and staying grounded in named pages (87 to 90 of 100). An AI judge (Claude Opus 4.6) scored tests, and two human reviewers agreed with only 38 and 27 of 40 judge decisions. The service sends the whole toolkit with each question, verifies quotes against pages, has no tools or web access, caps use at 600 model calls a day, and live answers are not routinely reviewed.","sledRelevance":"Why now: the record (opened Oct. 27, page modified Nov. 9) was not previously archived and is a rare, detailed public evaluation of a Claude-based government guidance assistant, including its failures. State and local agencies building policy or benefits-guidance assistants can copy the design (whole-corpus context, quote verification, refusal paths, usage caps) and the transparency format, while noting that even a narrow 40-page corpus did not reach full grounding and that AI-judge scoring needs human calibration.","evidence":"Government-published pre-launch test results with counts, dates and method (AI judge plus human agreement check, manual prompt-attack rounds, load test of 20 simultaneous questions). Small test sets; no live outcome data.","architectureImplications":"Single model call per question with the full ~40-page corpus (no retrieval), fixed output schema, quote-verification step, Bedrock guardrail, pinned model version, and a daily call cap. The record notes retrieval may be needed if the corpus grows.","governanceImplications":"Publishes missed thresholds and limits openly; DPIA screened out because no personal data is sought; security and platform readiness approvals recorded; no formal fairness assessment.","securityPrivacyImplications":"Bedrock guardrail masks some personal data before the model, but platform logs retain questions as typed (including any personal data) for 30 days in a restricted store; AWS processes prompts, and per Amazon documentation model makers cannot see them.","caveats":"Pre-launch evaluation by the operating team using an AI judge with weak human agreement; no live usage data at writing; UK national context; delivered via AWS Bedrock, so model behavior is Anthropic's but hosting terms are AWS's. Cross-tagged state-government for transferability of the evaluation and transparency method; UK national context limits direct applicability. Anthropic model delivered via Amazon Bedrock; kept in the Anthropic stream because the evidence concerns Claude model behavior.","streamIds":["anthropic","state-government"],"roles":{"sales":"Lead with transparency as a selling point. Defra published that its Claude-based guidance assistant missed two strict accuracy thresholds before launch, and explained its controls. For state and local leaders planning policy or benefits-guidance assistants, ask whether they will publish results and what accuracy bar they will set. Offer a design-and-evaluation package modelled on this record. Do not claim perfect grounding; even a 40-page corpus fell short.","engineering":"Engineering can reuse this design: send the full small corpus each call, force a structured answer, verify quotes against sources and drop mismatches, add refusal and referral states, pin the model version, and cap daily calls. The AI judge had weak human agreement, so calibrate it with human reviewers. Proof of value should report grounding and rule-selection rates on an agency test set, plus prompt-attack results, before launch.","delivery":"Delivery would follow Defra's sequence: build a fixed test set from the guidance, set pass marks, run repeated evaluations, calibrate the AI judge against people, complete privacy, security and accessibility checks, then publish a transparency record. Owner is the agency AI team. Dependencies are maintained source pages and log retention decisions. Risks include fluent but incomplete answers, logs holding personal data for 30 days, and no routine live review."},"roleDetails":{"sales":{"customerProblem":"Agencies want staff or public guidance assistants but lack a tested pattern for proving answers stay within official policy.","stakeholders":["State or county AI program lead","Policy or guidance owners","Privacy officer","Accessibility coordinator"],"discoveryQuestions":["What accuracy bar must answers meet before launch?","Will you publish evaluation results?","How large is the guidance corpus?","Who reviews live answers?"],"valueHypothesis":"A grounded, narrowly scoped assistant with published testing could help staff find rules faster while making limits visible.","potentialEngagement":"A design-and-evaluation engagement producing a test set, pass marks and a transparency record for one guidance corpus.","unsupportedClaims":["Full grounding accuracy","Fit for decisions about individuals","Usage or time-savings results (none yet)"]},"engineering":{"fit":"Strong fit for small, stable guidance corpora and staff-facing Q&A.","architectureIntegration":"Single model call with full corpus, structured output, quote verification, guardrail, pinned model, daily cap.","prerequisites":["Curated guidance corpus","Fixed test set and pass marks","Human calibration of AI judge"],"constraints":["Corpus size limits whole-context approach","UK context","Pre-launch evidence only"],"security":"Guardrail masks some personal data; platform logs retain raw questions 30 days; no tools or web access.","proofOfValue":["Report rule-selection and grounding rates on agency test set","Measure human-judge agreement","Run two rounds of prompt-attack tests"]},"delivery":{"work":"Build, evaluate and publish a grounded guidance assistant.","dependencies":["Guidance owners maintaining pages","Privacy and security approvals","Accessibility testing"],"ownership":"Agency AI team with policy owners.","skillsAdoption":"Team learns evaluation design and judge calibration.","governanceCheckpoints":["Pass marks agreed before build","Privacy screening","Publication of transparency record"],"acceptanceCriteria":["Proposed: grounding at or above agreed threshold on test set","Proposed: human-judge agreement measured and reported","Proposed: transparency record published at launch"],"risks":["Fluent incomplete answers","Personal data in logs","No live answer review"]}},"retrievedAt":"2026-11-11T04:22:12Z","enrichedAt":"2026-11-11T04:32:28Z","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Automated WCAG 2.2 AA testing found no failures; Defra's accessibility team will test with users after launch; Welsh and a representative public sample not tested.","procurementImplications":"Model accessed through AWS under the UK One Government Value Agreement, with no direct Anthropic contract; agencies can follow a similar cloud-marketplace path but must read Bedrock third-party model terms.","operatingModelImplications":"AI Capability and Enablement team maintains the service, retests a fixed question set before any change, and can stop it by configuration change and redeployment.","sourceVerification":{"openedUrl":"https://www.gov.uk/algorithmic-transparency-records/defra-ask-the-toolkit","referenceExcerpt":"It missed the 2 pass marks that need every answer to pass: quoting the expected rule (16 to 19 of 22)","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}