{"resourceId":"dib-bench-deployment-integrity-blindness-llm-evaluation-20261015","versions":[{"version":"external-2726b8946a415c8c055388b80cd3df82963946f8047e1aaf83fc96895e763dbc","resource":{"id":"dib-bench-deployment-integrity-blindness-llm-evaluation-20261015","title":"DIB-Bench preprint: output-only evaluators, including stronger LLM judges, accept up to 94% of tampered agent deployments; component hashes catch declared changes","organization":"Jianchang Su, Dawei Xiang, Yifan Zhang and Wei Zhang (arXiv preprint; affiliations not recorded in this inspection)","sector":"Evaluation methodology and reproducibility for LLM agents","geography":"Not stated in inspected text; applies to any institution publishing or relying on agent evaluations","publishedAt":"arXiv preprint 2610.21135v1, submitted 15 October 2026","publicationDate":"2026-10-15","eventDate":null,"sourceName":"arXiv","sourceLabel":"Academic preprint (not yet peer reviewed)","sourceUrl":"https://arxiv.org/abs/2610.21135","evidenceClass":"academic-research","outcomeClass":"cautionary","topics":["developers-agents","data-security","governance-procurement"],"finding":"DIB-Bench measures whether evaluators notice when a deployed LLM agent's runtime stack (model artifacts, tokenizer, system prompt, tool specifications, retrieval data, runtime software) differs from the evaluated one. Across 7,560 scored examples, six tamper types and an evaluator ladder from regex scorers through LLM judges to GPT-5-nano, stronger evaluators accepted more tampered outputs: mean acceptance rose from 88.6% (rule-based) to 94.1% (GPT-5-nano). A trigger-conditional tamper that is invisible on the evaluation suite produced 48-100% harmful outputs, rising with model capability. The authors give a PAC-style argument that tampers with zero output shift cannot be detected from outputs at any sample size, while a manifest-hash identity check detected declared component changes with zero samples. Behavioural fingerprinting identified the model component above chance (76.7%) but covers one of six components. They propose a graded deployment-identity descriptor for evaluation reports.","sledRelevance":"University labs increasingly publish agent benchmark results and research offices increasingly accept vendor evaluations. This preprint shows a score cannot tell you which system produced it: a changed prompt, tool or model can pass. For research reproducibility, evaluation reports need a record of the exact stack, the same way methods sections record software versions.","evidence":"Benchmark of 7,560 scored examples, six tamper types, evaluator ladder; acceptance rates 88.6% and 94.1%; trigger-conditional tamper 48-100% harmful outputs; fingerprinting 76.7%; formal bound plus manifest-hash comparison.","architectureImplications":"Agent deployments used in research should emit a manifest with hashes of model weights, tokenizer, prompts, tool specs, retrieval indexes and runtime packages, stored with evaluation results.","governanceImplications":"Research offices and journals reviewing AI-assisted results could require a deployment-identity descriptor alongside scores, and procurement could require vendors to supply one.","securityPrivacyImplications":"Shows trigger-conditional tampering can hide harmful behaviour from evaluations; manifest hashes help only for declared components and need protection against forgery.","caveats":"Synthetic, single-component tampering; small to mid-size models; no hardware attestation; preprint.","streamIds":["research"],"roles":{"sales":"The opening is evaluation provenance. This preprint shows that output scores, even from stronger LLM judges, accept most tampered agent deployments, so a benchmark result does not prove which system ran. For research integrity officers and CIOs buying research AI tools, a bounded offer is a review of how evaluations are recorded and what vendors must disclose. Do not claim the method stops attacks; it covers declared components only and was tested synthetically.","engineering":"Fit is a manifest of component hashes (weights, tokenizer, prompts, tools, retrieval data, runtime) stored with every evaluation and checked at deployment. It complements, not replaces, behavioural scoring. Constraints: only declared parts are covered and there is no hardware attestation. Proof of value should change one prompt or tool in a test deployment and confirm the manifest check fails while output scores pass, reproducing the paper's core contrast.","delivery":"Delivery is adding stack manifests to the institution's agent evaluation and deployment pipeline, plus a reporting template. Research computing would own the tooling and the research integrity office the reporting rule. Acceptance should be measured on coverage of components and detection of injected changes. Risks are undeclared components, forged manifests and treating a hash match as proof of safety."},"roleDetails":{"sales":{"customerProblem":"Institutions rely on agent evaluation scores without any evidence that the evaluated system is the one deployed or described in a paper.","stakeholders":["Research integrity officer","CIO or research computing director","AI tool procurement lead","Faculty publishing agent benchmarks"],"discoveryQuestions":["Do current evaluation reports record exact model, prompt, tool and data versions?","Do vendor contracts require notice when an AI tool's stack changes?","Who checks that a deployed research agent matches its evaluated version?"],"valueHypothesis":"Interpretation: recording stack manifests with evaluations may make agent-assisted results more reproducible and make silent changes detectable.","potentialEngagement":"A short review of evaluation and deployment records for research AI tools, with a manifest template and procurement language.","unsupportedClaims":["Protection against forged manifests or undeclared components","Detection of multi-component attacks","Results at frontier-model scale","That a passing hash means the system is safe or correct"]},"engineering":{"fit":"Relevant wherever agents are evaluated and then deployed for research tasks.","architectureIntegration":"Generate hashes for each stack component at evaluation; store with results; re-check at deployment and on updates; include a deployment-identity descriptor in reports.","prerequisites":["Inventory of agent stack components","Artifact storage for manifests","CI or deployment hooks"],"constraints":["Covers declared components only","No hardware-rooted attestation","Tested on synthetic tasks and 1.5B-20B models"],"security":"Protect manifests from tampering (signing, restricted write access) and alert on mismatches.","proofOfValue":["Inject a single-component change and confirm manifest check fails while output score passes","Measure share of stack components covered","Verify mismatch alerts reach owners"]},"delivery":{"work":"Add manifest generation and checks to evaluation and deployment pipelines and publish a reporting template.","dependencies":["Research computing pipelines","Vendor disclosure of stack components","Research integrity policy"],"ownership":"Research computing owns tooling; research integrity office owns reporting requirements.","skillsAdoption":"Train research software engineers and faculty on recording evaluation provenance.","governanceCheckpoints":["Template approved by research integrity office","Procurement clause reviewed by legal","Periodic audit of manifest coverage"],"acceptanceCriteria":["Proposal: manifests recorded for every published agent evaluation","Proposal: injected test changes detected","Proposal: vendor change notices received for contracted tools"],"risks":["Undeclared components escape checks","Manifests forged","False sense of safety from hash matches"]}},"retrievedAt":"2026-10-16T22:05:10-05:00","enrichedAt":"2026-10-16T22:12:24-05:00","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Research software engineers need to treat evaluation provenance as part of reproducibility training.","procurementImplications":"Contracts for AI research tools could require stack manifests and change notification so that a vendor update does not silently invalidate prior evaluation.","operatingModelImplications":"Evaluation becomes a continuing check tied to deployment identity, not a one-time score.","sourceVerification":{"openedUrl":"https://arxiv.org/abs/2610.21135","referenceExcerpt":"Stronger evaluators accept more tampered outputs: mean acceptance rises from 88.6% (rule-based) to 94.1% (GPT-5-nano)","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}