{"resourceId":"crossaudit-cross-vendor-agentic-science-20260805","versions":[{"version":"external-8c37e036a82d81edc2ca5349e3a54a18102c8eeba4a1f21040c32f4fb17a6285","resource":{"id":"crossaudit-cross-vendor-agentic-science-20260805","title":"CrossAudit: git-native cross-vendor audit loop so an AI scientist does not grade its own homework","organization":"University of Cambridge / University of Wisconsin–Madison (Dong & Chen)","sector":"AI scientist supervision / research integrity tooling","geography":"International academic authors; protocol transferable with vendor/API and confidentiality limits","publishedAt":"August 5, 2026 (arXiv; August 2026)","publicationDate":"2026-08-05","eventDate":null,"sourceName":"arXiv","sourceLabel":"Academic preprint — CrossAudit supervision protocol","sourceUrl":"https://arxiv.org/html/2608.28631","evidenceClass":"academic-research","outcomeClass":"emerging","topics":["governance-procurement","developers-agents","operating-model","data-security"],"finding":"CrossAudit specifies eight invariants for supervising agentic science: vendor-heterogeneous auditors, git ledger completeness (parsed tier), citation-valid reports, determinism-first checks, bounded revision, graded interruption, receipt binding, and fail-closed admission. Authors describe a reference implementation, a live computational-chemistry deployment, and an exploratory seeded-defect trial (30 increments, 43 defects) showing discordant configuration-level readings of the same rulebook; they explicitly state the protocol does not certify scientific truth.","sledRelevance":"Why now: Older than last run but newly relevant after 2026-09-20 capacity-vs-fidelity and this edition's WetRobo/SynAgent/Diamond agentic demos—supplies an independent supervision pattern campuses can adopt when AI scientists scale on university HPC and SDLs.","evidence":"Eight invariants; dual-repo separation; seeded trial 30/43; live chemistry deployment narrative; explicit non-claim of truth certification.","architectureImplications":"Split science and audit repositories; run DCL before LLM audit; gate protected actions on receipts.","governanceImplications":"Human principal owns Constitution; escalations interrupt humans; same-vendor self-critique is insufficient as sole control.","securityPrivacyImplications":"Fence increment content against prompt injection; second-vendor API confidentiality must be contracted; DCL hard fails non-waivable.","caveats":"Aug preprint with why-now; self-declared vendor identity; raw exchanges not ledgered; trial exploratory/blinding voided; observational deployment.","streamIds":["research"],"roles":{"sales":"Sales framing: Universities deploying AI scientists need supervision that is not the same vendor grading its own homework. CrossAudit evidence supports a bounded pitch around a git-native cross-vendor audit loop with deterministic checks first—reporting a protocol of eight invariants, a live computational-chemistry deployment, and an exploratory seeded-defect trial (30 increments, 43 defects) that shows discordant vendor readings of the same rulebook—not a claim that CrossAudit certifies scientific truth or eliminates correlated model error. Discovery should map generator/auditor vendors, Constitution ownership, and admission gates. Unsupported: cryptographic vendor attestation today, zero escalations economics without calibration, or fraud detection upstream of the repo.","engineering":"Engineering framing: Fit is a two-repository supervision protocol (Science vs Audit) with Constitution rules, deterministic check layer before LLM audit, receipts binding commit SHA/manifests, and fail-closed admission. Architecture integration points are GitHub Actions (or equivalent CI), OpenAI-compatible auditor endpoint, and append-only ledger commits—not an in-pipeline reflection agent alone. Prerequisites: two vendor API keys, branch protection for admission, and human-authored decidable rules. Constraints: vendor identity self-declared; raw model exchanges not yet ledgered; seeded trial blinding voided and exploratory; live deployment observational without control. Security: fence increment content; DCL non-overridable; escalate malformed audits. Proof of value is replayable ledger plus receipt-gated admission tests.","delivery":"Delivery framing: Stand up CrossAudit-style dual repos for one agentic research pipeline with a minimal Constitution and DCL, then measure escalation rate. Dependencies: research integrity, DevOps/CI, generator owners, and a human principal for escalations. Ownership: principal owns Constitution; platform owns admission wiring. Skills adoption teaches staff to write decidable rules and read ledgers instead of trusting internal critics. Governance checkpoints: Constitution change control, receipt verification before claim publication, and escalation triage. Acceptance: PASS/BLOCK receipts on pilot increments and written non-goals (no truth certification); risks include false-block floods, Goodharting the Constitution, and confidential data shipped to a second vendor API."},"roleDetails":{"sales":{"customerProblem":"Agentic research pipelines often use same-vendor critics and non-replayable logs, leaving institutions unable to inspect supervision.","stakeholders":["VPR/research integrity","Research computing","DevOps","Faculty running AI scientists"],"discoveryQuestions":["Which two vendors can be paired?","Who authors the Constitution?","What actions require receipt-gated admission?","What data may leave to a second API?"],"valueHypothesis":"Cross-vendor, git-ledgered audits with determinism-first checks can make supervision inspectable and reduce same-source self-preference exposure—without claiming scientific truth certification.","potentialEngagement":"Pilot dual-repo audit loop on one agentic pipeline with a small Constitution and admission gate on claim publication.","unsupportedClaims":["Certifies scientific truth","Eliminates all correlated model error","Guaranteed low escalation volume"]},"engineering":{"fit":"Git-native cross-vendor audit protocol with DCL precedence, citation-valid reports, receipts, and fail-closed admission.","architectureIntegration":"Science repo increments trigger audit-repo CI; auditor writes report+receipt; science side verifies receipt before protected actions.","prerequisites":["Two model vendors","CI + branch protection","Human Constitution","Deterministic check scripts"],"constraints":["Self-declared vendor identity","Parsed ledger without raw exchanges (rev1)","Exploratory trial limits","Confidentiality of increments to second vendor"],"security":"Fence untrusted increment text; never let models waive DCL hard fails; escalate empty/citation-free audits.","proofOfValue":["Receipt verification tests","Ledger replay of PASS/BLOCK cycles","Documented cross-vendor disagreement on identical inputs"]},"delivery":{"work":"Scaffold two repos, author Constitution v0, wire DCL+auditor, gate one protected action on receipts.","dependencies":["Research integrity principal","CI/DevOps","Generator team","Legal/data-governance for second-vendor API"],"ownership":"Principal owns rules; platform owns admission; generator owns increments.","skillsAdoption":"Train staff to write decidable rules and triage escalations asynchronously.","governanceCheckpoints":["Constitution change review","Admission binding check","Escalation SLA"],"acceptanceCriteria":["Receipt-gated protected action","Sample ledger published or retained","Written non-goals on truth certification"],"risks":["False-block storms","Goodharting rules","Data leakage via auditor API"]}},"retrievedAt":"2026-09-22T03:18:53Z","enrichedAt":"2026-09-22T03:18:53Z","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Redeploys scarce human attention to Constitutions and escalations rather than every increment; requires rule-authoring skills.","procurementImplications":"Require dual-vendor audit options and receipt-gated claim publication in AI-scientist procurements—not internal reflection modules alone.","operatingModelImplications":"Management-by-exception model only works if auditor calibration keeps escalations << increments.","sourceVerification":{"openedUrl":"https://arxiv.org/html/2608.28631","referenceExcerpt":"An AI scientist should not grade its own homework.","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}