{"resourceId":"intosyn-bioagent-df-decision-provenance-audit-20261101","versions":[{"version":"external-3e0b6b977c0fe359bf762e5839c5567e473d67691a8eba9fa8ec953de4b882b8","resource":{"id":"intosyn-bioagent-df-decision-provenance-audit-20261101","title":"From Tool State to Scientific Action: Auditing Decision Provenance in Biomedical Research Agents (BioAgent-DF)","organization":"INTOSYN AI Labs, with authors affiliated with Columbia University and Microsoft","sector":"Research enterprise (biomedical research agents, GPU reliability and audit)","geography":"International (industry lab with U.S. university co-author); laboratory-style experiments","publishedAt":"1 November 2026 (arXiv submission)","publicationDate":"2026-11-01","eventDate":null,"sourceName":"arXiv","sourceLabel":"arXiv 2611.01527, Yi, Zhang, Zhou, Bao, \"From Tool State to Scientific Action: Auditing Decision Provenance in Biomedical Research Agents,\" submitted 1 Nov 2026","sourceUrl":"https://arxiv.org/abs/2611.01527","evidenceClass":"independent-research","outcomeClass":"cautionary","topics":["developers-agents","infrastructure","data-security"],"finding":"Using values captured from faulty GPUs (silent data corruption that passes NaN/Inf checks), the authors show research agents can give the same verdict while reading and citing different evidence. In a hERG cardiac-safety assay allocation task, silent numerical faults changed testing decisions while the shortlist kept the same number of labeled blockers. Replacing the prediction file after the agent read faulty values recovered none of 39 changed decisions; restoring the earlier execution state with certified scores recovered all 39. Streaming arithmetic checks cut verification-plus-forward time by 42.4% versus the original FP64 implementation.","sledRelevance":"Relevant to research computing centers and autonomous-lab programs: audits of AI research agents should log evidence paths and execution state, and hardware faults on shared GPU clusters can silently alter scientific decisions. Limited relevance outside institutions running agents on GPU clusters.","evidence":"Controlled interventions on saved tool states with healthy-hardware continuation; SciFact retrieval with two 4B models across eight repeated batches; hERG allocation with 16 saved fault sources and 114 agent trajectories; recovery rules developed on a retrospective series then confirmed. Effects concentrated on selected captures: without selection on an upstream change, candidate discordance was no higher than healthy repeat variation. Preprint by an industry lab, not peer reviewed.","architectureImplications":"Verification belongs at the point where an agent decision is committed: arithmetic checks on tool outputs, trusted recomputation, saved execution state for rollback, and a commit service that only accepts certified actions.","governanceImplications":"Audit records should include what an agent read and cited, not only its final answer; a complete provenance record can faithfully preserve wrong values, so independent checks are needed.","securityPrivacyImplications":"Silent data corruption is an integrity risk rather than a confidentiality one; restoring visible history and invalidating caches is part of the remedy.","caveats":"Small models (4B) and specific faulty GPUs; repeatable evidence-path changes covered only two SciFact claims; R1+M margin rule is empirical; authors also release a GPU test tool, a possible interest; no prospective lab study.","streamIds":["research"],"roles":{"sales":"Integrity framing: a research agent can be right for the wrong reasons, and faulty GPUs can change which experiments get run without changing headline metrics. This supports a bounded conversation with research computing and autonomous-lab leads about decision-level verification and rollback. Interpretation: the evidence is controlled and narrow, so it cannot support claims that clusters are routinely corrupting science or that a product eliminates the risk.","engineering":"Fit is for agent pipelines on shared GPU clusters feeding experiment decisions. Integration points are arithmetic checks on tool outputs, trusted recomputation for flagged or near-boundary scores, saved agent execution state, and a commit gate. Prerequisites include GPU health testing and state checkpointing. Constraints: results used 4B models and selected fault captures. A proof of value should compare decision-level recovery with file-replacement recovery on the institution's own pipeline.","delivery":"Delivery work is adding decision provenance to an existing research-agent workflow: log reads, citations and actions; checkpoint state before tool reads; add a certified-commit step before instrument or assay instructions. Ownership sits with research computing and the lab running the agent. Checkpoints include GPU health screening and review of near-threshold decisions. Acceptance criteria are proposals such as complete evidence-path logs; risks include added token cost of about 29,000 per restored request."},"roleDetails":{"sales":{"customerProblem":"Labs adopting AI research agents lack ways to detect when silent hardware faults or tool errors change decisions without changing aggregate metrics.","stakeholders":["Research computing/HPC operations","Autonomous lab director","PI running agent pipelines","Research integrity office"],"discoveryQuestions":["Do your agent logs record evidence read and cited, or only final answers?","How are GPUs screened for silent data corruption?","Can agent state be rolled back before a tool read?","Which agent decisions trigger experiments or assays?"],"valueHypothesis":"Decision-level checks and state rollback could prevent wrong experiments from being committed, at modest verification overhead in the reported implementation.","potentialEngagement":"An assessment of one agent-driven workflow for provenance logging, GPU health screening and a commit gate.","unsupportedClaims":["Shared clusters routinely corrupt research results","Results generalize to large models","A single tool removes the risk"]},"engineering":{"fit":"Strong for agent pipelines that commit experiments from GPU-computed scores; low for chat-style assistants.","architectureIntegration":"Arithmetic checks on tool outputs, trusted recomputation, checkpointed agent state, certified commit service.","prerequisites":["GPU health/SDC screening","Agent state checkpointing","Structured logs of reads, citations and actions","Trusted hardware for recomputation"],"constraints":["Evidence from 4B models and selected faults","Margin rule is empirical, not a guarantee","Extra token and compute cost for recovery"],"security":"Treat tool outputs as untrusted until checked; invalidate caches and dependent state after rollback; restrict who can commit instrument instructions.","proofOfValue":["Inject known faulty values and confirm commit gate blocks changed decisions","Compare file replacement versus state restoration","Measure verification overhead"]},"delivery":{"work":"Instrument agent workflow with evidence-path logging and checkpoints; Add certified-commit step before experiment instructions","dependencies":["HPC operations","Agent framework support for state save/restore","Trusted recomputation capacity"],"ownership":"Research computing operations, with the lab PI accountable for committed experiments.","skillsAdoption":"HPC staff and lab users need training on SDC, provenance logs and rollback procedures.","governanceCheckpoints":["GPU health screening before agent workloads","Review of near-threshold decisions","Change control for commit-gate rules"],"acceptanceCriteria":["Proposed: every committed agent action has a logged evidence path","Proposed: fault-injection test blocks 100% of seeded changed decisions","Proposed: verification overhead measured and reported"],"risks":["False sense of security from provenance logs","Recovery token cost","Vendor or tool lock-in"]}},"retrievedAt":"2026-11-03T22:06:02-06:00","enrichedAt":"2026-11-03T22:07:06-06:00","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"HPC staff need SDC awareness and rollback skills; lab staff need to read evidence-path logs.","procurementImplications":"GPU purchases and cluster contracts could specify health testing and fault reporting; the paper names no procurement terms.","operatingModelImplications":"Places responsibility for agent decision integrity jointly on research computing and the lab, at the commit point.","peerReview":"preprint","sourceVerification":{"openedUrl":"https://arxiv.org/abs/2611.01527","referenceExcerpt":"After the agent had used faulty values, restoring its earlier execution state and supplying certified scores recovered all 39 changed decisions","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}