{"resourceId":"biorxiv-verifiable-agentic-science-galaxy-provenance-20261028","versions":[{"version":"external-1a8840ef1399d7586d36b7e0d55ff732f6806ec9fbde797e5d632d6381f9de96","resource":{"id":"biorxiv-verifiable-agentic-science-galaxy-provenance-20261028","title":"Preprint: 30 coding agents split into 18 different pipelines on one RNA-seq study; Galaxy team argues agentic science must run on public infrastructure that records provenance","organization":"Galaxy Project team at Johns Hopkins, Penn State, Temple, TACC, Moffitt Cancer Center and University of Freiburg","sector":"Research computing, bioinformatics and reproducibility","geography":"United States and Germany; public Galaxy infrastructure used internationally","publishedAt":"28 October 2026 (bioRxiv posting)","publicationDate":"2026-10-28","eventDate":null,"sourceName":"bioRxiv","sourceLabel":"Baker et al., \"Verifiable Agentic Science on Public Infrastructure,\" bioRxiv, posted 28 Oct 2026","sourceUrl":"https://www.biorxiv.org/content/biorxiv/early/2026/10/28/2026.10.26.761864.full.pdf","evidenceClass":"academic-research","outcomeClass":"cautionary","topics":["developers-agents","infrastructure","data-security"],"finding":"The authors asked 30 independent agents (10 each from Claude Code with Claude Opus 5.5, Codex with GPT-6 Astra and Google Antigravity with Gemini 3.8 Flash) to plan a replication of a published Bacillus cereus RNA-seq study whose methods left the reference assembly and sample pooling unstated. Twenty-nine of 30 plans diverged from the published workflow, splitting into 18 distinct eight-step pipelines and 11 designs across five result-changing choices. One agent per provider then ran its plan locally; all recovered the headline gene changes, but differing reference genomes silently changed the detected toxin genes, and the runs left 26 unversioned scripts totalling over 1,200 lines. The team then connected agents to public Galaxy servers and national supercomputers through the Galaxy MCP server and their Orbit desktop workbench, recording every tool, parameter and dataset, and reproduced a range of published studies, which they say exposed unstated choices in the originals.","sledRelevance":"Universities are seeing researchers adopt command-line coding agents as their main working environment. This study gives research computing and research integrity leaders concrete evidence that unguided local agent analyses diverge and leave no record, and a design for shared, provenance-recording infrastructure.","evidence":"Read the abstract, introduction, planning benchmark, local-execution comparison and discussion limitations in the full-text PDF. Later case studies were skimmed; supplements were not opened.","architectureImplications":"Agents call containerized tools and reviewed workflows on shared servers through MCP, so execution, versions and datasets are recorded server-side.","governanceImplications":"Provenance by default supports reproducibility and audit of AI-assisted analysis; agent-level metadata (which model and settings) is not yet recorded server-side.","securityPrivacyImplications":"Running on public servers suits public data; controlled or sensitive data would need institutional instances.","caveats":"Preprint, not peer reviewed, written by the developers of Galaxy and Orbit. The planning benchmark used 10 runs per model family on one dataset; live comparisons and case studies involved investigator steering. Public infrastructure does not remove model execution failures.","streamIds":["research"],"roles":{"sales":"Lead with reproducibility risk from researchers' coding agents. A 28 October bioRxiv preprint found 30 coding agents split into 18 different pipelines when planning the same RNA-seq replication, and local runs left 26 unversioned scripts. Research computing and integrity leaders need a sanctioned, recorded path for agent analyses. Offer a bounded assessment of agent use and provenance options. Avoid claiming any platform eliminates errors; the authors built the system they evaluate.","engineering":"Frame fit around routing agent execution through a workflow platform that records provenance. The team connected agents to Galaxy through an MCP server so tools, versions, parameters and datasets are logged server-side. A campus version needs a Galaxy or similar instance, containerized tools and HPC integration. Validate by giving several agents the same task with and without the platform and comparing pipeline divergence and recoverability of the record.","delivery":"Treat delivery as standing up a recorded execution path for agent-assisted analysis. Work covers deploying or connecting a workflow platform, exposing it to agents through MCP, onboarding labs and setting data policies. Ownership sits with research computing, with research integrity input. Governance checkpoints include adding model and settings metadata, which the paper says is missing. Acceptance criteria are proposals, such as every agent analysis yielding a shareable history, not reported results."},"roleDetails":{"sales":{"customerProblem":"Researchers run analyses with local coding agents that leave no verifiable record and choose different pipelines for the same data.","stakeholders":["Research computing director","Research integrity officer","Bioinformatics core lead","Vice president for research"],"discoveryQuestions":["How many labs use coding agents for data analysis today?","Is there a sanctioned workflow platform with provenance?","Which datasets cannot go to public servers?"],"valueHypothesis":"A recorded execution path may make agent-assisted analyses reviewable and reproducible.","potentialEngagement":"An assessment of agent use plus a pilot connecting agents to an institutional workflow platform.","unsupportedClaims":["That provenance prevents agent errors","Generalization beyond the tested bioinformatics tasks","Independent validation of the platform"]},"engineering":{"fit":"Fits institutions with life-science computing and growing agent use.","architectureIntegration":"Coding agents connected via MCP to a workflow platform running containerized tools on HPC, with histories linked to notebooks.","prerequisites":["Galaxy or comparable workflow platform","Containerized tool catalog","HPC integration"],"constraints":["Sensitive data needs institutional instances","Agent-level metadata not yet captured server-side","Evidence from the platform developers"],"security":"Use institutional instances for controlled data and authenticate agent connections.","proofOfValue":["Compare pipeline divergence with and without the platform","Check that each run yields a complete history","Measure researcher effort to review a run"]},"delivery":{"work":"Deploy or connect the platform, expose it to agents, onboard labs and define data policies.","dependencies":["Platform hosting","Tool containers","Lab participation"],"ownership":"Research computing with research integrity oversight.","skillsAdoption":"Researchers learn to steer agents and review recorded histories rather than trust summaries.","governanceCheckpoints":["Data classification before public-server use","Capture of model and settings metadata","Review of agent-registered custom tools"],"acceptanceCriteria":["Every pilot analysis produces a shareable history (proposal)","Agreed reduction in untracked scripts","Data policy compliance audited"],"risks":["Researchers bypassing the platform","Silent analytical choices persisting","Platform capacity limits"]}},"retrievedAt":"2026-10-29T22:01:29-05:00","enrichedAt":"2026-10-29T22:01:33-05:00","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Lets researchers without cluster credentials or pipeline skills use large compute, which raises the need for training in checking agent choices.","procurementImplications":"Builds on free public Galaxy servers and national supercomputers; institutions may run their own Galaxy instances.","operatingModelImplications":"Research computing could offer a sanctioned, recorded execution path for agent-driven analyses.","peerReview":"preprint","sourceVerification":{"openedUrl":"https://www.biorxiv.org/content/biorxiv/early/2026/10/28/2026.10.26.761864.full.pdf","referenceExcerpt":"After showing that 30 unguided local agents splinter into 18 incompatible pipelines on the same dataset","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}