From the Research Universities edition of October 29, 2026
Preprint: 30 coding agents split into 18 different pipelines on one RNA-seq study; Galaxy team argues agentic science must run on public infrastructure that records provenance
Galaxy Project team at Johns Hopkins, Penn State, Temple, TACC, Moffitt Cancer Center and University of Freiburg · Research computing, bioinformatics and reproducibility · United States and Germany; public Galaxy infrastructure used internationally
- Publisher
- bioRxiv
- Original publication
- 28 October 2026 (bioRxiv posting)
- Source retrieved
- 2026-10-30
What happened
The authors asked 30 independent agents (10 each from Claude Code with Claude Opus 5.5, Codex with GPT-6 Astra and Google Antigravity with Gemini 3.8 Flash) to plan a replication of a published Bacillus cereus RNA-seq study whose methods left the reference assembly and sample pooling unstated. Twenty-nine of 30 plans diverged from the published workflow, splitting into 18 distinct eight-step pipelines and 11 designs across five result-changing choices. One agent per provider then ran its plan locally; all recovered the headline gene changes, but differing reference genomes silently changed the detected toxin genes, and the runs left 26 unversioned scripts totalling over 1,200 lines. The team then connected agents to public Galaxy servers and national supercomputers through the Galaxy MCP server and their Orbit desktop workbench, recording every tool, parameter and dataset, and reproduced a range of published studies, which they say exposed unstated choices in the originals.
Why it matters
Universities are seeing researchers adopt command-line coding agents as their main working environment. This study gives research computing and research integrity leaders concrete evidence that unguided local agent analyses diverge and leave no record, and a design for shared, provenance-recording infrastructure.
Evidence and measured results
Read the abstract, introduction, planning benchmark, local-execution comparison and discussion limitations in the full-text PDF. Later case studies were skimmed; supplements were not opened.
Limitations and uncertainty
Preprint, not peer reviewed, written by the developers of Galaxy and Orbit. The planning benchmark used 10 runs per model family on one dataset; live comparisons and case studies involved investigator steering. Public infrastructure does not remove model execution failures.
Put this evidence to work
Lighthouse Advisory interpretation, grounded in this source. Enriched 2026-10-29; this does not change the original publication date. Each role follows the same checklist so takeaways compare across sources.
Sales
Role takeaway
- Customer problem
- Researchers run analyses with local coding agents that leave no verifiable record and choose different pipelines for the same data.
- Stakeholders
- Research computing director
- Research integrity officer
- Bioinformatics core lead
- Vice president for research
- Discovery questions
- How many labs use coding agents for data analysis today?
- Is there a sanctioned workflow platform with provenance?
- Which datasets cannot go to public servers?
- Value hypothesis
- A recorded execution path may make agent-assisted analyses reviewable and reproducible.
- Potential engagement
- An assessment of agent use plus a pilot connecting agents to an institutional workflow platform.
- Unsupported claims
- That provenance prevents agent errors
- Generalization beyond the tested bioinformatics tasks
- Independent validation of the platform
Pre-sales engineering
Role takeaway
- Fit
- Fits institutions with life-science computing and growing agent use.
- Architecture and integration
- Coding agents connected via MCP to a workflow platform running containerized tools on HPC, with histories linked to notebooks.
- Prerequisites
- Galaxy or comparable workflow platform
- Containerized tool catalog
- HPC integration
- Constraints
- Sensitive data needs institutional instances
- Agent-level metadata not yet captured server-side
- Evidence from the platform developers
- Security
- Use institutional instances for controlled data and authenticate agent connections.
- Proof of value
- Compare pipeline divergence with and without the platform
- Check that each run yields a complete history
- Measure researcher effort to review a run
Delivery
Role takeaway
- Work
- Deploy or connect the platform, expose it to agents, onboard labs and define data policies.
- Dependencies
- Platform hosting
- Tool containers
- Lab participation
- Ownership
- Research computing with research integrity oversight.
- Skills and adoption
- Researchers learn to steer agents and review recorded histories rather than trust summaries.
- Governance checkpoints
- Data classification before public-server use
- Capture of model and settings metadata
- Review of agent-registered custom tools
- Acceptance criteria
- Every pilot analysis produces a shareable history (proposal)
- Agreed reduction in untracked scripts
- Data policy compliance audited
- Risks
- Researchers bypassing the platform
- Silent analytical choices persisting
- Platform capacity limits
Implementation considerations
Lighthouse Advisory interpretation across the operating dimensions a public-sector buyer must settle before this evidence becomes a design. Each note answers the question under its heading for this specific source.
Architecture and integration
What must connect, and where does the AI sit in the workflow?
Agents call containerized tools and reviewed workflows on shared servers through MCP, so execution, versions and datasets are recorded server-side.
Governance
Who approves, reviews and stays accountable for outcomes?
Provenance by default supports reproducibility and audit of AI-assisted analysis; agent-level metadata (which model and settings) is not yet recorded server-side.
Security and privacy
What data, permissions and controls need testing?
Running on public servers suits public data; controlled or sensitive data would need institutional instances.
Accessibility and workforce
Who is affected, and what skills or accommodations follow?
Lets researchers without cluster credentials or pipeline skills use large compute, which raises the need for training in checking agent choices.
Procurement
What should contracts, pricing and exit terms secure?
Builds on free public Galaxy servers and national supercomputers; institutions may run their own Galaxy instances.
Operating model
Which teams own the service once it runs?
Research computing could offer a sanctioned, recorded execution path for agent-driven analyses.
Publication history
- 2026-10-29Research Universities · Issue 514 resources
Stable resource ID: biorxiv-verifiable-agentic-science-galaxy-provenance-20261028