From the Research Universities edition of October 1, 2026
YouRA: persistent verification state beats MLR-Agent and AI Scientist V2 on MLR-Bench Overall across three backbones
Electronics and Telecommunications Research Institute (ETRI) and University of Science and Technology (UST), Republic of Korea — arXiv preprint (cs.AI) · Higher education / national lab AI research agents for autonomous ML research workflows · Republic of Korea (authors); transferable methods for campus research-agent platforms with supervision limits
- Publisher
- arXiv 2610.01097
- Original publication
- October 1, 2026 (arXiv 2610.01097v1)
- Source retrieved
- 2026-10-02
What happened
YouRA (arXiv 2610.01097, submitted 1 Oct 2026) argues manuscript claims often diverge from executed experiments because research state, failure histories, and claim–evidence alignment are not persistent. It combines a Verification State Architecture (hypotheses, gates, evidence pointers), an Independent Controller separating control from execution, and Stateful Reflection for bounded repair/redesign/reset. On MLR-Bench’s predefined ten-task end-to-end subset, YouRA ranks first in 4-judge Overall on every matched backbone versus MLR-Agent and AI Scientist V2 (e.g., Sonnet 4.6 Overall 5.05 ± 0.92 vs 4.42 ± 1.11 and 3.88 ± 0.88; Opus 4.5 4.45 vs 3.62 and 4.28). Pairwise preferences also favor YouRA; ablations of VSA, controller, MCP tools, or reflection each lower Overall. Automated hallucination diagnostics still show substantial fabricated/unverified experimental-result flags across systems—so scalar Overall gains are not proof of trustworthy autonomy.
Why it matters
Campus research computing and research administration evaluating “AI scientist” platforms need persistent claim–evidence state and separated control—not chat-history agents that write fluent papers over unverified runs.
Evidence and measured results
arXiv abs HTML and PDF retrieved via HTTPS curl (User-Agent SLED-research-bot/1.0); Table 1 Overall scores and abstract claims confirmed in pdftotext; check_sources showed not archived.
Limitations and uncertainty
Preprint; MLR-Bench ML-workshop tasks (ten-task end-to-end subset); LLM-judge rubrics; not a wet-lab or multi-campus HPC deployment study; hallucination flags remain non-trivial.
Put this evidence to work
Lighthouse Advisory interpretation, grounded in this source. Enriched 2026-10-01; this does not change the original publication date. Each role follows the same checklist so takeaways compare across sources.
Sales
Role takeaway
- Customer problem
- End-to-end research agents produce papers whose claims diverge from executed experiments when state is only in chat history.
- Stakeholders
- VPR / research computing
- Research software engineers
- PI computational groups
- Research integrity offices
- Discovery questions
- Where do hypothesis, experiment, and claim artifacts live today?
- Which claim–evidence checks gate manuscript release?
- What fabricated-result rate is tolerable in a pilot?
- Value hypothesis
- Persistent verification state and separated control can raise end-to-end Overall versus released scaffolds while exposing provenance diagnostics.
- Potential engagement
- Six-week supervised pilot on a ten-task MLR-Bench-style campus subset with Overall, pairwise preference, and provenance reports.
- Unsupported claims
- Guaranteed elimination of fabricated results
- Automatic wet-lab discovery
- Campus-wide unattended autonomy
Pre-sales engineering
Role takeaway
- Fit
- Stateful evidence-traceable research agent with VSA, Independent Controller, MCP tools, and Stateful Reflection.
- Architecture and integration
- Externalize hypotheses/gates/evidence pointers; separate control from execution dialogue; retain failure lessons for bounded repair/redesign/reset.
- Prerequisites
- Dev/GPU sandbox
- MCP tool allowlist
- Artifact/object store
- MLR-Judge or equivalent rubric
- Constraints
- Preprint evidence
- Ten-task end-to-end subset
- LLM-judge Overall
- ML research tasks not wet-lab
- Security
- Keep credentials and destructive tools behind controller policy; audit evidence pointers before accepting manuscript claims.
- Proof of value
- Overall gain vs matched baselines
- Hallucination taxonomy counts
- Real-data provenance share
- Ablation drops when VSA/controller removed
Delivery
Role takeaway
- Work
- Deploy supervised YouRA-like sandbox, encode claim–evidence gates, run matched backbone bake-off, review failure logs weekly.
- Dependencies
- Compute sandbox
- Research computing IAM
- RSE reviewers
- Benchmark task pack
- Ownership
- Research computing + research software engineering jointly own the agent runtime; PIs own scientific acceptance.
- Skills and adoption
- Train RSEs on VSA artifacts and judges; train PIs to reject fluent papers lacking evidence pointers.
- Governance checkpoints
- Pre-pilot security review
- Mid-pilot provenance audit
- Release gate on claim–evidence sample
- Acceptance criteria
- Highest Overall on matched backbones vs campus baseline
- Documented reduction in faked-result flags on sample
- Ablation confirms state components matter
- Risks
- Judge-score gaming
- Persuasive fabrication
- Scope creep to unsupervised publication
Implementation considerations
Lighthouse Advisory interpretation across the operating dimensions a public-sector buyer must settle before this evidence becomes a design. Each note answers the question under its heading for this specific source.
Architecture and integration
What must connect, and where does the AI sit in the workflow?
Research-agent platforms should externalize hypothesis/experiment/claim state, separate lifecycle control from code-execution agents, and retain MCP/tool artifacts for audit—not rely on growing chat transcripts.
Governance
Who approves, reviews and stays accountable for outcomes?
Gate any autonomous paper-producing loop on claim–evidence reviews and provenance diagnostics; treat LLM-judge Overall as decision support, not publication authority.
Security and privacy
What data, permissions and controls need testing?
Independent controllers and tool allowlists reduce uncontrolled tool use; still require credential isolation and audit of evidence pointers before accepting results.
Accessibility and workforce
Who is affected, and what skills or accommodations follow?
Stateful agent tooling can offload workflow toil for computational researchers if RSEs own the control plane and faculty retain scientific acceptance.
Procurement
What should contracts, pricing and exit terms secure?
When buying AI-scientist stacks, require persistent state, ablation-backed components, and provenance diagnostics—not demos of paper generation alone.
Operating model
Which teams own the service once it runs?
Operate research agents like scientific instruments: supervised pilots, artifact retention, and explicit stop/repair/reset authorities.
Publication history
- 2026-10-01Research Universities · Issue 244 resources
Stable resource ID: youra-evidence-traceable-research-agents-20261001