{"resourceId":"youra-evidence-traceable-research-agents-20261001","versions":[{"version":"external-f2871f38079ad41923fe89efc7847ffff203a3a0a401d6f5bb1e2d8214a6ff38","resource":{"id":"youra-evidence-traceable-research-agents-20261001","title":"YouRA: persistent verification state beats MLR-Agent and AI Scientist V2 on MLR-Bench Overall across three backbones","organization":"Electronics and Telecommunications Research Institute (ETRI) and University of Science and Technology (UST), Republic of Korea — arXiv preprint (cs.AI)","sector":"Higher education / national lab AI research agents for autonomous ML research workflows","geography":"Republic of Korea (authors); transferable methods for campus research-agent platforms with supervision limits","publishedAt":"October 1, 2026 (arXiv 2610.01097v1)","publicationDate":"2026-10-01","eventDate":null,"sourceName":"arXiv 2610.01097","sourceLabel":"Academic preprint (arXiv)","sourceUrl":"https://arxiv.org/abs/2610.01097","evidenceClass":"academic-research","outcomeClass":"mixed","topics":["developers-agents","operating-model","governance-procurement","infrastructure"],"finding":"YouRA (arXiv 2610.01097, submitted 1 Oct 2026) argues manuscript claims often diverge from executed experiments because research state, failure histories, and claim–evidence alignment are not persistent. It combines a Verification State Architecture (hypotheses, gates, evidence pointers), an Independent Controller separating control from execution, and Stateful Reflection for bounded repair/redesign/reset. On MLR-Bench’s predefined ten-task end-to-end subset, YouRA ranks first in 4-judge Overall on every matched backbone versus MLR-Agent and AI Scientist V2 (e.g., Sonnet 4.6 Overall 5.05 ± 0.92 vs 4.42 ± 1.11 and 3.88 ± 0.88; Opus 4.5 4.45 vs 3.62 and 4.28). Pairwise preferences also favor YouRA; ablations of VSA, controller, MCP tools, or reflection each lower Overall. Automated hallucination diagnostics still show substantial fabricated/unverified experimental-result flags across systems—so scalar Overall gains are not proof of trustworthy autonomy.","sledRelevance":"Campus research computing and research administration evaluating “AI scientist” platforms need persistent claim–evidence state and separated control—not chat-history agents that write fluent papers over unverified runs.","evidence":"arXiv abs HTML and PDF retrieved via HTTPS curl (User-Agent SLED-research-bot/1.0); Table 1 Overall scores and abstract claims confirmed in pdftotext; check_sources showed not archived.","architectureImplications":"Research-agent platforms should externalize hypothesis/experiment/claim state, separate lifecycle control from code-execution agents, and retain MCP/tool artifacts for audit—not rely on growing chat transcripts.","governanceImplications":"Gate any autonomous paper-producing loop on claim–evidence reviews and provenance diagnostics; treat LLM-judge Overall as decision support, not publication authority.","securityPrivacyImplications":"Independent controllers and tool allowlists reduce uncontrolled tool use; still require credential isolation and audit of evidence pointers before accepting results.","caveats":"Preprint; MLR-Bench ML-workshop tasks (ten-task end-to-end subset); LLM-judge rubrics; not a wet-lab or multi-campus HPC deployment study; hallucination flags remain non-trivial.","streamIds":["research"],"roles":{"sales":"Sales framing: treat claim–evidence drift in end-to-end research agents as a research-computing reliability buy, not a chatbot seat. YouRA (ETRI/UST; arXiv 2610.01097, 1 Oct 2026) externalizes hypotheses, gates, and evidence pointers in a Verification State Architecture, separates an Independent Controller from execution, and logs structured failure recovery. On MLR-Bench’s predefined ten-task end-to-end subset it beats MLR-Agent and AI Scientist V2 on scalar Overall across three matched backbones (e.g., Sonnet 4.6 Overall 5.05 vs 4.42/3.88). Pitch a supervised pilot that audits claim–evidence alignment—not paper fluency alone.","engineering":"Engineering framing: persistent research state plus an independent control plane is the integration surface. Wire a Verification State Architecture for hypotheses/gates/evidence pointers, keep lifecycle control out of the execution dialogue, and attach MCP tools with artifact stores. Match evaluation to MLR-Bench-style end-to-end Overall plus hallucination/provenance diagnostics; ablations show removing core-state components drops performance. Do not equate LLM-judge Overall gains with verified wet-lab or campus HPC safety.","delivery":"Delivery framing: stand up a supervised autonomous-research sandbox with persistent state, failure logs, and claim–evidence review gates before any unattended paper-producing loop on campus clusters. Dependencies include GPU/dev sandboxes, MCP tool allowlists, and research-software reviewers who can reject fluent manuscripts lacking evidence pointers. Acceptance should require backbone-matched Overall improvement and lower fabricated-result rates versus baseline scaffolds—not manuscript length or novelty rhetoric. Risks include judge-only scoring and persuasive fabrication that still passes scalar Overall."},"roleDetails":{"sales":{"customerProblem":"End-to-end research agents produce papers whose claims diverge from executed experiments when state is only in chat history.","stakeholders":["VPR / research computing","Research software engineers","PI computational groups","Research integrity offices"],"discoveryQuestions":["Where do hypothesis, experiment, and claim artifacts live today?","Which claim–evidence checks gate manuscript release?","What fabricated-result rate is tolerable in a pilot?"],"valueHypothesis":"Persistent verification state and separated control can raise end-to-end Overall versus released scaffolds while exposing provenance diagnostics.","potentialEngagement":"Six-week supervised pilot on a ten-task MLR-Bench-style campus subset with Overall, pairwise preference, and provenance reports.","unsupportedClaims":["Guaranteed elimination of fabricated results","Automatic wet-lab discovery","Campus-wide unattended autonomy"]},"engineering":{"fit":"Stateful evidence-traceable research agent with VSA, Independent Controller, MCP tools, and Stateful Reflection.","architectureIntegration":"Externalize hypotheses/gates/evidence pointers; separate control from execution dialogue; retain failure lessons for bounded repair/redesign/reset.","prerequisites":["Dev/GPU sandbox","MCP tool allowlist","Artifact/object store","MLR-Judge or equivalent rubric"],"constraints":["Preprint evidence","Ten-task end-to-end subset","LLM-judge Overall","ML research tasks not wet-lab"],"security":"Keep credentials and destructive tools behind controller policy; audit evidence pointers before accepting manuscript claims.","proofOfValue":["Overall gain vs matched baselines","Hallucination taxonomy counts","Real-data provenance share","Ablation drops when VSA/controller removed"]},"delivery":{"work":"Deploy supervised YouRA-like sandbox, encode claim–evidence gates, run matched backbone bake-off, review failure logs weekly.","dependencies":["Compute sandbox","Research computing IAM","RSE reviewers","Benchmark task pack"],"ownership":"Research computing + research software engineering jointly own the agent runtime; PIs own scientific acceptance.","skillsAdoption":"Train RSEs on VSA artifacts and judges; train PIs to reject fluent papers lacking evidence pointers.","governanceCheckpoints":["Pre-pilot security review","Mid-pilot provenance audit","Release gate on claim–evidence sample"],"acceptanceCriteria":["Highest Overall on matched backbones vs campus baseline","Documented reduction in faked-result flags on sample","Ablation confirms state components matter"],"risks":["Judge-score gaming","Persuasive fabrication","Scope creep to unsupervised publication"]}},"retrievedAt":"2026-10-01T22:08:28-05:00","enrichedAt":"2026-10-01T22:14:00-05:00","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Stateful agent tooling can offload workflow toil for computational researchers if RSEs own the control plane and faculty retain scientific acceptance.","procurementImplications":"When buying AI-scientist stacks, require persistent state, ablation-backed components, and provenance diagnostics—not demos of paper generation alone.","operatingModelImplications":"Operate research agents like scientific instruments: supervised pilots, artifact retention, and explicit stop/repair/reset authorities.","sourceVerification":{"openedUrl":"https://arxiv.org/abs/2610.01097","referenceExcerpt":"YouRA improves over both MLR-Agent and AI Scientist V2 on scalar Overall","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}