{"resourceId":"rac-runtime-ai-scientist-coordination-20261001","versions":[{"version":"external-f2871f38079ad41923fe89efc7847ffff203a3a0a401d6f5bb1e2d8214a6ff38","resource":{"id":"rac-runtime-ai-scientist-coordination-20261001","title":"RAC: runtime agent selection tops native means on ResearchClawBench; contracts/verification cut those means under budgets","organization":"University of Oxford, KAUST, University of Sydney, University of Cambridge — arXiv preprint (cs.MA)","sector":"Higher education multi-agent AI scientist platforms / research workflow orchestration","geography":"United Kingdom / Saudi Arabia / Australia (authors); methods transferable to campus agent platforms with budget limits","publishedAt":"October 1, 2026 (arXiv 2610.00980v1)","publicationDate":"2026-10-01","eventDate":null,"sourceName":"arXiv 2610.00980","sourceLabel":"Academic preprint (arXiv)","sourceUrl":"https://arxiv.org/abs/2610.00980","evidenceClass":"academic-research","outcomeClass":"mixed","topics":["developers-agents","operating-model","infrastructure","governance-procurement"],"finding":"Runtime Agent Coordination (RAC; arXiv 2610.00980, 1 Oct 2026) asks whether AI scientists can reallocate work at runtime like humans. RAC selects agents from existing hosts (Agent Laboratory, EvoScientist, ARK), assigns scoped contracts, and adds artifact-grounded verification without blocking transitions. In a single-seed exploratory evaluation on ResearchClawBench under host-calibrated budgets, runtime selection (R2) yielded the highest observed mean score for each host (Agent Lab 12.08, EvoScientist 18.53, ARK 18.42 vs native 5.47 / 15.99 / 16.66). Full RAC with contracts/verification (R3) reduced those means (9.88 / 15.96 / 17.98) with host-dependent token/cost changes—Agent Lab cost fell while ARK and EvoScientist costs rose. Task-level results mix gains and declines; authors frame results as motivating runtime coordination while exposing limits of heavier mechanisms under constrained budgets.","sledRelevance":"Universities already running heterogeneous AI-scientist hosts should test runtime selection before buying heavier contract/verification layers that may spend the research budget without raising scores.","evidence":"arXiv abs HTML and PDF via HTTPS curl; Tables 2–3 mean scores and abstract claims confirmed in pdftotext; check_sources not archived.","architectureImplications":"Prefer host-preserving runtime bridges (messaging → selection → optional contracts/verification) over replacing campus agent stacks; meter tokens/cost per condition.","governanceImplications":"Treat single-seed exploratory scores as directional; require multi-seed and domain regression reviews before operationalizing coordination policies.","securityPrivacyImplications":"Scoped contracts must not widen host permissions; verification should consume artifacts already produced under existing IAM.","caveats":"Preprint; single-seed exploratory evaluation; host-calibrated budgets; mixed per-task outcomes; not a campus production deployment study.","streamIds":["research"],"roles":{"sales":"Sales framing: runtime coordination across existing AI-scientist hosts is a research-platform ops question, not a greenfield multi-agent rewrite. RAC (Oxford/KAUST/Sydney/Cambridge; arXiv 2610.00980, 1 Oct 2026) selects agents at runtime, assigns scoped contracts, and adds artifact-grounded verification without blocking transitions. In a single-seed exploratory evaluation on ResearchClawBench, runtime selection (R2) produced the highest mean score for Agent Laboratory, EvoScientist, and ARK; adding contracts/verification (R3) reduced those means and showed host-dependent cost tradeoffs. Offer a bounded bake-off before committing to heavier coordination layers.","engineering":"Engineering framing: bridge existing hosts rather than replacing them. Implement runtime communication, agent selection, then optional contracts and non-blocking artifact verification under host-calibrated budgets. Preserve host models, tools, and permissions. Expect mixed task-level gains and declines even when means rise (R2 +6.61 Agent Lab, +2.55 EvoScientist, +1.76 ARK vs native). Measure tokens/cost per host; R3 raised costs for ARK and EvoScientist while cutting Agent Laboratory spend.","delivery":"Delivery framing: run a single-seed comparative evaluation of native versus R1/R2/R3 coordination on a fixed ResearchClawBench-like task pack before any production rollout. Dependencies include host bridges, budget meters, and artifact stores owned by research computing. Acceptance should show mean gains without unacceptable task regressions or cost blowups across domains. Risks include single-seed exploratory results, host-specific failures when verification overhead consumes budget, and non-uniform benefits that look good in averages only."},"roleDetails":{"sales":{"customerProblem":"Fixed design-time multi-agent workflows underperform human scientists who reallocate work at runtime.","stakeholders":["Research computing directors","AI platform owners","PI groups using agent hosts","FinOps / research admin"],"discoveryQuestions":["Which AI-scientist hosts are already licensed?","What is the per-task budget ceiling?","Which domains show regressions under heavier coordination?"],"valueHypothesis":"Runtime agent selection on existing hosts can raise ResearchClawBench means without replacing native tools—if budgets absorb verification overhead.","potentialEngagement":"Four-week single-seed bake-off of native vs runtime selection vs contracts/verification on a ten-task campus pack.","unsupportedClaims":["Uniform gains on every task","Production readiness from single-seed results","Cost reductions for every host"]},"engineering":{"fit":"Runtime Agent Coordination layered onto Agent Laboratory / EvoScientist / ARK-style hosts.","architectureIntegration":"Add host bridges for runtime messaging, selection, scoped contracts, and non-blocking artifact-grounded verification under calibrated budgets.","prerequisites":["Existing AI-scientist host","Budget/token metering","Artifact store","Task pack with weighted scores"],"constraints":["Single-seed exploratory evaluation","Host-calibrated budgets","Mixed task-level gains/declines","Preprint"],"security":"Scoped work contracts and verification should not expand tool permissions beyond the host’s native policy.","proofOfValue":["R2 mean > N0 per host","Task-level win/loss tally","Token and USD deltas","DiscoveryBench transfer subset if used"]},"delivery":{"work":"Instrument hosts, run N0/R1/R2/R3 conditions at fixed seed, review regressions and cost, decide whether to keep selection-only or full RAC.","dependencies":["Host licenses","GPU/API budgets","Evaluation harness","Research ops owner"],"ownership":"AI platform / research computing owns coordination layer; domain PIs accept scientific outputs.","skillsAdoption":"Train operators to read contract/verification artifacts and to roll back when R3 regresses domain scores.","governanceCheckpoints":["Budget approval","Mid-bake-off regression review","Go/no-go on R2 vs R3"],"acceptanceCriteria":["Highest mean under R2 or justified R3","No critical domain score collapse","Cost within approved band"],"risks":["Single-seed overfit","Verification overhead starving execution","Opaque host-bridge bugs"]}},"retrievedAt":"2026-10-01T22:08:28-05:00","enrichedAt":"2026-10-01T22:14:00-05:00","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Runtime coordination can let smaller teams reuse existing agent hosts if operators are trained to interpret selection and verification artifacts.","procurementImplications":"Buy coordination layers only after a bake-off shows mean gains without cost blowups; do not procure on design-time workflow diagrams alone.","operatingModelImplications":"Operate multi-agent science like a scheduler problem: selection under budget, with explicit rollback when verification overhead regresses domain scores.","sourceVerification":{"openedUrl":"https://arxiv.org/abs/2610.00980","referenceExcerpt":"Runtime selection yields the highest observed mean score for each host","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}