{"resourceId":"vibe-researching-systematic-review-silent-failure-20261021","versions":[{"version":"external-5d5f56ff8fefcd7869fe04249a2625bfdd083da3f0bdafb4db724b99bb148326","resource":{"id":"vibe-researching-systematic-review-silent-failure-20261021","title":"\"Vibe Researching\" systematic review (MetaArXiv, 21 Oct): across 496 studies, AI research agents present erroneous outputs as complete without warning; safeguards worked in 16 of 41 independent tests versus 56 of 82 developer tests","organization":"University of Hamburg Institute for Computational Systems Biology / Galagos GmbH (single author, Fernando M. Delgado-Chaves)","sector":"Research integrity and AI-assisted research methods","geography":"Germany (international evidence base; applicable to U.S. research institutions)","publishedAt":"21 October 2026 (OSF MetaArXiv preprint v1)","publicationDate":"2026-10-21","eventDate":null,"sourceName":"MetaArXiv (OSF Preprints)","sourceLabel":"Delgado-Chaves, \"Vibe Researching: A PRISMA 2020 Systematic Review of Silent Failure, Reproducibility, and Competence in LLM-Assisted Research\", MetaArXiv, 21 October 2026","sourceUrl":"https://osf.io/preprints/metaarxiv/7vsbw_v1/","evidenceClass":"academic-research","outcomeClass":"cautionary","topics":["developers-agents","knowledge-work","governance-procurement"],"finding":"A PRISMA 2020 systematic review of 496 studies (searched to 7 Oct 2026) on LLM agents that search, analyse and draft research under human direction. 256 measured outcomes on research tasks, and 167 of 364 empirical studies were evaluated by the system's own developers. With moderate certainty (high on a strict subset of independent research-task studies, mostly autonomous runs), AI systems doing research tasks presented erroneous outputs as complete without any error signal. Evidence that they make consequential errors was low certainty overall because over half the supporting studies evaluated their own systems. Final outputs omitted process errors visible in traces. Of empirical studies discussing a safeguard, 48% left it untested; tested safeguards were effective in 56 of 82 developer evaluations but only 16 of 41 independent ones. Findings about the people directing these systems and about benefits had very low certainty. In the review's own AI-agent-produced workflow, 9.5% of drafted claims were not fully supported by their sources until checked. How often such errors occur in everyday research, and how this compares with human-only work, is unmeasured.","sledRelevance":"Gives research offices, integrity officers and graduate schools a systematic evidence base for policies on AI agents in research: unsignalled error is documented, developer-reported safeguard success overstates independent results, and benefits remain weakly evidenced. It supports requiring traces, repeated runs and citation checks while being honest that these are precautions, not proven interventions.","evidence":"Read the OSF API record (published 21 Oct 2026) and the full preprint PDF: abstract (496 studies; 256 on research tasks; 167 of 364 developer-evaluated; 48% safeguards untested; 56/82 vs 16/41; 9.5% unsupported drafted claims; kappa 0.59 and 0.42), methods section 2.1 (internal unregistered protocol, author's company builds tools in a recommended product category), AI-use disclosure (screening, extraction and coding by agents of one model without human dual review; author spot checks not systematic) and the limitations section headings.","architectureImplications":"Research AI workflows should retain execution traces, support repeated runs, and include automated citation-to-source checks, since final outputs omitted errors visible in traces.","governanceImplications":"Institutional AI-in-research guidance should require disclosure, trace retention and independent verification of AI-produced claims, and should discount developer-reported safeguard effectiveness.","securityPrivacyImplications":"Limited direct relevance; the review concerns correctness and integrity rather than data protection.","caveats":"Single-author preprint, not peer reviewed and not registered (retrospective registration prepared but not submitted). The review was itself produced largely by AI agents of one model without human dual review; inter-coder agreement was moderate (kappa 0.59 and 0.42) and the author's spot checks were not systematic. The author's company builds tools in a product category some recommendations favour (disclosed). Real-world error rates and comparison with human-only research are unmeasured.","streamIds":["research"],"roles":{"sales":"Lead with verification, not productivity. This 21 October systematic review of 496 studies finds AI research agents often present wrong outputs as complete without warning, and safeguards that worked in 56 of 82 developer tests worked in only 16 of 41 independent ones. Research integrity officers and vice-presidents for research need defensible AI-in-research policies. Interpretation: a bounded policy and controls review fits; this evidence cannot support claims that any safeguard prevents errors or that agents improve research quality.","engineering":"Frame the fit around observability for research agents. The review found final outputs omitted process errors visible in traces, and recommends kept traces, repeated runs and citation checks as precautions. Pre-sales should ask whether current tools log full agent trajectories, can rerun analyses reproducibly and can check citations against sources. Interpretation: build these controls into any agent deployment, but note the review itself was agent-produced, unregistered and not peer reviewed, so treat its certainty ratings cautiously.","delivery":"Treat delivery as operationalising verification in research workflows. Work covers drafting AI-use and disclosure guidance, configuring trace retention and citation-checking steps, and training researchers to review agent outputs. The research integrity office owns policy, with research computing operating the tooling. Interpretation: acceptance should track the share of AI-assisted outputs with retained traces and completed checks, since the review shows error rates in everyday research are not yet measured."},"roleDetails":{"sales":{"customerProblem":"Research leaders lack an evidence-based basis for policies on AI agents that search, analyse and draft research.","stakeholders":["Vice-president for research","Research integrity officer","Graduate school dean","Research computing director"],"discoveryQuestions":["How are researchers using AI agents for analysis or drafting today?","What disclosure and verification rules exist?","Do tools retain traces that reviewers can inspect?"],"valueHypothesis":"Clear verification controls may reduce the risk of unsignalled AI errors reaching publications, without banning useful tools.","potentialEngagement":"An AI-in-research policy and controls review mapping current use against trace, rerun and citation-check requirements.","unsupportedClaims":["Any specific safeguard prevents AI errors","Agents improve research quality","Error rates in everyday research are known"]},"engineering":{"fit":"Fits institutions where researchers already use coding or research agents for analysis and writing.","architectureIntegration":"Agent tools configured to log trajectories, store code and outputs with provenance, rerun analyses and run automated citation-to-source checks.","prerequisites":["Inventory of AI tools in use","Storage for traces and artefacts","Citation-checking tooling"],"constraints":["Preprint evidence, not peer reviewed","Developer-evaluated studies dominate","No real-world error rate data"],"security":"Trace retention must respect data classification for any sensitive research data captured in logs.","proofOfValue":["Pilot trace retention on one group's agent workflows","Measure unsupported claims caught by citation checks","Compare repeated-run stability on a sample analysis"]},"delivery":{"work":"Draft guidance, configure logging and checks, train researchers and integrity staff, and monitor compliance.","dependencies":["Faculty governance approval","Tool vendor logging support","Integrity office capacity"],"ownership":"Research integrity office, with research computing operating tooling.","skillsAdoption":"Researchers need practice reading agent traces and verifying citations; reviewers need guidance on proportional checks.","governanceCheckpoints":["Policy approval by research council","Disclosure requirement in submission workflows","Annual review of safeguard evidence"],"acceptanceCriteria":["Proposed: target share of AI-assisted outputs with retained traces","Proposed: citation checks completed before submission","Proposed: researchers trained in verification within a set period"],"risks":["Checks seen as burdensome and skipped","False confidence in untested safeguards","Logs capturing sensitive data"]}},"retrievedAt":"2026-10-24T22:02:44-05:00","enrichedAt":"2026-10-24T22:02:45-05:00","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Researchers directing agents need verification skills (reading traces, checking citations); evidence about human oversight competence is very low certainty.","procurementImplications":"When buying research-agent tools, require independent evaluation evidence rather than developer benchmarks, given the gap between developer and independent safeguard results.","operatingModelImplications":"Treat AI-produced research outputs as drafts requiring verification steps proportional to autonomy, with integrity offices setting minimum checks.","sourceVerification":{"openedUrl":"https://osf.io/preprints/metaarxiv/7vsbw_v1/","referenceExcerpt":"tested safeguards were effective in 56 of 82 developer evaluations and 16 of 41 independent ones.","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}