Higher Education & ResearchIssue 404 records
Research Universities
Four newly inspected sources (9–15 Oct 2026). The European Commission's fourth ERA Living Guidelines say AI agents are research tools, not independent scientists, bar AI from merit judgements in review, and urge institutions to offer governed AI. Anthropic's vendor-authored preprint shows Claude agents flagging a new phage enzyme family in a 949-session campaign, though ten reruns missed the key feature and 14 of 17 candidates were rejected. Northwestern's Autonomous Science Bench finds agents passed 31 of 144 budgeted lab campaigns, with the best setup costing 25 times the cheapest. An Eye study finds LLMs caught 81.7% of planted manuscript inconsistencies. Two patterns: agent results need repeat runs and run records, and AI in review is settling into a supervised support role.
- Evidence records
- 4
- Cross-source patterns
- 2
- Evidence classes
- 2 academic research1 standards or public-body guidance1 vendor claim
- Outcomes
- 2 mixed1 emerging1 cautionary
- Source freshness
- 4 new this fortnight
- Research completed
- 2026-10-19
Choose a role to see its takeaway beside every record in the ledger.
Patterns across the evidence
Synthesis · Lighthouse Advisory interpretation
Agent research results vary by run and configuration, so repeat runs and run records are becoming part of the method
Three sources this week point the same way. Anthropic's own report shows a campaign that found a new enzyme family once, then missed the key repeat array in ten reruns. Autonomous Science Bench finds pass rates, failure modes and costs vary sharply by agent configuration and campaign. The Commission's updated guidance asks researchers to weigh non-determinism and keep prompts, logs and model versions where needed to audit agent work. Interpretation: a vendor report, a small preprint and non-binding guidance, not outcome evidence.
Operating questionShould research offices require replicate runs and retained run records (prompts, logs, model versions) before agent-assisted findings are reported or acted on?
Supporting evidenceAnthropic (Yoon, Athukoralage, Ameisen, Kauderer-Abrams, Perry, Durrant)Northwestern University (Wen and Liu, arXiv cs.AI)European Commission, Directorate-General for Research and Innovation (ERA Forum)
AI in research review is settling into a support role with human judgement and adjudication
The Commission now says AI must not judge scientific merit or draft evaluation reports, and should be limited to supporting tasks in governed tools. The Eye study measures one such supporting task: LLMs caught 81.7% of planted manuscript inconsistencies and most extra flags were real, but some errors were missed. Interpretation: together they describe a screening role with people adjudicating flags; one is guidance and one is a small single-field study.
Operating questionWhich review tasks will the institution allow AI to support (consistency checks, language), and who adjudicates its flags?
Supporting evidenceEuropean Commission, Directorate-General for Research and Innovation (ERA Forum)Adelaide University and Royal Adelaide Hospital (La, Paris, Agarwal, Chan, Selva)
Evidence ledger
Full record · every source keeps its link and limitations
European Commission updates ERA Living Guidelines on generative AI in research: agents stay tools, merit judgements stay human, institutions should offer governed AI
The fourth version of the ERA Living Guidelines (published 15 October 2026; first version March 2024, previous revision May 2026) adds agentic AI and expands peer-review and proposal-evaluation guidance. It says AI agents must be used as research tools, not independent scientists, with human validation of agent-generated hypotheses, data processing and conclusions. Researchers should disclose substantial AI use with oversight and verification methods, and consider retaining prompts, logs, model versions and parameters because outputs are non-deterministic. Reviewers and evaluators must not delegate scientific-merit judgement to AI or put unpublished proposals into non-secured tools. Research organisations should, where possible, provide locally hosted or cloud tools they govern themselves.
Why it matters, evidence and limitations
- Why it matters
- U.S. research universities face the same questions about agents, disclosure and AI in internal review, and many collaborate with EU partners or Horizon-funded projects. The document is a ready reference for a research-office policy update, though it is non-binding and EU-specific.
- Evidence and measured results
- Opened the fourth-version PDF and the 16 October Commission news item. Read the recommendations for researchers, research organisations and funders, the agentic AI and peer-review sections, and the revision history (fourth version published 15 October 2026; next revision foreseen Q3 2027).
- Limitations and uncertainty
- Principles, not an implementation standard or law; no outcome data that the recommendations reduce misconduct or bias; EU legal context differs from U.S. federal and state rules; interpretation of what counts as substantial use is left to disciplines and institutions.
Anthropic preprint: Claude agents surveyed 1.9 billion protein clusters and flagged a new phage enzyme family, but ten reruns missed the key repeat array
Anthropic ran an autonomous Claude Code harness (worker and supervisor agents with a shared record) on a written research brief. The campaign ran 119 tasks and 949 agent sessions, 77 agent-hours and 215.6 million tokens over 21.5 hours without human intervention, recovered about 200,000 RT clusters, and ended with 19 reports. Of 17 promoted candidate partner families, only three were confirmed as new RT associations; 14 were artifacts, known systems or bystanders. An agent spotted a repeat array by reading raw DNA, leading to a new family (ART). When the same campaign was run ten more times, the array was missed in every rerun. In fixed-input tests the strongest models described the array in at least 90% of attempts with DNA in the prompt but as low as 32% with files and tools. ART function remains unknown.
Why it matters, evidence and limitations
- Why it matters
- University genomics cores and research-computing centres are being pitched agent harnesses for data mining. This is a detailed, vendor-authored account of what such a campaign costs in sessions and tokens, how much human curation it still needs, and how unreliable a single run is.
- Evidence and measured results
- Opened the full technical report PDF on the Anthropic CDN (results, replicate-campaign methods, fixed-input benchmark) and the Anthropic research page; bioRxiv v1 date (11 October 2026) confirmed through the bioRxiv details API. The Drug Discovery News article of 17 October returned HTTP 403 and was not used.
- Limitations and uncertainty
- Vendor-authored preprint about its own models, not peer reviewed; model versions used are proprietary; dollar cost not reported; biological function of ART not established; replicate search could miss ART loci on contigs outside the original identifier set.
Autonomous Science Bench (Northwestern preprint): agents pass only 31 of 144 budgeted lab campaigns, and the best setup costs 25 times the cheapest
Autonomous Science Bench v0.1 gives agents six budgeted design-build-test-learn campaigns (two each in biology, chemistry and materials) built on published experiments, with a lab service that returns measurements and keeps a ledger. A campaign passes only if both a discovery gate and a learning gate pass. In 144 trials across eight agent-model configurations, only 31 passed; the best configuration resolved 67% of its trials and the hardest campaign was solved in three of 24 attempts. Higher success came at steeply higher cost: total cost rose 25-fold from $6.91 to $172.90 between the cheapest and best configurations, and cost per resolved trial from $1.38 to $14.41. Agents failed in different ways: some ranked well but found too few hits, others found hits but could not predict unseen designs.
Why it matters, evidence and limitations
- Why it matters
- Research offices and self-driving-lab programmes need ways to judge agent experiment planning before committing instrument time. This benchmark shows success, failure modes and cost vary widely by configuration and campaign.
- Evidence and measured results
- Opened the arXiv abstract page and the full HTML paper: campaign table, experimental setup, cost and performance results, failure analysis and limitations.
- Limitations and uncertainty
- Small pilot (six campaigns, eight configurations); replay and digital-twin backends, not live labs; public data may be memorised by models; preprint not peer reviewed; cost figures are API costs for these runs only.
Eye study (Adelaide): LLMs caught 81.7% of planted manuscript inconsistencies and flagged mostly genuine errors, but missed some systematically
Researchers planted one inconsistency (percentage/count, sample size/subgrouping or statistical reporting) in 30 of 40 published ophthalmic articles and had four LLMs review all 40 with one prompt. Across 120 assessments of manipulated manuscripts, 98 planted errors were detected (81.7%), ranging from 96.7% (GPT-5.6) to 63.3% (GPT-5.5). Once detected, all were localised and 97 of 98 correctly explained and corrected. Of 355 additional flags, 337 (94.9%) were genuine pre-existing inconsistencies, 158 unique after de-duplication, mostly rounding discrepancies for the top model. The authors conclude that missing errors, not interpreting them, is the main limit and position LLMs as complementary pre-submission or editorial screening.
Why it matters, evidence and limitations
- Why it matters
- Research offices, journals hosted by universities and pre-submission support services are considering AI checks. This is measured evidence for a narrow support task of the kind the new EU guidance allows, not for judging merit.
- Evidence and measured results
- Opened the full open-access brief communication on nature.com: methods, Tables 1 and 2 as described in the text, and conclusions.
- Limitations and uncertainty
- Small sample (40 articles, 30 planted errors, one discipline); planted errors were of three simple types; model versions change quickly; datasets available only on request.
How to read this edition
Source findings, measured results and limitations come from the cited publications. Patterns, operating questions, role takeaways and implementation considerations are Lighthouse Advisory interpretation, stated as questions to validate locally rather than guaranteed outcomes. Vendor and operator claims are labeled as claims. Full research method.
- Academic research
- Research produced through an academic institution or peer-reviewed venue.
- Standards or public-body guidance
- Normative or advisory guidance from a standards body or public institution.
- Vendor claim
- A supplier-provided assertion that has not been upgraded to independent evidence.