Higher Education & ResearchIssue 114 records
Research Universities
Four sources contrast a same-week NSF Expanse2 capacity award and Pitt research-computing throughput metrics with agent reproducibility scrutiny. Expanse2 funds 2027 national HPC/AI access without measured outcomes yet; Pitt reports record FY2026 jobs and hours while planning private LMaaS. AutoResearchEval finds metacognitive failures across 800 trajectories; Faraday improves figure-replication win-rates versus Claude/Codex but authors warn replication remains early-stage. Role guidance separates capacity onboarding from fidelity and process audits.
- Evidence records
- 4
- Cross-source patterns
- 2
- Evidence classes
- 2 vendor claim2 academic research
- Outcomes
- 2 mixed1 emerging1 cautionary
- Source freshness
- 2 recent1 new this fortnight1 undated
- Research completed
- 2026-09-17
Choose a role to see its takeaway beside every record in the ledger.
Patterns across the evidence
Synthesis · Lighthouse Advisory interpretation
National AI capacity awards still leave campus benefits prospective beside measured local throughput
Expanse2 funds 2027 national AI/HPC access without Expanse2 outcome metrics, while Pitt already publishes FY2026 job and hour totals. Institutions should demand comparable telemetry before treating awards as research benefit. Neither source measures the other's system.
Operating questionWhich utilization and service metrics must a new AI/HPC award demonstrate before leadership claims research benefit?
Supporting evidenceSan Diego Supercomputer Center at UC San Diego (NSF award)University of Pittsburgh Center for Research Computing and Data
Replication training gains do not erase lifecycle metacognitive failures
Faraday raises figure-replication win-rates versus Claude/Codex, while AutoResearchEval shows process failures—especially uncorrected self-awareness—across 800 trajectories. Keep separate gates for relative eval scores and process-faithful release. Neither study is a campus production deployment.
Operating questionWhat process checks must still pass after a research agent posts a strong replication or endpoint score?
Supporting evidenceInherent (Faraday / Replica preprint)Multi-institution author team (AutoResearchEval / ARFT preprint)
Evidence ledger
Full record · every source keeps its link and limitations
NSF Expanse2 award funds 2027 SDSC capacity for AI/HPC research; outcomes still prospective
NSF awarded $10M for Expanse2 to expand AI/HPC capacity and NAIRR-facing access, but production is planned for 2027 and no Expanse2 scientific-outcome metrics are published yet.
Why it matters, evidence and limitations
- Why it matters
- Directly relevant to U.S. university research computing and NAIRR-linked infrastructure planning; announcement defines capacity and education goals, not measured discovery gains.
- Evidence and measured results
- Published September 15, 2026: $10M NSF award (2614012) to SDSC/UC San Diego to deploy NSF Expanse2 replacing Expanse (since late 2020); AMD processors/accelerators; all-NVMe AI/ML storage plus capacity tier; Jupyter and AI/ML frameworks; EOT for universities, teaching colleges, high schools; industry discretionary allocations; production in 2027; intended NAIRR contribution. No utilization, time-to-science, or reproducibility metrics for Expanse2 reported.
- Limitations and uncertainty
- Prospective system entering production in 2027. Institutional announcement, not an independent evaluation. Architecture details are high-level. Prior Expanse user counts do not prove Expanse2 outcomes.
Pitt CRCD reports record FY2026 jobs and hours; LMaaS still prospective
Pitt reports record research-computing throughput and storage-cost cuts, plus AI-assisted ops analysis, while private Language Model as a Service remains a planned—not measured—offering.
Why it matters, evidence and limitations
- Why it matters
- Newly relevant measured campus research-computing telemetry contrasting prospective national AI capacity awards; U.S. public research university setting. Does not prove scientific reproducibility or LMaaS efficacy.
- Evidence and measured results
- FY2026: 6,588,805 jobs and 3,425,520 compute hours (GPU 1,151,851 jobs / 542,690 hours); 184 grants / $95.6M expenditures; ~$4M upgrades; warm-tier storage $65→$30 per TB/year (~54%); 13 RTX 6000 Pro Blackwell nodes (104 GPUs) and 2 H200 nodes (16 GPUs); performance review used Claude on curated scheduler data; LMaaS and cloud bursting listed as services in development. No controlled evaluation of Claude analysis quality or released LMaaS outcomes.
- Limitations and uncertainty
- Operator self-report without independent audit or clear PDF publication date. Throughput is not scientific outcome. LMaaS unreleased. Claude analysis methodology thin.
AutoResearchEval maps 45 agent failure patterns to a missing metacognitive loop
Across 800 end-to-end research trajectories, agent failures concentrate on missing metacognitive self-checks rather than any single scaffold—so process audits must go beyond endpoint scores.
Why it matters, evidence and limitations
- Why it matters
- Newly relevant after 2026-09-15 ABE-Ralph fidelity coverage: expands scrutiny from experimental protocol fidelity to full-lifecycle failure taxonomy for campuses piloting research agents. International preprint; not a U.S. campus deployment study.
- Evidence and measured results
- 100 tasks / 7 domains / 6 lifecycle stages; 8 harness–model combinations → 800 trajectories; ARFT 45 patterns. Authors: agents lack metacognitive loop. F.4 Uncorrected self-awareness HIT in 660/800 (82.5%); C.1 Circular-Valid 552; A.5 Metric-Misalign 545; F.2 Fail-Gate 502. Patterns recur across all eight systems including strongest models. Orchestration fixes not tested.
- Limitations and uncertainty
- Preprint with author-defined tasks/judge. HIT% is annotation frequency. No campus productivity study. Does not prove orchestration cannot help. Covered now as deferred complementary scrutiny after ABE-Ralph.
Faraday improves figure-replication win-rates vs Claude/Codex; authors still call replication early-stage
Post-training a 27B research agent improves rubric win-rates on figure replication versus strong coding agents, yet authors warn failures persist and humans must keep inspecting results.
Why it matters, evidence and limitations
- Why it matters
- Newly relevant alongside AutoResearchEval after recent fidelity coverage: shows training can raise replication scores without authorizing unattended scientific acceptance on campuses.
- Evidence and measured results
- Replica: 310 figure-replication tasks from 100 ML/AI-for-science papers. Faraday (Qwen3.6-27B + coding-agent tools; turn-level GRPO) outperforms Claude Opus 4.8 and GPT-5.5 on 73% of in-distribution ML tasks and 60% of held-out AI-for-science tasks; +6% vs Claude and +8% vs Codex average on test; human preference 29/41 rollouts. Limitations section: early-stage; Faraday failed some rigorous originals; do not treat agent non-replication as indictment of originals; humans must retain skills and inspect outputs.
- Limitations and uncertainty
- Lab/vendor preprint; auto-generated rubric judge; figure-replication scope narrower than full papers or discovery; compute environment may not match campus clusters. Covered for mixed training gains vs process-failure scrutiny.
How to read this edition
Source findings, measured results and limitations come from the cited publications. Patterns, operating questions, role takeaways and implementation considerations are Lighthouse Advisory interpretation, stated as questions to validate locally rather than guaranteed outcomes. Vendor and operator claims are labeled as claims. Full research method.
- Vendor claim
- A supplier-provided assertion that has not been upgraded to independent evidence.
- Academic research
- Research produced through an academic institution or peer-reviewed venue.