AgentActionBench exposes the gap between generated code and executed research

Discuss unreliable paper-to-code handoffs with research computing leaders, principal investigators and research integrity staff. Ask how often generated implementations actually run, what counts as a faithful result and who resolves disputed evaluations. Offer a bounded reproduction assessment on a small, permitted local paper set. The value hypothesis is identifying failure stages before researchers build on unsupported outputs. Do not present rubric scores as discovery success rates or promise reduced staffing. Include reviewer time and dependency repair in the scope, and establish a manual baseline for total effort before estimating economic value.

Build a sandbox with mediated file and command access, immutable external logging and versioned environments. Integrate campus identity and approved storage; select cloud, local or hybrid execution according to data permissions and workload requirements. Preserve exit status, outputs and failed attempts separately from the agent's narrative. A proof of value should compare trace-based judgments with independent human reruns, deliberately including plausible code that was never executed. Calibrate any automated judge locally because agreement between rubric sources cannot guarantee correct judgments. Restrict outbound access and confirm logs cannot disclose confidential data.

Assign a research software engineering lead and an independent disciplinary reviewer. Dependencies include lawful dataset access, runnable environments, an agreed rubric and protected review time. Onboard researchers through an accessible example showing the difference between writing code and validating its result. Proposed acceptance criteria are a complete action trace for every attempted task, documented treatment of each execution failure and an independent comparison against the expected scientific result. Track elapsed time, reviewer effort and unresolved discrepancies. These are proposed measures, not benchmark observations. Risks include judge errors, environment drift and overfitting to checklist wording.