{"resourceId":"ecas-alcf-validation-gated-hpc-agents-20260913","versions":[{"version":"external-603645f71399153cb28e8beb6dd935f2d74ad304a0b364ddcd2dc8abf2a13718","resource":{"id":"ecas-alcf-validation-gated-hpc-agents-20260913","title":"ECAS edge control and validation gates raise ALCF injected-fault success from 0/6 to 6/6","organization":"Argonne National Laboratory and Oregon State University (authors Sun, Xia, Zheng)","sector":"Research computing / HPC agent automation","geography":"United States (ALCF Crux and Sophia; transferable to university research computing)","publishedAt":"September 13, 2026, arXiv 2609.14211v1","publicationDate":"2026-09-13","eventDate":null,"sourceName":"arXiv","sourceLabel":"Author preprint; abs and html methods/results inspected","sourceUrl":"https://arxiv.org/html/2609.14211","evidenceClass":"academic-research","outcomeClass":"mixed","topics":["developers-agents","infrastructure","data-security","operating-model","governance-procurement"],"finding":"On two ALCF systems, an edge-controlled agent with validation-gated scale-up and private skills improved injected-fault HPC campaign success from 0/6 (one-shot) to 6/6, while keeping credentials and site skills off the cloud.","sledRelevance":"Newly relevant measured control pattern for university research computing adopting LLM agents on HPC: separates cloud reasoning from edge authority. Injected-fault study—not scientific discovery ROI.","evidence":"ECAS: cloud proposes plans/repairs; edge retains credentials, policy gate, sanitization, and expert-distilled skill library never disclosed to cloud; HPC executes. Workloads IOR/DLIO/E3SM-IO on Crux/Sophia; six injected faults; B_rep=3; N=5 on key runs. Closed-loop 0/6→6/6 vs one-shot; validation gating prevents all three observed target-scale failures; skill conditioning 4/6→6/6. E1: all three workloads complete on Crux with composite oracles; E3SM-IO 1112 MiB/s at 512-proc target. Threat model: fallible not adversarial cloud.","architectureImplications":"Interpretation: put SSH/scheduler, credentials, and skill libraries on a user-controlled edge; allow cloud LLMs only sanitized, site-agnostic intents; require small-scale validation before target-scale promotion.","governanceImplications":"Interpretation: set resource-policy gates (nodes, walltime, account, remaining node-hours) as non-negotiable edge checks; treat skill libraries as sensitive operational assets.","securityPrivacyImplications":"Interpretation: never grant cloud models direct HPC credentials; sanitize logs of usernames, absolute paths, and project IDs before they leave the trust boundary.","caveats":"Preliminary injected-fault experiments on two ALCF systems and three I/O workloads; author preprint; success is application/oracle completion, not scientific insight quality.","streamIds":["research"],"roles":{"sales":"Interpretation: Talk with research computing and cyber directors worried about giving LLM agents HPC credentials. Ask where one-shot job scripts fail today, what node-hour waste they will tolerate, and who may hold edge control. Offer a bounded validation-gated pilot on one queue with injected-fault style drills—not a promise of autonomous science. Value hypothesis is fewer wasted target-scale failures. Unsupported: discovery acceleration or adversarial cloud defenses.","engineering":"Interpretation: Fit is campus/national HPC wanting agent assistance without cloud-held credentials. Integrate edge controller (static checks, debug-queue validation, policy gate, log sanitization) with institutional LLM gateway and MCP/scheduler tools. Prerequisites: PBS/Slurm access patterns and skill capture process. Constraints: site-specific ABI/launcher knowledge; repair budgets. Security: edge-private skills and credentials. Proof: fault-injection success rates and target-scale waste avoided—not paper counts.","delivery":"Interpretation: Research computing owns edge control and skills; PIs own scientific goals; security reviews cloud egress. Work: build skill library with experts, instrument validation gates, and run controlled fault drills. Dependencies: inference gateway and scheduler APIs. Train operators on escalation when repair budgets exhaust. Checkpoints: policy gate config review; sanitization audit. Acceptance: documented 0→success under injected faults on a pilot workload. Risks: treating I/O benchmark success as discovery proof and leaking site identifiers in unsanitized logs."},"roleDetails":{"sales":{"customerProblem":"LLM agents that write HPC jobs can waste allocations or expose credentials if given direct system access.","stakeholders":["Research computing directors","Cybersecurity / research security","HPC facilitators","Allocation owners"],"discoveryQuestions":["Where do one-shot agent job scripts fail today?","What target-scale waste is unacceptable?","Who may hold credentials and skill libraries?","Which queues allow validation/debug runs?"],"valueHypothesis":"Edge control plus validation gating can convert expensive scale-out failures into cheap small-scale repairs when skills cover site conventions.","potentialEngagement":"Bounded pilot on one ALCF-like or campus queue: static checks, debug validation, and fault-injection drills for a single workload class.","unsupportedClaims":["Autonomous scientific discovery","Defense against adversarial cloud models","Guaranteed node-hour savings on all workloads"]},"engineering":{"fit":"LLM-assisted HPC campaign automation with edge trust boundary and validation-gated promotion.","architectureIntegration":"Cloud LLM for plans/repairs; edge agent for SSH/scheduler, policy gate, sanitization, skill retrieval; HPC execution via MCP/IRI-style tools.","prerequisites":["User-controlled edge host","Scheduler access","Institutional LLM gateway","Oracle definitions per workload"],"constraints":["Site-specific MPI/launcher ABIs","Repair/time budgets","Injected-fault evidence may not match organic failure mix"],"security":"Keep credentials and skills on edge; sanitize logs; policy-gate resource requests.","proofOfValue":["Injected-fault success vs one-shot","Target-scale failures prevented by gating","Autonomous completion without expert intervention"]},"delivery":{"work":"Stand up edge controller, capture expert skills, define oracles, and run controlled validation-gated pilots.","dependencies":["HPC experts for skill building","Security review of sanitization","Scheduler/API access"],"ownership":"Research computing owns edge ops; PIs define scientific goals and oracles.","skillsAdoption":"Train facilitators to escalate novel failures into new skills rather than granting cloud SSH.","governanceCheckpoints":["Policy gate parameter approval","Sanitization audit","Skill-library access control review"],"acceptanceCriteria":["Pilot fault set reaches documented success threshold","No cloud-held credentials","Target-scale jobs only after validation pass"],"risks":["Benchmark success mistaken for discovery","Skill library leakage","Overfitting repairs to injected faults"]}},"retrievedAt":"2026-09-18T03:07:35Z","enrichedAt":"2026-09-18T03:10:00Z","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Interpretation: edge-private skills can encode facilitator expertise so less experienced users benefit—if escalation paths remain for novel failures.","procurementImplications":"Interpretation: require edge-held credentials, validation gates, and sanitization in any agentic HPC product contract.","operatingModelImplications":"Interpretation: experts build skills once per failure class; day-to-day runs stay automated with bounded repair budgets.","sourceVerification":{"openedUrl":"https://arxiv.org/html/2609.14211","referenceExcerpt":"closed-loop repair improves application success from 0/6 to 6/6 over one-shot generation.","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}