From the Research Universities edition of September 21, 2026
Diamond Agent: federated HPC placement cuts median extra completion 42s→4s (10.5×) across four supercomputers
Rutgers / NCSA / University of Chicago / Argonne / UW–Madison / University of Rochester (Xie et al.) · University/national research computing / agentic HPC orchestration · United States (NCSA, TACC, RCAC and related); federation-limited to authorized sites
- Publisher
- arXiv
- Original publication
- September 5, 2026 (arXiv v1)
- Source retrieved
- 2026-09-22
What happened
Diamond Agent provides an agent-facing control plane over Diamond/Globus for multi-supercomputer workflows with ResourceEnvelopes, deterministic feasibility, live best-feasible placement, and event-driven continuation. Evaluation reports 27 hours of federation telemetry and 19 matched multi-site submission rounds (83 jobs) across Delta, DeltaAI, Anvil, and Vista; relative to a fixed-Delta baseline, best-feasible reduced median additional completion time versus the fastest observed placement from 42 seconds to 4 seconds (10.5×). Two coding-agent workflows resumed after 37 and 84 minutes with zero model-side status calls.
Why it matters
Why now: Unarchived 5 Sep measured multi-site placement outcomes—newly actionable after 2026-09-20 NAIRR/campus capacity stories—show federation telemetry and event-driven agents can cut fixed-site wait penalties without putting long-lived agents on every login node.
Evidence and measured results
19 rounds/83 jobs; 42s→4s median additional completion; 10.5×; 543 snapshots/27h; 0 status polls over 37–84 min intervals.
Limitations and uncertainty
Large p95 tails; no user-code sandbox; transfer duration not jointly optimized; four-system study; preprint.
Put this evidence to work
Lighthouse Advisory interpretation, grounded in this source. Enriched 2026-09-22; this does not change the original publication date. Each role follows the same checklist so takeaways compare across sources.
Sales
Role takeaway
- Customer problem
- Users with multiple HPC allocations still pin work to one site and burn wait time while other authorized capacity sits freer.
- Stakeholders
- Research computing directors
- Multi-allocation PIs
- HPC center operators
- Workflow engineers
- Discovery questions
- Which sites/allocations are authorized?
- What Globus endpoints exist?
- What is the fixed-site baseline wait?
- Which jobs can be re-placed when pending?
- Value hypothesis
- Live-state best-feasible placement across authorized sites can cut typical extra completion time versus a fixed favorite site (authors: 42s→4s median additional completion) when envelopes and deterministic rules are enforced.
- Potential engagement
- Pilot federated placement on one multi-site workflow with matched-round measurement against fixed-site baseline.
- Unsupported claims
- Always selects the absolute fastest site
- Sandboxed untrusted code
- NAIRR-wide production guarantee
Pre-sales engineering
Role takeaway
- Fit
- Agent-facing federated HPC control plane with envelopes, live snapshots, deterministic placement, and event-driven continuation.
- Architecture and integration
- Sit above Globus Compute/Transfer and site Slurm dialects; persist tasks/events outside the model context.
- Prerequisites
- ≥2 authorized systems
- Globus auth
- Site profiles
- Persistent orchestrator host
- Constraints
- Large tail estimation error
- No joint transfer+queue optimization yet
- No code sandbox
- Four-system evaluation
- Security
- Never put cluster credentials in prompts; enforce authorization/credit; audit placement traces.
- Proof of value
- Matched multi-site rounds vs fixed baseline
- Async continuation with zero status polls
- UNSCHEDULABLE rejection of impossible envelopes
Delivery
Role takeaway
- Work
- Deploy control service, configure site profiles, run pilot workloads with fixed vs best-feasible comparison.
- Dependencies
- Globus/identity
- Center policies
- RC platform
- PI workflow definition
- Ownership
- RC owns control plane; centers own admission; PI owns scientific workflow.
- Skills and adoption
- Train users to write ResourceEnvelopes and interpret decision traces instead of hard-coding sites.
- Governance checkpoints
- Credit/allowlist review
- Blocked partition policy
- Event/continuation audit
- Acceptance criteria
- Documented median extra-completion improvement
- Placement traces retained
- Written limits on estimator accuracy
- Risks
- Stale federation snapshots
- Policy conflicts on login nodes
- Overtrusting AI-drafted envelopes
Implementation considerations
Lighthouse Advisory interpretation across the operating dimensions a public-sector buyer must settle before this evidence becomes a design. Each note answers the question under its heading for this specific source.
Architecture and integration
What must connect, and where does the AI sit in the workflow?
Research computing should expose typed federation skills and persistent orchestrators above Slurm/Globus rather than per-login coding agents.
Governance
Who approves, reviews and stays accountable for outcomes?
Keep site admission/credit authority; treat AI-drafted envelopes as auditable proposals under deterministic rules.
Security and privacy
What data, permissions and controls need testing?
Exclude credentials from prompts; enforce authorization/credit; audit placement and continuation events.
Accessibility and workforce
Who is affected, and what skills or accommodations follow?
Helps multi-allocation users without login-node agent sprawl; still requires Globus literacy and envelope authoring skills.
Procurement
What should contracts, pricing and exit terms secure?
Procure federation control planes with matched-round measurement requirements against fixed-site baselines—not agent demos alone.
Operating model
Which teams own the service once it runs?
Operating model separates active agent time from batch lifetime and funds persistent orchestration as shared RC services.
Publication history
- 2026-09-21Research Universities · Issue 164 resources
Stable resource ID: diamond-agent-federated-hpc-placement-20260905