Lighthouse AdvisorySLED AI Adoption Intelligence

Strategic PartnersIssue 032 records

VAST Data

State and higher education: New York's Empire AI issued a vendor-neutral Gamma RFI on October 29 (up to 5 MW, about $160 million across two tranches) that records Beta's 20 PB of VAST storage and asks how new storage would interoperate with it, including across two sites, while its user guide says no data is backed up. Research computing: an LLNL study accepted at PDSW'26 found VAST spared GPU communication from contention but still stalled 12% of training iterations. The CSCS evaluation remains inaccessible.

Evidence records
2
Cross-source patterns
1
Evidence classes
1 public-sector association guidance1 academic research
Outcomes
1 emerging1 mixed
Source freshness
1 new this fortnight1 recent
Research completed
2026-11-02

Choose a role to see its takeaway beside every record in the ledger.

Patterns across the evidence

Synthesis · Lighthouse Advisory interpretation

1 pattern, each supported by at least two sources

  1. Public AI compute buyers need measured storage-path evidence, not rated VAST capacity or throughput

    Empire AI's Gamma RFI asks respondents for sustained throughput, metadata performance, data protection and interoperability with Beta's 20 PB of VAST, and warns that a cross-site link cannot fix latency. LLNL's Tuolumne study shows why: VAST in a separate traffic class avoided all-reduce contention yet still had multi-second stalls in 12% of training iterations. Interpretation: state and university buyers should specify tail-latency, fabric placement and protection tests with representative workloads.

    Operating questionFor a public AI cluster extending or adding VAST storage, which representative-workload tests of tail latency, fabric placement, cross-site access and restore will gate acceptance?

    Supporting evidenceEmpire AI Consortium, Inc. (New York State public-private research consortium; University at Buffalo host site)Lawrence Livermore National Laboratory (LLNL) researchers (paper accepted at PDSW'26)

Evidence ledger

Full record · every source keeps its link and limitations

2 records
  1. Public-sector association guidanceEmergingPeer review status unknownNew this fortnight

    New York's Empire AI issues a $160M Gamma RFI that names Beta's 20 PB of VAST as incumbent storage Gamma must interoperate with

    Empire AI's Gamma RFI (issued October 29, 2026; responses due December 4) seeks input on up to 5 MW of AI/HPC compute and white space for its UB North Campus facility, with a roughly $160 million budget split across Gamma and Delta tranches. It records that Beta (online August 2026) includes 20 PB of VAST and 10 PB of DDN storage, and asks respondents to describe storage capacity, sustained throughput, metadata performance, data protection and integration with the existing VAST and DDN systems, whether or not Beta is relocated. It reports 1,631 active users, 314 projects and over 80 percent utilization on Alpha and Beta.

    Empire AI Consortium, Inc. (New York State public-private research consortium; University at Buffalo host site)United States (Buffalo and Amherst, New York)2026-10-29

    Why it matters, evidence and limitations
    Why it matters
    Direct state and higher-education relevance: a New York State-established consortium serving ten institutions is planning its next AI system, and VAST is the incumbent all-flash platform. The RFI is vendor-neutral, so it signals an interoperability and data-strategy requirement, not a VAST award.
    Evidence and measured results
    Primary procurement document from a state-sponsored public-private consortium (requirements and context, not an evaluation). Retrieved as GovTribe's text version (pages 1-24 of 40 shown); posting date and responses-due date corroborated by the Govly listing of New York State Contract Reporter #2140363. UB CCR's Empire AI user guide independently states that no data on Empire AI systems is backed up. No VAST performance or outcome data is provided.
    Limitations and uncertainty
    The RFI is not a solicitation and awards nothing; naming VAST reflects the installed base, not a preference. The text was read through a third-party portal's extracted version, which showed only part of the 40-page document; the original portal posting requires an account. Interpretation (not source finding): interoperability language may favour incumbents in a later RFP, but Empire AI explicitly states it does not prescribe a manufacturer.
    Empire AI Gamma Phase RFI (state procurement document) Role takeaways & implementation
  2. Academic researchMixedPreprint, not peer reviewedRecent

    LLNL study on the Tuolumne production system finds VAST avoids all-reduce contention but still shows heavy-tailed DataLoader stalls in 12% of iterations

    On LLNL's Tuolumne (HPE Slingshot-11, four nodes with 16 AMD MI300A GPUs, a real GNN training workload), DataLoader waits were 15 ms median on VAST but spiked to multiple seconds in 12% of VAST iterations versus 28% for Lustre. VAST, mounted over NFSv3/TCP, sits in a separate Slingshot traffic class, which largely spared all-reduce from contention, but its data path still crosses the shared fabric. Node-local NVMe staging (DYAD) was 1.06x faster than VAST over one epoch and 1.31x once the cache warmed, and 7.4x faster than Lustre.

    Lawrence Livermore National Laboratory (LLNL) researchers (paper accepted at PDSW'26)United States (Livermore, California)2026-09-19

    Why it matters, evidence and limitations
    Why it matters
    Relevant to university, state-funded and national-lab-adjacent AI clusters that pair VAST with a parallel file system, including the Empire AI Gamma RFI issued October 29, 2026, which asks how storage will interoperate with Beta's VAST tier. Why now: the paper was revised in September and is accepted at PDSW'26 (held with SC26 in November); it offers independent measured evidence on exactly the storage-fabric questions that state AI consortia are now putting into procurements.
    Evidence and measured results
    Independent measured study by national-lab researchers on a production system, comparing Lustre, VAST and node-local staging with the same workload. Sample is narrow: one GNN workload, four nodes, one fabric and VAST over NFSv3/TCP. DYAD is the authors' own system, so the comparison is not neutral toward it. The arXiv version is a preprint of a workshop paper.
    Limitations and uncertainty
    Single workload and small scale; results depend on Slingshot traffic classes and NFSv3/TCP mounts and may differ with RDMA or other fabrics. Authors evaluate their own mitigation. Not a SLED deployment; transferability to university or state clusters is an interpretation. Original submission September 6, 2026; v2 September 19.
    Wang, Wu, Kim and Yeom, 'Sharing a Fabric with Collective Communication' (accepted at PDSW'26) Role takeaways & implementation

How to read this edition

Source findings, measured results and limitations come from the cited publications. Patterns, operating questions, role takeaways and implementation considerations are Lighthouse Advisory interpretation, stated as questions to validate locally rather than guaranteed outcomes. Vendor and operator claims are labeled as claims. Full research method.

Public-sector association guidance
Practitioner guidance or an association-supplied case; not independent outcome evidence.
Academic research
Research produced through an academic institution or peer-reviewed venue.