From the Everpure edition of October 29, 2026
What Production AI Factories Need from the Data Plane
Everpure (Philip Ninan) · AI data infrastructure vendor (with Red Hat and NVIDIA) · Global
- Publisher
- Everpure Blog
- Original publication
- 2026-10-20
- Source retrieved
- 2026-10-30
- Event date
- 2026-10-20
What happened
Everpure describes its Data Architecture for Red Hat AI Factory with NVIDIA: FlashBlade//S serving S3 and NFS, Portworx FlashBlade Direct Access for shared read-write-many volumes, Kueue admission, and a capstone fine-tune of Qwen3-32B with DeepSpeed ZeRO-3. It discloses two build failures: a checkpoint silently incomplete because rank 0 uploaded only local shards, and jobs that queued silently when admission quotas covered GPUs but not CPU and memory. It states FlashBlade//S earned NVIDIA-Certified Storage for Vera Rubin and that the inference reference architecture is still being completed.
Why it matters
Directly useful to university research computing and state AI centres building shared GPU clusters on OpenShift/Kubernetes: the silent-failure lessons are platform-agnostic operational checks. Governance points on lineage for model promotion map to public-sector accountability expectations.
Evidence and measured results
Vendor engineering blog describing internal builds; no customer deployment, sample or measured production outcome. Self-reported failures add credibility but are not independently verified. The blog calls MLPerf results 'independent evidence'; they are audited vendor submissions.
Limitations and uncertainty
Marketing context (GTC Berlin demos); inference and hybrid reference architectures are not complete. Not specific to government or education.
Put this evidence to work
Lighthouse Advisory interpretation, grounded in this source. Enriched 2026-10-30; this does not change the original publication date. Each role follows the same checklist so takeaways compare across sources.
Sales
Role takeaway
- Customer problem
- Research and state AI centres move from pilots to shared GPU platforms and hit silent operational failures and weak model lineage.
- Stakeholders
- Research computing director
- AI platform lead
- Research data governance office
- CIO
- Discovery questions
- How do you verify checkpoint completeness today?
- What metric do you use for GPU efficiency?
- Can you show the data and code behind a promoted model?
- Value hypothesis
- Applying documented checks may reduce wasted GPU time and improve auditability of models, magnitude unknown.
- Potential engagement
- A bounded production-readiness review of an existing Kubernetes/OpenShift AI platform against checkpoint, quota and lineage checks.
- Unsupported claims
- Completed inference or hybrid reference architectures
- Production customer outcomes
- That MLPerf results are fully independent tests
Pre-sales engineering
Role takeaway
- Fit
- Fits OpenShift/Kubernetes GPU clusters for fine-tuning and inference in research settings.
- Architecture and integration
- Red Hat AI on bare metal, FlashBlade//S (S3+NFS), Portworx FlashBlade Direct Access, Kueue, DeepSpeed, llm-d for inference.
- Prerequisites
- OpenShift/Kubernetes platform
- FlashBlade//S
- Portworx
- Kueue configured
- Constraints
- Inference and hybrid architectures incomplete
- Three-vendor support boundaries
- Vera Rubin certification is forward-looking for most campuses
- Security
- Respect existing permissions during data preparation and treat embeddings as governed derived data.
- Proof of value
- Checkpoint completeness verification across ranks
- Quota test covering CPU, memory and GPU
- Lineage record for a promoted model
Delivery
Role takeaway
- Work
- Implement and test checkpoint, admission-quota and lineage controls in a shared GPU platform.
- Dependencies
- Platform engineering team
- Vendor reference documentation
- Research data governance approval
- Ownership
- Research computing or central AI platform team.
- Skills and adoption
- Requires Kubernetes, Kueue and distributed training skills; runbooks for silent-failure checks.
- Governance checkpoints
- Model promotion review with lineage
- Data-preparation access review
- Post-run resource release check
- Acceptance criteria
- Proposed: automated check flags any missing checkpoint shard
- Proposed: no job waits silently when requested resources exist
- Proposed: every promoted model has data, code and evaluation lineage
- Risks
- Silent failures recur in new workloads
- Support gaps between vendors
- Over-reliance on vendor-run benchmarks
Implementation considerations
Lighthouse Advisory interpretation across the operating dimensions a public-sector buyer must settle before this evidence becomes a design. Each note answers the question under its heading for this specific source.
Architecture and integration
What must connect, and where does the AI sit in the workflow?
Shared namespace for rank-sharded checkpoints, S3-triggered fine-tuning, cache-aware routing with llm-d and FlashBlade as persistent KV tier; Portworx provides Kubernetes data mobility across hybrid environments.
Governance
Who approves, reviews and stays accountable for outcomes?
Emphasises defensible lineage (data set version, code commit, evaluation evidence) before model promotion, and governance extending to derived data such as embeddings under sovereignty or privacy requirements.
Security and privacy
What data, permissions and controls need testing?
Governed access must respect existing permissions through data preparation; embeddings treated as derived data in scope for privacy controls.
Accessibility and workforce
Who is affected, and what skills or accommodations follow?
No workforce data; implies need for platform engineering skills (Kubernetes, Kueue, DeepSpeed).
Procurement
What should contracts, pricing and exit terms secure?
Reference architectures span three vendors; buyers should clarify support boundaries and which components are certified versus in development.
Operating model
Which teams own the service once it runs?
Promotes goodput over GPU utilisation as the efficiency metric and returning accelerators to a shared pool, relevant to campus chargeback models.
Publication history
- 2026-10-29Everpure · Issue 017 resources
Stable resource ID: everpure-ai-factory-data-plane-lessons-2026-10-20