{"resourceId":"semianalysis-inferencex-rubin-agentic-20260914","versions":[{"version":"external-065051557cb29b36d65bba0d4456f1ee4fc2aecd8caa7f1eb3df9235436033be","resource":{"id":"semianalysis-inferencex-rubin-agentic-20260914","title":"SemiAnalysis InferenceX: early Rubin AgentX shows large tok/s/MW gains; MaxLPS not isolated","organization":"SemiAnalysis / InferenceX","sector":"Independent semiconductor / inference analysis","geography":"Global (benchmark fleet)","publishedAt":"September 14, 2026","publicationDate":"2026-09-14","eventDate":"2026-09-14","sourceName":"InferenceX by SemiAnalysis","sourceLabel":"Independent benchmark analysis","sourceUrl":"https://inferencex.semianalysis.com/blog/vera-rubin-nvl72-agentic-inference","evidenceClass":"independent-research","outcomeClass":"emerging","topics":["developers-agents","infrastructure","operating-model"],"finding":"Sep 14, 2026 InferenceX port reports first verified AgentX agentic-inference results for Vera Rubin NVL72 on early pre-release software: large TCO and tok/s/MW advantages vs GB300 depending on interactivity target and engine (examples include ~67x throughput per owning-TCO vs GB300 Dynamo TRTLLM at 170 TPS on a dense NVFP4 path; ~7.2x tok/s/MW vs GB300 SGLang at 150 TPS; engine label critical—at 170 TPS vs GB300 TRTLLM the MW multiplier is ~62.9x but vs SGLang ~5.56x). Authors argue GTC 3x per MW sandbagged vs measured up to ~7x. MaxLPS is described as first-class dynamic power shifting to fit more GPUs from real workload power profiles, with PowerX integration forthcoming—AgentX curves are not presented as isolated MaxLPS measurements.","sledRelevance":"Independent counterweight to vendor AgentX/MaxLPS marketing for SLED capacity planning: multipliers are SLA- and engine-scoped, early-software, and not MaxLPS-isolated. Limited direct public-sector deployment; relevance is procurement literacy for tokens/MW slides.","evidence":"AgentX real-world coding trajectories; owning vs 3-year rental TCO frames; tok/s/MW table across TPS targets; MaxLPS qualitative description; early TRTLLM Rubin software caveat; subscriber-only continuation noted.","architectureImplications":"Compare engines at matched P90 TPS; do not collapse MaxLPS densification into AgentX efficiency curves; expect software maturity to move frontiers.","governanceImplications":"Require engine/SLA/corpus pins on any Rubin multiplier used in budget cases.","securityPrivacyImplications":"Not addressed.","caveats":"Public port of newsletter; early pre-release software; NVIDIA collaboration acknowledged; some fleet economics are modeled; MaxLPS not separately ablated.","streamIds":["nvidia"],"roles":{"sales":"When buyers wave 30x/67x AgentX slides, re-anchor on InferenceX matched-TPS and engine-label discipline. Celebrate measured Rubin efficiency where pinned, but refuse MaxLPS=AgentX conflation and early-silicon universalization. Offer a matched-SLA bakeoff rather than selling a single multiplier as planning truth. Keep claims bounded to the inspected evidence and require site-matched proof before any capacity or interconnect commitment. Keep claims bounded to the inspected evidence and require site-matched proof before any capacity or interconnect commitment.","engineering":"Fit for teams selecting serving stacks for agentic coding/research agents. Integrate TRTLLM/vLLM/SGLang comparisons at target TPS, with power metering. Proof of value: reproduce relative tok/s/MW at the buyer P90 interactivity—not GTC chart tracing—and keep MaxLPS as a separate test plan. Document configuration pins and failure modes so the next upgrade cannot silently invalidate the measured result. Document configuration pins and failure modes so the next upgrade cannot silently invalidate the measured result.","delivery":"Work is benchmark governance: pin model, engine, quantization, TPS, and power meter method. Ownership: inference platform. Risks: early software churn invalidates pins; TCO model assumptions diverge from local power rates; MaxLPS wrongly inferred from AgentX curves. Record acceptance evidence in the program archive so later editions can cite what was actually proven. Record acceptance evidence in the program archive so later editions can cite what was actually proven."},"roleDetails":{"sales":{"customerProblem":"Agentic tokens/MW marketing numbers are being treated as portable planning constants.","stakeholders":["AI platform","CFO","facilities"],"discoveryQuestions":["Which P90 TPS and engine are in the slide?","Is MaxLPS ablated or assumed?","Early-software or GA stack?"],"valueHypothesis":"Pinned AgentX-style comparisons can justify Rubin TCO—if SLA/engine are matched.","potentialEngagement":"Matched-SLA InferenceX-style bakeoff on buyer corpus.","unsupportedClaims":["67x everywhere","MaxLPS measured in AgentX curves","GTC 3x is the ceiling"]},"engineering":{"fit":"Teams optimizing agentic serving efficiency under power caps.","architectureIntegration":"Engine choice + interactivity target + power telemetry; MaxLPS evaluated separately.","prerequisites":["power meters","AgentX or equivalent trace replay","engine pins"],"constraints":["Early Rubin software","modeled TCO components","public article incomplete vs subscriber"],"security":"N/A.","proofOfValue":["tok/s/MW at matched TPS","Engine A/B logged","MaxLPS separate test plan"]},"delivery":{"work":"Stand up pinned benchmark jobs and review board for multiplier use in planning.","dependencies":["SemiAnalysis/InferenceX methods or internal equivalent","facilities metering"],"ownership":"Inference platform lead.","skillsAdoption":"Analysts must read engine labels and TPS axes.","governanceCheckpoints":["No unpinned multipliers in budget decks","MaxLPS claims require separate PoC","Early-software watermark"],"acceptanceCriteria":["Pinned result archive","Sensitivity to engine documented","Planning number approved"],"risks":["Software churn","Wrong SLA target","Conflating TCO model with cash cost"]}},"retrievedAt":"2026-09-20T03:04:30Z","enrichedAt":"2026-09-20T03:13:48Z","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"None stated.","procurementImplications":"Require pinned SLA/engine exhibits alongside any Rubin efficiency claims in proposals.","operatingModelImplications":"Capacity economics tracked in tokens per megawatt at stated interactivity, not peak FLOPS.","sourceVerification":{"openedUrl":"https://inferencex.semianalysis.com/blog/vera-rubin-nvl72-agentic-inference","referenceExcerpt":"At 150 TPS, Rubin retains nearly 37 million tok/s/MW, approximately 7.2x GB300 SGLang.","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}