{"resourceId":"pinterest-dynamo-vlm-b200-20260910","versions":[{"version":"external-065051557cb29b36d65bba0d4456f1ee4fc2aecd8caa7f1eb3df9235436033be","resource":{"id":"pinterest-dynamo-vlm-b200-20260910","title":"Pinterest builds production VLM serving on Dynamo + Blackwell B200 with projection embeddings","organization":"Pinterest Engineering","sector":"Internet / multimodal AI serving","geography":"Global (Pinterest production; AWS EKS)","publishedAt":"September 10, 2026","publicationDate":"2026-09-10","eventDate":"2026-09-10","sourceName":"Pinterest Engineering Blog (Medium)","sourceLabel":"Operator engineering case study","sourceUrl":"https://medium.com/pinterest-engineering/building-pinterests-vlm-serving-stack-on-nvidia-dynamo-0dce6e93d0f3","evidenceClass":"independent-reporting","outcomeClass":"effective","topics":["developers-agents","infrastructure","operating-model"],"finding":"Pinterest Engineering (Sep 10, 2026) details a production VLM stack on NVIDIA Blackwell B200 GPUs and NVIDIA Dynamo (DynamoGraphDeployment on EKS), with vLLM, LMCache KV offload, encoder/prefill/decode disaggregation, and multimodal KV-aware routing. Custom PinCLIP projection embeddings avoid pixel-path vision encoding: vs pixel inputs they report ~85x average TTFT, ~7.3x E2E latency, and ~1.1x TPOT speedups (peaks ~369x TTFT / ~44x E2E), enabling ~250-image contexts with latency comparable to ~10-image pixel requests (~25x visual context). B200 showed >2x latency improvement vs Hopper in preliminary Assistant benchmarks. AIPerf DAG benchmarks mirror multi-turn multimodal traffic.","sledRelevance":"Strong transfer for SLED multimodal assistants (image+text) on managed Kubernetes: shows Dynamo is an operator stack requiring payload design, KV tiers, and disaggregation—not a drop-in NIM. Limited direct public-sector deployment evidence; relevance is architectural pattern and measured serving levers.","evidence":"Dynamo operator via Helm; DynamoGraphDeployment CRDs; OpenAI Chat Completions + Model Router; LMCache GPU/CPU/NVMe tiers; multimodal frontend image decode; PinCLIP projector path; TTFT p99 up to 5.92x with MM KV routing (~1.42x average); LoRA hot-loading; AIPerf DAG for agentic workflows; use cases: Assistant, rerank, OCR, safety.","architectureImplications":"Plan E/P/D worker ratios, multimodal KV routing, and embedding vs pixel payload contracts before citing Dynamo for campus VLM; PinCompute-style dedicated EKS/GPU pools and sidecar injection are part of the pattern.","governanceImplications":"Require production-like multimodal SLOs (multi-image, multi-turn) in acceptance; treat vendor Summit mentions of Pinterest as marketing unless operator metrics are reviewed.","securityPrivacyImplications":"Image URL/base64 ingestion and embedding stores expand data-handling surface; source does not claim confidential computing.","caveats":"Single large operator; metrics are Pinterest-internal benchmarks on Assistant/VLM traffic; not a SLED agency deployment; embedding path depends on proprietary PinCLIP.","streamIds":["nvidia"],"roles":{"sales":"Open with the buyer multimodal assistant latency and image-count problem, then cite Pinterest Dynamo+B200 stack as existence proof that disaggregated VLM serving and projection embeddings can unlock large visual context without proportional TTFT blowups. Bound the pitch to Kubernetes-native operators willing to own Dynamo CRDs and KV tiers—do not sell Summit slides as a turnkey SLED appliance. Keep claims bounded to the inspected evidence and require site-matched proof before any capacity or interconnect commitment.","engineering":"Fit for teams already on GPU Operator/Kubernetes and needing multimodal Chat Completions. Integrate DynamoGraphDeployment, vLLM, LMCache offload, and optional projector modality; prerequisites include EKS-class cluster, model artifact contracts, and AIPerf-style multimodal benchmarks. Proof of value: reproduce TTFT/E2E under multi-image multi-turn load with and without KV-aware routing. Document configuration pins and failure modes so the next upgrade cannot silently invalidate the measured result.","delivery":"Work is platform enablement: Helm install Dynamo operator, define CRDs, inject mesh/sidecars, stand up embedding artifact pipeline, and gate go-live on AIPerf DAG SLOs. Ownership: AI platform SRE with ML infra. Risks: pixel-path cost surprises, KV eviction under board-scale contexts, and over-claiming proprietary embedding gains. Record acceptance evidence in the program archive so later editions can cite what was actually proven."},"roleDetails":{"sales":{"customerProblem":"Multimodal assistants stall on TTFT/cost when requests carry many images and long visual history.","stakeholders":["AI platform","product","infra/SRE","security"],"discoveryQuestions":["How many images per request in peak traffic?","Do you already run Dynamo or only NIM?","Can you serve precomputed embeddings instead of pixels?"],"valueHypothesis":"Dynamo disaggregation plus embedding payloads can cut TTFT enough to make campus multimodal assistants operable on B200-class GPUs.","potentialEngagement":"Four-week Dynamo VLM pilot on one assistant workflow with AIPerf DAG baselines.","unsupportedClaims":["85x TTFT for every agency","PinCLIP available off-the-shelf","Turnkey SLED appliance"]},"engineering":{"fit":"Strong for K8s GPU platforms; weak for VM-only or unmanaged bare metal without operators.","architectureIntegration":"DynamoGraphDeployment with E/P/D workers, vLLM backend, LMCache tiers, multimodal router, optional projector path.","prerequisites":["GPU Operator","Dynamo operator Helm","artifact registry","multimodal benchmark harness"],"constraints":["Operator-specific embeddings","EKS/sidecar conventions","prefill-heavy VLM traffic"],"security":"Harden image fetch paths and embedding stores; no CC claims in source.","proofOfValue":["TTFT/E2E under multi-image load","KV hit-rate logs","pixel vs embedding A/B"]},"delivery":{"work":"Stand up Dynamo serving path, projector contract, and AIPerf regression gates.","dependencies":["NVIDIA Dynamo SE","ML training for projector artifacts","platform networking"],"ownership":"AI platform engineering with product ML co-owner.","skillsAdoption":"Teams need Dynamo CRD literacy and multimodal payload design, not only model fine-tuning.","governanceCheckpoints":["No production without multi-turn multimodal SLO","Embedding PII review","Rollback to pixel path documented"],"acceptanceCriteria":["p99 TTFT within product budget","Success rate under target QPS","KV offload stable"],"risks":["KV pressure under board-scale contexts","Frontend parse latency on large embeddings","Vendor roadmap drift"]}},"retrievedAt":"2026-09-20T03:05:00Z","enrichedAt":"2026-09-20T03:13:48Z","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Faster multimodal assistants can improve discovery UX; no accessibility conformance claims.","procurementImplications":"Buy platform enablement and GPU capacity, not a Pinterest-clone SKU; require operator-owned benchmarks in RFP.","operatingModelImplications":"Shifts GenAI from per-team serving snowflakes to a shared Dynamo paved path.","sourceVerification":{"openedUrl":"https://medium.com/pinterest-engineering/building-pinterests-vlm-serving-stack-on-nvidia-dynamo-0dce6e93d0f3","referenceExcerpt":"We built Pinterest’s VLM serving stack on top of NVIDIA Blackwell GPUs and NVIDIA Dynamo.","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}