Lighthouse AdvisorySLED AI Adoption Intelligence

Strategic PartnersIssue 093 records

Red Hat

A small, constraint-heavy Red Hat AI edition. Independent TechTarget coverage (5 Nov) of the Red Hat Government Symposium reports a hybrid model strategy: open-weight models on government-controlled infrastructure for routine work, frontier models for selected analysis, with speaker claims (VA OIG: 85% of about 300 AI processes on open-weight models) and an explicit caveat that hosting costs can offset savings. Red Hat's 2 Nov OpenShift AI 3.5 benchmark reports tiered KV-cache offloading gains of up to 5.8x throughput for one long-context model, with smaller or regressing results elsewhere. Red Hat's 3.5 known-issues documentation shows that shared Models-as-a-Service gateways can allow route hijacking by default settings and that rate limiting and llm-d scheduling can fail silently. No new state, local, K-12 or higher-education Red Hat AI deployment evidence was found for 6-7 Nov; all items require local validation.

Evidence records
3
Cross-source patterns
2
Evidence classes
2 vendor claim1 independent reporting
Outcomes
2 mixed1 cautionary
Source freshness
2 new this fortnight1 undated
Research completed
2026-11-08

Choose a role to see its takeaway beside every record in the ledger.

Patterns across the evidence

Synthesis · Lighthouse Advisory interpretation

2 patterns, each supported by at least two sources

  1. OpenShift AI 3.5 inference gains and controls must be verified on the customer's stack

    Red Hat's KV-offload benchmark shows large but conditional gains (up to 5.8x for one model, a TTFT regression for another, single-copy CephFS), and the 3.5 known issues show that intelligent scheduling and rate limiting can be silently disabled by gateway configuration. Interpretation: neither performance nor governance features should be accepted on configuration alone; both require workload-specific measurement and enforcement tests.

    Operating questionWhich measured tests (trace replay, cache-hit rate, 429 checks, route-attachment checks) will prove that the configured inference optimizations and gateway controls are actually in effect?

    Supporting evidenceRed Hat Developer: Benchmarking tiered KV cache offloading on H100 GPUs in OpenShift AI 3.5Red Hat OpenShift AI Self-Managed 3.5 release notes, Chapter 7 Known issues: MaaS gateway route hijacking, silent rate-limit and scheduler bypass, FIPS limits

  2. Self-hosted open-weight economics depend on platform engineering, not token price alone

    Government Symposium panelists argued routine work can move to open-weight models on owned infrastructure but conceded infrastructure and maintenance can offset savings; Red Hat's benchmark shows that unit capacity on fixed GPUs swings severalfold with KV-cache tiering and working-set fit. Interpretation: the cost case for self-hosting on Red Hat AI hinges on measured utilization and tuning, which agencies must budget skills for.

    Operating questionWhat fully loaded cost per request, including GPU, storage tiers and staff, does the agency achieve on its own routine workloads versus negotiated commercial pricing?

    Supporting evidenceTechTarget (reporting on Red Hat Government Symposium panel; Red Hat, U.S. Department of Veterans Affairs OIG speakers)Red Hat Developer: Benchmarking tiered KV cache offloading on H100 GPUs in OpenShift AI 3.5

Evidence ledger

Full record · every source keeps its link and limitations

3 records
  1. Independent reportingMixedNew this fortnight

    TechTarget: Government uses open-weight AI to control costs and meet mission needs (Red Hat Government Symposium coverage)

    Source finding: TechTarget reports that panelists at the October 2026 Red Hat Government Symposium in Washington, D.C. advocated a hybrid model strategy: frontier models for complex analysis and mission-critical reasoning, smaller or open-weight models run on government-controlled infrastructure for routine tasks. Operator claims (not independently verified): Red Hat's chief architect for health and life sciences, Ben Cushing, said federal health CIOs exhausted annual cloud token credits within about a month; the VA OIG CIO, Lance Jenkinson, said the office has about 300 AI-driven processes, 85% using open-weight models, and created 42,000 patches over several months primarily with open-weight models. The article also states that panelists recommended evaluating models by task and acknowledged open-weight AI can introduce infrastructure and maintenance expenses that offset apparent savings, and that GSA OneGov discounts (e.g., a 40% Anthropic token discount announced October 2026) are an alternative cost lever.

    TechTarget (reporting on Red Hat Government Symposium panel; Red Hat, U.S. Department of Veterans Affairs OIG speakers)United StatesNov. 5, 2026

    Why it matters, evidence and limitations
    Why it matters
    Limited but direct: the panel is federal, but the cost problem (token budgets exhausted by developers, long-context and document workloads) and the routing pattern (task-based model selection between open-weight models on owned infrastructure and frontier models under negotiated contracts) transfer to state CIO offices, county IT and university research computing that are evaluating Red Hat AI / OpenShift AI as a self-hosted inference layer. State and local buyers lack GSA OneGov leverage in the same form, so the comparison set differs.
    Evidence and measured results
    Trade journalism summarizing panel remarks at a vendor-hosted event. Quantities (300 processes, 85% open-weight, 42,000 patches, one-month credit burn) are speaker statements with no method, baseline, accuracy measure or cost comparison published. The DHS January 2025 playbook and NIST CAISI references are cited by the reporter but were not opened in this run.
    Limitations and uncertainty
    Vendor-hosted event; Red Hat speakers have a commercial interest in open-weight self-hosting. Federal sample; no state/local data. No total-cost-of-ownership comparison, no accuracy evaluation, and the VA figures are self-reported by the operator. Event date is reported only as October 2026.
    Independent trade reporting Role takeaways & implementation
  2. Vendor claimMixedNew this fortnight

    Red Hat Developer: Benchmarking tiered KV cache offloading on H100 GPUs in OpenShift AI 3.5

    Vendor-reported benchmark: on OpenShift AI 3.5 (vLLM 0.24.0, llm-d 0.8.0, OpenShift 4.22.2, 8xH100 nodes) replaying an agentic trace averaging about 50,600 input tokens, tiered KV cache offloading raised Nemotron-Ultra-253B single-replica throughput from 0.13 to 0.76-0.77 req/s (up to 5.8x) and cut mean TTFT from 59.7 s to 9.6 s. CPU offload alone gave 3.3x for Nemotron; for Qwen3.6-35B-A3B, whose working set fit the CPU tier, CPU offload captured about 90% of the gain. Results were mixed elsewhere: Gemma 4 31B gains were constrained by GPU memory and its 2-replica mean TTFT regressed from 3.3 s to 4.4 s; at concurrency 128, p95 TTFT exceeded 400 s in every configuration. CephFS results used single-copy (size 1) replication. Red Hat recommends a customer proof of concept using FIO p99 latency and a restore-versus-recompute crossover calculation.

    Red HatGlobal (tests on Red Hat OpenShift and IBM Cloud VPC)November 2, 2026

    Why it matters, evidence and limitations
    Why it matters
    Relevant to universities and state shared-services data centers planning on-premises long-context or agentic inference on OpenShift AI, where GPU memory is the scarce resource. It provides a reproducible PoC method rather than a SLED deployment; no public-sector workload was tested. Why now: published after the archive's last coverage of OpenShift AI 3.5 inference features and alongside the 5 Nov Red Hat/NVIDIA AI Factory inference reference design, and it was not previously archived.
    Evidence and measured results
    Vendor-run benchmark with stated hardware, software versions, models, trace (SemiAnalysis Weka AgentX trace, 30-minute runs, seed 42), FIO storage method and per-configuration results. No independent replication; single trace; no cost-per-request figures; CephFS minimal replication; preemption contribution not isolated (authors state an ablation would be required).
    Limitations and uncertainty
    Vendor self-benchmark; not peer reviewed or independently reproduced. Results depend on model, block size, concurrency and storage tail latency. CephFS used size-1 replication, so durability differs from production defaults. No energy or cost measurements. Not a SLED deployment.
    Vendor engineering benchmark Role takeaways & implementation
  3. Vendor claimCautionaryUndated source

    Red Hat OpenShift AI Self-Managed 3.5 release notes, Chapter 7 Known issues: MaaS gateway route hijacking, silent rate-limit and scheduler bypass, FIPS limits

    Source finding (vendor-documented defects): Red Hat lists known issues in OpenShift AI 3.5 that affect governed model serving. RHOAIENG-80360: default Gateway settings such as allowedRoutes.namespaces.from: All let a user with standard namespace-level permissions attach an HTTPRoute that intercepts traffic to a legitimate model endpoint, exposing API keys, prompts and responses. RHOAIENG-76586: with Red Hat Connectivity Link 1.4.0/1.4.1, rate limiting for batch gateways silently stops. INFERENG-6962: any extra HTTPRoute on a wildcard listener bypasses the llm-d EndpointPicker, silently disabling prefix-cache and load-aware scheduling. RHOAIENG-71638: MaaS returns HTTP 500/503 under high concurrency due to a 200 ms auth timeout (affects 3.4 and 3.5). RHOAIENG-58765: llm-d prefill/decode disaggregation fails on FIPS-enabled clusters in 3.4 EA2. RHOAIENG-82694: MCP Lifecycle Operator can run out of memory on large clusters, with no workaround. Workarounds are documented for most items.

    Red HatGlobalUndated documentation page (release notes describe OpenShift AI 3.5.2)

    Why it matters, evidence and limitations
    Why it matters
    Directly relevant to state agencies, counties and universities considering OpenShift AI Models-as-a-Service as a shared, multi-department model gateway: the documented route-hijack risk and silent rate-limit failure bear on tenant isolation and chargeback, and FIPS constraints matter for CJIS- or federal-data-adjacent environments. Why now: the 5 Nov Red Hat/NVIDIA AI Factory expansion (archived) promotes a MaaS + AI gateway reference design with access policies and rate limits; these known issues define the operating conditions for that design. Not previously archived.
    Evidence and measured results
    Vendor product documentation listing defect IDs, symptoms, affected versions and workarounds. No exploitation reports, prevalence data or severity scores are given on this page.
    Limitations and uncertainty
    Undated living documentation; contents can change as fixes ship. Some items are from early-access builds (EA1/EA2) and may not apply to GA deployments. Vendor-reported; no independent validation or incident data. Severity and exploitability are not rated here.
    Vendor product documentation Role takeaways & implementation

How to read this edition

Source findings, measured results and limitations come from the cited publications. Patterns, operating questions, role takeaways and implementation considerations are Lighthouse Advisory interpretation, stated as questions to validate locally rather than guaranteed outcomes. Vendor and operator claims are labeled as claims. Full research method.

Vendor claim
A supplier-provided assertion that has not been upgraded to independent evidence.
Independent reporting
Independent reporting with attributable sources but without a formal evaluation design.