{"resourceId":"atos-german-public-sector-slm-fine-tuning-openshift-ai-pov-20261027","versions":[{"version":"external-61384c947475d54a4e0f4324629ab04595144eecdb3de53a683da0915bff878f","resource":{"id":"atos-german-public-sector-slm-fine-tuning-openshift-ai-pov-20261027","title":"Atos proof of value for a German public-sector customer: fine-tuned small models on OpenShift AI for structured extraction from social media","organization":"Atos (Red Hat partner) for an unnamed German public-sector customer; published by Red Hat","sector":"Public sector (unnamed German government organization); hate-speech and violence-signal analysis of public social media","geography":"Germany (non-US; transferability caveat applies)","publishedAt":"Red Hat Developer article, Oct. 27, 2026 (last updated Oct. 30, 2026); companion Red Hat blog Oct. 28, 2026","publicationDate":"2026-10-27","eventDate":null,"sourceName":"Red Hat Developer","sourceLabel":"Smaller model, sharper insights: How Atos approached structured outputs","sourceUrl":"https://developers.redhat.com/articles/2026/10/27/smaller-model-sharper-insights-how-atos-approached-structured-outputs","evidenceClass":"vendor-claim","outcomeClass":"emerging","topics":["knowledge-work","infrastructure","data-security","developers-agents"],"finding":"Vendor and partner claim: in a proof of value for a German public-sector customer that required self-hosted models only, Atos used OpenShift AI (workbenches, Kubeflow Trainer) to fine-tune Phi-4 Mini 3.8B, Qwen 2.5 1.5B and Qwen 2.5 7B on synthetic data generated by an API-hosted teacher model, to extract 14 features from German social media comments into JSON and detect calls to violence. Training used 3 NVIDIA L40 GPUs. On a custom 34-point rubric across 39 evaluation runs, fine-tuned Qwen 2.5 7B scored 30.8 versus 30.5 for the teacher with a detailed prompt, and fine-tuned Phi-4 Mini about 30. Phi-4 Mini JSON validity rose from 1% (base) to about 99%, against 96% for the teacher. The companion Red Hat blog says fine-tuned Qwen 2.5 7B outperformed GPT-4.1 and Phi-4 Mini came very close; neither page explicitly states that GPT-4.1 was the teacher. Production deployment is described as a next step, not achieved.","sledRelevance":"Directly relevant to state and local agencies, fusion and public-safety analysts, and university research groups that cannot send sensitive text to hosted frontier APIs. It shows a pattern (synthetic data plus small-model fine-tuning on a self-hosted platform) for high-volume structured extraction. The customer is German and the use case is content monitoring, which in US SLED raises civil-liberties and First Amendment questions that the source does not address.","evidence":"Read the full developer article and the companion blog. The article reports the rubric design (28 points extraction, 3 JSON, 3 violence detection), 27 fine-tuned variants and 12 baselines, and headline scores read from figures. It does not report the size of the evaluation set, who graded the extraction answers, inter-rater agreement, confidence intervals, or whether evaluation data was independent of the teacher that produced training data. The article's lead says a 1-billion-parameter model beat the teacher, but the smallest model listed is 1.5B and the reported top score is the 7B model, so that claim is internally inconsistent.","architectureImplications":"A self-hosted pattern: an API-hosted teacher generates synthetic training data, then small models are trained on modest on-premises GPUs and served locally. Agencies must decide whether using an external teacher API on source data is itself acceptable under their data rules, since the article says the teacher was accessed by API.","governanceImplications":"Custom, task-specific rubrics replace generic benchmarks, so the agency must own the rubric, the gold standard and periodic re-evaluation. Use of AI to flag violent or hateful speech needs documented policy, human review and appeal paths before any operational use.","securityPrivacyImplications":"Keeping inference on premises reduces exposure of sensitive text to third parties, but the synthetic-data step sent source-derived prompts to an API-hosted teacher. Data classification of the seed data determines whether that step is allowed.","caveats":"Vendor and partner self-report of a proof of value; unnamed customer; non-US context; no independent replication; evaluation set size and grading method not disclosed; internal inconsistency about a '1-billion parameter' model; production deployment not yet achieved.","streamIds":["red-hat"],"roles":{"sales":"Sales framing: this is a sovereignty-and-cost story for agencies that cannot send sensitive text to hosted frontier models. Atos reports that small models tuned on OpenShift AI matched or slightly beat a frontier teacher on one narrow extraction task. Position it as a bounded proof of value for one high-volume structured-extraction workload, such as records triage or form intake, not as proof that small models beat frontier models generally. Avoid content-monitoring use cases unless legal and civil-liberties review is already in place.","engineering":"Engineering framing: the reusable pattern is synthetic data from a teacher model, supervised and orthogonal-subspace fine-tuning in OpenShift AI workbenches with Kubeflow Trainer, and a task-specific rubric. Validate three things before trusting the numbers: that evaluation data is independent of teacher-generated training data, that the rubric is human-graded with agreement measured, and that JSON validity holds on real production inputs. Confirm whether calling an external teacher API is allowed for the customer's data classification.","delivery":"Delivery framing: plan this as a proof of value with explicit go/no-go criteria, not a production deployment. Work includes defining a gold-standard evaluation set with the agency, generating and reviewing synthetic data, running tuning on a small GPU pool and scoring against a baseline. The agency must own the rubric and retraining after go-live. Budget for human review of model outputs and for drift checks, since the source reports no production operation yet."},"roleDetails":{"sales":{"customerProblem":"Agencies need to extract structured data from large volumes of sensitive text but cannot use hosted frontier APIs because of data-sovereignty or classification rules, and per-token costs at scale are high.","stakeholders":["Agency CIO or CDO","Program owner for the extraction workload","Privacy officer and legal counsel","CISO","Systems integrator partner"],"discoveryQuestions":["Which high-volume text workload needs structured output, and what does an error cost?","Are hosted model APIs prohibited for this data, including for synthetic-data generation?","Do you have, or can you build, a labeled gold-standard evaluation set?","What GPU capacity exists on premises today?","Who would own retraining and evaluation after a pilot?"],"valueHypothesis":"For one narrow, well-defined extraction task, a tuned small model on self-hosted OpenShift AI may reach accuracy comparable to a frontier model with lower inference cost and no data leaving agency control, subject to the agency's own evaluation.","potentialEngagement":"A 6 to 8 week proof of value on one extraction workload with an agency-owned evaluation set, comparing a tuned small model against a frontier or existing baseline.","unsupportedClaims":["That small models outperform frontier models in general","Any specific cost savings or latency figures for a SLED workload","That the approach is production-proven (the source describes a proof of value)","That the evaluation was independent or human-graded at scale","That a 1-billion-parameter model beat the teacher (the source is inconsistent)"]},"engineering":{"fit":"Good fit for customers already on OpenShift with some GPU capacity and a narrow, schema-bound text task; poor fit for open-ended assistant use cases.","architectureIntegration":"OpenShift AI workbenches and Kubeflow Trainer jobs for SFT/OSFT; synthetic data generation via an SDG flow calling a teacher model; tuned models served on OpenShift AI model serving and consumed by downstream systems expecting JSON.","prerequisites":["OpenShift with OpenShift AI installed","At least a few data-center GPUs (the source used 3 NVIDIA L40)","A representative, labeled evaluation set","Approval to use (or replace) an external teacher model for synthetic data"],"constraints":["Non-US data and language; results may not transfer to English-language SLED records","Teacher API use may be prohibited for CJIS, HIPAA or FERPA data","Interpretive tasks such as tonality were harder for all models"],"security":"Inference stays on premises, but the synthetic-data stage sends source-derived prompts to an API-hosted teacher unless a self-hosted teacher is used. Treat tuned model weights and training data as sensitive assets.","proofOfValue":["Score tuned and baseline models on a held-out, human-labeled set not produced by the teacher","Measure JSON validity and field-level accuracy on real production samples","Report inter-rater agreement for rubric scoring","Measure inference cost and latency against the current approach"]},"delivery":{"work":"Define the task schema and rubric, build the gold set, generate and review synthetic data, run fine-tuning variants, evaluate against baselines and decide go/no-go.","dependencies":["GPU availability on the cluster","Data access approvals for seed data","Subject-matter experts to label and grade","Partner or internal ML engineering capacity"],"ownership":"The agency program owner owns the rubric and acceptance; the platform team owns OpenShift AI and model serving; a named ML owner handles retraining.","skillsAdoption":"Train agency staff on rubric design, evaluation and reading model error reports; platform staff on Kubeflow Trainer and model serving.","governanceCheckpoints":["Privacy and legal review of the use case before data collection","Approval of synthetic-data generation method","Evaluation sign-off before any operational use","Periodic re-evaluation after deployment"],"acceptanceCriteria":["Proposed: tuned model meets an agency-set field-accuracy threshold on the held-out gold set","Proposed: JSON validity of at least 98% on production samples","Proposed: inference cost per 1,000 records below the agreed baseline","Proposed: all outputs used for decisions reviewed by a person"],"risks":["Evaluation leakage from teacher-generated data inflating scores","Model drift as language and topics change","Civil-liberties exposure if used for speech monitoring","Lack of in-house skills to retrain"]}},"retrievedAt":"2026-10-31T03:11:38Z","enrichedAt":"2026-10-31T03:19:23Z","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Requires ML engineering skills for fine-tuning and evaluation design that many SLED IT teams lack; partner-led delivery is likely. No end-user accessibility discussion.","procurementImplications":"Shows a partner-delivered proof of value model (systems integrator plus Red Hat platform). Buyers should require the evaluation set, rubric and scoring method as deliverables and avoid paying for unproven production savings.","operatingModelImplications":"Moves cost from per-token API spend to owned GPU capacity and an internal model-maintenance function (retraining, drift monitoring, rubric upkeep).","sourceVerification":{"openedUrl":"https://developers.redhat.com/articles/2026/10/27/smaller-model-sharper-insights-how-atos-approached-structured-outputs","referenceExcerpt":"Before fine-tuning, the Phi-4 Mini base model produced valid JSON only 1% of the time.","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}