Atos proof of value for a German public-sector customer: fine-tuned small models on OpenShift AI for structured extraction from social media

Atos (Red Hat partner) for an unnamed German public-sector customer; published by Red Hat · Public sector (unnamed German government organization); hate-speech and violence-signal analysis of public social media · Germany (non-US; transferability caveat applies)

Red Hat

Publisher: Red Hat Developer
Original publication: Red Hat Developer article, Oct. 27, 2026 (last updated Oct. 30, 2026); companion Red Hat blog Oct. 28, 2026
Source retrieved: 2026-10-31

Read original source

What happened

Vendor and partner claim: in a proof of value for a German public-sector customer that required self-hosted models only, Atos used OpenShift AI (workbenches, Kubeflow Trainer) to fine-tune Phi-4 Mini 3.8B, Qwen 2.5 1.5B and Qwen 2.5 7B on synthetic data generated by an API-hosted teacher model, to extract 14 features from German social media comments into JSON and detect calls to violence. Training used 3 NVIDIA L40 GPUs. On a custom 34-point rubric across 39 evaluation runs, fine-tuned Qwen 2.5 7B scored 30.8 versus 30.5 for the teacher with a detailed prompt, and fine-tuned Phi-4 Mini about 30. Phi-4 Mini JSON validity rose from 1% (base) to about 99%, against 96% for the teacher. The companion Red Hat blog says fine-tuned Qwen 2.5 7B outperformed GPT-4.1 and Phi-4 Mini came very close; neither page explicitly states that GPT-4.1 was the teacher. Production deployment is described as a next step, not achieved.

Why it matters

Directly relevant to state and local agencies, fusion and public-safety analysts, and university research groups that cannot send sensitive text to hosted frontier APIs. It shows a pattern (synthetic data plus small-model fine-tuning on a self-hosted platform) for high-volume structured extraction. The customer is German and the use case is content monitoring, which in US SLED raises civil-liberties and First Amendment questions that the source does not address.

Evidence and measured results

Read the full developer article and the companion blog. The article reports the rubric design (28 points extraction, 3 JSON, 3 violence detection), 27 fine-tuned variants and 12 baselines, and headline scores read from figures. It does not report the size of the evaluation set, who graded the extraction answers, inter-rater agreement, confidence intervals, or whether evaluation data was independent of the teacher that produced training data. The article's lead says a 1-billion-parameter model beat the teacher, but the smallest model listed is 1.5B and the reported top score is the 7B model, so that claim is internally inconsistent.

Limitations and uncertainty

Vendor and partner self-report of a proof of value; unnamed customer; non-US context; no independent replication; evaluation set size and grading method not disclosed; internal inconsistency about a '1-billion parameter' model; production deployment not yet achieved.

Put this evidence to work

Lighthouse Advisory interpretation, grounded in this source. Enriched 2026-10-31; this does not change the original publication date. Each role follows the same checklist so takeaways compare across sources.

Sales

Role takeaway

Customer problem: Agencies need to extract structured data from large volumes of sensitive text but cannot use hosted frontier APIs because of data-sovereignty or classification rules, and per-token costs at scale are high.
Stakeholders: Agency CIO or CDO
Program owner for the extraction workload
Privacy officer and legal counsel
CISO
Systems integrator partner
Discovery questions: Which high-volume text workload needs structured output, and what does an error cost?
Are hosted model APIs prohibited for this data, including for synthetic-data generation?
Do you have, or can you build, a labeled gold-standard evaluation set?
What GPU capacity exists on premises today?
Who would own retraining and evaluation after a pilot?
Value hypothesis: For one narrow, well-defined extraction task, a tuned small model on self-hosted OpenShift AI may reach accuracy comparable to a frontier model with lower inference cost and no data leaving agency control, subject to the agency's own evaluation.
Potential engagement: A 6 to 8 week proof of value on one extraction workload with an agency-owned evaluation set, comparing a tuned small model against a frontier or existing baseline.
Unsupported claims: That small models outperform frontier models in general
Any specific cost savings or latency figures for a SLED workload
That the approach is production-proven (the source describes a proof of value)
That the evaluation was independent or human-graded at scale
That a 1-billion-parameter model beat the teacher (the source is inconsistent)

Pre-sales engineering

Role takeaway

Fit: Good fit for customers already on OpenShift with some GPU capacity and a narrow, schema-bound text task; poor fit for open-ended assistant use cases.
Architecture and integration: OpenShift AI workbenches and Kubeflow Trainer jobs for SFT/OSFT; synthetic data generation via an SDG flow calling a teacher model; tuned models served on OpenShift AI model serving and consumed by downstream systems expecting JSON.
Prerequisites: OpenShift with OpenShift AI installed
At least a few data-center GPUs (the source used 3 NVIDIA L40)
A representative, labeled evaluation set
Approval to use (or replace) an external teacher model for synthetic data
Constraints: Non-US data and language; results may not transfer to English-language SLED records
Teacher API use may be prohibited for CJIS, HIPAA or FERPA data
Interpretive tasks such as tonality were harder for all models
Security: Inference stays on premises, but the synthetic-data stage sends source-derived prompts to an API-hosted teacher unless a self-hosted teacher is used. Treat tuned model weights and training data as sensitive assets.
Proof of value: Score tuned and baseline models on a held-out, human-labeled set not produced by the teacher
Measure JSON validity and field-level accuracy on real production samples
Report inter-rater agreement for rubric scoring
Measure inference cost and latency against the current approach

Delivery

Role takeaway

Work: Define the task schema and rubric, build the gold set, generate and review synthetic data, run fine-tuning variants, evaluate against baselines and decide go/no-go.
Dependencies: GPU availability on the cluster
Data access approvals for seed data
Subject-matter experts to label and grade
Partner or internal ML engineering capacity
Ownership: The agency program owner owns the rubric and acceptance; the platform team owns OpenShift AI and model serving; a named ML owner handles retraining.
Skills and adoption: Train agency staff on rubric design, evaluation and reading model error reports; platform staff on Kubeflow Trainer and model serving.
Governance checkpoints: Privacy and legal review of the use case before data collection
Approval of synthetic-data generation method
Evaluation sign-off before any operational use
Periodic re-evaluation after deployment
Acceptance criteria: Proposed: tuned model meets an agency-set field-accuracy threshold on the held-out gold set
Proposed: JSON validity of at least 98% on production samples
Proposed: inference cost per 1,000 records below the agreed baseline
Proposed: all outputs used for decisions reviewed by a person
Risks: Evaluation leakage from teacher-generated data inflating scores
Model drift as language and topics change
Civil-liberties exposure if used for speech monitoring
Lack of in-house skills to retrain

Implementation considerations

Lighthouse Advisory interpretation across the operating dimensions a public-sector buyer must settle before this evidence becomes a design. Each note answers the question under its heading for this specific source.

Architecture and integration

What must connect, and where does the AI sit in the workflow?

A self-hosted pattern: an API-hosted teacher generates synthetic training data, then small models are trained on modest on-premises GPUs and served locally. Agencies must decide whether using an external teacher API on source data is itself acceptable under their data rules, since the article says the teacher was accessed by API.

Governance

Who approves, reviews and stays accountable for outcomes?

Custom, task-specific rubrics replace generic benchmarks, so the agency must own the rubric, the gold standard and periodic re-evaluation. Use of AI to flag violent or hateful speech needs documented policy, human review and appeal paths before any operational use.

Security and privacy

What data, permissions and controls need testing?

Keeping inference on premises reduces exposure of sensitive text to third parties, but the synthetic-data step sent source-derived prompts to an API-hosted teacher. Data classification of the seed data determines whether that step is allowed.

Accessibility and workforce

Who is affected, and what skills or accommodations follow?

Requires ML engineering skills for fine-tuning and evaluation design that many SLED IT teams lack; partner-led delivery is likely. No end-user accessibility discussion.

Procurement

What should contracts, pricing and exit terms secure?

Shows a partner-delivered proof of value model (systems integrator plus Red Hat platform). Buyers should require the evaluation set, rubric and scoring method as deliverables and avoid paying for unproven production savings.

Operating model

Which teams own the service once it runs?

Moves cost from per-token API spend to owned GPU capacity and an internal model-maintenance function (retraining, drift monitoring, rubric upkeep).

Publication history

2026-10-30Red Hat · Issue 026 resources

Read preserved resource versions (JSON)

Stable resource ID: atos-german-public-sector-slm-fine-tuning-openshift-ai-pov-20261027