{"resourceId":"nvidia-hugging-face-nemotron-transcribe-granary-v2-adaptation-case-study-20261028","versions":[{"version":"external-3b71819e9fb9b5041a701366fc1201336b5bfef9e610f75dfb3c6a61aa6d6215","resource":{"id":"nvidia-hugging-face-nemotron-transcribe-granary-v2-adaptation-case-study-20261028","title":"NVIDIA's Hugging Face post details Granary v2 and a medical fine-tuning case study, and advises keeping 100 real labelled clips and spot-checking output","organization":"NVIDIA (published on Hugging Face)","sector":"Speech AI platform (vendor technical blog)","geography":"Global","publishedAt":"2026-10-28","publicationDate":"2026-10-28","eventDate":null,"sourceName":"Hugging Face Blog","sourceLabel":"NVIDIA Nemotron 3.5 Transcribe and Granary v2: Intelligent Multilingual Transcription You Can Prompt and Adapt","sourceUrl":"https://huggingface.co/blog/nvidia/nemotron-transcribe-granary","evidenceClass":"vendor-claim","outcomeClass":"emerging","topics":["knowledge-work","developers-agents","accessibility-workforce"],"finding":"NVIDIA reports the model translates 31 languages and attributes speakers in 13, handles up to eight speakers, and ranks first among open-weight models on the English Open ASR Leaderboard at 5.33% average WER (API models excluded). In a medical case study on Eka Care's Indian doctor benchmark, a better prompt cut WER from 30.8% to 23.8% and fine-tuning to 11.8%; on 2,764 held-out clips WER fell from 22.44% to 12.17% (45.8% relative), while LibriSpeech WER rose from 1.54% to 1.69% and a medical specialist model still led on brand-name drugs. Granary v2's public manifest covers 463,062 hours in 48 languages; source audio must be downloaded under each corpus's licence.","sledRelevance":"Shows the adaptation path an agency or university would follow to tune transcription for local place names, legal or clinical terms and accents, and NVIDIA's own advice to hold out real labelled clips and spot-check names, speaker changes and answers against the recording.","evidence":"Vendor technical blog with benchmark figures and a held-out-speaker case study on a public medical benchmark; NVIDIA-run evaluation.","architectureImplications":"Adaptation trains the speech encoder and uses LoRA on the LLM, about 14 GPU-hours on one GB300; serving via vLLM or SGLang, with hosted endpoints from Baseten and DeepInfra as alternatives.","governanceImplications":"Fine-tuned variants become agency-specific models that need versioning, evaluation records and a rule for when they replace the base model.","securityPrivacyImplications":"Fine-tuning on real recordings means sensitive audio enters training pipelines; hosted endpoints would move audio outside agency control.","caveats":"Case study is medical and Indian-English, not U.S. public-sector audio; general-speech accuracy declined slightly after specialisation; leaderboard figures exclude API models.","streamIds":["nvidia"],"roles":{"sales":"Use this post to show buyers a realistic path, not a finished product. NVIDIA's own case study needed a careful prompt, roughly 245 hours of mixed training audio and a held-out test to cut medical errors, and it still trailed a specialist model on brand names. For a court, county or university, the pitch is a measured tuning project on their vocabulary and accents, priced with evaluation and review included. Avoid quoting the 45.8% figure as a likely local result.","engineering":"Engineering should replicate NVIDIA's method rather than its numbers: set a development prompt naming language and script, measure, then fine-tune only if errors remain. Hold out speakers, not just clips, so results generalise. Track general-speech regression alongside domain gains, since LibriSpeech error rose after tuning. Budget about 14 GPU-hours per adaptation on GB300-class hardware and check licences for any Granary v2 source audio. Avoid hosted endpoints for sensitive recordings unless contracts cover retention and access.","delivery":"Delivery should treat tuning as a governed project with a labelled evaluation set of at least 100 real local clips, as NVIDIA advises. Record the base model, prompt, training data and evaluation results for each released variant. Require reviewers to spot-check names, speaker changes and numbers. Set a rollback rule if a tuned model regresses on ordinary speech. Keep training audio under the same retention and access controls as the source recordings, and document licence terms for any public corpora used."},"roleDetails":{"sales":{"customerProblem":"Generic transcription misses local names, legal or clinical terms and regional accents.","stakeholders":["Court or clerk operations lead","University disability services","IT data science team","General counsel (licensing)"],"discoveryQuestions":["Which terms and names does your current transcription get wrong?","Do you have human-verified transcripts to build a test set?","Can training audio legally be reused under your retention rules?"],"valueHypothesis":"A measured tuning project could reduce domain-term errors on the customer's audio, validated on held-out speakers.","potentialEngagement":"Prompt-then-tune evaluation on a held-out local test set.","unsupportedClaims":["That NVIDIA's 45.8% medical gain transfers to public-sector audio","That tuning removes the need for human review","Cost of hosted endpoints"]},"engineering":{"fit":"Domains with stable vocabularies such as courts, councils and lectures.","architectureIntegration":"NeMo Speech fine-tuning (encoder plus LoRA), vLLM or SGLang serving, optional hosted endpoints.","prerequisites":["Labelled local clips with speaker-level holdout","GPU time for adaptation","Licence review for training corpora"],"constraints":["Case study is medical Indian-English only","General-speech accuracy can regress","Granary v2 audio must be fetched under source licences"],"security":"Keep sensitive training audio on premises; avoid hosted endpoints without contractual controls.","proofOfValue":["Held-out-speaker WER before and after tuning","General-speech regression check","Entity accuracy for local names"]},"delivery":{"work":"Build a local evaluation set, test prompts, tune if needed and release a versioned model.","dependencies":["Labelled clips","GPU capacity","Licence and records sign-off"],"ownership":"Shared IT data team with business-owner sign-off.","skillsAdoption":"Train staff to label clips and review transcripts.","governanceCheckpoints":["Licence review before training","Model release approval with evaluation record"],"acceptanceCriteria":["Proposed: held-out WER improves vs best-prompt baseline","Proposed: no material regression on general speech"],"risks":["Overfitting to tuning clips","Licence breach on training audio","Sensitive audio exposure in pipelines"]}},"retrievedAt":"2026-10-29T02:59:07Z","enrichedAt":"2026-10-29T03:06:28Z","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Spot-checking guidance implies human reviewers remain necessary; agencies need staff who can build and label evaluation sets.","procurementImplications":"Granary v2 audio comes under each source corpus's own licence, so any agency training effort needs licence review; hosted endpoints are separate commercial services.","operatingModelImplications":"A tuning workflow adds data curation, evaluation and model lifecycle tasks to IT or a shared-services team.","sourceVerification":{"openedUrl":"https://huggingface.co/blog/nvidia/nemotron-transcribe-granary","referenceExcerpt":"For your own domain, keep at least 100 real, labeled clips for checkpoint selection","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}