{"resourceId":"rutgers-machine-translation-government-vital-documents-study-2026","versions":[{"version":"external-33cbf2bf2d006145c848e723dbd9db11dee74f0eb222a757e4d4419d4b464a51","resource":{"id":"rutgers-machine-translation-government-vital-documents-study-2026","title":"Rutgers study with 27 professional translators finds GPT-4 beat DeepL and Google Translate and, with human post-editing, matched human translation quality for government documents","organization":"Rutgers University (Stephanie A. Rodríguez, Miguel A. Jiménez-Crespo)","sector":"Government language access research","geography":"United States (English to Spanish)","publishedAt":"Published online 2026-07-09; Digital Translation 13:2 (2026), pp. 125-153","publicationDate":"2026-07-09","eventDate":null,"sourceName":"John Benjamins, Digital Translation","sourceLabel":"Digital Translation journal article (open access)","sourceUrl":"https://benjamins.com/catalog/dt.25018.rod","evidenceClass":"academic-research","outcomeClass":"effective","topics":["accessibility-workforce","knowledge-work"],"finding":"Twenty-seven professional translators performed human translation, machine translation post-editing and quality evaluation of government legal documents from English to Spanish. Quality was scored with an adapted MQM framework and effort measured with keylogging. Post-editing significantly reduced completion time compared with human translation; GPT-4 scored higher overall than DeepL and Google Translate; and post-edited output and human translation performed similarly in overall quality.","sledRelevance":"Gives state and local language-access programs evidence that LLM translation plus professional post-editing can meet quality needs for vital documents faster. Included as newly relevant because several cities are starting AI language-access pilots this fall. One language pair only.","evidence":"Controlled study with 27 professional translators, English to Spanish, government legal documents; MQM quality scoring; keylogging for effort. Abstract read; detailed effect sizes not extracted in this run.","architectureImplications":"Supports a workflow of LLM draft plus professional post-editing rather than raw machine output.","governanceImplications":"Evidence favors keeping qualified human post-editors accountable for vital documents.","securityPrivacyImplications":"Using commercial LLMs for documents with personal data would need data-handling review; not addressed in the abstract.","caveats":"Single language pair; GPT-4 specifically, newer models untested; abstract-level reading; peer-review status not stated on the page.","streamIds":["citizen-engagement"],"roles":{"sales":"This is rare controlled evidence for public-sector translation: LLM drafts post-edited by professionals matched human quality and saved time on government legal documents. It supports a bounded pitch for faster vital-document translation with human accountability. It does not support raw AI translation for residents or claims for languages other than Spanish.","engineering":"The tested workflow is LLM draft, professional post-edit, MQM-based review. Engineering should design for translator-facing editing tools, terminology control and audit trails of edits, and plan to re-test with current models and other languages. Data handling for documents with personal information needs separate assessment.","delivery":"Delivery would set up a post-editing pipeline: model drafts, translator queues, MQM-style quality sampling and turnaround tracking. Ownership sits with the language-access program. Proposed acceptance criteria compare post-edited quality scores and turnaround against current human baselines per language. Interpretation of the study's findings."},"roleDetails":{"sales":{"customerProblem":"Language-access mandates are growing faster than translation capacity for vital documents.","stakeholders":["Language access coordinator","Agency legal or compliance staff","Contracted translation vendors"],"discoveryQuestions":["What is current turnaround for vital documents?","Which languages are most backlogged?","Are certified translators available for post-editing?"],"valueHypothesis":"LLM drafting with professional post-editing may cut turnaround while keeping quality similar, based on this English-Spanish study.","potentialEngagement":"A post-editing workflow pilot on a set of Spanish vital documents with quality sampling.","unsupportedClaims":["Results in other languages","Quality of unedited AI output","Specific percentage time savings, which were not extracted here"]},"engineering":{"fit":"Fits document translation workflows with professional reviewers.","architectureIntegration":"LLM translation API feeding a translator editing environment with terminology management and quality scoring.","prerequisites":["Professional post-editors","Terminology glossary","Quality rubric (MQM-style)"],"constraints":["Single language pair evidence","Model versions change"],"security":"Review data sent to LLM providers for personal information.","proofOfValue":["Blind MQM scoring of post-edited vs human translations per language","Timed effort comparison"]},"delivery":{"work":"Introduce LLM-plus-post-editing for vital document translation.","dependencies":["Translator contracts","LLM access approved for document types","Quality rubric"],"ownership":"Language access program manager.","skillsAdoption":"Translators trained in post-editing practice.","governanceCheckpoints":["Data handling approval","Quality sampling each cycle"],"acceptanceCriteria":["Proposed: post-edited quality scores not lower than human baseline","Proposed: measured turnaround reduction per document type"],"risks":["Quality lower in other languages","Translator workforce concerns","Model changes altering quality"]}},"retrievedAt":"2026-10-30T03:09:17Z","enrichedAt":"2026-10-30T03:17:06Z","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Shifts translator work toward post-editing; does not support removing professional translators.","procurementImplications":"Supports buying translation services that combine LLM drafts with certified post-editing.","operatingModelImplications":"Post-editing workflows could raise throughput of existing language-access staff.","peerReview":"unknown","sourceVerification":{"openedUrl":"https://benjamins.com/catalog/dt.25018.rod","referenceExcerpt":"MTPE and HT perform similarly in overall quality.","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}