{"resourceId":"penn-plos-one-facial-recognition-degraded-police-images-error-rates-20261021","versions":[{"version":"external-10f49c08ab01594855a72640951f5a604751b47fa30305a2fd9b84146a3773d3","resource":{"id":"penn-plos-one-facial-recognition-degraded-police-images-error-rates-20261021","title":"Peer-reviewed University of Pennsylvania experiment finds facial recognition error rates shift sharply with blur, low resolution and lighting in police-style images, with persistent race and gender disparities","organization":"University of Pennsylvania (Department of Criminology; Wharton Statistics and Data Science; School of Engineering)","sector":"Policing / facial recognition accuracy and forensic reliability","geography":"United States","publishedAt":"PLOS One, Oct. 21, 2026","publicationDate":"2026-10-21","eventDate":"2026-10-21","sourceName":"PLOS One","sourceLabel":"Accuracy and fairness of facial recognition technology in real-world, low-quality police images: An experiment","sourceUrl":"https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0353793","evidenceClass":"academic-research","outcomeClass":"cautionary","topics":["data-security","governance-procurement","operating-model"],"finding":"Cuellar, To and Mehrotra generated 50,000 synthetic faces with StyleGAN3, labelled them with FairFace, and built race-and-gender-balanced samples (a 167-face control database and 167 treated probes, 83 target-present and 84 target-absent). They degraded probes by contrast, brightness, motion blur, pose and resolution and ran 1:n identification with DeepFace using ArcFace. Low resolution, high blur and contrast, and low brightness substantially increased the false negative identification rate (FNIR), with reduced resolution the largest effect. At the selected operating point the false positive identification rate (FPIR) was substantially higher than FNIR and highest near baseline image quality, generally falling as degradation increased. Disparities persisted: FPIR was often higher for Black individuals, especially Black women, while the group with the highest FNIR varied by degradation type. The authors argue FRT should not be summarised with one accuracy figure and should be evaluated on the low-quality images used in investigations.","sledRelevance":"Gives police agencies, prosecutors and courts a peer-reviewed basis for asking for image-quality-specific error rates before relying on a facial recognition lead, especially as courts begin to require disclosure of FRT use.","evidence":"Opened the PLOS One article and read the methods, results, discussion and limitations sections in full. Peer-reviewed journal article; data are deposited at Harvard Dataverse (not opened).","architectureImplications":"Suggests attaching image-quality measures (resolution, blur, brightness, contrast, pose) to each probe and reporting condition-specific error rates rather than a single vendor benchmark figure.","governanceImplications":"Supports policies that treat FRT results as investigative leads only and that require validation on agency-like image conditions before use or court reliance.","securityPrivacyImplications":"False positives, higher for some demographic groups, translate into risk of wrongful suspicion; the authors cite at least seven confirmed misidentification cases.","caveats":"Synthetic faces and controlled single-factor degradations, which may not capture real surveillance imagery; one open-source pipeline (DeepFace/ArcFace), not commercial police systems; small 167-face gallery; FairFace labels and simplified race categories; FFHQ-derived faces skewed white before balancing. U.S. academic study.","streamIds":["public-safety"],"roles":{"sales":"Sales framing: this peer-reviewed Penn study shows that facial recognition error rates depend heavily on image quality, and that one accuracy figure hides that. The buyer problem for police and prosecutors is knowing when a match from blurry or low-resolution footage is too risky to act on. A bounded engagement could design a validation protocol using the agency’s own image conditions. The evidence cannot support claims about any commercial police product, because the study tested one open-source pipeline on synthetic faces.","engineering":"Engineering framing: the practical lesson is to measure image quality before a search and report error rates for that condition. Pre-sales should ask which FRT product the agency uses, what threshold it applies, how large the gallery is, and whether probe images are logged with quality metadata. A proof of value would score a sample of past probe images for resolution and blur and compare outcomes against known resolved cases. Do not generalise the study’s numbers to commercial systems or larger galleries.","delivery":"Delivery framing: the work is adding an image-quality gate and documentation step to the investigative FRT workflow: score the probe, record the condition, and flag low-quality searches for extra review. The operational owner is the agency’s FRT unit with the prosecutor’s office for disclosure. Acceptance criteria are proposals: every search logged with quality scores, a written threshold below which results need supervisor review, and disclosure-ready records. Main risk: treating a quality score as proof of reliability."},"roleDetails":{"sales":{"customerProblem":"Police and prosecutors rely on facial recognition leads from degraded footage without knowing how error rates change with image quality.","stakeholders":["Police FRT or real-time crime centre units","Prosecutors and defence counsel","Courts","Civil rights oversight bodies"],"discoveryQuestions":["Which product, threshold and gallery size does the agency use?","Are probe images logged with quality information?","How are FRT leads disclosed in court?"],"valueHypothesis":"Condition-specific validation can reduce wrongful leads and strengthen court-defensible records.","potentialEngagement":"Validation protocol and image-quality gate design for the FRT workflow.","unsupportedClaims":["Error rates for any commercial police FRT product","That quality scoring eliminates bias"]},"engineering":{"fit":"Fits agencies already using FRT for investigative leads.","architectureIntegration":"Image-quality scoring before search; logging of score, threshold and candidate list with the case record.","prerequisites":["FRT search logs","Sample of resolved cases","Image-quality metrics"],"constraints":["Synthetic faces","Single open-source pipeline","Small gallery","Single-factor degradations"],"security":"Search logs contain biometric data and must stay in CJIS-compliant systems with access controls.","proofOfValue":["Retrospective scoring of past probes against resolved outcomes."]},"delivery":{"work":"Add quality scoring, condition logging and supervisor review to the FRT workflow.","dependencies":["Vendor export of search logs","Prosecutor disclosure requirements","Policy approval"],"ownership":"Agency FRT unit with the prosecutor’s office","skillsAdoption":"Analyst training on how image quality shifts false positive and false negative risk.","governanceCheckpoints":["Policy update approval","Periodic audit of low-quality searches","Disclosure review with prosecutors"],"acceptanceCriteria":["All searches logged with quality scores","Review threshold documented","Disclosure-ready records"],"risks":["False confidence in quality scores","Vendor resistance to log export","Over-generalising study numbers"]}},"retrievedAt":"2026-10-30T03:00:22Z","enrichedAt":"2026-10-30T03:06:22Z","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Human reviewers of candidate lists need training on how image quality changes the error profile.","procurementImplications":"Agencies can require vendors to provide error rates under degraded-image conditions and by demographic group, not only NIST-style high-quality benchmarks.","operatingModelImplications":"Moves validation from one-time vendor selection to per-image judgement about whether a probe is fit for search.","peerReview":"peer-reviewed","sourceVerification":{"openedUrl":"https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0353793","referenceExcerpt":"FPIR is often higher for Black individuals, particularly Black women, whereas the demographic groups with the highest FNIR differ across degradation conditions.","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}