{"resourceId":"columbia-pnas-fabricated-references-500k-science-engineering-papers-20261026","versions":[{"version":"external-811c4c0996178595c7f5e0e11794d31b577dac5652de8b41c34fd77714fd15df","resource":{"id":"columbia-pnas-fabricated-references-500k-science-engineering-papers-20261026","title":"Columbia-led PNAS audit: about 1 in 435 science and engineering papers from early 2026 cite references that cannot be found, up from 1 in 1,600 in 2023","organization":"Columbia University School of Nursing and Data Science Institute (Topaz, Gupta, Zhang) with University of Eastern Finland and University of Turku (Peltonen)","sector":"Research integrity, scholarly publishing and research administration","geography":"International literature (OpenAlex corpus); directly transferable to U.S. university research offices, libraries and presses","publishedAt":"26 October 2026 (Columbia University Irving Medical Center release as republished by Mirage News)","publicationDate":"2026-10-26","eventDate":null,"sourceName":"Mirage News (Columbia University Irving Medical Center release)","sourceLabel":"Columbia University Irving Medical Center, \"Fake References Plague Physics, Engineering and Computer Science Journals, Columbia Audit Finds,\" via Mirage News, 26 Oct 2026; study: Topaz et al., PNAS, DOI 10.1073/pnas.2623869123","sourceUrl":"https://www.miragenews.com/fake-references-plague-physics-engineering-and-1766729/","evidenceClass":"academic-research","outcomeClass":"cautionary","topics":["knowledge-work","governance-procurement"],"finding":"The team drew 500,000 chemistry, physics, materials science, engineering and computer science papers published January 2023 to March 2026 from OpenAlex. An automated tool checked the 21.6 million references that carried a DOI and classified 1,135 references across 619 papers as fabricated when the cited title could not be found online. Two independent experts reviewing a random sample of 400 flagged references confirmed 85.8% as fabricated. Affected papers rose from 6.3 per 10,000 (about 1 in 1,600) in 2023 to 15.4 per 10,000 in 2025 and 23.0 per 10,000 (about 1 in 435) in Q1 2026. Field rates per 10,000 papers: physics 25.6, materials science 18.4, computer science and AI 11.1, engineering 10.3, chemistry 4.3. Most affected papers had one or two fabricated references, but 27 papers with five or more held over a third of all flagged references. The authors extrapolate about 8,500 affected papers among roughly 7 million in these fields; two of the 619 had been retracted at audit time. They recommend automated reference checks at submission, retroactive screening, and recording fabricated references as a distinct integrity category. A follow-on HHS OASH award (ORIIR260088, $249,999) funds screening of up to 12 million medical papers and development of screening protocols with editors and research integrity officers.","sledRelevance":"Directly relevant to vice presidents for research, research integrity officers, libraries and university presses. It extends the team's earlier biomedical audit to physical sciences, engineering and computing and gives a measurable, automatable control (reference verification) that institutions can run on theses, grant applications and faculty submissions.","evidence":"Read the full release text: corpus, DOI-only scope, automated classification rule, 400-reference expert validation (85.8% precision), yearly and per-field rates, distribution of fabricated references per paper, extrapolation, retraction count, recommendations and the follow-on HHS project. The PNAS article page returned 403 to the agent, so methods detail beyond the release was not inspected.","architectureImplications":"Reference checking can be implemented as a pipeline that resolves DOIs and titles against bibliographic databases (OpenAlex, Crossref) at submission time for theses, internal reviews or institutional repositories; non-DOI references need separate handling.","governanceImplications":"The authors argue that one unverifiable reference and five or more should be treated differently; institutional policy should set proportionate responses and record fabricated references as their own category rather than folding them into plagiarism or falsification.","securityPrivacyImplications":"Screening unpublished manuscripts or proposals should run on approved infrastructure; the check needs only reference lists, which limits data exposure.","caveats":"The release attributes no cause to AI use directly; fabricated references are inferred from titles not found online, and 14.2% of the expert-reviewed sample were not confirmed. Only DOI-bearing references were checked, so non-DOI fabrication is unmeasured. Extrapolation to 8,500 papers is an estimate. The PNAS paper itself was not opened (403).","streamIds":["research"],"roles":{"sales":"Lead with an automatable integrity control. A Columbia-led PNAS audit published 26 October found suspected fabricated references in about 1 in 435 science and engineering papers from early 2026, more than triple the 2023 rate, with expert review confirming 85.8% of a sample. Research offices and libraries can act on this with a bounded screening pilot. Do not imply that the audit proves AI caused the problem or that a tool eliminates misconduct; the evidence supports early detection, not guaranteed outcomes.","engineering":"Frame fit around a reference-resolution pipeline. The audit checked only DOI-bearing references against online databases, and 14.2% of expert-reviewed flags were not confirmed, so any deployment needs a human review queue and a plan for non-DOI citations. Integrate with thesis submission, institutional repository ingest or internal grant review. Keep unpublished documents on approved infrastructure and log each check. Validate precision locally against a sample reviewed by librarians before scaling.","delivery":"Treat delivery as standing up a proportionate screening workflow. Work covers choosing an integration point, configuring resolution against OpenAlex or Crossref, defining severity tiers (one flag versus five or more, as the authors suggest), and training reviewers. Ownership sits with the research integrity office and library. Governance checkpoints should cover due process before any finding. Acceptance criteria are proposals, such as measured precision on a local sample, not results observed in the study."},"roleDetails":{"sales":{"customerProblem":"Research offices, presses and graduate schools have no routine way to detect fabricated references before work is published or funded.","stakeholders":["Vice president or provost for research","Research integrity officer","University librarian","Graduate school dean","University press director"],"discoveryQuestions":["Where are references checked today, if anywhere, before submission or deposit?","How would you handle a paper with one unverifiable reference versus five or more?","Which repositories or workflows could host an automated check?"],"valueHypothesis":"An automated reference check at one submission point may surface a small number of high-risk documents early, with human review deciding outcomes.","potentialEngagement":"A scoped pilot screening one document stream (for example theses or internal pre-submission review) with a librarian-validated precision estimate and a severity-tier policy.","unsupportedClaims":["That AI use caused every fabricated reference","That screening eliminates research misconduct","That precision measured in this audit will hold locally","Any cost or time savings"]},"engineering":{"fit":"Fits institutions with thesis, repository or internal review workflows that handle structured reference lists.","architectureIntegration":"Reference extraction plus DOI and title resolution against OpenAlex and Crossref, with a flag queue for human review and an audit log.","prerequisites":["Structured reference extraction from submitted documents","Access to bibliographic APIs","A human review role for flags"],"constraints":["Audit covered only DOI-bearing references","Title-not-found rules produce false positives (14.2% unconfirmed in the expert sample)","Unpublished work must stay on approved infrastructure"],"security":"Process only reference lists where possible, keep unpublished documents on institution-approved systems and log every check.","proofOfValue":["Measure precision on a librarian-reviewed sample of local flags","Test coverage on non-DOI references","Compare flag rates across document types before setting policy thresholds"]},"delivery":{"work":"Select integration point, configure resolution pipeline, define severity tiers and review procedure, train reviewers and run a pilot term.","dependencies":["Library metadata expertise","Research integrity policy owner","Document submission system access"],"ownership":"Research integrity office with the university library operating the screening service.","skillsAdoption":"Reviewers learn to triage flags and distinguish citation errors from fabrication; authors receive guidance on verifying AI-assisted references.","governanceCheckpoints":["Policy approval for proportionate responses by severity","Due-process review before any misconduct referral","Term-end review of false-positive rate"],"acceptanceCriteria":["Locally measured precision reported for a reviewed sample (proposal: at least 80% before expansion)","All flags reviewed by a human within a set time","Documented handling for non-DOI references"],"risks":["False positives harming authors","Overreliance on a single automated rule","Scope creep from screening into surveillance of individual researchers"]}},"retrievedAt":"2026-10-27T03:24:06-05:00","enrichedAt":"2026-10-27T03:24:07-05:00","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Editors, librarians and research integrity staff need training to interpret automated flags and to separate citation errors from fabrication.","procurementImplications":"When licensing reference-checking or manuscript-screening tools, ask for measured precision against expert review and coverage of non-DOI references.","operatingModelImplications":"Treat reference verification as a routine pre-submission and pre-award check owned jointly by libraries and research integrity offices, with escalation for papers with multiple flags.","sourceVerification":{"openedUrl":"https://www.miragenews.com/fake-references-plague-physics-engineering-and-1766729/","referenceExcerpt":"It classified 1,135 references across 619 papers as fabricated when the cited title could not be found in online databases.","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}