From the Research Universities edition of October 26, 2026
Columbia-led PNAS audit: about 1 in 435 science and engineering papers from early 2026 cite references that cannot be found, up from 1 in 1,600 in 2023
Columbia University School of Nursing and Data Science Institute (Topaz, Gupta, Zhang) with University of Eastern Finland and University of Turku (Peltonen) · Research integrity, scholarly publishing and research administration · International literature (OpenAlex corpus); directly transferable to U.S. university research offices, libraries and presses
- Publisher
- Mirage News (Columbia University Irving Medical Center release)
- Original publication
- 26 October 2026 (Columbia University Irving Medical Center release as republished by Mirage News)
- Source retrieved
- 2026-10-27
What happened
The team drew 500,000 chemistry, physics, materials science, engineering and computer science papers published January 2023 to March 2026 from OpenAlex. An automated tool checked the 21.6 million references that carried a DOI and classified 1,135 references across 619 papers as fabricated when the cited title could not be found online. Two independent experts reviewing a random sample of 400 flagged references confirmed 85.8% as fabricated. Affected papers rose from 6.3 per 10,000 (about 1 in 1,600) in 2023 to 15.4 per 10,000 in 2025 and 23.0 per 10,000 (about 1 in 435) in Q1 2026. Field rates per 10,000 papers: physics 25.6, materials science 18.4, computer science and AI 11.1, engineering 10.3, chemistry 4.3. Most affected papers had one or two fabricated references, but 27 papers with five or more held over a third of all flagged references. The authors extrapolate about 8,500 affected papers among roughly 7 million in these fields; two of the 619 had been retracted at audit time. They recommend automated reference checks at submission, retroactive screening, and recording fabricated references as a distinct integrity category. A follow-on HHS OASH award (ORIIR260088, $249,999) funds screening of up to 12 million medical papers and development of screening protocols with editors and research integrity officers.
Why it matters
Directly relevant to vice presidents for research, research integrity officers, libraries and university presses. It extends the team's earlier biomedical audit to physical sciences, engineering and computing and gives a measurable, automatable control (reference verification) that institutions can run on theses, grant applications and faculty submissions.
Evidence and measured results
Read the full release text: corpus, DOI-only scope, automated classification rule, 400-reference expert validation (85.8% precision), yearly and per-field rates, distribution of fabricated references per paper, extrapolation, retraction count, recommendations and the follow-on HHS project. The PNAS article page returned 403 to the agent, so methods detail beyond the release was not inspected.
Limitations and uncertainty
The release attributes no cause to AI use directly; fabricated references are inferred from titles not found online, and 14.2% of the expert-reviewed sample were not confirmed. Only DOI-bearing references were checked, so non-DOI fabrication is unmeasured. Extrapolation to 8,500 papers is an estimate. The PNAS paper itself was not opened (403).
Put this evidence to work
Lighthouse Advisory interpretation, grounded in this source. Enriched 2026-10-27; this does not change the original publication date. Each role follows the same checklist so takeaways compare across sources.
Sales
Role takeaway
- Customer problem
- Research offices, presses and graduate schools have no routine way to detect fabricated references before work is published or funded.
- Stakeholders
- Vice president or provost for research
- Research integrity officer
- University librarian
- Graduate school dean
- University press director
- Discovery questions
- Where are references checked today, if anywhere, before submission or deposit?
- How would you handle a paper with one unverifiable reference versus five or more?
- Which repositories or workflows could host an automated check?
- Value hypothesis
- An automated reference check at one submission point may surface a small number of high-risk documents early, with human review deciding outcomes.
- Potential engagement
- A scoped pilot screening one document stream (for example theses or internal pre-submission review) with a librarian-validated precision estimate and a severity-tier policy.
- Unsupported claims
- That AI use caused every fabricated reference
- That screening eliminates research misconduct
- That precision measured in this audit will hold locally
- Any cost or time savings
Pre-sales engineering
Role takeaway
- Fit
- Fits institutions with thesis, repository or internal review workflows that handle structured reference lists.
- Architecture and integration
- Reference extraction plus DOI and title resolution against OpenAlex and Crossref, with a flag queue for human review and an audit log.
- Prerequisites
- Structured reference extraction from submitted documents
- Access to bibliographic APIs
- A human review role for flags
- Constraints
- Audit covered only DOI-bearing references
- Title-not-found rules produce false positives (14.2% unconfirmed in the expert sample)
- Unpublished work must stay on approved infrastructure
- Security
- Process only reference lists where possible, keep unpublished documents on institution-approved systems and log every check.
- Proof of value
- Measure precision on a librarian-reviewed sample of local flags
- Test coverage on non-DOI references
- Compare flag rates across document types before setting policy thresholds
Delivery
Role takeaway
- Work
- Select integration point, configure resolution pipeline, define severity tiers and review procedure, train reviewers and run a pilot term.
- Dependencies
- Library metadata expertise
- Research integrity policy owner
- Document submission system access
- Ownership
- Research integrity office with the university library operating the screening service.
- Skills and adoption
- Reviewers learn to triage flags and distinguish citation errors from fabrication; authors receive guidance on verifying AI-assisted references.
- Governance checkpoints
- Policy approval for proportionate responses by severity
- Due-process review before any misconduct referral
- Term-end review of false-positive rate
- Acceptance criteria
- Locally measured precision reported for a reviewed sample (proposal: at least 80% before expansion)
- All flags reviewed by a human within a set time
- Documented handling for non-DOI references
- Risks
- False positives harming authors
- Overreliance on a single automated rule
- Scope creep from screening into surveillance of individual researchers
Implementation considerations
Lighthouse Advisory interpretation across the operating dimensions a public-sector buyer must settle before this evidence becomes a design. Each note answers the question under its heading for this specific source.
Architecture and integration
What must connect, and where does the AI sit in the workflow?
Reference checking can be implemented as a pipeline that resolves DOIs and titles against bibliographic databases (OpenAlex, Crossref) at submission time for theses, internal reviews or institutional repositories; non-DOI references need separate handling.
Governance
Who approves, reviews and stays accountable for outcomes?
The authors argue that one unverifiable reference and five or more should be treated differently; institutional policy should set proportionate responses and record fabricated references as their own category rather than folding them into plagiarism or falsification.
Security and privacy
What data, permissions and controls need testing?
Screening unpublished manuscripts or proposals should run on approved infrastructure; the check needs only reference lists, which limits data exposure.
Accessibility and workforce
Who is affected, and what skills or accommodations follow?
Editors, librarians and research integrity staff need training to interpret automated flags and to separate citation errors from fabrication.
Procurement
What should contracts, pricing and exit terms secure?
When licensing reference-checking or manuscript-screening tools, ask for measured precision against expert review and coverage of non-DOI references.
Operating model
Which teams own the service once it runs?
Treat reference verification as a routine pre-submission and pre-award check owned jointly by libraries and research integrity offices, with escalation for papers with multiple flags.
Publication history
- 2026-10-26Research Universities · Issue 482 resources
Stable resource ID: columbia-pnas-fabricated-references-500k-science-engineering-papers-20261026