From the Anthropic edition of November 7, 2026
aiXiv preprint audit: in a Claude-written education research draft, 13 of 14 references existed but only 9 were correct in every checked field
aiXiv preprint (behavioural audit of LLM-assisted education research) · Education research / higher education · Not stated on inspected page
- Publisher
- aiXiv
- Original publication
- 2026-11-07
- Source retrieved
- 2026-11-08
What happened
Source finding (preprint, not peer reviewed; abstract-level inspection): a retrospective audit of twelve published AI-in-education papers drafted with substantial language-model help (records name Claude, plus Gemini for some planning material). Evidence combined the human orchestrator's error records, a Crossref check of 21 DOIs, an independent lookup of the 14 references in one Claude-written draft, and an audit of 35 items from a second assistant. In the Claude draft, 13 references resolved to real works but only 9 were correct in every field; errors were paraphrased or truncated titles, a wrong initial, stale status or no match. Errors clustered in fine details while the register stayed confident; the checker also produced a false alarm. Authors propose a nine-type error taxonomy and an independence test for verification. Why now: posted Nov 7.
Why it matters
Relevant to universities, schools of education and K-12 research offices where Claude-drafted research reaches teachers and policymakers: existence checks alone overstate citation accuracy, so verification must check fields and claims.
Evidence and measured results
Small retrospective case study; one 14-reference Claude draft; no access to model internals; authors state limits. Only the abstract page was accessible; full methods and author details were not inspected.
Limitations and uncertainty
Preprint on aiXiv (not arXiv), not peer reviewed; very small sample; abstract-only inspection; authorship and model versions not confirmed on the inspected page.
Put this evidence to work
Lighthouse Advisory interpretation, grounded in this source. Enriched 2026-11-08; this does not change the original publication date. Each role follows the same checklist so takeaways compare across sources.
Sales
Role takeaway
- Customer problem
- Researchers and policy staff publish Claude-assisted drafts whose references may exist yet contain wrong details.
- Stakeholders
- Research integrity officer
- Dean of education school
- Institutional research office
- Policy communications lead
- Discovery questions
- How are AI-drafted references checked before publication?
- Do checks verify fields or only existence?
- Who signs off on AI-assisted policy briefs?
- Value hypothesis
- Independent field-level checks can catch errors that existence checks miss.
- Potential engagement
- Verification workflow pilot for AI-assisted research and policy documents.
- Unsupported claims
- Error rates beyond a single 14-reference draft
- Comparisons between Claude and other models
- Peer-reviewed status
Pre-sales engineering
Role takeaway
- Fit
- Fits any group drafting with Claude and citing scholarly sources.
- Architecture and integration
- Pipeline that extracts references, resolves them against Crossref/OpenAlex and diffs fields before human review.
- Prerequisites
- Reference extraction tooling
- Crossref or OpenAlex API access
- Review queue
- Constraints
- Coverage gaps in bibliographic databases
- Checker false positives
- Security
- Minimal; avoid sending unpublished manuscripts to unapproved services.
- Proof of value
- Run field-level checks on recent AI-assisted drafts and count discrepancies
- Measure checker false-alarm rate
Delivery
Role takeaway
- Work
- Add a citation verification gate to AI-assisted research publication.
- Dependencies
- Library or research office support
- Author buy-in
- Tooling access
- Ownership
- Research integrity office with the library.
- Skills and adoption
- Short guidance for authors on the nine error types and verification steps.
- Governance checkpoints
- Verification evidence attached before publication
- Periodic review of error trends
- Acceptance criteria
- Proposed: all AI-assisted outputs pass field-level checks before release
- Proposed: error log maintained by type
- Risks
- Checks skipped under deadline pressure
- Overtrust in automated checkers
Implementation considerations
Lighthouse Advisory interpretation across the operating dimensions a public-sector buyer must settle before this evidence becomes a design. Each note answers the question under its heading for this specific source.
Architecture and integration
What must connect, and where does the AI sit in the workflow?
Citation verification should be an independent step using a different tool or database than the drafter.
Governance
Who approves, reviews and stays accountable for outcomes?
Supports disclosure and verification requirements for AI-assisted research and policy documents.
Security and privacy
What data, permissions and controls need testing?
Limited; primarily research integrity rather than data security.
Accessibility and workforce
Who is affected, and what skills or accommodations follow?
Researchers need time and training for verification; drafting speed gains may be offset.
Procurement
What should contracts, pricing and exit terms secure?
Limited; may inform selection of bibliographic verification tools.
Operating model
Which teams own the service once it runs?
Introduces a verification step owned outside the drafting author.
Publication history
- 2026-11-07Anthropic · Issue 106 resources
Stable resource ID: aixiv-llm-assisted-education-research-reference-audit-claude-20261107