{"resourceId":"dartmouth-herron-levin-ai-column-evaluation-20260929","versions":[{"version":"external-432fed95ad3eb9ead4be197e240a8cf6a62c8b315815b3e212ab482b802fc94e","resource":{"id":"dartmouth-herron-levin-ai-column-evaluation-20260929","title":"Dartmouth faculty evaluation finds substantive AI-characteristic errors in provost Washington Post column","organization":"The Dartmouth (Herron and Levin)","sector":"Higher education faculty governance / administrative communications integrity","geography":"United States (Dartmouth College)","publishedAt":"September 29, 2026","publicationDate":"2026-09-29","eventDate":"2026-09-29","sourceName":"The Dartmouth","sourceLabel":"Faculty guest analysis independently evaluating substantive content of provost AI-linked column","sourceUrl":"https://www.thedartmouth.com/article/2026/09/herron-and-levin-our-independent-evaluation-of-provost-schnells-washington-post-column","evidenceClass":"independent-reporting","outcomeClass":"cautionary","topics":["governance-procurement","knowledge-work","operating-model"],"finding":"On September 29, 2026 Dartmouth professors Michael Herron and Andrew Levin published an independent substantive evaluation of Provost Santiago Schnell Washington Post guest column, arguing that definitive conclusions cannot rest on proprietary detectors alone. They report content failures they consider characteristic of AI-generated text and practically inconceivable from a human measurement expert: a non sequitur shift from academic-integrity framing to capability revelation; misstatement of the Barcaui 2025 study effective sample (abstract n=120 versus 85 retention-test completers underlying key results); omission of convenience-sampling limits and later related studies; near-reproduction of Dartmouth press-release language without citation; and a conflict between acknowledged ChatGPT refinement and The Washington Post attestation that submissions were not created or manipulated with AI. They warn AI-related rule enforcement at Dartmouth may become unworkable without transparent resolution. Authors state AI was used only for background research, not analysis, writing, or editing.","sledRelevance":"Independent faculty scrutiny of administrative public AI use sharpens campus-operations controls for disclosure, attestation, and quality review of leadership communications. Why now: published September 29, 2026 and not previously archived; complements same-day IHE reporting with content-level evidence.","evidence":"Attributed faculty guest analysis with specific methodological critiques of one published column. Not a formal audit or peer-reviewed study; personnel process ongoing.","architectureImplications":"Admin AI writing stacks need source-citation checks, sample-size/methods verification steps, and attestation gates before external publication—detector scores are not a substitute architecture.","governanceImplications":"Shows why role-appropriate rules must define substantial AI contribution, required disclosure, and publisher attestation handling; also why detector-only adjudication is inadequate for high-stakes leadership cases.","securityPrivacyImplications":"Primary risk is integrity and false attestation rather than data breach; still implies need to retain prompt/edit histories for accountable review when leaders dispute AI extent.","caveats":"Single-essay faculty analysis; campus-ops relevance limited to admin communications governance. Contested personnel matter. Transferability is about policy design, not a general AI accuracy benchmark.","streamIds":["campus-operations"],"roles":{"sales":"When campuses buy AI writing aids for administrators, buyers need quality-control and attestation workflows—not detector theater. Use this analysis to ask how sample-size/methods checks and publisher rules are enforced. Bound engagement to disclosure/QC design; do not sell that any detector would have prevented these substantive errors.","engineering":"Design pre-publication checks for citations, numeric claims against primary sources, and attestation confirmation. Prerequisites: access to source papers and a human methods reviewer for leadership pieces. Constraints: one contested column. Proof of value is a checklist that catches abstract-vs-body sample mismatches before publish.","delivery":"Stand up a lightweight leadership-communications QC lane: disclosure statement, primary-source verification for cited studies, and publisher-attestation confirmation. Ownership: communications with faculty methods advisors as needed. Acceptance: checklist used on next external op-ed. Risks include politicized enforcement and over-blocking legitimate editing assistance."},"roleDetails":{"sales":{"customerProblem":"Administrative AI writing can introduce factual/method errors and false publisher attestations that detectors alone will not catch.","stakeholders":["Communications / public affairs","Provost office","Faculty governance","General counsel","CIO / academic technology"],"discoveryQuestions":["Who verifies cited study methods before leadership op-eds publish?","What counts as AI manipulation under target publishers rules?","Are prompt/edit histories retained for contested pieces?","Is detector output ever used as sole evidence?"],"valueHypothesis":"A human methods-and-attestation QC lane for AI-assisted leadership writing can catch abstract/body mismatches and disclosure failures this analysis highlights—without claiming automated perfection.","potentialEngagement":"Ten-day communications QC design producing a pre-publication checklist and attestation workflow for executive external writing.","unsupportedClaims":["Detectors alone prove authorship percentage","All AI-assisted editing is unethical","This analysis adjudicates the personnel outcome"]},"engineering":{"fit":"Quality and attestation controls for AI-assisted administrative public writing.","architectureIntegration":"Drafting tools with disclosure metadata; citation/source-check workflow; human methods review queue; attestation confirmation before submission.","prerequisites":["Publisher rule library","Access to cited primary sources","Defined substantial-contribution threshold","Human reviewer availability"],"constraints":["Single-essay evidence base","Contested facts in personnel case","No vendor architecture specified"],"security":"Limit access to draft histories; protect personnel investigation materials; avoid covert monitoring of routine staff AI assistance.","proofOfValue":["Checklist flags an abstract/body sample mismatch in a test draft","Attestation confirmation recorded before mock submission","Policy states detectors are insufficient alone"]},"delivery":{"work":"Publish checklist, train communications staff, route high-visibility pieces through methods/attestation review, and document outcomes for governance.","dependencies":["Communications","Faculty reviewers","Provost office","Counsel for attestation language"],"ownership":"Communications owns the QC lane; provost owns policy thresholds; faculty advisors support methods checks.","skillsAdoption":"Short training for speechwriters/comms on verifying empirical claims and disclosure language.","governanceCheckpoints":["Policy threshold for mandatory review","Pre-submission attestation sign-off","Post-publication correction protocol"],"acceptanceCriteria":["Checklist adopted for executive external AI-assisted pieces","At least one dry-run completed","Detector-only rule prohibited in writing"],"risks":["Selective enforcement","Review bottleneck delays publishing","Dispute over what counts as substantial AI contribution"]}},"retrievedAt":"2026-09-29T21:58:00-05:00","enrichedAt":"2026-09-29T22:04:43-05:00","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Faculty authors model transparent AI-use norms (background research only) while criticizing opaque leadership practice—workforce implication is legitimacy of shared academic standards, not measured staffing change.","procurementImplications":"Discourages procurement narratives that equate AI-detector licenses with authorship governance; favors process/tooling for citation verification and attestation management.","operatingModelImplications":"Adds a quality-control operating lane for AI-assisted executive communications alongside broader campus AI adoption programs.","sourceVerification":{"openedUrl":"https://www.thedartmouth.com/article/2026/09/herron-and-levin-our-independent-evaluation-of-provost-schnells-washington-post-column","referenceExcerpt":"This analysis identifies substantive errors and omissions that are characteristic of AI-generated text","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}