{"resourceId":"arxiv-ai-detector-editing-style-confound-20260827","versions":[{"version":"external-eb1ea8c9a9f751f8830643e6a926de98446b6f9cbd3752bd9b89319d7b61139e","resource":{"id":"arxiv-ai-detector-editing-style-confound-20260827","title":"Large paired-edit study: AI detector false positives span 0 to 100% and track editing style","organization":"Chung-Ang University and Wordvice (Park, Jeong, Kim)","sector":"Academic AI/NLP research","geography":"International academic writing corpus (implications for multilingual learners)","publishedAt":"27 August 2026 (arXiv 2608.26710v1)","publicationDate":"2026-08-27","eventDate":null,"sourceName":"arXiv","sourceLabel":"Preprint PDF","sourceUrl":"https://arxiv.org/pdf/2608.26710","evidenceClass":"academic-research","outcomeClass":"cautionary","topics":["governance-procurement","accessibility-workforce","knowledge-work","data-security"],"finding":"Park, Jeong, and Kim analyze 135,389 original vs professionally edited non-native academic manuscript pairs across 13 AI text detectors. False-positive rates on human-written texts ranged from 0.0% to 100.0%. The same edits increased AI scores in some detectors and decreased them in others, with score changes correlating with editing intensity; practical significance thresholds were met in only 7 of 13 detectors. Authors identify professional editing style as a key confound rather than cleanly separating origin from linguistic form, raising fairness and reliability concerns in academic settings. Why now: as districts tighten integrity rules alongside AI pauses/pilots, detector-only misconduct findings are especially risky for multilingual K-12 writers.","sledRelevance":"Supports K-12 academic-integrity policy that forbids sole reliance on AI detectors—aligned with developmental fairness for multilingual students and with process/evidence-based misconduct review.","evidence":"Large-scale paired natural experiment holding content/authorship fixed while varying professional editing; detector outputs compared pre/post; academic manuscripts 2018 to 2025, not K-12 classroom essays.","architectureImplications":"If detectors are used at all, treat scores as triage signals requiring human review and process evidence; avoid automated grade penalties from detector APIs.","governanceImplications":"Update integrity codes: detectors are not proof; disclose limitations to teachers/families; prefer assignment redesign and oral/process defenses over surveillance scoring.","securityPrivacyImplications":"Sending student writing to third-party detectors creates additional data-sharing risk; prefer local process review where possible.","caveats":"Corpus is professional academic editing, not K-12 essays; one coauthor affiliated with editing vendor Wordvice; detector landscape changes quickly; transfer to schools requires caution.","streamIds":["k12"],"roles":{"sales":"Sales framing: do not sell AI-detector suites as definitive cheating proof. This study shows human FPRs can be extreme and that polished style itself moves scores. Offer integrity-process redesign (assignment design, process evidence, limited detector triage)—without guaranteeing detection accuracy.","engineering":"Engineering framing: if detectors exist, wire them as advisory flags with mandatory human review queues; log model/version; do not auto-fail. Prerequisites: privacy review of detector vendors. Constraints: K-12 transferability limits. Proof of value: human overturn rates and disparity monitoring for multilingual students—not raw AI-percent dashboards.","delivery":"Delivery framing: revise misconduct SOPs, train teachers on detector limits, and monitor false-positive disparities. Ownership: academics and equity/ELL leads. Acceptance criteria: written ban on detector-only sanctions; documented review checklist. Risks include discriminatory impacts on multilingual writers and vendor lock-in to unreliable scores."},"roleDetails":{"sales":{"customerProblem":"Schools want AI-cheating controls but risk false accusations, especially for multilingual writers.","stakeholders":["Principals","Teachers","ELL/equity leads","Students/families","Counsel"],"discoveryQuestions":["Are detector scores used as sole evidence today?","How are multilingual students represented in flagged cases?","What process evidence is required before sanctions?","Which detector vendors receive student text?"],"valueHypothesis":"An integrity-process engagement can reduce false positives by demoting detectors to advisory signals.","potentialEngagement":"2 to 3 week academic-integrity AI review: SOP rewrite, teacher briefing, and disparity monitoring plan.","unsupportedClaims":["Any detector is accurate enough for automatic sanctions","Results transfer unchanged to K-12 essays","Zero false positives after process changes"]},"engineering":{"fit":"Advisory detection with human review; poor fit for automated grade invalidation.","architectureIntegration":"Optional detector API behind teacher review UI; version pinning; retain human decision records; minimize data sent offsite.","prerequisites":["Integrity SOP update","Privacy/DPA for detector vendor","Teacher training module"],"constraints":["Academic corpus is not K-12","Detector drift","Vendor conflict disclosures"],"security":"Minimize student-text egress; prefer on-prem/process alternatives; purge detector vendor copies per DPA.","proofOfValue":["Human overturn rate of flags","Flag rates by language status","Elimination of auto-sanctions"]},"delivery":{"work":"Rewrite misconduct procedures, train staff, stand up disparity dashboards, and communicate limits to families.","dependencies":["Counsel","ELL office","LMS admins","Communications"],"ownership":"Academic integrity lead owns SOP; equity office monitors disparate impact.","skillsAdoption":"Train teachers to gather process evidence and to interpret detector scores as hypotheses only.","governanceCheckpoints":["SOP adoption","Training completion","Quarterly disparity review","Vendor DPA check"],"acceptanceCriteria":["No detector-only sanctions","Documented human review on all flags","Public FAQ on limitations"],"risks":["Discriminatory false positives","Silent auto-grading integrations","Parent trust loss"]}},"retrievedAt":"2026-09-25T03:10:55Z","enrichedAt":"2026-09-25T03:17:50Z","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"False positives tied to polished/non-native style threaten multilingual and edited student writing; equity monitoring is essential.","procurementImplications":"If purchasing detectors, require transparency on FPR testing, human-review workflows, and data deletion—not marketing accuracy claims.","operatingModelImplications":"Shift integrity ops from surveillance scores to assignment design and process evidence.","sourceVerification":{"openedUrl":"https://arxiv.org/pdf/2608.26710","referenceExcerpt":"FPRs for human-written texts varied widely, from 0.0% to 100.0%.","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}