From the Student Success edition of October 31, 2026
Study reported by ThinkPol: ChatGPT ranks essay feedback differently from students (rank correlation 0.34), and undergraduates in focus groups mostly oppose instructors using AI to mark their work
Fong et al. (Computers and Education Open), reported by ThinkPol · Higher education assessment feedback and grading · United States (large public university in the Southwest; undergraduate samples)
- Publisher
- ThinkPol
- Original publication
- ThinkPol, 31 October 2026, reporting a Computers and Education Open article published in October 2026
- Source retrieved
- 2026-11-01
What happened
ThinkPol (31 October 2026) reports a Computers and Education Open study by Carlton J. Fong and co-authors combining two studies of US undergraduates. In the first, 392 students compared 45 instructor-written feedback comments in pairs to produce a ranking, and ChatGPT (GPT-3.5) ranked the same comments; the rank correlation was 0.34. Specificity explained most of the students' ranking, while none of the measured features explained ChatGPT's. In the second, five focus groups with 20 undergraduates generally could not tell AI feedback from an instructor's, and most disapproved of AI marking, citing connection, expertise and fairness. Most accepted AI for grammar checks and a hybrid where instructors review AI drafts, and wanted disclosure.
Why it matters
Feedback quality and trust in grading are core to persistence and learning; the study warns that AI-generated marking may not match what students find useful and may erode trust if undisclosed.
Evidence and measured results
Two-part mixed-methods study as summarised by a news outlet: a ranking comparison and qualitative focus groups.
Limitations and uncertainty
Original article not opened because of a captcha; older model; single-session ranking; small single-site focus groups on a hypothetical scenario.
Put this evidence to work
Lighthouse Advisory interpretation, grounded in this source. Enriched 2026-11-01; this does not change the original publication date. Each role follows the same checklist so takeaways compare across sources.
Sales
Role takeaway
- Customer problem
- Faculty want relief from marking load, but students distrust AI marking and may not find AI-judged feedback useful.
- Stakeholders
- Dean of undergraduate education
- Teaching and learning center
- Faculty senate
- Academic integrity office
- Student government
- Discovery questions
- Are any instructors already using AI to draft feedback?
- Do we have a disclosure rule for staff AI use in grading?
- Which assignments would students accept AI help on?
- Value hypothesis
- Interpretation: a disclosed hybrid feedback model may save instructor time while keeping student trust.
- Potential engagement
- Design of a disclosed AI-drafted feedback pilot in a few large courses, with student ratings of usefulness.
- Unsupported claims
- AI feedback matches instructor quality
- Students accept AI marking
- Results generalise to current models
- Time savings of a specific size
Pre-sales engineering
Role takeaway
- Fit
- Instructor-reviewed AI drafting of feedback in the LMS; not autonomous grading.
- Architecture and integration
- AI drafting step inside the LMS grading view with labelled AI comments, instructor edit and approval, and audit logs.
- Prerequisites
- LMS integration for feedback drafting
- Labelling of AI-drafted comments
- Institution-approved model with data protections
- Constraints
- Single-site evidence
- Older model in the ranking test
- Original paper not reviewed
- Security
- Student submissions must stay under institutional data agreements with no vendor training on student work.
- Proof of value
- Student ratings of feedback specificity for AI-drafted versus instructor-only comments
- Instructor edit rates on AI drafts
- Disclosure shown on every AI-assisted comment
Delivery
Role takeaway
- Work
- Set disclosure rules and run a hybrid AI-drafted, instructor-edited feedback pilot with student evaluation.
- Dependencies
- Faculty volunteers in large courses
- Approved AI tool
- Student survey instrument
- Ownership
- Teaching and learning center, with academic affairs and the academic integrity office.
- Skills and adoption
- Instructors need practice editing AI drafts for specificity and explaining their AI use to students.
- Governance checkpoints
- Disclosure policy approval
- Exclusion list for high-stakes work
- End-of-term student trust review
- Acceptance criteria
- Proposed: 100% of AI-assisted feedback carries a disclosure label
- Proposed: student usefulness ratings no lower than instructor-only feedback
- Proposed: no AI drafting on major projects without opt-in
- Risks
- Undisclosed AI marking
- Generic feedback that lacks specificity
- Perceived double standard between staff and students
Implementation considerations
Lighthouse Advisory interpretation across the operating dimensions a public-sector buyer must settle before this evidence becomes a design. Each note answers the question under its heading for this specific source.
Architecture and integration
What must connect, and where does the AI sit in the workflow?
Any AI feedback tool should keep the instructor in review and record when AI drafted feedback.
Governance
Who approves, reviews and stays accountable for outcomes?
Students wanted to be told when AI is involved in feedback; the authors caution against quietly using AI feedback.
Security and privacy
What data, permissions and controls need testing?
Not addressed in the report; sending student work to AI marking tools raises data handling questions that need institutional review.
Accessibility and workforce
Who is affected, and what skills or accommodations follow?
Instructors teaching large classes face marking load; students accepted AI help mainly for low-stakes and grammar tasks.
Procurement
What should contracts, pricing and exit terms secure?
AI grading features should be evaluated against student judgments of feedback usefulness, not speed alone.
Operating model
Which teams own the service once it runs?
A disclosed hybrid model, where AI drafts and instructors edit, matches what students in the focus groups would accept.
Publication history
- 2026-10-31Student Success · Issue 544 resources
Stable resource ID: thinkpol-fong-chatgpt-feedback-ranking-diverges-students-reject-ai-marking-20261031