From the Student Success edition of October 30, 2026
Manchester engineering academic replaces AI-proofing with a do, augment, challenge, defend sequence, citing ChatGPT accuracy falling to about 41% on open-ended engineering problems
The University of Manchester (chemical engineering teaching) · Higher education engineering teaching and assessment design · Manchester, United Kingdom
- Publisher
- Times Higher Education Campus
- Original publication
- THE Campus, 30 October 2026
- Source retrieved
- 2026-10-31
What happened
Salman Shahid (University of Manchester) writes that his research found about 95% of ChatGPT's answers to knowledge-recall chemical engineering questions were correct, falling to about 41% on complex, open-ended problems needing professional judgement, with some wrong answers plausible enough that spotting them required real engineering knowledge. He has moved from AI-proofing to AI-authentic learning in four stages: students attempt the problem first, then use AI as a tutor told not to give the answer, then assess the AI output (sometimes a deliberately flawed solution), then defend their decisions. He argues AI literacy cannot be separated from disciplinary knowledge.
Why it matters
Offers a concrete assignment sequence for STEM courses that keeps student thinking ahead of AI use, with a cited accuracy gap that explains why verification skills matter.
Evidence and measured results
Author's own teaching practice and a summary of his prior research; no evaluation of the four-stage approach.
Limitations and uncertainty
Single-author practitioner piece; accuracy figures come from earlier research not detailed here; no outcomes; published in a section created in partnership with a sponsor.
Put this evidence to work
Lighthouse Advisory interpretation, grounded in this source. Enriched 2026-10-31; this does not change the original publication date. Each role follows the same checklist so takeaways compare across sources.
Sales
Role takeaway
- Customer problem
- AI-proofing assessment is a losing battle, and students accept plausible but wrong AI answers on open-ended problems.
- Stakeholders
- Engineering dean
- Department heads
- Teaching and learning center
- Accreditation leads
- Discovery questions
- How do engineering courses currently handle AI use?
- Could students spot a flawed AI solution in your discipline?
- Do instructors have time to build critique tasks?
- Value hypothesis
- Interpretation: an attempt-first, critique-and-defend sequence may build judgement students need to use AI safely in professional work.
- Potential engagement
- Adaptation and evaluation of the four-stage sequence in a small set of engineering courses.
- Unsupported claims
- Learning gains from the sequence
- That the cited accuracy figures hold for current models
- Applicability outside engineering
Pre-sales engineering
Role takeaway
- Fit
- Courses using general-purpose AI tools with instructor-configurable assistants.
- Architecture and integration
- Shared restricted-tutor assistant configured on the campus AI platform plus a repository of flawed solutions.
- Prerequisites
- Campus AI tool with shared assistants
- Discipline experts to build flawed solutions
- Defence rubric
- Constraints
- Model behaviour changes over time
- Students can use unrestricted tools
- UK practitioner context
- Security
- Keep flawed-solution banks and rubrics in access-controlled course spaces.
- Proof of value
- Restricted prompt withholds answers under pushback
- Flawed-solution tasks discriminate understanding
- Compare open-ended problem scores with prior cohorts
Delivery
Role takeaway
- Work
- Package the four-stage sequence, build a shared tutor prompt and flawed-solution bank, and run an evaluated pilot.
- Dependencies
- Instructor time
- Campus AI platform configuration
- Assessment office input
- Ownership
- Teaching and learning center with engineering departments.
- Skills and adoption
- Instructor workshop on building critique tasks; student briefing on using AI as a restricted tutor.
- Governance checkpoints
- Assessment rules updated for AI-after-attempt
- Mid-pilot review
- Evaluation before scale-up
- Acceptance criteria
- Proposed: pilot in an agreed number of courses
- Proposed: measurable comparison on open-ended problems
- Proposed: student feedback on fairness collected
- Risks
- Students skip the attempt stage
- Instructor workload
- Outdated accuracy assumptions
Implementation considerations
Lighthouse Advisory interpretation across the operating dimensions a public-sector buyer must settle before this evidence becomes a design. Each note answers the question under its heading for this specific source.
Architecture and integration
What must connect, and where does the AI sit in the workflow?
None specific; the approach works with general-purpose AI tools given a constrained tutor prompt.
Governance
Who approves, reviews and stays accountable for outcomes?
Implies course rules that permit AI only after an initial student attempt and require critique of AI output.
Security and privacy
What data, permissions and controls need testing?
Not addressed in the source.
Accessibility and workforce
Who is affected, and what skills or accommodations follow?
Instructors need disciplinary expertise and time to prepare flawed AI solutions and run defence activities.
Procurement
What should contracts, pricing and exit terms secure?
None.
Operating model
Which teams own the service once it runs?
Teaching centres can package the four-stage sequence as a reusable assignment pattern.
Publication history
- 2026-10-30Student Success · Issue 535 resources
Stable resource ID: the-campus-manchester-ai-authentic-engineering-do-augment-challenge-defend-20261030