From the Student Success edition of October 1, 2026
Melbourne/Monash GenAI practice platform: human-upstream answer verification; users prefer immediate imperfect feedback
University of Melbourne & Monash University (Chen, Buskes, Ren, Leong) · Higher education engineering teaching / formative feedback · Australia (University of Melbourne / Monash; large electrical engineering probability subject)
- Publisher
- arXiv
- Original publication
- Submitted to arXiv 1 October 2026 (AAEE 2026 conference paper)
- Source retrieved
- 2026-10-02
What happened
Pilot of a GenAI practice platform that combines a demonstrator-verified question bank, AI-generated exam-style variants, and an iterative attempt–feedback–revise loop in a core electrical-engineering probability subject. Design relocates human oversight upstream to answer verification rather than real-time grading. Of 95 registrants from 157 enrolled, 34 attempted at least one question (230 submissions; median three, max 40). End-of-semester survey (n=53): all five constructs above neutral midpoint; engagement/active learning highest (3.97/5; 79% agreement); lowest for preferring the system over the existing problem booklet (42%). Platform users were more convinced than non-users that immediate imperfect AI feedback is more useful than delayed demonstrator feedback (79% vs 59%). Progress-test scores for a subset illustrate trajectories only—not causal learning effects.
Why it matters
Complementary HE design pattern to access-only tutors: scale formative feedback by verifying solutions up front and keeping GenAI in practice loops. Useful for student-success/teaching teams facing large-class feedback delays, but evidence is engagement/perception—not an RCT on grades.
Evidence and measured results
Platform analytics (95/34/230), survey of five constructs plus open responses, illustrative progress-test trajectories. No randomized control for learning impact; preprint/conference paper.
Limitations and uncertainty
Voluntary uptake; survey self-report; no causal learning estimate; single Australian engineering subject; preprint.
Put this evidence to work
Lighthouse Advisory interpretation, grounded in this source. Enriched 2026-10-02; this does not change the original publication date. Each role follows the same checklist so takeaways compare across sources.
Sales
Role takeaway
- Customer problem
- Large STEM classes cannot deliver timely formative feedback, and GenAI grading raises trust and accountability concerns.
- Stakeholders
- engineering faculty
- teaching & learning
- academic technology
- student success
- demonstrators/TAs
- Discovery questions
- Which items will humans verify before AI feedback goes live?
- Is the tool formative-only or grading assessed work?
- How will we measure active practice versus registration?
- Value hypothesis
- Upstream human verification plus GenAI formative loops can raise perceived usefulness of immediate feedback without claiming graded-assessment automation.
- Potential engagement
- One-subject mid-semester formative pilot with verified bank, analytics, and user/non-user survey.
- Unsupported claims
- Causal exam-score gains
- Replacement of demonstrators
- Safe auto-grading of high-stakes work
Pre-sales engineering
Role takeaway
- Fit
- Formative GenAI practice platforms for large undergraduate engineering subjects with verified solution keys.
- Architecture and integration
- Verified answer bank → AI variants → attempt–feedback–revise loop → analytics; human path is key verification, not live grading.
- Prerequisites
- demonstrator-verified items
- LMS/SSO registration
- formative-only policy
- Constraints
- voluntary uptake
- imperfect AI feedback
- no RCT learning evidence in source
- Security
- Attempt/feedback logs as education records; isolate from misconduct tooling.
- Proof of value
- Active-attempt rate among registrants
- Revision-loop completion
- Later controlled learning evaluation
Delivery
Role takeaway
- Work
- Build verified bank, configure prompts, launch mid-semester, monitor submissions, survey users/non-users, iterate feed-forward prompts.
- Dependencies
- subject faculty
- demonstrators for keys
- IT hosting
- Ownership
- Subject coordinator with teaching-support engineering.
- Skills and adoption
- Demonstrators verify keys and tune prompts; students practice attempt-then-revise habits.
- Governance checkpoints
- formative-scope signoff
- key-verification complete before release
- end-term claims audit
- Acceptance criteria
- No summative auto-grade
- Verified keys for released items
- Active-use metrics reported
- Risks
- Low active use
- Overtrust in imperfect feedback
- Scope creep into grading
Implementation considerations
Lighthouse Advisory interpretation across the operating dimensions a public-sector buyer must settle before this evidence becomes a design. Each note answers the question under its heading for this specific source.
Architecture and integration
What must connect, and where does the AI sit in the workflow?
Separate verified answer store from generative variant/feedback generation; enforce attempt–feedback–revise before revealing solutions; keep humans out of real-time grading path for formative practice.
Governance
Who approves, reviews and stays accountable for outcomes?
Do not use this design for high-stakes grading without additional assurance. Disclose that feedback can be imperfect; prefer formative practice scopes with verified keys.
Security and privacy
What data, permissions and controls need testing?
Practice attempts and AI feedback logs are education records; minimize retention and avoid feeding them into misconduct systems.
Accessibility and workforce
Who is affected, and what skills or accommodations follow?
Immediate feedback may help stuck learners in large classes, but 34/95 active use and 42% preference versus booklet show access is not adoption. Demonstrators still needed for verified keys and edge cases.
Procurement
What should contracts, pricing and exit terms secure?
Require demonstrator-verified item banks, exportable attempt logs, and formative-only deployment clauses—not auto-grading of assessed work.
Operating model
Which teams own the service once it runs?
Staff time shifts to upstream key verification and prompt/feed-forward design; monitor active-use rates weekly rather than registration counts.
Publication history
- 2026-10-01Student Success · Issue 253 resources
Stable resource ID: chen-melbourne-genai-practice-feedback-20261001