From the Student Success edition of October 27, 2026
Controlled exam experiment with 95 engineering and computer science undergraduates: GenAI access raised short-exam scores, mostly for lower-baseline students, and self-reported full verification tracked larger gains
Ruppin Academic Center and Holon Institute of Technology (authors); two engineering and computer science courses · Higher education assessment and learning outcomes · Israel
- Publisher
- Education and Information Technologies (Springer Nature)
- Original publication
- Education and Information Technologies (Springer), published 26 October 2026
- Source retrieved
- 2026-10-28
What happened
In a within-subjects experiment, 95 undergraduate STEM students from two engineering and computer science courses each completed two comparable short in-class exams, one with and one without generative AI assistance. Scores improved significantly overall with AI, with larger gains among students with lower baseline performance and minimal improvement among higher-baseline students. After accounting for baseline performance, students who reported fully verifying AI output improved significantly more than those who reported partial verification. Students mainly used AI for task-oriented solution-seeking and accuracy checks rather than deepening conceptual understanding. The authors call for assessment to shift toward critical verification competence and evaluative judgment.
Why it matters
Shows that AI-allowed exam scores measure something different from independent mastery, especially for weaker students, which matters for campuses deciding which assessments permit AI.
Evidence and measured results
Within-subjects experiment (n=95, two courses); exam scores plus post-exam questionnaire on usage and verification. Published 26 October 2026 after acceptance on 20 October.
Limitations and uncertainty
Abstract-level inspection only; short exams; order and difficulty controls not verified; self-reported verification; Israeli institutions and STEM courses limit transfer.
Put this evidence to work
Lighthouse Advisory interpretation, grounded in this source. Enriched 2026-10-28; this does not change the original publication date. Each role follows the same checklist so takeaways compare across sources.
Sales
Role takeaway
- Customer problem
- Programmes allowing AI in assessments cannot tell whether scores reflect independent competence, especially for weaker students.
- Stakeholders
- Provost or vice provost for academic affairs
- Assessment and accreditation director
- STEM department chairs
- Teaching and learning center
- Discovery questions
- Which assessments in your programmes currently allow AI, and what are they meant to certify?
- Do you compare assisted and unassisted performance for the same students?
- How do you teach and assess verification of AI output?
- Value hypothesis
- Interpretation: separating AI-allowed and AI-free assessment, and grading verification explicitly, could make grades more interpretable without banning AI.
- Potential engagement
- An assessment-mapping review for one programme with a paired AI-allowed versus AI-free pilot in a gateway course.
- Unsupported claims
- That AI access improves learning or retention
- That results transfer beyond short STEM exams
- Specific effect sizes, which were not inspected in the full text
Pre-sales engineering
Role takeaway
- Fit
- Courses that run in-class or proctored digital exams where AI access can be enabled or disabled.
- Architecture and integration
- Exam delivery environment with a sanctioned AI tool toggled per assessment and optional usage logging linked to submissions.
- Prerequisites
- Controlled exam devices or lockdown environment
- Sanctioned AI tool with logging
- Comparable paired exam forms
- Constraints
- Short exams in two STEM courses
- Israeli higher education context
- Self-reported verification data
- Security
- Exam-time AI logs are assessment records; restrict access and set retention limits.
- Proof of value
- Paired comparison of AI-allowed and AI-free scores by baseline group
- Log-based measure of verification behaviour
- Check exam form equivalence and order effects
Delivery
Role takeaway
- Work
- Classify assessments, add verification-focused rubrics to AI-allowed tasks, keep AI-free checkpoints and pilot a paired comparison.
- Dependencies
- Faculty willingness to run paired exams
- Assessment office support
- Exam technology team
- Ownership
- Academic programme leads with the assessment office.
- Skills and adoption
- Faculty development on designing verification tasks; student guidance on checking AI output.
- Governance checkpoints
- Programme committee approval of assessment categories
- Ethics or data review for linking logs and scores
- End-of-term review of score gaps
- Acceptance criteria
- Proposed: every course assessment labelled AI-allowed or AI-free
- Proposed: assisted-unassisted gap reported by baseline quartile
- Proposed: verification rubric used in all AI-allowed exams
- Risks
- Grade inflation from AI-allowed assessments
- Misreading assisted scores as mastery
- Faculty workload for paired exams
Implementation considerations
Lighthouse Advisory interpretation across the operating dimensions a public-sector buyer must settle before this evidence becomes a design. Each note answers the question under its heading for this specific source.
Architecture and integration
What must connect, and where does the AI sit in the workflow?
Limited; AI was used inside in-class exams, which requires controlled access and device rules.
Governance
Who approves, reviews and stays accountable for outcomes?
Supports explicit decisions on which assessments allow AI and what they certify; verification behaviour becomes an assessable skill.
Security and privacy
What data, permissions and controls need testing?
Not discussed in the accessible text.
Accessibility and workforce
Who is affected, and what skills or accommodations follow?
Larger AI-assisted gains for lower-baseline students could mask gaps in independent skill if AI-allowed scores are used for progression decisions.
Procurement
What should contracts, pricing and exit terms secure?
None stated.
Operating model
Which teams own the service once it runs?
Assessment policy needs separate measures for AI-assisted performance and AI-free competence.
Publication history
- 2026-10-27Student Success · Issue 506 resources
Stable resource ID: springer-eit-ruppin-genai-allowed-exam-within-subjects-verification-20261026