{"resourceId":"springer-eit-ruppin-genai-allowed-exam-within-subjects-verification-20261026","versions":[{"version":"external-d72efa6debbd22e8676fc5b304f05f60c02586169b18f53a8a2cdfd0976ce851","resource":{"id":"springer-eit-ruppin-genai-allowed-exam-within-subjects-verification-20261026","title":"Controlled exam experiment with 95 engineering and computer science undergraduates: GenAI access raised short-exam scores, mostly for lower-baseline students, and self-reported full verification tracked larger gains","organization":"Ruppin Academic Center and Holon Institute of Technology (authors); two engineering and computer science courses","sector":"Higher education assessment and learning outcomes","geography":"Israel","publishedAt":"Education and Information Technologies (Springer), published 26 October 2026","publicationDate":"2026-10-26","eventDate":null,"sourceName":"Education and Information Technologies (Springer Nature)","sourceLabel":"Peer-reviewed journal article (within-subjects experiment)","sourceUrl":"https://link.springer.com/article/10.1007/s10639-026-14178-8","evidenceClass":"academic-research","outcomeClass":"mixed","topics":["knowledge-work","governance-procurement"],"finding":"In a within-subjects experiment, 95 undergraduate STEM students from two engineering and computer science courses each completed two comparable short in-class exams, one with and one without generative AI assistance. Scores improved significantly overall with AI, with larger gains among students with lower baseline performance and minimal improvement among higher-baseline students. After accounting for baseline performance, students who reported fully verifying AI output improved significantly more than those who reported partial verification. Students mainly used AI for task-oriented solution-seeking and accuracy checks rather than deepening conceptual understanding. The authors call for assessment to shift toward critical verification competence and evaluative judgment.","sledRelevance":"Shows that AI-allowed exam scores measure something different from independent mastery, especially for weaker students, which matters for campuses deciding which assessments permit AI.","evidence":"Within-subjects experiment (n=95, two courses); exam scores plus post-exam questionnaire on usage and verification. Published 26 October 2026 after acceptance on 20 October.","architectureImplications":"Limited; AI was used inside in-class exams, which requires controlled access and device rules.","governanceImplications":"Supports explicit decisions on which assessments allow AI and what they certify; verification behaviour becomes an assessable skill.","securityPrivacyImplications":"Not discussed in the accessible text.","caveats":"Abstract-level inspection only; short exams; order and difficulty controls not verified; self-reported verification; Israeli institutions and STEM courses limit transfer.","streamIds":["student-success"],"roles":{"sales":"Sales framing: this peer-reviewed experiment shows AI access in exams lifts weaker students most, so AI-allowed scores can hide gaps in independent mastery. Use it with provosts, assessment directors and STEM chairs deciding which exams allow AI. A bounded engagement could map a programme's assessments into AI-allowed and AI-free categories and define what each certifies. Do not claim AI improves learning; the study measured assisted exam scores, not durable learning, and the full paper was not reviewed.","engineering":"Engineering framing: technical fit is about controlled exam environments, not tutoring tools. Check whether the campus can provide sanctioned AI access during in-class exams, log use for later review, and switch it off for AI-free sittings. Proof of value should compare AI-allowed and AI-free scores for the same students and capture whether students verified outputs, ideally from logs rather than self-report, given the study's reliance on questionnaires.","delivery":"Delivery framing: delivery is assessment redesign with faculty, not system rollout. Work would include classifying course assessments, writing rubrics that reward verification and evaluative judgment where AI is allowed, and keeping AI-free checkpoints for core competencies. Run a paired comparison in one or two STEM courses to see whether gaps between assisted and unassisted scores are largest for lower-baseline students, and share results with programme committees before scaling."},"roleDetails":{"sales":{"customerProblem":"Programmes allowing AI in assessments cannot tell whether scores reflect independent competence, especially for weaker students.","stakeholders":["Provost or vice provost for academic affairs","Assessment and accreditation director","STEM department chairs","Teaching and learning center"],"discoveryQuestions":["Which assessments in your programmes currently allow AI, and what are they meant to certify?","Do you compare assisted and unassisted performance for the same students?","How do you teach and assess verification of AI output?"],"valueHypothesis":"Interpretation: separating AI-allowed and AI-free assessment, and grading verification explicitly, could make grades more interpretable without banning AI.","potentialEngagement":"An assessment-mapping review for one programme with a paired AI-allowed versus AI-free pilot in a gateway course.","unsupportedClaims":["That AI access improves learning or retention","That results transfer beyond short STEM exams","Specific effect sizes, which were not inspected in the full text"]},"engineering":{"fit":"Courses that run in-class or proctored digital exams where AI access can be enabled or disabled.","architectureIntegration":"Exam delivery environment with a sanctioned AI tool toggled per assessment and optional usage logging linked to submissions.","prerequisites":["Controlled exam devices or lockdown environment","Sanctioned AI tool with logging","Comparable paired exam forms"],"constraints":["Short exams in two STEM courses","Israeli higher education context","Self-reported verification data"],"security":"Exam-time AI logs are assessment records; restrict access and set retention limits.","proofOfValue":["Paired comparison of AI-allowed and AI-free scores by baseline group","Log-based measure of verification behaviour","Check exam form equivalence and order effects"]},"delivery":{"work":"Classify assessments, add verification-focused rubrics to AI-allowed tasks, keep AI-free checkpoints and pilot a paired comparison.","dependencies":["Faculty willingness to run paired exams","Assessment office support","Exam technology team"],"ownership":"Academic programme leads with the assessment office.","skillsAdoption":"Faculty development on designing verification tasks; student guidance on checking AI output.","governanceCheckpoints":["Programme committee approval of assessment categories","Ethics or data review for linking logs and scores","End-of-term review of score gaps"],"acceptanceCriteria":["Proposed: every course assessment labelled AI-allowed or AI-free","Proposed: assisted-unassisted gap reported by baseline quartile","Proposed: verification rubric used in all AI-allowed exams"],"risks":["Grade inflation from AI-allowed assessments","Misreading assisted scores as mastery","Faculty workload for paired exams"]}},"retrievedAt":"2026-10-28T03:22:58Z","enrichedAt":"2026-10-28T03:28:46Z","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Interpretation: larger AI-assisted gains for lower-baseline students could mask gaps in independent skill if AI-allowed scores are used for progression decisions.","procurementImplications":"None stated.","operatingModelImplications":"Interpretation: assessment policy needs separate measures for AI-assisted performance and AI-free competence.","sourceVerification":{"openedUrl":"https://link.springer.com/article/10.1007/s10639-026-14178-8","referenceExcerpt":"students who reported full verification showed significantly greater improvement in exam performance than those who reported partial verification","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}