{"resourceId":"bocconi-openai-rct-chatgpt-edu-causal-reasoning-202608","versions":[{"version":"external-64ee63f1ee5cb8b0e36d8db6f2ab9a39fc1df742ab53984ffdbfcf5edd23fcee","resource":{"id":"bocconi-openai-rct-chatgpt-edu-causal-reasoning-202608","title":"RCT of 1,053 Bocconi undergraduates: ChatGPT Edu raised rubric scores; causal-reasoning training broadened ideas but did not","organization":"Bocconi University; OpenAI Economic Research","sector":"Higher education","geography":"Milan, Italy","publishedAt":"August 2026","publicationDate":null,"eventDate":null,"sourceName":"OpenAI (working paper PDF)","sourceLabel":"Training novices to think, or giving them LLMs? Evidence from an RCT","sourceUrl":"https://cdn.openai.com/pdf/novices-and-llm-august-2026.pdf","evidenceClass":"academic-research","outcomeClass":"mixed","topics":["knowledge-work","accessibility-workforce"],"finding":"In a 2x2 randomized trial of 1,053 first-year students, ChatGPT Edu (GPT-4o) access raised expert rubric scores on a business case, while causal-reasoning training increased mechanism identification (about 0.5 SD) and idea diversity without raising rubric scores. The combined condition showed more coherent and falsification logic.","sledRelevance":"Gives universities experimental evidence that AI access improves graded output on well-defined tasks but that critical-thinking instruction adds distinct benefits the rubric did not capture, informing how ChatGPT Edu rollouts pair with curriculum.","evidence":"RCT across 13 classes with expert graders, inter-rater reliability reported in the appendix, and OLS with clustered errors; some covariates failed balance checks and were controlled.","architectureImplications":"Limited: used standard ChatGPT Edu access; no integration.","governanceImplications":"Assessment design matters: rubrics rewarding closeness to expert answers may favor AI-assisted work and penalize original reasoning.","securityPrivacyImplications":"Limited relevance; study data handling not a focus.","caveats":"Co-authored with OpenAI employees and contractors; single Italian university and task; GPT-4o rather than current models; publication month only (August 2026); not shown as peer-reviewed.","streamIds":["openai"],"roles":{"sales":"Sales framing: ChatGPT Edu access improved graded work in this trial, and pairing it with critical-thinking training gave the broadest gains. That supports bundling access with pedagogy support. Interpretation: do not claim learning gains generally, since the study measured one task with vendor co-authors.","engineering":"Engineering framing: the intervention was plain ChatGPT Edu access, so the technical lift is provisioning and controls rather than integration. Limited applicability: findings depend on model version and task type; any proof of value should use local assignments and blinded grading.","delivery":"Delivery framing: rollouts should include faculty guidance on assessment design and optional reasoning training. The operational owner is the teaching and learning center with academic leadership. Proposed acceptance: a local pilot comparing rubric and originality measures across sections."},"roleDetails":{"sales":{"customerProblem":"Universities rolling out AI need evidence on whether access helps or harms student work quality and thinking.","stakeholders":["Provost","Teaching and learning center","Faculty","Institutional research"],"discoveryQuestions":["Which courses would pilot AI access with assessment changes?","Do your rubrics reward originality as well as quality?","Can you run a controlled comparison across sections?"],"valueHypothesis":"AI access paired with reasoning instruction may improve work quality while preserving idea diversity.","potentialEngagement":"A bounded course-level pilot design with blinded grading and originality measures.","unsupportedClaims":["Long-term learning gains","Generalization beyond well-defined tasks","Independence from vendor influence"]},"engineering":{"fit":"Fits institutions with ChatGPT Edu or equivalent access able to run course pilots.","architectureIntegration":"Standard ChatGPT Edu workspace access; no integration.","prerequisites":["Provisioned Edu workspace","IRB or research approval for pilots","Grading rubric design"],"constraints":["Model version differences","Single-task evidence","Vendor co-authorship"],"security":"Pilot data handling should follow institutional research policies.","proofOfValue":["Section-level randomized pilot","Blinded expert grading","Measure idea diversity alongside rubric scores"]},"delivery":{"work":"Design and run course pilots that pair AI access with reasoning training and revised rubrics.","dependencies":["Faculty participation","Research approval","Grading capacity"],"ownership":"Teaching and learning center with institutional research.","skillsAdoption":"Faculty development on assessment design and AI-aware pedagogy.","governanceCheckpoints":["Research approval","Mid-term review","Results review before scaling"],"acceptanceCriteria":["Pilot completed with blinded grading (proposed)","Originality measure reported (proposed)"],"risks":["Homogenized student work","Overreliance on AI","Misreading vendor-coauthored results"]}},"retrievedAt":"2026-10-30T03:08:05Z","enrichedAt":"2026-10-30T03:11:31Z","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Targets novice workers and students; suggests training in reasoning remains valuable alongside AI access.","procurementImplications":"Limited relevance to procurement.","operatingModelImplications":"Supports pairing tool access with curriculum and assessment redesign.","peerReview":"preprint","sourceVerification":{"openedUrl":"https://cdn.openai.com/pdf/novices-and-llm-august-2026.pdf","referenceExcerpt":"causal reasoning does not improve evaluations, unlike ChatGPT.","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}