{"resourceId":"uc-berkeley-denero-ai-homework-answers-feel-productive-61a-bot-interviews-20261020","versions":[{"version":"external-b6b4feace80a4c58e62d50b84a25e4666f291e76dd17ce24ae0608d580281e97","resource":{"id":"uc-berkeley-denero-ai-homework-answers-feel-productive-61a-bot-interviews-20261020","title":"UC Berkeley's John DeNero: AI that explains homework answers feels productive but may not build practice; Berkeley guides students to mistakes with 61A Bot and now interviews students after correct answers","organization":"University of California, Berkeley (EECS and Data Science Undergraduate Studies)","sector":"Higher education computing education / AI tutoring and course support","geography":"Berkeley, California, United States","publishedAt":"UC Berkeley Research news, 20 October 2026 (talk given 9 October 2026)","publicationDate":"2026-10-20","eventDate":"2026-10-09","sourceName":"UC Berkeley Office of the Vice Chancellor for Research","sourceLabel":"University news article summarising a faculty talk","sourceUrl":"https://vcresearch.berkeley.edu/news/how-ai-transforming-computing-education-uc-berkeley","evidenceClass":"vendor-claim","outcomeClass":"mixed","topics":["knowledge-work","developers-agents","operating-model"],"finding":"Reporting DeNero's 9 October talk, the 20 October article says AI that explains how to do homework, usually including the answers, can feel productive but in his view does not give practice, show how ideas apply or help students assess themselves. He cited a 2026 Anthropic study in which Claude Code session success ranged from 39% for novices to 65% for experts. Berkeley's 61A Bot, deployed in CS 61A as a homework tutor, explains conceptually why an answer is wrong and what to do next rather than giving answers. Berkeley researchers are also testing interviews with students about their answer each time they solve a homework problem correctly. No outcome figures are given.","sledRelevance":"A same-day institutional account from a large U.S. public university that distinguishes answer-giving AI from guided tutoring and adds a verification step after correct answers, directly relevant to course-integrated tutor design.","evidence":"Faculty practitioner account in university news. The cited studies and 61A Bot findings are linked but not quantified in the article.","architectureImplications":"Interpretation: tutor design choices (withhold answers, explain errors, interview after success) matter more than model choice, and need course-specific prompts and assignment data.","governanceImplications":"Supports course policies that separate guided AI help from answer generation; how interviews are graded is not described.","securityPrivacyImplications":"Homework submissions and interview transcripts would be student records; handling is not described in the article.","caveats":"Talk summary by the university, not an evaluation. The interview intervention is under investigation, and the student report of improved understanding is a single anecdote.","streamIds":["student-success"],"roles":{"sales":"Sales framing: DeNero gives a credible faculty voice for a key distinction: AI that hands out answers feels productive but may not build skill, while guided tutors that explain mistakes aim to. Use it in discovery about how institutions configure course tutors and verify learning. Offer bounded help with tutor design and evaluation. Do not cite it as measured evidence of learning gains: the article summarises a talk and gives no outcome figures.","engineering":"Engineering framing: 61A Bot shows a tutor configured to explain why an answer is wrong and what to try next, not to supply answers, and Berkeley is adding interviews after correct answers. Engineering work centres on course-specific prompts, access to assignment and autograder context, answer-withholding tests and logging. Validate that the tutor does not leak solutions, and design the interview step so it can be scored consistently and at scale.","delivery":"Delivery framing: course tutors need ongoing ownership by course staff, who must write problems, tune tutor behaviour and decide how verification interviews count. Pilot in one large course, track solution leakage, student use and exam performance against prior terms, and review each term. Communicate to students why the tutor withholds answers so the design is not seen as a defect."},"roleDetails":{"sales":{"customerProblem":"Students use general AI to get homework answers, which can feel productive without building skill.","stakeholders":["Department chairs in high-enrolment courses","Course staff and teaching assistants","Teaching and learning centre","Undergraduate studies dean"],"discoveryQuestions":["Which large courses see the most answer-seeking AI use?","Can current tutors be configured to withhold answers?","How would learning be verified after correct submissions?","Who owns tutor configuration each term?"],"valueHypothesis":"Interpretation: guided tutors with verification steps could preserve practice while still giving individual feedback, if exam performance is tracked.","potentialEngagement":"A bounded course-tutor design and evaluation in one gateway course, with solution-leakage tests and a comparison against prior terms.","unsupportedClaims":["Measured learning gains","Higher pass rates","Reduced teaching assistant workload","Elimination of answer-seeking"]},"engineering":{"fit":"Fits large programming or problem-set courses with autograded assignments and active course staff.","architectureIntegration":"Course-deployed tutor tied to assignments, explaining errors conceptually, plus an interview step after correct answers.","prerequisites":["Assignment and autograder context","Course-specific prompts","Logging of tutor sessions","Staff time for problem design"],"constraints":["Practitioner account, not an evaluation","Interview step still under investigation","Cited statistics are second-hand","Computing course context may not transfer to all disciplines"],"security":"Treat submissions, tutor logs and interview transcripts as student records with defined retention and access.","proofOfValue":["Solution-leakage red-team test","Exam performance compared with prior terms","Interview scoring consistency check","Student use and satisfaction sampling"]},"delivery":{"work":"Configure and pilot a guided tutor with a post-answer verification step in one course, reviewed each term.","dependencies":["Course staff","Tutor platform","Autograder integration","Assessment policy"],"ownership":"Course instructor and teaching team, supported by the teaching and learning centre.","skillsAdoption":"Teaching assistant training on tutor configuration and interview scoring; student guidance on why answers are withheld.","governanceCheckpoints":["Course AI policy update","Data review of tutor logs","Mid-term leakage review","End-of-term outcome review"],"acceptanceCriteria":["Proposed: no full solutions returned in red-team tests","Proposed: interview scores agree across graders on a sample","Proposed: exam performance at least equal to prior terms","Proposed: student use reported each term"],"risks":["Tutor leaking answers","Interviews adding workload without scale","Students bypassing to general AI","Assuming results without measurement"]}},"retrievedAt":"2026-10-21T03:21:36Z","enrichedAt":"2026-10-21T03:27:54Z","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Human-written problems remain necessary, so faculty and staff effort shifts to problem design and tutor configuration.","procurementImplications":"Interpretation: favours configurable tutors that can withhold answers and use course content over generic chatbot licences.","operatingModelImplications":"Course staff own tutor behaviour and verification steps, which implies ongoing iteration each term.","sourceVerification":{"openedUrl":"https://vcresearch.berkeley.edu/news/how-ai-transforming-computing-education-uc-berkeley","referenceExcerpt":"For example, students are now interviewed about their answer every time they complete a homework problem correctly.","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}