{"resourceId":"wang-columbia-voice-ai-discussion-partner-mba-field-experiment-20261007","versions":[{"version":"external-b7d39d9830d8a58e5297ffee9049b2f3e8859c4dc0550f879ade8deee973973c","resource":{"id":"wang-columbia-voice-ai-discussion-partner-mba-field-experiment-20261007","title":"Columbia field experiment: two sessions with a voice AI debate partner raised later voluntary class participation about 31%, but the first use did not","organization":"Management Division, Columbia Business School, Columbia University (Dan J. Wang, Neelam Modi Jain and colleagues)","sector":"Higher education student success / AI preparation tools and in-class engagement","geography":"United States (one required first-year MBA course, ten sections)","publishedAt":"arXiv preprint v1 submitted 7 October 2026 (experiment run in fall 2025)","publicationDate":"2026-10-07","eventDate":null,"sourceName":"arXiv (preprint)","sourceLabel":"Academic preprint, preregistered randomized field experiment with observed class behavior and surveys; not yet peer reviewed","sourceUrl":"https://arxiv.org/abs/2610.10463","evidenceClass":"academic-research","outcomeClass":"emerging","topics":["knowledge-work","accessibility-workforce","operating-model"],"finding":"In fall 2025, 759 MBA students in ten sections of a required strategy course were each randomly assigned two of ten case sessions to prepare for with CAiSEY, a purpose-built voice AI that takes the opposite side of the case question and argues back, followed by a written reflection and generated feedback. The design randomized when, not whether, students used it. Take-up was high: 92.2% to 98.9% of assigned students held at least one conversation, and 38.1% to 67.0% held one of ten minutes or more. Teaching assistants blind to assignment recorded voluntary contributions (mean 0.73 per session). Use did not change participation in the session prepared for, and after a first use participation in the next session was slightly lower. After a second use, participation rose in later sessions; the qualifying-use estimate equals about one extra voluntary contribution every four to five sessions, roughly 31% of the mean, and held at every use threshold from four minutes. Among 627 students who completed both surveys, later use was associated with more comfort speaking up and more perceived learning (about a tenth of a standard deviation per use), with no change in focus or motivation.","sledRelevance":"New today and one of few randomized studies of a purpose-built AI tool in a live U.S. course measuring behavior rather than test scores. It is a counterpoint to the archived UMD trial: a narrow, structured, opposing-argument design with high take-up, versus an open study assistant few chose to use. The stream brief asks for durable learning and equity, and this study measures participation and self-reports only.","evidence":"Preregistered within-course randomization of timing across 759 students (4,787 student-sessions from 504 students in the main models), intention-to-treat and instrumented qualifying-use (LATE) estimates at three horizons, robustness across use-length thresholds, midterm and end-of-term surveys (627 students), Columbia IRB exempt protocol with informed consent.","architectureImplications":"The tool is a voice debate partner scoped to one case question per session, with logged transcripts, positions, durations, reflections and generated summaries linked to student IDs. The paper does not name the underlying models or speech services.","governanceImplications":"Interpretation: purpose-built tools tied to a specific pedagogical move (prepare to argue a position) may avoid the answer-seeking seen with open assistants; institutions should require that kind of design rationale before adoption.","securityPrivacyImplications":"The platform recorded voice-session transcripts and linked them to participation records, surveys and resumes; production use would need consent, retention limits and clear rules on who can see transcripts.","caveats":"Single elite MBA program and one case-method course; every student got two uses, so there is no never-used comparison and no human-partner comparison; outcomes are participation counts and self-reports, not grades or learning; first use showed a short-term dip; mean reported as 0.73 in results and 0.70 in the discussion; tool vendor and models not disclosed; preprint, not peer reviewed.","streamIds":["student-success"],"roles":{"sales":"The framing point is design over access. Columbia's randomized study found a voice AI debate partner, used twice before class, lifted later voluntary participation about 31%, while a single use did nothing. Use it with institutions that want AI to deepen in-class engagement rather than replace work. Keep it bounded: one MBA course, participation and self-reports rather than grades, and no comparison with students who never used the tool or practiced with a person.","engineering":"For engineering, the framing point is a narrow, structured tool: a voice partner that takes the opposite side of one case question, then collects a reflection and returns a summary. Check latency and speech quality for live debate, restrict scope to the assigned case, and log duration, positions and transcripts linked to student IDs so effects can be measured. Plan consent, retention and access rules for voice transcripts before any production rollout.","delivery":"Delivery's framing point is repetition and measurement. Assign the AI preparation at least twice per student, tie it to the next discussion question, and send reminders as the study did. Have teaching assistants record voluntary contributions without knowing who used the tool. Add a never-used comparison group and a grade or learning measure, which this study lacked, and report participation by student group before scaling beyond one course."},"roleDetails":{"sales":{"customerProblem":"Faculty fear AI lets students skip preparation, and few tools are shown to improve live class engagement.","stakeholders":["Dean or associate dean of teaching","Teaching and learning center","Faculty in discussion-based courses","Academic technology","Institutional research"],"discoveryQuestions":["Which courses depend on class discussion or case method?","Do you record voluntary participation today?","Would faculty assign AI preparation more than once per student?"],"valueHypothesis":"Repeated use of a structured voice AI debate partner may raise voluntary participation in discussion courses; learning effects must be measured locally.","potentialEngagement":"A one-term pilot in two or three discussion-based courses with randomized timing, blinded participation recording and a learning measure.","unsupportedClaims":["That the tool improves grades or learning","That a generic voice chatbot has the same effect","That results transfer from an MBA program to undergraduates"]},"engineering":{"fit":"Courses built on discussion or case method with an LMS and SSO.","architectureIntegration":"Voice debate service scoped to a per-session question, reflection capture and generated summary, event logs linked to student IDs and exported to analytics.","prerequisites":["Voice service with acceptable latency","Consent flow for recorded conversations","Per-session question authoring by faculty"],"constraints":["Underlying models undisclosed in the study","Cost of real-time voice at scale","Single-program evidence base"],"security":"Minimize transcript retention, restrict instructor and vendor access to transcripts, and exclude student data from model training.","proofOfValue":["Blinded participation counts after first and second use","Comparison against a never-used group","Learning or grade outcome alongside participation"]},"delivery":{"work":"Pilot a structured voice AI preparation activity with repeated assignments and blinded participation measurement.","dependencies":["Faculty-authored discussion questions","Teaching assistant time to record participation","IR support for analysis"],"ownership":"Course faculty own the activity; teaching and learning center owns pilot design; IR owns evaluation.","skillsAdoption":"Brief students on how the debate partner works; train teaching assistants on consistent participation recording.","governanceCheckpoints":["Privacy review of voice transcripts","Accessibility review for students who cannot use voice","End-of-term review of results by student group"],"acceptanceCriteria":["Each student assigned at least two AI preparation sessions","Participation recorded blind to assignment","Learning outcome reported alongside participation"],"risks":["Mistaking engagement for learning","Short-term dip after first use discouraging adoption","Voice access barriers for some students"]}},"retrievedAt":"2026-10-08T03:21:13Z","enrichedAt":"2026-10-08T03:25:13Z","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Comfort speaking up rose with use, which may matter for students anxious about public speaking, but effects were not reported by student group.","procurementImplications":"Value depends on a purpose-built, course-aligned design rather than a generic voice chatbot, which the authors say their results do not cover.","operatingModelImplications":"Assign AI preparation more than once, tie it to the next class's discussion question, and measure participation with blinded recording.","sourceVerification":{"openedUrl":"https://arxiv.org/abs/2610.10463","referenceExcerpt":"After two uses of the AI discussion partner, students made about 31% more voluntary contributions in each later class session.","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}