From the Student Success edition of October 7, 2026
Columbia field experiment: two sessions with a voice AI debate partner raised later voluntary class participation about 31%, but the first use did not
Management Division, Columbia Business School, Columbia University (Dan J. Wang, Neelam Modi Jain and colleagues) · Higher education student success / AI preparation tools and in-class engagement · United States (one required first-year MBA course, ten sections)
- Publisher
- arXiv (preprint)
- Original publication
- arXiv preprint v1 submitted 7 October 2026 (experiment run in fall 2025)
- Source retrieved
- 2026-10-08
What happened
In fall 2025, 759 MBA students in ten sections of a required strategy course were each randomly assigned two of ten case sessions to prepare for with CAiSEY, a purpose-built voice AI that takes the opposite side of the case question and argues back, followed by a written reflection and generated feedback. The design randomized when, not whether, students used it. Take-up was high: 92.2% to 98.9% of assigned students held at least one conversation, and 38.1% to 67.0% held one of ten minutes or more. Teaching assistants blind to assignment recorded voluntary contributions (mean 0.73 per session). Use did not change participation in the session prepared for, and after a first use participation in the next session was slightly lower. After a second use, participation rose in later sessions; the qualifying-use estimate equals about one extra voluntary contribution every four to five sessions, roughly 31% of the mean, and held at every use threshold from four minutes. Among 627 students who completed both surveys, later use was associated with more comfort speaking up and more perceived learning (about a tenth of a standard deviation per use), with no change in focus or motivation.
Why it matters
New today and one of few randomized studies of a purpose-built AI tool in a live U.S. course measuring behavior rather than test scores. It is a counterpoint to the archived UMD trial: a narrow, structured, opposing-argument design with high take-up, versus an open study assistant few chose to use. The stream brief asks for durable learning and equity, and this study measures participation and self-reports only.
Evidence and measured results
Preregistered within-course randomization of timing across 759 students (4,787 student-sessions from 504 students in the main models), intention-to-treat and instrumented qualifying-use (LATE) estimates at three horizons, robustness across use-length thresholds, midterm and end-of-term surveys (627 students), Columbia IRB exempt protocol with informed consent.
Limitations and uncertainty
Single elite MBA program and one case-method course; every student got two uses, so there is no never-used comparison and no human-partner comparison; outcomes are participation counts and self-reports, not grades or learning; first use showed a short-term dip; mean reported as 0.73 in results and 0.70 in the discussion; tool vendor and models not disclosed; preprint, not peer reviewed.
Put this evidence to work
Lighthouse Advisory interpretation, grounded in this source. Enriched 2026-10-08; this does not change the original publication date. Each role follows the same checklist so takeaways compare across sources.
Sales
Role takeaway
- Customer problem
- Faculty fear AI lets students skip preparation, and few tools are shown to improve live class engagement.
- Stakeholders
- Dean or associate dean of teaching
- Teaching and learning center
- Faculty in discussion-based courses
- Academic technology
- Institutional research
- Discovery questions
- Which courses depend on class discussion or case method?
- Do you record voluntary participation today?
- Would faculty assign AI preparation more than once per student?
- Value hypothesis
- Repeated use of a structured voice AI debate partner may raise voluntary participation in discussion courses; learning effects must be measured locally.
- Potential engagement
- A one-term pilot in two or three discussion-based courses with randomized timing, blinded participation recording and a learning measure.
- Unsupported claims
- That the tool improves grades or learning
- That a generic voice chatbot has the same effect
- That results transfer from an MBA program to undergraduates
Pre-sales engineering
Role takeaway
- Fit
- Courses built on discussion or case method with an LMS and SSO.
- Architecture and integration
- Voice debate service scoped to a per-session question, reflection capture and generated summary, event logs linked to student IDs and exported to analytics.
- Prerequisites
- Voice service with acceptable latency
- Consent flow for recorded conversations
- Per-session question authoring by faculty
- Constraints
- Underlying models undisclosed in the study
- Cost of real-time voice at scale
- Single-program evidence base
- Security
- Minimize transcript retention, restrict instructor and vendor access to transcripts, and exclude student data from model training.
- Proof of value
- Blinded participation counts after first and second use
- Comparison against a never-used group
- Learning or grade outcome alongside participation
Delivery
Role takeaway
- Work
- Pilot a structured voice AI preparation activity with repeated assignments and blinded participation measurement.
- Dependencies
- Faculty-authored discussion questions
- Teaching assistant time to record participation
- IR support for analysis
- Ownership
- Course faculty own the activity; teaching and learning center owns pilot design; IR owns evaluation.
- Skills and adoption
- Brief students on how the debate partner works; train teaching assistants on consistent participation recording.
- Governance checkpoints
- Privacy review of voice transcripts
- Accessibility review for students who cannot use voice
- End-of-term review of results by student group
- Acceptance criteria
- Each student assigned at least two AI preparation sessions
- Participation recorded blind to assignment
- Learning outcome reported alongside participation
- Risks
- Mistaking engagement for learning
- Short-term dip after first use discouraging adoption
- Voice access barriers for some students
Implementation considerations
Lighthouse Advisory interpretation across the operating dimensions a public-sector buyer must settle before this evidence becomes a design. Each note answers the question under its heading for this specific source.
Architecture and integration
What must connect, and where does the AI sit in the workflow?
The tool is a voice debate partner scoped to one case question per session, with logged transcripts, positions, durations, reflections and generated summaries linked to student IDs. The paper does not name the underlying models or speech services.
Governance
Who approves, reviews and stays accountable for outcomes?
Purpose-built tools tied to a specific pedagogical move (prepare to argue a position) may avoid the answer-seeking seen with open assistants; institutions should require that kind of design rationale before adoption.
Security and privacy
What data, permissions and controls need testing?
The platform recorded voice-session transcripts and linked them to participation records, surveys and resumes; production use would need consent, retention limits and clear rules on who can see transcripts.
Accessibility and workforce
Who is affected, and what skills or accommodations follow?
Comfort speaking up rose with use, which may matter for students anxious about public speaking, but effects were not reported by student group.
Procurement
What should contracts, pricing and exit terms secure?
Value depends on a purpose-built, course-aligned design rather than a generic voice chatbot, which the authors say their results do not cover.
Operating model
Which teams own the service once it runs?
Assign AI preparation more than once, tie it to the next class's discussion question, and measure participation with blinded recording.
Publication history
- 2026-10-07Student Success · Issue 314 resources
Stable resource ID: wang-columbia-voice-ai-discussion-partner-mba-field-experiment-20261007