From the Student Success edition of October 6, 2026
Cornell field study in India: AI dialogue beats written responses on engagement but not learning, and fewer students finish it
Cornell University (Deepak Varuvel Dennison, Connie Zhang, Rene Kizilcec, Aditya Vashistha) with an India implementation partner · Higher education student success / AI dialogic learning activities, multilingual access · India (Karnataka; English-medium undergraduate institutions); limited transferability to U.S. settings
- Publisher
- arXiv (preprint)
- Original publication
- arXiv preprint v1 submitted 5 October 2026
- Source retrieved
- 2026-10-07
What happened
305 native Kannada-speaking undergraduates at English-medium institutions were randomly assigned to six conditions: written response, text dialogue with an AI tutor, or voice dialogue with an AI tutor, each in English-only or bilingual Kannada-English, within an optional AI-literacy module (Gemini 2.5 Flash tutor; GPT-4o Transcribe and Cartesia speech for voice). 173 completed. Completion was 68.6% for written response versus 52.3% for text dialogue and 48.4% for voice dialogue (p = .009); language did not affect completion (57.9% vs 55.6%). Knowledge rose from 5.45 to 8.89 on a 15-point test across conditions, with no reliable difference by format (0.07 points, p = .88) or language. Completers of dialogic activities spent more time, contributed more and reported greater interest and self-efficacy. Students abandoned 86.7% of English voice conversations before completion versus 65.4% of bilingual voice conversations; bilingual voice scored 1.89 points higher than English voice (p = .052, inconclusive).
Why it matters
Released this week and relevant to institutions replacing written assignments with AI dialogue to blunt AI-written answers. It shows the trade-off a redesign can bring: richer engagement for those who finish, lower completion overall, and no extra short-term learning. The bilingual finding bears on multilingual and English-learner students in U.S. colleges, with transfer limits.
Evidence and measured results
Between-subjects randomized field study across two deployments (25 community digital centres, then one college), pre/post knowledge, interest and self-efficacy surveys one week apart, LMS interaction logs, discourse analysis of 1,604 conversations, and 16 interviews. Outcome analyses use the 173 completers; the first deployment was cut short when centres closed after funding cuts.
Limitations and uncertainty
Optional, low-stakes module; outcomes analysed only for completers, so differential attrition may bias comparisons; small cells (about 20 to 30 per group); one-week horizon; Indian English-medium context; first deployment disrupted by centre closures; preprint, not peer reviewed.
Put this evidence to work
Lighthouse Advisory interpretation, grounded in this source. Enriched 2026-10-07; this does not change the original publication date. Each role follows the same checklist so takeaways compare across sources.
Sales
Role takeaway
- Customer problem
- Written assignments are easy to complete with AI, and institutions are exploring AI dialogue as an alternative without evidence on completion or learning.
- Stakeholders
- Teaching and learning center
- Faculty redesigning assessments
- Multilingual and English-learner support services
- Academic technology
- Accessibility office
- Discovery questions
- Which courses are replacing written work with AI conversation activities?
- What share of your students learn in a second language?
- Do you measure activity completion and abandonment by student group?
- Value hypothesis
- Offering AI dialogue as an option, with language choice, may raise engagement among completers while keeping written paths for others; learning effects need local measurement.
- Potential engagement
- A one-term pilot comparing written and dialogic AI activities in selected courses with completion, abandonment and learning measures by group.
- Unsupported claims
- That AI dialogue improves learning over written work
- That voice tutoring is worth its extra cost
- That findings from Karnataka transfer directly to U.S. colleges
Pre-sales engineering
Role takeaway
- Fit
- Institutions with multilingual student populations and LMS-integrated activity tools.
- Architecture and integration
- LMS-embedded activity with text or voice dialogue; voice uses streaming speech-to-text, LLM tutor and speech synthesis; turn-level logs flow to analytics.
- Prerequisites
- Model evaluation on target languages
- Consent flow for voice recording
- Turn-level interaction logging
- Constraints
- Latency and cost of real-time voice
- Vendor data terms across several services
- Small-sample, single-context evidence
- Security
- Minimise retention of voice recordings and transcripts, restrict vendor training use, and document each service's data handling.
- Proof of value
- Completion and abandonment by modality and language versus written baseline
- Pre/post knowledge gain comparison
- Per-student cost of voice versus text dialogue
Delivery
Role takeaway
- Work
- Run an optional dialogic AI activity pilot beside written work, with language choice, and evaluate completion and learning.
- Dependencies
- Faculty willing to offer both formats
- Language support staff input
- Analytics access to activity logs
- Ownership
- Course instructors own activities; teaching and learning center owns pilot design; IR owns evaluation.
- Skills and adoption
- Brief faculty on reading completion and abandonment data; orient students on modality and language options.
- Governance checkpoints
- Privacy review of voice data
- Accessibility review of interfaces
- Mid-term completion check by group
- Acceptance criteria
- Completion of dialogic activities within an agreed margin of written completion
- Abandonment reported by language and modality
- Knowledge gain at least equal to the written baseline
- Risks
- Lower completion widening gaps
- Attrition hiding true effects
- Voice costs exceeding value
Implementation considerations
Lighthouse Advisory interpretation across the operating dimensions a public-sector buyer must settle before this evidence becomes a design. Each note answers the question under its heading for this specific source.
Architecture and integration
What must connect, and where does the AI sit in the workflow?
Voice tutoring needed a real-time pipeline of speech-to-text, LLM and text-to-speech services; text dialogue is simpler. Models were chosen for quality, latency and Kannada support, so language coverage is an architecture requirement.
Governance
Who approves, reviews and stays accountable for outcomes?
Replacing written work with AI dialogue changes who completes; any redesign should track completion and abandonment by group, not only scores, and keep a non-dialogic path.
Security and privacy
What data, permissions and controls need testing?
Voice conversations and transcripts pass through several third-party services; recordings of students need consent, retention limits and vendor terms, which the paper does not detail.
Accessibility and workforce
Who is affected, and what skills or accommodations follow?
Bilingual voice reduced abandonment and articulation difficulty for non-native English speakers, while students also valued English for career reasons; offering choice of modality and language supports access.
Procurement
What should contracts, pricing and exit terms secure?
Voice AI tutoring carries extra service components and costs; ask vendors for language coverage evidence and completion data, not only satisfaction.
Operating model
Which teams own the service once it runs?
Treat dialogic AI activities as an option alongside written work, give students control over modality and language, and monitor completion as a primary metric.
Publication history
- 2026-10-06Student Success · Issue 304 resources
Stable resource ID: dennison-kizilcec-dialogic-ai-vs-written-karnataka-20261005