{"resourceId":"dennison-kizilcec-dialogic-ai-vs-written-karnataka-20261005","versions":[{"version":"external-664425108929b7e799fe9df903aa67a37cb35d991b0f74a6b25a8e8216785142","resource":{"id":"dennison-kizilcec-dialogic-ai-vs-written-karnataka-20261005","title":"Cornell field study in India: AI dialogue beats written responses on engagement but not learning, and fewer students finish it","organization":"Cornell University (Deepak Varuvel Dennison, Connie Zhang, Rene Kizilcec, Aditya Vashistha) with an India implementation partner","sector":"Higher education student success / AI dialogic learning activities, multilingual access","geography":"India (Karnataka; English-medium undergraduate institutions); limited transferability to U.S. settings","publishedAt":"arXiv preprint v1 submitted 5 October 2026","publicationDate":"2026-10-05","eventDate":null,"sourceName":"arXiv (preprint)","sourceLabel":"Academic preprint, IRB-approved randomized field study with logs and interviews; not yet peer reviewed","sourceUrl":"https://arxiv.org/abs/2610.07483","evidenceClass":"academic-research","outcomeClass":"mixed","topics":["knowledge-work","accessibility-workforce","operating-model"],"finding":"305 native Kannada-speaking undergraduates at English-medium institutions were randomly assigned to six conditions: written response, text dialogue with an AI tutor, or voice dialogue with an AI tutor, each in English-only or bilingual Kannada-English, within an optional AI-literacy module (Gemini 2.5 Flash tutor; GPT-4o Transcribe and Cartesia speech for voice). 173 completed. Completion was 68.6% for written response versus 52.3% for text dialogue and 48.4% for voice dialogue (p = .009); language did not affect completion (57.9% vs 55.6%). Knowledge rose from 5.45 to 8.89 on a 15-point test across conditions, with no reliable difference by format (0.07 points, p = .88) or language. Completers of dialogic activities spent more time, contributed more and reported greater interest and self-efficacy. Students abandoned 86.7% of English voice conversations before completion versus 65.4% of bilingual voice conversations; bilingual voice scored 1.89 points higher than English voice (p = .052, inconclusive).","sledRelevance":"Released this week and relevant to institutions replacing written assignments with AI dialogue to blunt AI-written answers. It shows the trade-off a redesign can bring: richer engagement for those who finish, lower completion overall, and no extra short-term learning. The bilingual finding bears on multilingual and English-learner students in U.S. colleges, with transfer limits.","evidence":"Between-subjects randomized field study across two deployments (25 community digital centres, then one college), pre/post knowledge, interest and self-efficacy surveys one week apart, LMS interaction logs, discourse analysis of 1,604 conversations, and 16 interviews. Outcome analyses use the 173 completers; the first deployment was cut short when centres closed after funding cuts.","architectureImplications":"Voice tutoring needed a real-time pipeline of speech-to-text, LLM and text-to-speech services; text dialogue is simpler. Models were chosen for quality, latency and Kannada support, so language coverage is an architecture requirement.","governanceImplications":"Interpretation: replacing written work with AI dialogue changes who completes; any redesign should track completion and abandonment by group, not only scores, and keep a non-dialogic path.","securityPrivacyImplications":"Voice conversations and transcripts pass through several third-party services; recordings of students need consent, retention limits and vendor terms, which the paper does not detail.","caveats":"Optional, low-stakes module; outcomes analysed only for completers, so differential attrition may bias comparisons; small cells (about 20 to 30 per group); one-week horizon; Indian English-medium context; first deployment disrupted by centre closures; preprint, not peer reviewed.","streamIds":["student-success"],"roles":{"sales":"The framing point is a trade-off, not a win. Cornell's field study found AI dialogue raised engagement and confidence among students who finished, but fewer finished, and learning gains matched plain written responses. Use it with institutions redesigning assignments around AI to discuss completion, language access and choice of format. Do not position dialogic AI as improving learning outcomes; this evidence supports a monitored pilot that tracks completion and abandonment by student group.","engineering":"For engineering, the framing point is that voice dialogue is a multi-service pipeline: speech-to-text, the tutoring model and speech synthesis, each with latency, language-coverage and data-handling implications. Test candidate models on the languages your students actually use, since bilingual interaction changed abandonment rates. Log turn-level events so completion, abandonment and time on task can be measured per condition, and plan consent and retention controls for voice recordings and transcripts.","delivery":"Delivery's framing point is that completion is the metric to watch. Pilot dialogic AI activities as an option alongside written work in a graded or credit-bearing context, and track who completes and who abandons by modality, language and student group. Offer bilingual support where students learn in a second language. Compare knowledge gains against the written baseline before scaling, since this study found no learning advantage in the short term."},"roleDetails":{"sales":{"customerProblem":"Written assignments are easy to complete with AI, and institutions are exploring AI dialogue as an alternative without evidence on completion or learning.","stakeholders":["Teaching and learning center","Faculty redesigning assessments","Multilingual and English-learner support services","Academic technology","Accessibility office"],"discoveryQuestions":["Which courses are replacing written work with AI conversation activities?","What share of your students learn in a second language?","Do you measure activity completion and abandonment by student group?"],"valueHypothesis":"Offering AI dialogue as an option, with language choice, may raise engagement among completers while keeping written paths for others; learning effects need local measurement.","potentialEngagement":"A one-term pilot comparing written and dialogic AI activities in selected courses with completion, abandonment and learning measures by group.","unsupportedClaims":["That AI dialogue improves learning over written work","That voice tutoring is worth its extra cost","That findings from Karnataka transfer directly to U.S. colleges"]},"engineering":{"fit":"Institutions with multilingual student populations and LMS-integrated activity tools.","architectureIntegration":"LMS-embedded activity with text or voice dialogue; voice uses streaming speech-to-text, LLM tutor and speech synthesis; turn-level logs flow to analytics.","prerequisites":["Model evaluation on target languages","Consent flow for voice recording","Turn-level interaction logging"],"constraints":["Latency and cost of real-time voice","Vendor data terms across several services","Small-sample, single-context evidence"],"security":"Minimise retention of voice recordings and transcripts, restrict vendor training use, and document each service's data handling.","proofOfValue":["Completion and abandonment by modality and language versus written baseline","Pre/post knowledge gain comparison","Per-student cost of voice versus text dialogue"]},"delivery":{"work":"Run an optional dialogic AI activity pilot beside written work, with language choice, and evaluate completion and learning.","dependencies":["Faculty willing to offer both formats","Language support staff input","Analytics access to activity logs"],"ownership":"Course instructors own activities; teaching and learning center owns pilot design; IR owns evaluation.","skillsAdoption":"Brief faculty on reading completion and abandonment data; orient students on modality and language options.","governanceCheckpoints":["Privacy review of voice data","Accessibility review of interfaces","Mid-term completion check by group"],"acceptanceCriteria":["Completion of dialogic activities within an agreed margin of written completion","Abandonment reported by language and modality","Knowledge gain at least equal to the written baseline"],"risks":["Lower completion widening gaps","Attrition hiding true effects","Voice costs exceeding value"]}},"retrievedAt":"2026-10-07T03:12:18Z","enrichedAt":"2026-10-07T03:17:13Z","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Bilingual voice reduced abandonment and articulation difficulty for non-native English speakers, while students also valued English for career reasons; offering choice of modality and language supports access.","procurementImplications":"Voice AI tutoring carries extra service components and costs; ask vendors for language coverage evidence and completion data, not only satisfaction.","operatingModelImplications":"Treat dialogic AI activities as an option alongside written work, give students control over modality and language, and monitor completion as a primary metric.","sourceVerification":{"openedUrl":"https://arxiv.org/abs/2610.07483","referenceExcerpt":"Knowledge increased across all conditions, with no reliable differences in gains between activity formats or languages.","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}