From the Student Success edition of November 10, 2026
Duke CTL (2 Nov): six-course pilot of Coursera's Dialogue AI coach gets about 84% would-use-again among respondents, but needed heavy authoring and framing and split learners
Duke University Center for Teaching and Learning · Higher education online and MOOC instruction · United States (North Carolina)
- Publisher
- Duke Center for Teaching and Learning
- Original publication
- Duke Center for Teaching and Learning blog, 2 November 2026 (updated 4 November)
- Source retrieved
- 2026-11-11
What happened
Duke's Learning Experience Design team piloted Coursera's Dialogue tool, which has learners hold ungraded, real-time conversations with an AI "Coach" built from course material and instructor input, in six self-paced Coursera courses. About 84 percent of learners who engaged with Dialogue and took the optional survey somewhat or strongly agreed they would use it again. Open-ended responses varied widely, from wholesale endorsement to refusal, with ethical qualms, overreliance and privacy concerns alongside appreciation for interactivity. The authors say the activities took significantly more time to author and test than a typical assessment and needed extra scaffolding; an introductory reading that explained the pilot's values measurably improved learner perception. They call the pilot a success, say AI tools are "far from a silver bullet," and note long-term learning effects remain unanswered.
Why it matters
Shows an elite university piloting a vendor AI coach at MOOC scale, with a candid account of authoring cost, learner variation and framing, and with perception rather than learning outcomes as the success metric.
Evidence and measured results
Operator-authored summary of an optional learner survey in six courses; no denominators, comparison group or learning-outcome data are reported, and the tool is from the platform partner.
Limitations and uncertainty
Self-selected survey respondents; satisfaction rather than learning measured; partner-vendor tool; MOOC context may not transfer to credit courses.
Put this evidence to work
Lighthouse Advisory interpretation, grounded in this source. Enriched 2026-11-11; this does not change the original publication date. Each role follows the same checklist so takeaways compare across sources.
Sales
Role takeaway
- Customer problem
- Large asynchronous courses struggle to give individualized feedback and practice at scale.
- Stakeholders
- Online learning leadership
- Course instructors
- Instructional designers
- Platform partner
- Learners
- Discovery questions
- How much authoring time per AI activity is acceptable?
- What learning measures beyond satisfaction exist?
- What alternative do objecting learners get?
- Value hypothesis
- Interpretation: well-framed AI practice activities may add low-stakes interaction at scale; the source reports perceived value only.
- Potential engagement
- Pilot design for two or three courses: authoring template, learner framing, opt-out path and learning-outcome measurement.
- Unsupported claims
- Learning gains
- Retention or completion improvement
- Cost savings
Pre-sales engineering
Role takeaway
- Fit
- Vendor AI coach embedded in an online course platform.
- Architecture and integration
- Course-platform embedding, instructor-supplied content and prompts, survey capture and optional learner analytics.
- Prerequisites
- Course content permissions
- Platform partner configuration
- Conversation testing protocol
- Constraints
- Vendor-hosted tool
- No data handling detail in source
- MOOC context
- Security
- Establish what conversation data the vendor retains and how it is used before scaling.
- Proof of value
- Pre-release adversarial testing of conversations
- A/B completion and quiz comparison
- Survey with reported denominators
Delivery
Role takeaway
- Work
- Co-design AI activities with instructors, test them, write learner framing, and review feedback.
- Dependencies
- Instructors
- Instructional designers
- Platform partner
- Privacy office
- Ownership
- Teaching and learning center with instructors
- Skills and adoption
- Instructors need examples and templates for AI-supported activity design; learners need explanation of how and why AI is used.
- Governance checkpoints
- Instructor approval of each activity
- Privacy review
- Post-term review of learner feedback
- Acceptance criteria
- Proposed: authoring hours per activity tracked
- Proposed: completion and quiz comparison with non-users
- Proposed: survey response rate and denominators reported
- Risks
- Authoring cost outweighs benefit
- Learners disengage over AI ethics concerns
- Satisfaction mistaken for learning
Implementation considerations
Lighthouse Advisory interpretation across the operating dimensions a public-sector buyer must settle before this evidence becomes a design. Each note answers the question under its heading for this specific source.
Architecture and integration
What must connect, and where does the AI sit in the workflow?
A vendor-hosted conversational coach embedded in a course platform, configured per activity from course material and instructor inputs; no data architecture detail is given.
Governance
Who approves, reviews and stays accountable for outcomes?
Instructor co-design and approval gate each activity, and pre-reading explains the institution's values on AI use; learners could choose how to engage.
Security and privacy
What data, permissions and controls need testing?
Learners raised privacy concerns; the article does not detail data handling by the vendor.
Accessibility and workforce
Who is affected, and what skills or accommodations follow?
Authoring effort burdens instructional designers and faculty; some learners will disengage entirely, so an alternative path matters.
Procurement
What should contracts, pricing and exit terms secure?
Embedded in an existing platform partnership; cost is mostly design labor, which procurement conversations should surface.
Operating model
Which teams own the service once it runs?
A central teaching center co-designs with faculty and the platform vendor, using learner and instructor feedback as pilot metrics.
Publication history
- 2026-11-10Student Success · Issue 645 resources
Stable resource ID: duke-ctl-coursera-dialogue-ai-coach-pilot-20261102