{"resourceId":"the-city-st-georges-duologue-socratic-ai-continuous-assessment-pilot-20261019","versions":[{"version":"external-b6b4feace80a4c58e62d50b84a25e4666f291e76dd17ce24ae0608d580281e97","resource":{"id":"the-city-st-georges-duologue-socratic-ai-continuous-assessment-pilot-20261019","title":"City St George's, University of London pilots Duologue, an in-house AI platform that leads Socratic conversations to assess understanding continuously instead of detecting AI-written essays","organization":"City St George's, University of London; Times Higher Education (publisher)","sector":"Higher education assessment and academic integrity / AI literacy","geography":"London, United Kingdom","publishedAt":"Times Higher Education opinion, published 19 October 2026, last updated 20 October 2026","publicationDate":"2026-10-19","eventDate":null,"sourceName":"Times Higher Education","sourceLabel":"Opinion article by the platform developers (university digital transformation director and agency partner)","sourceUrl":"https://www.timeshighereducation.com/opinion/what-if-ai-could-strengthen-academic-integrity","evidenceClass":"vendor-claim","outcomeClass":"emerging","topics":["knowledge-work","governance-procurement","operating-model","developers-agents"],"finding":"Baba Sheba (City St George's) and Mike Bond (Bond & Coyne) describe Duologue, an AI platform built and piloted at City St George's in which the AI leads a Socratic conversation probing what a student knows, so assessment happens throughout learning rather than at the end. Game-like labs, often with two or three students at a screen, teach how prediction algorithms, tokens and classifiers work. Development was mainly funded through the university's Digital Innovation and Transformation Office, with module input from the Raspberry Pi Foundation, Sage Publishing and Wolfram. A safeguarding layer once flagged an innocent juicing-diet reference and needed local recalibration. The authors say the platform is still in pilot and they are wary of overclaiming.","sledRelevance":"A developer account of replacing detection with continuous, conversation-based assessment, published as U.S. and UK institutions debate in-person exams versus AI-integrated assessment. The previous run opened it but deferred it because the page carried a 20 October date; it is not archived.","evidence":"Developer description and design rationale. No outcome, validity or reliability data; the detector false-positive point links to regulator casework, while the claim of bias against non-native English speakers is stated without supporting detail.","architectureImplications":"Interpretation: an in-house conversational assessment platform with a safeguarding classifier needs local tuning, logging of conversations and human review of flags.","governanceImplications":"Continuous AI-led assessment raises questions about marking validity, appeals and how conversation evidence is weighed; the article does not say how results feed grades.","securityPrivacyImplications":"A safeguarding layer monitors student language for concerning content, which implies sensitive data handling and welfare escalation procedures not described in the piece.","caveats":"Opinion by the developers, one of whom is an agency partner; single-institution pilot with no published results. Conversations with other universities are reported, not verified.","streamIds":["student-success"],"roles":{"sales":"Sales framing: Duologue reframes integrity as continuous evidence of understanding rather than detection. Use it to open discovery on assessment redesign, AI literacy goals and the institution's appetite to build versus buy. Offer a bounded assessment-design pilot with a validation plan. Do not claim reduced misconduct, better learning or reliable grading: the article is written by the developers, reports no results and says the platform is still in pilot.","engineering":"Engineering framing: an AI that leads Socratic questioning and a safeguarding classifier that watches for concerning language both need local tuning, transcripts and human review. Key checks are whether conversational judgements are consistent across students, how false safeguarding flags are handled, and how evidence is stored for appeals. Validate scoring agreement against academic markers and test the classifier on local language before any assessment weight is attached.","delivery":"Delivery framing: moving from essays to AI-led conversational assessment is a teaching change, not a tool rollout. Start with low-stakes modules, define how conversation evidence informs marks, and set a review route for safeguarding flags and student appeals. Involve sceptical staff early, as the authors did, and measure marker agreement, student experience and staff workload before scaling beyond the single-university pilot."},"roleDetails":{"sales":{"customerProblem":"Essays and AI detectors no longer give institutions confidence about what students understand.","stakeholders":["Pro vice-chancellor for education","Academic integrity lead","Learning design team","Digital transformation office","Student welfare and safeguarding lead"],"discoveryQuestions":["Which modules rely most on take-home essays?","Would the institution build or buy a conversational assessment tool?","How would conversation evidence be weighed in marks and appeals?","Who reviews safeguarding flags?"],"valueHypothesis":"Interpretation: continuous conversational assessment could give earlier evidence of understanding than end-of-term essays, if its judgements are validated against academic marking.","potentialEngagement":"A bounded low-stakes pilot design: module selection, marker-agreement study and safeguarding review process.","unsupportedClaims":["Reduced misconduct","Valid or reliable grading","Learning gains","Lower marking workload"]},"engineering":{"fit":"Fits institutions with in-house learning technology capacity and a willingness to redesign assessment.","architectureIntegration":"In-house platform with AI-led Socratic dialogue, game-like labs and a safeguarding layer, plus partner-contributed module content.","prerequisites":["Learning designers and technologists","Module content and rubrics","Conversation logging","Safeguarding escalation procedures"],"constraints":["UK context and institutional build","No published validity evidence","Partner IP in some modules","Classifier needs local recalibration"],"security":"Protect conversation transcripts as sensitive student records, restrict access to safeguarding flags and document retention for appeals.","proofOfValue":["Agreement between platform judgements and academic markers","False-positive rate of safeguarding flags","Student experience survey","Consistency across student groups including non-native speakers"]},"delivery":{"work":"Pilot conversational assessment in low-stakes modules with marker comparison and a safeguarding review route.","dependencies":["Academic staff buy-in","Learning design capacity","Platform engineering team","Student welfare services"],"ownership":"Education or digital education office, with module leaders owning assessment decisions.","skillsAdoption":"Staff training on interpreting conversation evidence and student guidance on how the dialogue is used.","governanceCheckpoints":["Assessment board approval before any weighting","Safeguarding procedure review","Equality impact review","End-of-pilot evaluation"],"acceptanceCriteria":["Proposed: substantial agreement between platform and academic judgements on a sample","Proposed: every safeguarding flag reviewed by a person within two working days","Proposed: no assessment weight until validity review passes","Proposed: student experience measured against the essay baseline"],"risks":["Unvalidated judgements affecting grades","Over-sensitive safeguarding flags","Staff resistance","Bias against particular student groups"]}},"retrievedAt":"2026-10-21T03:21:32Z","enrichedAt":"2026-10-21T03:27:54Z","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Some staff were sceptical that a chatbot can stand in for a seminar; workload effects on academics are not reported.","procurementImplications":"The authors argue for building inside the university rather than buying plug-in products; buyers should weigh build capacity and partner IP contributions.","operatingModelImplications":"Shifts integrity work from detection and discipline to designing conversational assessment, which needs learning designers, technologists and academics working together.","sourceVerification":{"openedUrl":"https://www.timeshighereducation.com/opinion/what-if-ai-could-strengthen-academic-integrity","referenceExcerpt":"Instead of the student asking the AI questions, the AI leads a Socratic conversation probing what the student already knows and where the gaps are","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}