{"resourceId":"elliott-cogniti-genai-agents-pilot-2026","versions":[{"version":"external-ab9281bcc03778714c4073ac52ab4eb0517ef6bdb207c83e3156e785e215da83","resource":{"id":"elliott-cogniti-genai-agents-pilot-2026","title":"Cogniti GenAI academic-agent pilot: perceived learning/success up; engagement and wellbeing flat","organization":"Flinders University / Technology, Knowledge and Learning (Springer)","sector":"Higher education teaching and learning / assessment support","geography":"Australia (Flinders University; undergraduate courses)","publishedAt":"Accepted 17 December 2025; © The Author(s) 2026 (online OA)","publicationDate":"2026-01-01","eventDate":null,"sourceName":"Technology, Knowledge and Learning (Springer Open)","sourceLabel":"Elliott et al. (2026) — Cogniti AI personalized GenAI agents pilot","sourceUrl":"https://link.springer.com/content/pdf/10.1007/s10758-025-09942-7.pdf","evidenceClass":"academic-research","outcomeClass":"mixed","topics":["knowledge-work","developers-agents","operating-model","accessibility-workforce"],"finding":"Pilot study of course-embedded Cogniti GenAI academic agents for assessment preparation across four undergraduate courses (education, psychology, social work, sport). N=139 survey completers (ages 18–55) pre/post agent deployment. Mann–Whitney U tests: significant improvements in perceived student learning and perceived success (both p=0.04; w≈1642 and 1639.5), with no significant change in engagement (p=0.30) or wellbeing (p=0.59). Qualitative themes valued 24/7 context-specific support and confidence; students also stressed retaining human academic interaction. Authors conclude agents may enhance targeted aspects of experience when carefully integrated to complement—not replace—human-centred learning.","sledRelevance":"Newly relevant after 2026-09-16 archived hybrid/reflective feedback and RAG-tutor nulls: this unarchived 2026 OA pilot supplies HE evidence on personalized academic agents’ effects on student-success perceptions versus engagement/wellbeing. Why now: fills agent-product gap for advising/tutoring-adjacent on-demand support without overclaiming objective grades.","evidence":"Source findings: pre/post perception surveys on one assessment; Mann–Whitney and rank-biserial effect sizes; content analysis of open responses. Distinguishes perception gains (learning/success) from null engagement/wellbeing. Limitations: pilot/non-randomized; self-report only; single-assessment window; Australian one-university sample; response rates vary by course (e.g., psychology 9.6% of enrolled).","architectureImplications":"Interpretation: prefer course-scoped agent configs grounded in assessment briefs over generic chat; instrument both perception and objective performance; keep human escalation paths. Integration: LMS/course sites with agent endpoints and usage/outcome logging.","governanceImplications":"Interpretation: approve agents as supplemental academic support with disclosure, academic-integrity boundaries, and faculty ownership of prompts/knowledge—not unmanaged consumer bots.","securityPrivacyImplications":"Source emphasizes institutional integration; does not detail FERPA-equivalent flows. Interpretation: treat prompts/uploads as education records; require vetted campus tools, retention limits, and no-train vendor terms.","caveats":"Perception outcomes ≠ measured GPA/retention. Non-experimental design cannot support causal claims. Not a U.S. community-college advising RCT. Publication accepted late 2025 / ©2026; included as newly archived for this stream.","streamIds":["student-success"],"roles":{"sales":"Sales framing: position Cogniti-style course agents as bounded assessment-support aids that may lift perceived learning/success, not as proven engagement or wellbeing solutions. This Flinders pilot (N=139) found p=0.04 gains on learning and success perceptions with flat engagement/wellbeing and strong student demand to keep humans in the loop. Bound proposals to instrumented course pilots with human escalation—unsupported: guaranteed retention, wellness ROI, or replacement of faculty contact.","engineering":"Engineering framing: implement course-scoped GenAI agents tied to assessment briefs with pre/post instrumentation covering learning, success, engagement, and wellbeing perceptions plus objective marks. Prerequisites: faculty-authored knowledge, LMS placement, and analytics. Constraints: self-selection bias in pilots; cross-border model hosting. Proof of value: replicate perception contrasts and add grade/time-on-task arms before scale.","delivery":"Delivery framing: delivery is instructional design—prompt libraries, student onboarding, and office-hour escalation—not a chatbot flip. Dependencies: faculty champions in each course and integrity guidance. Ownership: teaching/learning with departments. Acceptance: live agents in pilot courses, measured pre/post bundles, documented human-support path; risks include mistaking perception lifts for learning and eroding staff contact."},"roleDetails":{"sales":{"customerProblem":"Campuses want always-on academic agents but lack evidence separating perception gains from engagement/wellbeing and objective learning.","stakeholders":["Teaching & learning center","Faculty course leads","Student success / retention","Academic technology"],"discoveryQuestions":["Will you measure objective marks as well as perceptions?","Is human escalation required in the workflow?","Which courses own agent knowledge updates?","What integrity rules apply to agent use on assessments?"],"valueHypothesis":"Course-embedded GenAI agents can improve perceived learning/success for assessment prep when faculty-scoped and paired with humans; engagement/wellbeing gains are not evidenced here.","potentialEngagement":"Bounded multi-course pilot with pre/post surveys plus grade checks and explicit no-replacement staffing commitment.","unsupportedClaims":["Guaranteed retention or wellness improvements","That perception gains equal learning","Staff-cost savings from replacing tutors/advisors"]},"engineering":{"fit":"Fits LMS-embedded, course-specific academic agents for assessment clarification—not enterprise early-alert alone.","architectureIntegration":"Per-course agent configs + retrieval over approved materials + LMS entry points + logging; human ticket/escalation channel.","prerequisites":["Faculty-authored briefs/rubrics","Campus-approved GenAI endpoint","Survey/analytics pipeline","Access control by course roster"],"constraints":["Non-random pilot bias","Variable course response rates","Data residency for student prompts"],"security":"Prompts may include personal academic struggles; minimize retention and block consumer tools for roster data.","proofOfValue":["Replicate learning/success perception contrasts","Add objective assessment outcomes","Track escalation rates to humans"]},"delivery":{"work":"Faculty prompt design, student communications, integrity guidance, and helpdesk escalation playbooks.","dependencies":["Course coordinator time","LMS admin","Integrity office language","Support staffing"],"ownership":"Center for teaching and learning with participating departments.","skillsAdoption":"Faculty need agent-authoring skills; students need disclosure/how-to-use guidance.","governanceCheckpoints":["Pedagogy approval of agent scope","Privacy review of prompts","Integrity syllabus statements"],"acceptanceCriteria":["Agents live in ≥2 pilot courses","Pre/post instruments completed","Human escalation path documented"],"risks":["Overclaiming from self-report","Uneven adoption across disciplines","Students substituting agents for staff"]}},"retrievedAt":"2026-09-18T03:06:28Z","enrichedAt":"2026-09-18T03:15:52Z","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Interpretation: 24/7 agents may help students balancing work/caregiving, but flat wellbeing and student insistence on human contact caution against using agents as disability or counseling substitutes; co-design with accessibility offices.","procurementImplications":"Interpretation: procure course-configurable agents with faculty authoring rights and evaluation clauses—not seat licenses sold on engagement/wellbeing claims unsupported here.","operatingModelImplications":"Interpretation: operating model keeps faculty as knowledge owners and agents as supplemental capacity; staffing models should reserve human contact for complex cases.","sourceVerification":{"responseId":"curl-springer-cogniti-2026-09-17","openedUrl":"https://link.springer.com/content/pdf/10.1007/s10758-025-09942-7.pdf","referenceExcerpt":"student learning [w = 1642, p = 0.04] and student suc-","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}