{"resourceId":"ates-hybrid-genai-feedback-multisite-2026","versions":[{"version":"external-3a00719833df833a8bd842c6b4cd8c4e8a5b23036ddaecee7aa1f6f25a5d78a6","resource":{"id":"ates-hybrid-genai-feedback-multisite-2026","title":"Multisite GenAI feedback RCT: reflective and hybrid designs beat direct AI on AI-free transfer","organization":"Kırşehir Ahi Evran University / International Journal of Educational Technology in Higher Education","sector":"Higher education teaching and learning / scientific argumentation","geography":"Four universities (science domains; Turkey-affiliated corresponding author; multisite HE)","publishedAt":"Online 16 July 2026; journal volume 2026","publicationDate":"2026-07-16","eventDate":null,"sourceName":"International Journal of Educational Technology in Higher Education (Springer Open)","sourceLabel":"Ates (2026) IJETHE 23:38 — human-centered GenAI feedback designs","sourceUrl":"https://link.springer.com/content/pdf/10.1186/s41239-026-00614-9.pdf","evidenceClass":"academic-research","outcomeClass":"mixed","topics":["knowledge-work","operating-model","accessibility-workforce"],"finding":"Cluster-randomized longitudinal field experiment (N=1,176 first-year undergraduates; 48 sections; four universities; three science domains) compared peer-only, direct GenAI, reflective GenAI, and hybrid (self-evaluation + peer + GenAI) feedback. Direct GenAI improved immediate four-dimension argument-quality gain versus peer (adjusted contrast b=0.57, d=0.25). Reflective and hybrid further outperformed direct GenAI on argument-quality gain (d=0.29 and 0.40). Hybrid showed the clearest conceptual-learning advantage versus direct GenAI (b=0.25, d=0.23). On delayed AI-free transfer, reflective and hybrid significantly outperformed direct GenAI (d=0.39 and 0.48); direct GenAI did not beat peer on transfer. Process measures: reflective/hybrid raised feedback uptake and self-regulated learning versus direct GenAI.","sledRelevance":"Newly relevant for U.S./international student-success and teaching-learning offices after the 2026-09-15 student-success edition flagged durable learning and FeedbackWriter post-test nulls: this previously inaccessible (Cloudflare-blocked) open article supplies multisite causal evidence that GenAI feedback design—not access alone—drives AI-independent transfer. Directly informs LMS-integrated feedback and writing-support pilots.","evidence":"Source findings: multilevel models with section random intercepts and institution fixed effects; baseline equivalence; high implementation fidelity (95.8%); Holm-adjusted contrasts; mediation via uptake and SRL. Distinguishes immediate product gains from delayed AI-free transfer. Author limitations: unequal time-on-task/training across conditions; peer-comment quality unmeasured; first-year science only; mediation not cycle-lagged causal proof.","architectureImplications":"Interpretation: architectures should expose sequenced workflows (self-eval → optional peer → GenAI critique) rather than a single chat endpoint; log revision artifacts for uptake metrics; keep a supervised AI-free assessment path to measure transfer. Integration points: LMS assignment + rubric tools + GenAI critique service with audit trails.","governanceImplications":"Interpretation: governance should approve feedback-design standards (when direct AI is allowed vs required reflection), human oversight of GenAI critique quality, and research consent separate from compulsory coursework—as the study did.","securityPrivacyImplications":"Source notes GenAI-supported conditions received GenAI-use training; does not detail FERPA/vendor data flows. Interpretation: treat student drafts sent to GenAI as sensitive academic data; prefer institution-controlled models or contractual no-train clauses and retention limits.","caveats":"Online 2026-07-16 (before last stream run) but newly accessible after prior Cloudflare rejection on 2026-09-15. Reflective/hybrid advantages may partly reflect extra evaluative workload and GenAI training, not hybrid sequencing alone. Four institutions modeled as fixed effects; generalization beyond first-year science uncertain. Not a U.S. community-college advising trial.","streamIds":["student-success"],"roles":{"sales":"Sales framing: do not sell “GenAI feedback access” as equivalent to durable learning. This multisite RCT shows direct GenAI can lift immediate argument quality versus peer feedback, but reflective and hybrid designs outperform direct GenAI on delayed AI-free transfer and process mediators (uptake, self-regulated learning). Use the paper to bound proposals around feedback workflow design, LMS sequencing, and AI-free transfer KPIs—not seat licenses alone. Unsupported: guaranteed GPA/retention lift or that any chatbot equals hybrid pedagogy.","engineering":"Engineering framing: implement sequenced feedback states (self-evaluation artifacts, peer rubric input, GenAI critique) with measurable uptake logging, not a standalone tutor chat. Prerequisites include shared rubrics, LMS APIs, and a supervised AI-free transfer task path. Constraints: unequal time-on-task confounds if you only A/B “AI on/off”; jurisdictional limits on sending drafts to third-party models. Proof of value: replicate argument-quality and delayed AI-free transfer contrasts with pre-registered equal-time arms.","delivery":"Delivery framing: delivery work is instructional redesign—faculty rubric training, student GenAI-use onboarding, and revision cycles—not a plug-in install. Dependencies: section scheduling, peer-feedback facilitation capacity, and assessment integrity rules for AI-free tasks. Ownership: teaching/learning center with department chairs. Acceptance criteria: documented hybrid/reflective workflows live in LMS plus measured transfer task; risks include workload spikes and mistaking immediate draft polish for learning."},"roleDetails":{"sales":{"customerProblem":"Institutions buying GenAI writing/feedback tools need evidence on durable learning versus immediate draft improvement.","stakeholders":["Provost/teaching & learning","Faculty senate / department chairs","LMS and academic-technology leads","Student-success assessment offices"],"discoveryQuestions":["Will you measure AI-free delayed transfer, or only revision scores?","Is self-evaluation required before GenAI critique?","What equal-time control will you use against hybrid workflows?","Where do student drafts go contractually?"],"valueHypothesis":"Sequenced reflective/hybrid GenAI feedback can improve argument quality and AI-free transfer versus direct GenAI access, if pedagogy and measurement are in scope.","potentialEngagement":"Bounded pilot: redesign one multi-section science/writing course’s feedback workflow with hybrid sequence and a delayed AI-free transfer assessment.","unsupportedClaims":["Guaranteed retention or GPA gains","That GenAI access alone matches hybrid outcomes","Cost savings from replacing faculty feedback"]},"engineering":{"fit":"Fits LMS-centered writing/argumentation courses needing rubric-aligned GenAI critique with human/peer stages.","architectureIntegration":"Workflow engine over LMS assignments: self-eval form → peer review → GenAI critique service → revision upload; separate AI-free exam mode.","prerequisites":["Shared four-dimension (or local) rubric","LMS APIs for assignment states","GenAI critique endpoint with logging","Consent/research data pipeline if evaluating"],"constraints":["Extra student time in reflective/hybrid arms","Peer feedback quality variance","Cross-border model hosting / data residency"],"security":"Drafts are education records risk; require vendor no-train, retention caps, and role-based instructor access to logs.","proofOfValue":["Pre-register transfer and conceptual-learning contrasts","Equal-time sensitivity arm","Uptake/SRL process metrics as leading indicators"]},"delivery":{"work":"Faculty development on hybrid feedback sequencing, student GenAI training module, rubric calibration, and transfer-task administration.","dependencies":["Section scheduling","Peer-review facilitation","Assessment office approval of AI-free tasks","Support staffing for GenAI how-to"],"ownership":"Center for teaching and learning jointly with course coordinators.","skillsAdoption":"Instructors need rubric coaching and GenAI critique review skills; students need evaluative-judgment prompts before AI use.","governanceCheckpoints":["Pedagogy board sign-off on feedback designs","Privacy review of draft routing","Academic integrity alignment for AI-free assessments"],"acceptanceCriteria":["Hybrid/reflective workflow live in LMS for pilot sections","Delayed AI-free transfer administered","Fidelity checklist ≥90% of steps"],"risks":["Workload backlash","Ceiling effects on strong writers","Overclaiming from immediate gain scores"]}},"retrievedAt":"2026-09-17T03:08:30Z","enrichedAt":"2026-09-17T03:15:56Z","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Interpretation: reflective prompts and hybrid designs may help diverse learners practice evaluative judgment, but extra workload can burden students with disabilities or jobs unless time is accommodated; co-design lighter reflective minima as authors suggest for future work.","procurementImplications":"Interpretation: procure GenAI critique as a configurable workflow component with logging and no-train terms, not a generic chatbot seat; budget faculty development and equal-time evaluation, which the paper flags as needed for cost-effectiveness claims.","operatingModelImplications":"Interpretation: operating model shifts from “turn on AI feedback” to owned instructional sequences across sections, with teaching-center ownership and multi-cycle revision calendars.","sourceVerification":{"responseId":"curl-springer-pdf-2026-09-16","openedUrl":"https://link.springer.com/content/pdf/10.1186/s41239-026-00614-9.pdf","referenceExcerpt":"conditions outperformed direct GenAI-supported feedback on delayed AI-free","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}