{"resourceId":"fan-genai-he-three-level-meta-2026","versions":[{"version":"external-2127cf1174f89fa41888a6820a22947a0691b240840dde8a5e9f73fca2f63c80","resource":{"id":"fan-genai-he-three-level-meta-2026","title":"Three-level meta-analysis: GenAI shows medium HE learning effects; pedagogy is the key moderator","organization":"Xiamen University / NingboTech University / Shenyang Normal University / Frontiers in Psychology","sector":"Higher education learning sciences / GenAI outcomes synthesis","geography":"International HE literature (China-affiliated authors; studies synthesized globally)","publishedAt":"Published 15 May 2026","publicationDate":"2026-05-15","eventDate":null,"sourceName":"Frontiers in Psychology (open access systematic review)","sourceLabel":"Fan, Ke, Chen & Lv (2026) — GenAI HE three-level meta-analysis","sourceUrl":"https://www.frontiersin.org/journals/psychology/articles/10.3389/fpsyg.2026.1758670/pdf","evidenceClass":"academic-research","outcomeClass":"mixed","topics":["knowledge-work","operating-model","accessibility-workforce"],"finding":"Three-level random-effects meta-analysis of 36 empirical studies (132 effect sizes; 7,229 participants) finds a significant medium overall GenAI effect on higher-education learning outcomes (Hedges’ g=0.499, 95% CI 0.372–0.627, p<0.0001). Stronger effects for understanding/cognitive/creative outcomes (g=0.669) and higher-order learning (g=0.504); moderate for dispositions (g=0.452) and attainments (g=0.363). Membership/inclusion/self-worth and ‘using’ categories lacked sufficient evidence for significant pooled effects. Teaching method was the only significant moderator, with collaborative learning (g=1.026) and blended learning (g=0.633) strongest.","sledRelevance":"Newly archived synthesis after 2026-09-17’s Liu intellectual/social–emotional meta: this May 2026 three-level analysis adds LOTG outcome granularity and a clear pedagogy-moderator finding useful for student-success portfolio design. Why now: complements prior pooled view with teaching-method actionable strata and flags inclusion-evidence gaps.","evidence":"Source findings: three-level models; moderator tests; sensitivity near g=0.48 after outlier exclusion. Limitations: construct heterogeneity; insufficient k for some LOTG categories; studies vary in quality; not U.S.-only advising/retention specific.","architectureImplications":"Interpretation: prioritize architectures that embed GenAI in collaborative/blended course designs with shared LOTG-tagged outcome logging rather than generic chatbot access.","governanceImplications":"Interpretation: research governance should require pedagogy documentation and outcome-family pre-registration when citing pooled GenAI effects for investment.","securityPrivacyImplications":"Meta-analysis does not evaluate privacy. Interpretation: outcome claims cannot substitute for FERPA/vendor reviews.","caveats":"Medium pooled g is not a guaranteed local effect. Inclusion/self-worth evidence insufficient—do not claim belonging ROI from this paper. Not a community-college persistence RCT.","streamIds":["student-success"],"roles":{"sales":"Sales framing: cite this May 2026 three-level meta-analysis (36 studies, 7,229 participants) as cautious support for medium GenAI learning effects (g=0.499) that are strongest under collaborative and blended teaching—not as a universal lift. Emphasize that teaching method was the only significant moderator and that membership/inclusion evidence was insufficient. Bound proposals to pedagogy-redesign pilots with separate attainment and higher-order measures rather than undifferentiated seat licenses.","engineering":"Engineering framing: meta-analytic results imply PoCs must tag teaching method and LOTG outcome family, not only tool vendor. Prerequisites include instrument libraries, comparable control conditions, and faculty pedagogy design support. Constraints include insufficient evidence for membership/inclusion outcomes and cross-study construct heterogeneity. Proof of value: local effects with collaborative/blended arms, confidence intervals, and sensitivity checks before scale. Label recommendations as local interpretation pending campus instruments.","delivery":"Delivery framing: delivery is instructional redesign plus evaluation—not chatbot rollout alone. Ownership sits with institutional research and teaching-learning centers. Acceptance requires LOTG-aligned instruments, teaching-method strata in reports, and documented human pedagogy alongside GenAI. Risks include marketing g=0.499 as an expected local lift and neglecting belonging outcomes where pooled evidence remains insufficient. Treat vendor marketing claims as non-evidence until local LOTG reports exist."},"roleDetails":{"sales":{"customerProblem":"Leaders need a pedagogy-aware pooled estimate of GenAI learning effects beyond single pilots.","stakeholders":["Provost","Teaching & learning","IR/assessment","Faculty senate"],"discoveryQuestions":["Will collaborative/blended designs be required for GenAI tools?","Which LOTG outcome families will you instrument?","How will you treat membership/inclusion outcomes with insufficient evidence?","What local effect size is decision-relevant given g≈0.5?"],"valueHypothesis":"GenAI shows a medium pooled learning effect in HE when pedagogy aligns, especially collaborative and blended designs; attainments are smaller and inclusion evidence is insufficient.","potentialEngagement":"Evidence briefing plus a pedagogy-conditioned multi-course pilot with LOTG-aligned instruments—not a seat-license sale on g alone.","unsupportedClaims":["Guaranteed g=0.499 local gains","Membership/belonging improvements from GenAI","That tool brand matters more than teaching method"]},"engineering":{"fit":"Fits institutions building shared evaluation architectures across GenAI tutoring/writing tools with pedagogy tags.","architectureIntegration":"Outcome warehouse tagged by LOTG categories and teaching method; comparable logging across tools; study registry.","prerequisites":["Outcome instrument library mapped to LOTG","IRB/ethics path","Faculty pedagogy design support","Vendor data exports"],"constraints":["Cross-study construct heterogeneity","Insufficient evidence on inclusion/self-worth outcomes","China-affiliated synthesis of international literature"],"security":"Outcome data may include sensitive disposition items—classify and restrict access accordingly.","proofOfValue":["Pre-register primary LOTG endpoints","Report teaching-method strata","Power for medium local effects near g=0.5 not large single-study claims"]},"delivery":{"work":"Stand up LOTG-aligned evaluation playbooks and faculty redesign support for collaborative/blended GenAI use.","dependencies":["IR capacity","Faculty development time","Tool logging","Assessment redesign"],"ownership":"Institutional research jointly with teaching and learning centers.","skillsAdoption":"Faculty need GenAI-pedagogy design skills; IR needs moderator-aware reporting literacy.","governanceCheckpoints":["Pre-registration of campus pilots","Ethics review for disposition measures","Publication of nulls and inclusion gaps"],"acceptanceCriteria":["Dual intellectual/attainment instrumentation live","Teaching-method stratum reported","Decision memo cites CI not point estimate alone"],"risks":["Overclaiming from pooled g","Ignoring inclusion evidence gaps","Tool sprawl without pedagogy redesign"]}},"retrievedAt":"2026-09-19T03:05:14Z","enrichedAt":"2026-09-19T03:11:00Z","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Interpretation: insufficient membership/inclusion pooled evidence cautions against claiming GenAI closes belonging or disability-related support gaps without targeted studies.","procurementImplications":"Interpretation: procure with pedagogy-integration and evaluation clauses—not seat licenses sold on undifferentiated learning gains.","operatingModelImplications":"Interpretation: operating model pairs GenAI with collaborative/blended instructional design capacity; tool access alone is a weak lever per the moderator result.","sourceVerification":{"responseId":"curl-frontiers-meta-1758670-2026-09-18","openedUrl":"https://www.frontiersin.org/journals/psychology/articles/10.3389/fpsyg.2026.1758670/pdf","referenceExcerpt":"learning outcomes (g = 0.499). Stronger effects were found for understanding,","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}