{"resourceId":"liu-genai-he-outcomes-meta-analysis-2026","versions":[{"version":"external-ab9281bcc03778714c4073ac52ab4eb0517ef6bdb207c83e3156e785e215da83","resource":{"id":"liu-genai-he-outcomes-meta-analysis-2026","title":"Meta-analysis: GenAI shows large but heterogeneous intellectual gains and smaller social–emotional effects in HE","organization":"Chuxiong Normal University / Sichuan University of Science & Engineering / Frontiers in Psychology","sector":"Higher education learning sciences / GenAI outcomes synthesis","geography":"International HE literature (search to 26 Jan 2026; China-affiliated authors)","publishedAt":"Published 08 June 2026","publicationDate":"2026-06-08","eventDate":null,"sourceName":"Frontiers in Psychology (open access systematic review)","sourceLabel":"Liu, Xie & Xu (2026) — GenAI HE intellectual and social–emotional meta-analysis","sourceUrl":"https://www.frontiersin.org/journals/psychology/articles/10.3389/fpsyg.2026.1848745/pdf","evidenceClass":"academic-research","outcomeClass":"mixed","topics":["knowledge-work","accessibility-workforce","operating-model"],"finding":"Systematic review and random-effects meta-analysis of 33 experimental/quasi-experimental studies (3,394 participants; studies published 2021–2024; search through 26 Jan 2026). Generative AI showed a significant pooled positive effect on intellectual outcomes (Hedges’ g=1.096, 95% CI 0.087–2.104, p=0.033) and a smaller significant effect on social–emotional outcomes (g=0.301, 95% CI 0.048–0.553, p=0.020). Domain difference was not statistically significant. Publication bias detected for intellectual outcomes; social–emotional findings more stable in sensitivity analyses. Authors conclude GenAI is promising but evidence remains heterogeneous and methodologically uneven (PROSPERO CRD420261361622).","sledRelevance":"Newly relevant synthesis for student-success leaders after single-study mixed RCT nights (2026-09-15/16): provides an unarchived 2026 pooled view of intellectual vs social–emotional effects with explicit bias/heterogeneity warnings useful for portfolio evaluation.","evidence":"Source findings: Stata 18 random-effects models; moderator analyses (functional type, duration, knowledge domain) not significant at omnibus level; funnel/sensitivity for bias. Limitations: wide CI on intellectual g; detected publication bias; included studies to 2024 only; HE contexts heterogeneous; social–emotional construct breadth.","architectureImplications":"Interpretation: do not treat one tutoring chatbot architecture as universally effective; portfolio architectures should instrument intellectual and social–emotional outcomes separately and plan for study-level variance.","governanceImplications":"Interpretation: research governance should require preregistration and bias-aware evaluation when citing pooled GenAI effect sizes for campus investment.","securityPrivacyImplications":"Meta-analysis does not evaluate privacy architectures. Interpretation: outcome claims cannot substitute for FERPA/vendor reviews when deploying studied tool classes.","caveats":"Large intellectual point estimate sits in a very wide CI with publication bias—do not market g≈1.1 as expected local effect. Not U.S.-only; not advising/retention specific. Published June 2026 (before last run) but not previously archived for this stream.","streamIds":["student-success"],"roles":{"sales":"Sales framing: use this 2026 meta-analysis to support cautious optimism—pooled GenAI effects are positive on intellectual outcomes and smaller on social–emotional ones—while refusing to quote g≈1.1 as a guaranteed campus lift. Emphasize heterogeneity, wide confidence intervals, and publication bias. Bound deals to evaluated pilots with separate intellectual and belonging/engagement metrics.","engineering":"Engineering framing: meta-analytic variance implies PoCs must pre-register outcome families (intellectual vs social–emotional) and power for small-to-moderate local effects, not assume large pooled g. Prerequisites: clear outcome instruments and comparable control conditions. Constraints: literature skew and bias. Proof of value: local effects with sensitivity to study exclusion rules.","delivery":"Delivery framing: delivery should include evaluation design support—outcome dictionaries, bias checks—not only tool training. Ownership: IR/assessment with teaching-learning. Risks: cherry-picking favorable studies; overstating social–emotional benefits."},"roleDetails":{"sales":{"customerProblem":"Leaders see conflicting GenAI learning claims and need a bias-aware synthesis before scaling.","stakeholders":["Provost","IR/assessment","Teaching & learning","Student affairs (wellbeing)"],"discoveryQuestions":["Which outcome family is primary—intellectual or social–emotional?","How will you handle publication-bias risk in vendor claims?","What local effect size is decision-relevant?","Will wellbeing metrics be instrumented separately?"],"valueHypothesis":"GenAI can improve intellectual outcomes on average across studies, with smaller social–emotional effects, if local evaluation accounts for heterogeneity and bias.","potentialEngagement":"Evidence-briefing plus a pre-registered multi-outcome pilot design workshop—not a campus-wide seat purchase.","unsupportedClaims":["Guaranteed g≈1.1 local learning gains","Strong wellbeing transformation from GenAI","Homogeneous effects across disciplines"]},"engineering":{"fit":"Fits institutions building an evaluation architecture across tutoring/writing/chat tools rather than a single product.","architectureIntegration":"Shared outcomes warehouse for intellectual and social–emotional metrics; study registry; comparable logging across tools.","prerequisites":["Outcome instrument library","IRB/ethics path","Analytics warehouse","Vendor data exports"],"constraints":["Cross-study construct mismatch","Publication bias in literature baselines","Short intervention durations in many studies"],"security":"Outcome data may include sensitive wellbeing items—apply higher classification and access controls.","proofOfValue":["Pre-register primary endpoints","Report CI and sensitivity","Separate intellectual vs social–emotional dashboards"]},"delivery":{"work":"Build evaluation playbooks, train faculty on instruments, and run bias-aware reporting for GenAI pilots.","dependencies":["IR capacity","Faculty survey fatigue management","Tool logging"],"ownership":"Institutional research jointly with teaching and learning.","skillsAdoption":"Staff need basic meta-analytic literacy to interpret vendor claims.","governanceCheckpoints":["Pre-registration of campus pilots","Ethics review for wellbeing measures","Publication of nulls"],"acceptanceCriteria":["Dual-outcome instrumentation live","Bias/sensitivity note in pilot reports","Decision memo uses CIs not point estimates alone"],"risks":["Overfitting to optimistic literature","Neglecting social–emotional harms","Tool sprawl without shared metrics"]}},"retrievedAt":"2026-09-18T03:06:28Z","enrichedAt":"2026-09-18T03:15:52Z","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Interpretation: smaller social–emotional pooled effects caution against claiming GenAI closes belonging or disability-related support gaps without targeted studies.","procurementImplications":"Interpretation: require vendors to map claims to intellectual vs social–emotional evidence and disclose evaluation quality; avoid sole-source scale-ups based on pooled point estimates.","operatingModelImplications":"Interpretation: create a standing evaluation operating model (IR + CTL) that absorbs heterogeneous GenAI pilots into comparable reporting.","sourceVerification":{"responseId":"curl-frontiers-meta-pdf-2026-09-17","openedUrl":"https://www.frontiersin.org/journals/psychology/articles/10.3389/fpsyg.2026.1848745/pdf","referenceExcerpt":"Hedges’ g = 1.096, 95% CI 0.087 to 2.104, p = 0.033","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}