{"resourceId":"hayat-human-guided-genai-stem-pse-asee-2026","versions":[{"version":"external-7a715863c2362fdf022d9b0815af1a0994b730ec0913bcb4542ad9df0b61db58","resource":{"id":"hayat-human-guided-genai-stem-pse-asee-2026","title":"ASEE NSF IUSE WIP: human-guided GenAI outreach linked to higher STEM Performance Self-Evaluation","organization":"University of Nebraska–Lincoln / Lehigh University (NSF IUSE)","sector":"Higher education / academic research","geography":"United States (undergraduate engineering; transferable with limits)","publishedAt":"2026","publicationDate":null,"eventDate":null,"sourceName":"ASEE PEER","sourceLabel":"ASEE 2026 Annual Conference paper (Paper ID #50902 / peer.asee.org/60325)","sourceUrl":"https://peer.asee.org/60325.pdf","evidenceClass":"academic-research","outcomeClass":"emerging","topics":["accessibility-workforce","operating-model","knowledge-work","developers-agents"],"finding":"Human-guided GenAI pipeline (LLM engagement forecast + counseling-expert messages + GPT-4o personalization) associated with higher positive Performance Self-Evaluation across the semester (75.4% vs 56.5% historical baseline; d=0.90). Why now: ASEE 2026 NSF grantee paper not in student-success archive; addresses proactive non-cognitive support gap after tutoring RCTs.","sledRelevance":"Offers U.S. STEM student-success teams a responsible pattern—experts author, AI personalizes—for scaling weekly outreach before midterm early-alerts fire.","evidence":"Quasi-experimental comparison of Spring/Fall 2025 intervention cohort (N=41) to three prior semesters (N=96); Welch t-test on PSE; WIP explicitly non-causal.","architectureImplications":"Two-stage pipeline: forecasting model on longitudinal self-reports; personalization LLM constrained to expert message cores; human review before send.","governanceImplications":"Keep counseling expertise in the loop for tone/safety; treat WIP as hypothesis-generating pending RCT.","securityPrivacyImplications":"Self-report engagement data and personalized messages are sensitive education records; minimize retention and vendor training.","caveats":"Historical controls; PSE self-report only; no grade/retention endpoints in this WIP.","streamIds":["student-success"],"roles":{"sales":"For STEM student-success and engineering deans, this ASEE WIP links a human-guided GenAI outreach system to substantially higher Performance Self-Evaluation versus prior semesters—but it is not a causal RCT. Sell discovery around expert-authored message libraries, forecast-triggered outreach, and planned within-semester randomization—not guaranteed retention. Bounded engagement is a capped pilot with PSE dashboards and an RCT design, not a campus-wide chatbot rollout.","engineering":"Engineering takeaway: build a forecast→expert-template→constrained personalization pipeline with full audit logs of model edits before human approve/send. Prerequisites include a weekly self-report instrument, counseling SME ownership of message cores, and an approved LLM tenancy that does not train on identifiable outreach. Constraints: quasi-experimental external evidence and a single PSE outcome in the WIP. Security: FERPA-aligned storage of surveys and messages with least-privilege access. Proof of value: pre-register a within-semester RCT and track PSE alongside grades and persistence.","delivery":"Delivery work: instrument weekly self-reports, stand up a counselor-owned message library, enable GPT personalization with mandatory human approve/send, and monitor PSE dashboards. Dependencies include counseling psychology capacity, institutional research, and a reliable LMS or notification channel. Ownership sits with the STEM student-success lead partnered with counseling. Skills: staff training to review AI-personalized copy for tone and accuracy. Governance checkpoints: message approval SOP and student opt-out. Acceptance: documented PSE trajectory plus a signed RCT protocol; risks include Hawthorne effects and overclaiming causality."},"roleDetails":{"sales":{"customerProblem":"STEM programs react too late on grades while confidence and self-evaluation erode unnoticed.","stakeholders":["engineering student success","counseling/wellness","IR/analytics","IT"],"discoveryQuestions":["Do you collect weekly non-cognitive self-reports?","Who authors outreach copy today?","Will you randomize within a semester?"],"valueHypothesis":"Bounded value: pilot human-guided personalized outreach and measure PSE before claiming retention impact.","potentialEngagement":"One-semester capped STEM pilot with PSE dashboard and RCT design workshop—not a retention guarantee.","unsupportedClaims":["Causal proof of retention gains","Guaranteed grade improvements","Replacement of human advisors/counselors"]},"engineering":{"fit":"Fits as proactive outreach layer atop LMS/survey pipelines with constrained LLM personalization.","architectureIntegration":"Survey store → forecast service → template vault → LLM personalizer → human approve → notify.","prerequisites":["Weekly PSE/engagement instrument","Counseling SME","Approved GenAI vendor"],"constraints":["Historical-control evidence only","Self-report bias","WIP status"],"security":"Encrypt self-reports and messages; no consumer tools for identifiable outreach.","proofOfValue":["PSE lift vs contemporaneous control","Message edit/audit completeness","Downstream grade/persistence correlation"]},"delivery":{"work":"Deploy survey cadence, message library, personalization workflow, and weekly review huddles.","dependencies":["Counseling capacity","Notification channel","Analytics"],"ownership":"STEM student-success office with counseling psychology partner.","skillsAdoption":"Train staff to review AI-personalized copy for tone and accuracy.","governanceCheckpoints":["Weekly message audit sample","Student opt-out path","IRB if research-framed"],"acceptanceCriteria":["Documented PSE trajectory","Signed RCT protocol for next term"],"risks":["Hawthorne effects","Message fatigue","Over-reliance on self-report"]}},"retrievedAt":"2026-09-23T03:19:40Z","enrichedAt":"2026-09-23T03:24:00Z","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Non-cognitive outreach may support persistence for students with declining confidence; ensure accessible survey/notification channels.","procurementImplications":"Procure forecasting and personalization as modules with human-approval gates; avoid unsupervised generative outreach SKUs.","operatingModelImplications":"Shift from midterm reactive alerts to weekly forecast-triggered, human-approved outreach cycles.","sourceVerification":{"responseId":"curl-asee-peer-60325","openedUrl":"https://peer.asee.org/60325.pdf","referenceExcerpt":"The intervention group maintains significantly higher Performance Self-Evaluation throughout the semester (75.4% vs. 56.5% positive responses)","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}