{"resourceId":"ibitoye-computing-llm-srl-strategies-20260927","versions":[{"version":"external-2aecc9ebf44bea3ca61198d763c821d17b3c60ee9b34afcb55a5ef38713481ff","resource":{"id":"ibitoye-computing-llm-srl-strategies-20260927","title":"Computing students’ LLM learning use maps to Probe–Monitor–Shape SRL strategies","organization":"University of Cincinnati School of Information Technology (Ibitoye & Popoola)","sector":"Higher education computing education / learning sciences","geography":"United States (University of Cincinnati computing programs; undergrad and graduate)","publishedAt":"Submitted to arXiv 27 September 2026 (cs.CY)","publicationDate":"2026-09-27","eventDate":null,"sourceName":"arXiv","sourceLabel":"Academic preprint (cs.CY)","sourceUrl":"https://arxiv.org/pdf/2609.33750","evidenceClass":"academic-research","outcomeClass":"mixed","topics":["knowledge-work","developers-agents","accessibility-workforce","operating-model"],"finding":"Survey of 225 computing students finds LLM use across nine learning functions (top open-coded: explain/clarify concepts n=31; debugging n=15; research support n=14). Of strategy respondents, detailed context (n=46) and iterative prompting (n=22) dominate. On Likert items, 83.87% reported incorrect or confusing answers; 61.29% worried about over-dependence; views on long-term conceptual grasp were split. Authors synthesize Probe–Monitor–Shape as an SRL heuristic and note monitoring/verification is less common than shaping via context. Evidence is self-reported practice, not causal learning gains.","sledRelevance":"Gives student-success and CS teaching teams an operable taxonomy for AI literacy curricula: teach students to probe, monitor, and shape model help—especially verification, which lags. Complements FGLI depth findings by showing even computing students struggle with incorrect outputs and dependency concerns.","evidence":"English survey with open/closed items; pilot-tested; thematic coding with Krippendorff’s α=0.85 on double-coded subset; descriptive Likert analysis. Optional items reduce n for some questions; preprint.","architectureImplications":"Tutor/help interfaces should prompt for context, require or nudge verification steps, and surface uncertainty—encoding Probe–Monitor–Shape in UX rather than relying on ambient chat. Log strategy scaffolds separately from answer dumps.","governanceImplications":"AI literacy policy should define expected student strategies (verification against course materials) and disclose model fallibility. Do not treat self-reported ‘understanding help’ as proof of learning outcomes.","securityPrivacyImplications":"Survey is anonymous academic research; production systems that log prompts for literacy analytics must treat transcripts as education records, minimize retention, and avoid punitive use of strategy telemetry.","caveats":"Self-report; optional items; computing-heavy sample at one U.S. institution; no experimental outcome trial; percentages on challenge items reflect respondents to those items.","streamIds":["student-success"],"roles":{"sales":"For CS and student-success buyers investing in AI tutors or literacy modules, this survey sells a curriculum and UX pattern—Probe, Monitor, Shape—not another unguided chatbot. Bound claims to n=225 self-reports: students mainly seek explanations and debugging help, yet 83.87% hit incorrect answers and monitoring lags. Position a bounded literacy pilot that grades verification habits, not seat utilization.","engineering":"Engineering should add interaction scaffolds that ask students for task context, encourage or require verification against course artifacts, and make model uncertainty visible in the tutor UI. Wire strategy events (probe, monitor, shape) into analytics separate from answer content. Constraints include self-report baseline evidence and a computing-centric respondent pool that may not transfer to all majors. Proof-of-value is rising verification actions and fewer blind-accept behaviors in telemetry—not chatbot satisfaction alone. Keep literacy logs out of automatic integrity pipelines.","delivery":"Delivery should embed Probe–Monitor–Shape practice in computing courses and tutoring centers, with faculty-authored verification checklists tied to weekly assignments. Ownership sits with teaching-and-learning alongside CS coordinators who maintain course-grounded examples. Train tutors to coach monitoring strategies rather than only fixing code. Risks include teaching prompt tricks without conceptual ownership, survey nonresponse bias in the source evidence, and scope creep when generalizing beyond computing majors without local revalidation."},"roleDetails":{"sales":{"customerProblem":"Students use campus GenAI heavily for explanations/debugging but distrust accuracy and under-verify outputs.","stakeholders":["CS department","teaching & learning","tutoring center","student success","academic technology"],"discoveryQuestions":["Do our AI tools nudge verification against course materials?","How do we teach monitoring, not only prompting?","Which courses see dependency concerns?"],"valueHypothesis":"Encoding Probe–Monitor–Shape in literacy and tutor UX can raise verification and reduce blind reliance without banning GenAI.","potentialEngagement":"One-term AI literacy module in CS1/IT with telemetry on context, iteration, and verification steps.","unsupportedClaims":["Causal exam-score lift","Cross-discipline generalizability","Elimination of incorrect model answers"]},"engineering":{"fit":"LLM tutor/help tools in computing courses with course-material grounding.","architectureIntegration":"Chat UX → context checklist → response → verification panel linked to LMS/resources → optional reflection log.","prerequisites":["course content connectors","student auth","analytics schema for strategy events"],"constraints":["self-report baseline","optional-item missingness","computing-sample bias"],"security":"Store strategy logs as education records; no punitive integrity auto-action from literacy telemetry; no training on prompts without contract.","proofOfValue":["Verification event rate","Student-reported incorrect-answer handling","Reduction in ‘accept without understanding’ Likert"]},"delivery":{"work":"Author verification checklists, train tutors/faculty, run literacy modules, review telemetry weekly.","dependencies":["faculty content","tutoring staff","LMS integration"],"ownership":"Teaching & learning with CS course coordination.","skillsAdoption":"Tutors coach monitoring strategies; students practice citing course sources against model claims.","governanceCheckpoints":["Syllabus AI-use statement","midterm strategy review","end-term literacy outcome check"],"acceptanceCriteria":["Probe–Monitor–Shape taught and assessed","Verification scaffold live in at least one course tool","Limitations of self-report disclosed"],"risks":["Prompt engineering without understanding","Survey nonresponse bias","Scope creep beyond computing"]}},"retrievedAt":"2026-09-30T03:28:22Z","enrichedAt":"2026-09-30T03:31:28Z","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Students already face incorrect/generic answers; literacy that teaches monitoring helps all learners, including those without strong peer debugging networks. Staff roles shift toward coaching verification strategies rather than only answering content questions.","procurementImplications":"Prefer AI learning tools with built-in verification scaffolds and strategy analytics over free-form chat alone. Require disclosure of known failure modes (incorrect answers) in vendor demos.","operatingModelImplications":"Make Probe–Monitor–Shape a standard tutoring/AI-literacy operating procedure; measure verification as an ops KPI alongside usage.","sourceVerification":{"openedUrl":"https://arxiv.org/pdf/2609.33750","referenceExcerpt":"The most common issue was receiving incorrect or confusing answers as noted by 83.87%","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}