From the Student Success edition of September 29, 2026
Computing students’ LLM learning use maps to Probe–Monitor–Shape SRL strategies
University of Cincinnati School of Information Technology (Ibitoye & Popoola) · Higher education computing education / learning sciences · United States (University of Cincinnati computing programs; undergrad and graduate)
- Publisher
- arXiv
- Original publication
- Submitted to arXiv 27 September 2026 (cs.CY)
- Source retrieved
- 2026-09-30
What happened
Survey of 225 computing students finds LLM use across nine learning functions (top open-coded: explain/clarify concepts n=31; debugging n=15; research support n=14). Of strategy respondents, detailed context (n=46) and iterative prompting (n=22) dominate. On Likert items, 83.87% reported incorrect or confusing answers; 61.29% worried about over-dependence; views on long-term conceptual grasp were split. Authors synthesize Probe–Monitor–Shape as an SRL heuristic and note monitoring/verification is less common than shaping via context. Evidence is self-reported practice, not causal learning gains.
Why it matters
Gives student-success and CS teaching teams an operable taxonomy for AI literacy curricula: teach students to probe, monitor, and shape model help—especially verification, which lags. Complements FGLI depth findings by showing even computing students struggle with incorrect outputs and dependency concerns.
Evidence and measured results
English survey with open/closed items; pilot-tested; thematic coding with Krippendorff’s α=0.85 on double-coded subset; descriptive Likert analysis. Optional items reduce n for some questions; preprint.
Limitations and uncertainty
Self-report; optional items; computing-heavy sample at one U.S. institution; no experimental outcome trial; percentages on challenge items reflect respondents to those items.
Put this evidence to work
Lighthouse Advisory interpretation, grounded in this source. Enriched 2026-09-30; this does not change the original publication date. Each role follows the same checklist so takeaways compare across sources.
Sales
Role takeaway
- Customer problem
- Students use campus GenAI heavily for explanations/debugging but distrust accuracy and under-verify outputs.
- Stakeholders
- CS department
- teaching & learning
- tutoring center
- student success
- academic technology
- Discovery questions
- Do our AI tools nudge verification against course materials?
- How do we teach monitoring, not only prompting?
- Which courses see dependency concerns?
- Value hypothesis
- Encoding Probe–Monitor–Shape in literacy and tutor UX can raise verification and reduce blind reliance without banning GenAI.
- Potential engagement
- One-term AI literacy module in CS1/IT with telemetry on context, iteration, and verification steps.
- Unsupported claims
- Causal exam-score lift
- Cross-discipline generalizability
- Elimination of incorrect model answers
Pre-sales engineering
Role takeaway
- Fit
- LLM tutor/help tools in computing courses with course-material grounding.
- Architecture and integration
- Chat UX → context checklist → response → verification panel linked to LMS/resources → optional reflection log.
- Prerequisites
- course content connectors
- student auth
- analytics schema for strategy events
- Constraints
- self-report baseline
- optional-item missingness
- computing-sample bias
- Security
- Store strategy logs as education records; no punitive integrity auto-action from literacy telemetry; no training on prompts without contract.
- Proof of value
- Verification event rate
- Student-reported incorrect-answer handling
- Reduction in ‘accept without understanding’ Likert
Delivery
Role takeaway
- Work
- Author verification checklists, train tutors/faculty, run literacy modules, review telemetry weekly.
- Dependencies
- faculty content
- tutoring staff
- LMS integration
- Ownership
- Teaching & learning with CS course coordination.
- Skills and adoption
- Tutors coach monitoring strategies; students practice citing course sources against model claims.
- Governance checkpoints
- Syllabus AI-use statement
- midterm strategy review
- end-term literacy outcome check
- Acceptance criteria
- Probe–Monitor–Shape taught and assessed
- Verification scaffold live in at least one course tool
- Limitations of self-report disclosed
- Risks
- Prompt engineering without understanding
- Survey nonresponse bias
- Scope creep beyond computing
Implementation considerations
Lighthouse Advisory interpretation across the operating dimensions a public-sector buyer must settle before this evidence becomes a design. Each note answers the question under its heading for this specific source.
Architecture and integration
What must connect, and where does the AI sit in the workflow?
Tutor/help interfaces should prompt for context, require or nudge verification steps, and surface uncertainty—encoding Probe–Monitor–Shape in UX rather than relying on ambient chat. Log strategy scaffolds separately from answer dumps.
Governance
Who approves, reviews and stays accountable for outcomes?
AI literacy policy should define expected student strategies (verification against course materials) and disclose model fallibility. Do not treat self-reported ‘understanding help’ as proof of learning outcomes.
Security and privacy
What data, permissions and controls need testing?
Survey is anonymous academic research; production systems that log prompts for literacy analytics must treat transcripts as education records, minimize retention, and avoid punitive use of strategy telemetry.
Accessibility and workforce
Who is affected, and what skills or accommodations follow?
Students already face incorrect/generic answers; literacy that teaches monitoring helps all learners, including those without strong peer debugging networks. Staff roles shift toward coaching verification strategies rather than only answering content questions.
Procurement
What should contracts, pricing and exit terms secure?
Prefer AI learning tools with built-in verification scaffolds and strategy analytics over free-form chat alone. Require disclosure of known failure modes (incorrect answers) in vendor demos.
Operating model
Which teams own the service once it runs?
Make Probe–Monitor–Shape a standard tutoring/AI-literacy operating procedure; measure verification as an ops KPI alongside usage.
Publication history
- 2026-09-29Student Success · Issue 233 resources
Stable resource ID: ibitoye-computing-llm-srl-strategies-20260927