From the Student Success edition of October 3, 2026
VT cross-subfield study: LLM adoption swings from 89.6% (algorithms) to 15.2% (software engineering) under one literacy intervention
Virginia Tech Department of Computer Science (Nizamani, Lee, Donekal Chandrashekar, Ellis, Ramakrishnan) · Higher education computing teaching / AI literacy and student GenAI practices · United States (Virginia Tech; undergraduate post-CS1 problem-solving course)
- Publisher
- arXiv
- Original publication
- Submitted to arXiv 1 October 2026 (cs.CY); accepted IEEE FIE 2026
- Source retrieved
- 2026-10-04
What happened
Among 211 undergraduates completing reflections across seven CS subfield assignments under a shared demystify–use–reflect intervention, LLM adoption varied from Algorithms 89.6% and Web Development 80.6% down to Software Engineering 15.2%, with Networking/Databases moderate (~58.3%) and HCI/ML lower (~40–43%). Students predominantly cast LLMs as Assistants rather than Experts. Among users, verification was nearly universal (93%–100%), typically multi-strategy; testing was highest in Algorithms (87.8%) and lowest in HCI (27.5%), while web search rose in open-ended tasks.
Why it matters
Shows one campus AI-literacy policy does not produce uniform GenAI practice: assignment structure drives adoption and verification mix. Complements yesterday's enjoyment-vs-actionable-feedback and engagement-gate findings by warning student-success teams not to treat 'students use Copilot' as a single behavior.
Evidence and measured results
Post-assignment reflection surveys; Kruskal–Wallis on prompt counts; χ² on testing rates. Self-report not telemetry; single course; preprint.
Limitations and uncertainty
Self-reported; assignment design confounded with subfield; no learning RCT; single R1 institution; preprint.
Put this evidence to work
Lighthouse Advisory interpretation, grounded in this source. Enriched 2026-10-04; this does not change the original publication date. Each role follows the same checklist so takeaways compare across sources.
Sales
Role takeaway
- Customer problem
- Institutions set one GenAI literacy rule while student use and verification differ radically by assignment type.
- Stakeholders
- CS faculty
- teaching centers
- academic technology
- student success
- academic integrity offices
- Discovery questions
- Do we measure GenAI adoption by assignment type?
- What verification strategies do students report?
- Is our literacy intervention reflective or ban-only?
- Value hypothesis
- Assignment-aware literacy analytics can target coaching where adoption is high and verification weak, without claiming grade guarantees.
- Potential engagement
- One-term reflection instrumentation pilot across a multi-module CS course with faculty readout.
- Unsupported claims
- Causal learning gains
- Telemetry-grade accuracy from surveys
- Uniform campus adoption
Pre-sales engineering
Role takeaway
- Fit
- Undergraduate CS courses seeking responsible GenAI literacy measurement across diverse assignment types.
- Architecture and integration
- LMS reflection forms → per-assignment analytics warehouse → faculty dashboards by subfield/activity family.
- Prerequisites
- LMS survey tooling
- roster linkage
- IRB/FERPA review
- Constraints
- self-report bias
- single-site evidence
- assignment confound
- Security
- Reflection and roster joins as education records; minimize retention; no automatic misconduct routing.
- Proof of value
- Stable per-assignment adoption reports
- Verification mix visible to faculty
- Coaching changes documented
Delivery
Role takeaway
- Work
- Deploy reflection prompts, clean response data, produce subfield adoption/verification briefings, iterate coaching.
- Dependencies
- course instructor
- teaching center
- IR/assessment
- Ownership
- Course instructor with teaching-center analytics support.
- Skills and adoption
- Faculty interpret subfield differences; students practice verification strategies matched to task type.
- Governance checkpoints
- FERPA/IRB signoff
- mid-term data quality check
- end-term faculty readout
- Acceptance criteria
- Adoption by assignment published internally
- Verification strategies summarized
- No ban-only policy assumed
- Risks
- Self-report inflation
- Over-policing high-adoption courses
- Ignoring open-ended task needs
Implementation considerations
Lighthouse Advisory interpretation across the operating dimensions a public-sector buyer must settle before this evidence becomes a design. Each note answers the question under its heading for this specific source.
Architecture and integration
What must connect, and where does the AI sit in the workflow?
Instrument GenAI use and verification by assignment type in LMS analytics; avoid one-size telemetry dashboards that average across subfields.
Governance
Who approves, reviews and stays accountable for outcomes?
Write AI-literacy policies that vary expectations by task type (testable coding vs open design) rather than a single allowed/banned rule.
Security and privacy
What data, permissions and controls need testing?
Reflection responses and prompt-count data are education records; avoid joining to misconduct systems without policy review.
Accessibility and workforce
Who is affected, and what skills or accommodations follow?
Students in open-ended tasks may need different verification scaffolds than those in unit-testable coding—uniform AI rules can under-serve or over-police subgroups.
Procurement
What should contracts, pricing and exit terms secure?
Prefer tools that report usage and verification proxies by course activity type, with faculty-configurable literacy modules—not only seat licenses.
Operating model
Which teams own the service once it runs?
Teaching centers should review GenAI reflection data by assignment family weekly and adjust prompting/verification coaching accordingly.
Publication history
- 2026-10-03Student Success · Issue 273 resources
Stable resource ID: nizamani-vt-llm-use-across-cs-subfields-20261001