{"resourceId":"nizamani-dbworkout-sql-formative-platform-20261001","versions":[{"version":"external-942a64f65e740b63ac88d1786439b65f2a34fef6b4f108e4d2b164b6984d0b20","resource":{"id":"nizamani-dbworkout-sql-formative-platform-20261001","title":"VT DBWorkout: undergrads enjoy gamified SQL practice, but only 42% find automated feedback actionable","organization":"Virginia Tech Department of Computer Science (Nizamani, Devaraj, Nguyen, Goyal, Nizamani, Hamouda, Goldberg)","sector":"Higher education computing teaching / formative practice platforms","geography":"United States (Virginia Tech; undergraduate database courses)","publishedAt":"Submitted to arXiv 1 October 2026 (cs.CY; WIP)","publicationDate":"2026-10-01","eventDate":null,"sourceName":"arXiv","sourceLabel":"Academic preprint / work-in-progress","sourceUrl":"https://arxiv.org/pdf/2610.01174","evidenceClass":"academic-research","outcomeClass":"mixed","topics":["knowledge-work","developers-agents","operating-model","accessibility-workforce"],"finding":"WIP evaluation of DBWorkout, a sandbox SQL practice platform with multi-dimensional result-based feedback and session gamification, plus LLM-assisted schema/question/solution generation under human-in-the-loop instructor control. Deployment with 170 undergraduates across two in-class sessions found 90% agreement on perceived learning value, 87% enjoyment, and low pressure (22%), yet only 42% found automated feedback sufficiently actionable—echoed in ~40% of open responses citing feedback depth. Authors plan richer next-step guidance (diffs, hints).","sledRelevance":"HE formative-practice signal complementary to Melbourne’s upstream-verified GenAI loops: enjoyment and perceived learning can look strong while actionable feedback—the feature students need to revise—lags. Warns student-success/teaching teams not to equate positive UX surveys with feedback quality.","evidence":"In-class deployment survey (Likert + open responses) with 170 students; TA pilot mentioned. No RCT on learning gains; WIP preprint.","architectureImplications":"Isolate live DB sandboxes per attempt; score multiple result dimensions; keep LLM generation behind instructor review before release to students.","governanceImplications":"Treat student enjoyment metrics as insufficient acceptance criteria; require actionable-feedback thresholds before scaling. Keep LLM-generated keys human-verified.","securityPrivacyImplications":"Sandbox queries and attempt logs are education records; isolate from grading/misconduct systems; review LLM prompts that may embed instructor solutions.","caveats":"WIP; perception not causal learning; single university; two sessions; LLM generation quality not separately evaluated.","streamIds":["student-success"],"roles":{"sales":"For CS teaching leads seeking scalable SQL practice, DBWorkout’s Virginia Tech pilot shows strong student enjoyment (87%) and perceived learning (90%) in 170 undergraduates—yet only 42% found automated feedback actionable enough to guide next steps. Sell a formative sandbox with human-verified LLM content and an actionable-feedback KPI, not a gamification vanity deployment. Bound claims to survey evidence and refuse guaranteed exam gains from WIP data.","engineering":"Engineering should ship isolated PostgreSQL sandboxes, multi-check result feedback strings, session gamification that resets cleanly, and an instructor-gated LLM pipeline for schemas, questions, and reference solutions. Instrument actionable-feedback ratings beside enjoyment and attempt counts. Constraints include WIP survey evidence and no learning RCT in the source. Proof-of-value is a rising share of students calling feedback actionable plus successful revisions after failed attempts—not enjoyment alone. Keep attempt logs purpose-limited and out of misconduct tooling.","delivery":"Delivery should run two in-class sessions, survey actionable feedback explicitly, and iterate hints/diffs before wider rollout. Ownership sits with the database course instructor and CS teaching support. Train TAs to verify LLM-generated schemas/keys. Acceptance means ≥target share finding feedback actionable and verified items only. Risks include mistaking enjoyment for learning and releasing unreviewed LLM keys."},"roleDetails":{"sales":{"customerProblem":"Undergraduate database courses lack iterative SQL practice with immediate, actionable feedback at scale.","stakeholders":["CS faculty","teaching & learning","academic technology","TAs/demonstrators","student success"],"discoveryQuestions":["What share of students call feedback actionable?","Who verifies LLM-generated schemas and keys?","Is the tool formative-only?"],"valueHypothesis":"Sandbox practice plus verified LLM content generation can raise engagement if actionable-feedback quality is an explicit success metric.","potentialEngagement":"One-course formative pilot with survey KPIs on actionable feedback and TA verification workflow.","unsupportedClaims":["Causal exam gains","Enjoyment equals learning","Ungated LLM keys are safe to release"]},"engineering":{"fit":"Formative SQL practice platforms for undergraduate database courses with optional LLM-assisted authoring.","architectureIntegration":"Sandbox DB per attempt → multi-dimensional feedback engine → gamification session layer → instructor LLM authoring with approval gate.","prerequisites":["course DB images","instructor review workflow","SSO/LMS optional"],"constraints":["WIP evidence base","feedback actionability gap","single-site deployment"],"security":"Attempt logs as education records; separate from grading; protect LLM prompts containing solutions.","proofOfValue":["Actionable-feedback agreement above target","Revision after failed attempts","100% LLM items human-approved"]},"delivery":{"work":"Deploy sandboxes, verify generated items, run in-class sessions, collect actionable-feedback survey, ship hint/diff improvements.","dependencies":["course instructor","TAs","IT hosting"],"ownership":"Database course instructor with CS teaching-support engineering.","skillsAdoption":"TAs verify LLM content; students practice iterate-on-feedback habits.","governanceCheckpoints":["formative-scope signoff","LLM content approval before release","post-session actionable-feedback review"],"acceptanceCriteria":["No unreviewed LLM keys live","Actionable-feedback metric reported","Formative-only scope held"],"risks":["Low actionable feedback","Overtrust in automated checks","Scope creep into grading"]}},"retrievedAt":"2026-10-03T03:19:26Z","enrichedAt":"2026-10-03T03:22:40Z","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Low-pressure gamified practice may help anxious learners enter practice, but weak actionable feedback leaves stuck students without clear next steps—especially those who already struggle with JOIN/GROUP BY.","procurementImplications":"Require sandbox isolation, multi-dimensional feedback with next-step guidance SLAs, instructor verification of LLM-generated content, and exportable attempt analytics.","operatingModelImplications":"Staff time shifts to verifying LLM-generated items and monitoring actionable-feedback ratings weekly, not only login or enjoyment scores.","sourceVerification":{"openedUrl":"https://arxiv.org/pdf/2610.01174","referenceExcerpt":"only 42% found automated feedback sufficient","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}