From the Student Success edition of October 3, 2026
UIC educator-guided ERD agent: staged disclosure yields 71.1% hidden-target incorporation among observed revisions
University of Illinois Chicago (Riazi, Rooshenas) · Higher education computing teaching / formative GenAI feedback · United States (University of Illinois Chicago; online mixed-level Database Systems)
- Publisher
- arXiv
- Original publication
- Submitted to arXiv 1 October 2026 (cs.AI); SIGCSE TS 2027 submission
- Source retrieved
- 2026-10-04
What happened
Educator-guided LLM agent for conceptual ERD feedback separates hidden artifact-grounded diagnosis from a four-stage learner-controlled disclosure workflow. Summer 2026 deployment: 383 feedback episodes; among error-positive episodes with an observed target-level ERD change, 71.1% fully or partially incorporated the hidden diagnostic target, including many after Stages 1–2 before explicit correction. Qualitative cases show staged disclosure can withhold inaccurate details or enable selective uptake, though incorrect targets can still be incorporated. Survey (self-selected ≤30 users): 64.3% felt they could make their own design decisions; 39.3% found feedback too indirect; 42.9% reported repetition.
Why it matters
Direct design answer to yesterday's DBWorkout actionable-feedback gap and Engage-to-Unlock productive friction: educator-set disclosure stages—not unrestricted LLM tutoring—can drive diagram revisions while preserving student authorship. Relevant for HE CS teaching, writing/STEM formative tools, and GenAI literacy policies.
Evidence and measured results
Classroom deployment logs + LLM incorporation classifier checked on 20-trajectory sample; optional survey. Not RCT; incorporation ≠ diagnosis correctness; preprint.
Limitations and uncertainty
Single course; no causal learning RCT; missing later ERDs for many episodes; self-selected survey; GPT-4o diagnosis error possible; preprint.
Put this evidence to work
Lighthouse Advisory interpretation, grounded in this source. Enriched 2026-10-04; this does not change the original publication date. Each role follows the same checklist so takeaways compare across sources.
Sales
Role takeaway
- Customer problem
- Large HE design courses lack timely formative feedback without over-revealing solutions via unrestricted GenAI tutors.
- Stakeholders
- CS faculty
- teaching & learning
- academic technology
- TAs
- student success
- Discovery questions
- Can faculty set disclosure stages for GenAI feedback?
- Do we measure revision after early-stage scaffolds?
- Is AI formative-only while humans grade?
- Value hypothesis
- Educator-controlled staged disclosure can increase target-aligned revisions while preserving student design agency, without guaranteeing grade lifts.
- Potential engagement
- One-course formative pilot with stage telemetry and revision endpoints on a design assignment.
- Unsupported claims
- Guaranteed exam gains
- Diagnosis always correct
- Campus-wide RCT equivalence
Pre-sales engineering
Role takeaway
- Fit
- Artifact-centered formative GenAI feedback for undergraduate conceptual design (ERD/database) courses.
- Architecture and integration
- Structured artifact parser → hidden diagnosis → educator disclosure controller → stage generators → episode memory linked to versioned artifacts.
- Prerequisites
- educator rubric/requirement authoring
- artifact versioning
- LLM with logged prompts
- Constraints
- single-course evidence
- diagnosis error risk
- no learning RCT
- Security
- Artifact and diagnosis logs as education records; minimize retention of hidden diagnoses; access-control TAs/instructors.
- Proof of value
- Target-aligned revision rate after early stages
- Low premature full-solution disclosure
- Exportable stage telemetry
Delivery
Role takeaway
- Work
- Author stage pools, configure credits, run homework pilot, review episode logs, survey indirectness/repetition, iterate transitions.
- Dependencies
- course instructor
- TAs for independent grading
- LMS/editor integration
- Ownership
- Course instructor with academic-technology operations support.
- Skills and adoption
- Faculty define when corrections may appear; students practice concept/application before requesting explicit fixes.
- Governance checkpoints
- formative-scope signoff
- disclosure-rule review
- mid-pilot indirectness check
- Acceptance criteria
- Disclosure rules documented
- Episode logs exportable
- Early stages cannot dump full solutions
- Risks
- Too-indirect UX
- Incorrect diagnosis incorporated
- Overclaiming learning from incorporation
Implementation considerations
Lighthouse Advisory interpretation across the operating dimensions a public-sector buyer must settle before this evidence becomes a design. Each note answers the question under its heading for this specific source.
Architecture and integration
What must connect, and where does the AI sit in the workflow?
Separate diagnosis service from disclosure controller; version ERD states per episode; educator-authored question pools/templates for early stages; credit/stage telemetry.
Governance
Who approves, reviews and stays accountable for outcomes?
Require educator-defined disclosure policies before campus LLM tutors may reveal corrections; log stage, context, and disclosed content for audit; keep AI formative-only when TAs grade independently.
Security and privacy
What data, permissions and controls need testing?
ERD artifacts, diagnoses, and chat are education records; purpose-limit logs; de-identify optional surveys; control who can view hidden diagnoses.
Accessibility and workforce
Who is affected, and what skills or accommodations follow?
Staged, learner-controlled disclosure may help students who need thinking time before correction, but indirectness and repetition can burden learners who need clearer next steps—test with accessibility services before scale.
Procurement
What should contracts, pricing and exit terms secure?
Require vendors to expose educator-configurable disclosure stages, exportable episode/telemetry, and formative-only modes—not only chatbot seats.
Operating model
Which teams own the service once it runs?
Operate GenAI formative tools with weekly metrics on stage-exit distribution, incorporation/revision rates, and student-reported indirectness—not chat volume alone.
Publication history
- 2026-10-03Student Success · Issue 273 resources
Stable resource ID: riazi-educator-guided-llm-erd-scaffolded-feedback-20261001