Large learning-assistant usage study highlights access confounding and inconsistent denominators

Teaching leaders may have adoption reports that obscure whether students could use the licensed tool in their actual courses. Include institutional research, learning technology, disability services and student representatives. Ask how the eligible population is defined and whether low use reflects missing coverage, access friction or preference. Offer an audit of one program's participation funnel with a reconciled denominator and qualitative follow-up. The value hypothesis is better targeting of support and a more defensible renewal decision. The descriptive evidence cannot support promises of higher grades or retention. Avoid treating demographic differences as fixed learner traits or importing German prevalence into local forecasts.

Start with an event dictionary and a versioned course-availability table. Prerequisites include stable identity joins, permitted aggregate demographic analysis and documented exclusions. Test whether duplicate enrollment, inactive accounts and content migrations change the denominator. Separate unique users, activity frequency and service failures in instrumentation.

Proposed proof of value

reconcile the dashboard against a manually reviewed sample, explain every exclusion and publish missingness before subgroup comparisons. Add independent assessment only through an approved evaluation design. Infrastructure sizing needs actual local concurrency and latency data; activity proportions from another institution cannot determine GPU requirements or cloud spending.

Have institutional research approve definitions and a platform administrator own instrumentation changes. Run a short reconciliation cycle with faculty confirming course availability and support staff reviewing reported barriers. Dependencies include data access approval, accessible feedback channels and analysts able to distinguish association from causation.

Proposed acceptance criteria

all reported denominators reconcile, each subgroup estimate has an exposure definition and small-cell rule, and unexplained discrepancies have owners before distribution. These are proposed quality gates. Train readers to interpret non-use cautiously. Risks include stigmatizing groups, optimizing logins instead of learning, and silently changing metrics during platform migration.