{"resourceId":"iu-syntea-usage-denominator-study-2026","versions":[{"version":"external-ac81a7103efea5a6d2ed8987f7e8d4be3ae8e096075c8f7da39cc5445cd15842","resource":{"id":"iu-syntea-usage-denominator-study-2026","title":"Large learning-assistant usage study highlights access confounding and inconsistent denominators","organization":"Kristina Schaaff, Quintus Stierstorfer and Valerie Hekkel; IU International University of Applied Sciences","sector":"Higher education teaching and student support","geography":"Germany; single distance-learning institution","publishedAt":"July 9, 2026","publicationDate":"2026-07-09","eventDate":null,"sourceName":"Using AI-based Learning Assistants in Higher Education: A Large-Scale Descriptive Analysis","sourceLabel":"Original arXiv preprint; authors employed by the deploying university","sourceUrl":"https://arxiv.org/html/2607.08748v1","evidenceClass":"academic-research","outcomeClass":"cautionary","topics":["knowledge-work","data-security","governance-procurement","accessibility-workforce","operating-model","infrastructure"],"finding":"Observed adoption differs across groups, but course availability can confound comparisons; use logs do not establish learning benefit.","sledRelevance":"Interpretation: Relevant to U.S. colleges planning participation measurement; German distance-study prevalence should not become a U.S. adoption forecast.","evidence":"February 2025 descriptive logs: abstract says 77,543 students, while methods and tables use 76,485 and report 44,035 users. The system was embedded in the learning platform and used GPT-4, GPT-4-Turbo and GPT-3.5-Turbo during observation.","architectureImplications":"Interpretation: Define eligible course exposure and stable event semantics before building adoption dashboards. Compare hosting options through local controls and service requirements; no infrastructure benchmark supports a choice here.","governanceImplications":"Interpretation: Separate availability, uptake and educational outcomes in reporting; do not infer student preference solely from a non-use flag.","securityPrivacyImplications":"Interpretation: Aggregate behavioral telemetry, suppress small cells and restrict linkage to demographic records; avoid sensitive student-level profiling for outreach without review.","caveats":"Single month, single operator-affiliated study, inconsistent sample reporting, small subgroups and uneven course coverage. No causal comparator, interaction-quality evaluation or direct learning outcome.","streamIds":["student-success"],"roles":{"sales":"Interpretation — Teaching leaders may have adoption reports that obscure whether students could use the licensed tool in their actual courses. Include institutional research, learning technology, disability services and student representatives. Ask how the eligible population is defined and whether low use reflects missing coverage, access friction or preference. Offer an audit of one program's participation funnel with a reconciled denominator and qualitative follow-up. The value hypothesis is better targeting of support and a more defensible renewal decision. The descriptive evidence cannot support promises of higher grades or retention. Avoid treating demographic differences as fixed learner traits or importing German prevalence into local forecasts.","engineering":"Interpretation — Start with an event dictionary and a versioned course-availability table. Prerequisites include stable identity joins, permitted aggregate demographic analysis and documented exclusions. Test whether duplicate enrollment, inactive accounts and content migrations change the denominator. Separate unique users, activity frequency and service failures in instrumentation. Proposed proof of value: reconcile the dashboard against a manually reviewed sample, explain every exclusion and publish missingness before subgroup comparisons. Add independent assessment only through an approved evaluation design. Infrastructure sizing needs actual local concurrency and latency data; activity proportions from another institution cannot determine GPU requirements or cloud spending.","delivery":"Interpretation — Have institutional research approve definitions and a platform administrator own instrumentation changes. Run a short reconciliation cycle with faculty confirming course availability and support staff reviewing reported barriers. Dependencies include data access approval, accessible feedback channels and analysts able to distinguish association from causation. Proposed acceptance criteria: all reported denominators reconcile, each subgroup estimate has an exposure definition and small-cell rule, and unexplained discrepancies have owners before distribution. These are proposed quality gates. Train readers to interpret non-use cautiously. Risks include stigmatizing groups, optimizing logins instead of learning, and silently changing metrics during platform migration."},"retrievedAt":"2026-09-13T03:00:47Z","enrichedAt":"2026-09-13T03:01:49Z","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Interpretation: Investigate access barriers through voluntary feedback and assistive-technology testing, rather than assuming an age or gender category explains non-use.","procurementImplications":"Interpretation: Require clear active-user definitions, course eligibility exports and model-version histories before paying or renewing on adoption metrics.","operatingModelImplications":"Interpretation: Institutional research should own denominator definitions jointly with learning-platform administrators and academic service owners.","updateExplanation":"New in the complete 247-resource archive and targeted identifier search. Explicit July preprint backfill adds objective usage measurement and a denominator warning; no September release is asserted.","sourceVerification":{"openedUrl":"https://arxiv.org/html/2607.08748v1","referenceExcerpt":"First, our study is descriptive and does not allow causal conclusions.","promptVersion":"sled-research-v3.1","model":null,"basis":"agent-reported inspection"}}}]}