{"resourceId":"nc-treasurer-chatgpt-pilot-survey-2025","versions":[{"version":"external-6046a5e99ec2cab53c5d7f84f5399d62fc0dd769b7e5caa9234fd7f064ade2da","resource":{"id":"nc-treasurer-chatgpt-pilot-survey-2025","title":"Treasurer pilot reports perceived savings while documenting incomplete comparisons and specialist-task errors","organization":"North Carolina Department of State Treasurer; analysis supported by North Carolina Central University","sector":"State government","geography":"North Carolina, United States","publishedAt":"July 2025; exact day unknown","publicationDate":null,"eventDate":null,"sourceName":"NC Treasurer / NCCU pilot report","sourceLabel":"Government pilot survey and interviews","sourceUrl":"https://www.nctreasurer.gov/nccunc-treasurer-ai-survey-report/open","evidenceClass":"government-evaluation","outcomeClass":"mixed","topics":["knowledge-work","developers-agents","data-security","accessibility-workforce","operating-model"],"finding":"Participants reported useful drafting and research assistance, with accuracy, completeness and specialist-task limitations.","sledRelevance":"Historical state workforce evidence newly added to qualify current rollout decisions; it does not demonstrate department-wide returns.","evidence":"The March–June 2025 timeline identifies 36 employees providing direct feedback. Participants estimated final savings of 30–60+ minutes per day. Recruitment included manual selection and interested volunteers; surveys and selected interviews, not a controlled time-and-quality comparison, underpin the findings.","architectureImplications":"Interpretation: preserve review and comparison checks around drafting and document analysis; separately validate coding and quantitative tasks. The report identifies ChatGPT Enterprise but supplies no reproducible infrastructure sizing or autonomous-agent evaluation.","governanceImplications":"Interpretation: distinguish perceived benefit from measured net effort and verified output completeness.","securityPrivacyImplications":"Participants remained uncertain about permitted inputs. Interpretation: give staff workflow-specific data rules and test that prohibited data cannot enter unsupported tools.","caveats":"Nonrepresentative cohort and incomplete survey participation. No causal baseline or measured net savings after verification. Exact report day unknown. Historical findings do not establish current model behavior.","streamIds":["state-government"],"roles":{"sales":"Interpretation: Discuss document-heavy work with division leaders, finance, IT and frontline staff. Ask where review time goes, how completeness is assessed, and whether employees can safely supply the needed documents. A bounded engagement could compare one recurring drafting or audit-comparison task with its existing process. The value hypothesis is less total effort at an acceptable quality level, including verification. Reported savings justify testing but do not substantiate guaranteed productivity or staffing reductions. Qualify training capacity and data access before proposing licenses. Treat specialist coding, legal and mathematical work as separate evaluation scopes rather than extending favorable drafting feedback to every division.","engineering":"Interpretation: Fit is assisted knowledge work under established human approval. Use approved enterprise access, explicit document boundaries and auditable source comparisons. Prerequisites include representative source pairs, expert reference answers and input-classification rules. The report's incomplete-comparison examples make omission detection especially relevant: check missing differences, not just whether identified differences are correct. Validate code in existing review and test pipelines and route legal conclusions to qualified staff. Proposed proof of value should measure task time plus review time, omissions and correction effort against unaided work. Deployment architecture and model versions require local verification; the study does not establish a secure agent or production integration pattern.","delivery":"Interpretation: Select a small workflow cohort, document the prior process and provide short guided exercises with approved inputs. The division manager owns quality and continuation decisions; IT operates access; a training lead supports hesitant users. Dependencies include protected learning time and expert review capacity. Record why staff abandon the tool, not just successful examples. Proposed acceptance criteria: compare total task effort and completeness against baseline, account for correction work, and resolve all identified prohibited-input paths before expansion. Run a governance checkpoint after onboarding and after material model changes. Risks include volunteer selection, self-report inflation and shifting effort from drafting into difficult verification."},"retrievedAt":"2026-09-10T03:02:13Z","enrichedAt":"2026-09-10T03:02:13Z","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Interpretation: reserve learning time and test role-specific onboarding with less confident users.","procurementImplications":"Interpretation: compare license and support costs with locally observed, quality-adjusted effort before renewal.","operatingModelImplications":"Interpretation: managers own output acceptance; IT owns approved access; workforce leads own training and adoption support.","updateExplanation":"No matching URL or NC Treasurer finding in full archive or targeted searches. July 2025 report newly inspected to qualify the department's April 2026 expansion announcement; no new September measurement asserted.","sourceVerification":{"openedUrl":"https://www.nctreasurer.gov/nccunc-treasurer-ai-survey-report/open","referenceExcerpt":"Pilot participants were not a representative sample of the state treasurer’s employees.","promptVersion":"sled-research-v3.1","model":null,"basis":"agent-reported inspection"}}}]}