Partnership for Public Service (Oct 8): Vermont AOT reports ChatVT at ~500 weekly users, plans graded test-prompt evaluation for a 75-document transportation knowledge repository

Vermont Agency of Transportation (Rose Vath, AI & BI and performance manager); Vermont Agency of Digital Services (ChatVT) · State transportation agency knowledge work and internal generative AI assistant · Vermont statewide

State Government

Publisher: Partnership for Public Service AI Center for Government
Original publication: 2026-10-08
Source retrieved: 2026-10-10

Read original source

What happened

In an Oct 8 interview, Vermont AOT's Rose Vath says ChatVT, the state's in-house employee assistant, has about 500 weekly users and was designed for cost minimization as a 'safety hatch' if token costs of public tools like ChatGPT, Claude and Gemini become burdensome; Vermont policy allows public AI tools with public data. AOT is only recently adopting Microsoft AI Hub (Power Automate) and Copilot Studio, deliberately preferring single AI steps inside deterministic workflows over agentic solutions. AOT is developing a knowledge repository over about 75 procedures, manuals and handbooks, with a planned evaluation using a test-prompt library graded by test users for completeness and accuracy. Vath runs a monthly 'On Ramp for AI at AOT' workshop.

Why it matters

Fresh adoption data point for a state-built assistant (up from nearly 200 users at end of 2024 per ADS's annual report) and an agency-level evaluation plan for a RAG-style knowledge tool, complementing Vermont ADS's Anthropic oversight story with ordinary knowledge-work use.

Evidence and measured results

Opened the Partnership for Public Service AI Center for Government blog post (article:published_time 2026-10-08) via curl and extracted full text.

Limitations and uncertainty

Interview-based association blog; usage figure is self-reported, with no denominator, cost data or measured savings. Knowledge repository is in development and its evaluation not yet run. Celebratory 'Bright Spot' framing.

Put this evidence to work

Lighthouse Advisory interpretation, grounded in this source. Enriched 2026-10-09; this does not change the original publication date. Each role follows the same checklist so takeaways compare across sources.

Sales

Role takeaway

Customer problem: Agency procedures are scattered across ~75 documents and commercial AI token costs may grow.
Stakeholders: Agency BI/AI manager
State chief data and AI officer
Agency project managers
ADS ChatVT team
Discovery questions: What does ChatVT cost per weekly user versus commercial licenses?
Who owns accuracy of the 75 source documents?
What accuracy threshold gates repository launch?
Which workflows may include an AI step?
Value hypothesis: A graded evaluation can show whether a retrieval tool reliably answers procedure questions for new staff.
Potential engagement: Short evaluation-harness build for one agency knowledge repository.
Unsupported claims: Measured cost savings from ChatVT
Accuracy of the knowledge repository
Statewide productivity gains

Pre-sales engineering

Role takeaway

Fit: Fits retrieval over a bounded agency document set inside existing Microsoft and in-house tooling.
Architecture and integration: Index ~75 AOT documents for ChatVT or Copilot Studio retrieval; embed AI as one step in deterministic Power Automate flows.
Prerequisites: Document inventory and owners
Test-prompt library
Approved platform choice
Constraints: Public tools only with public data
Agentic designs disfavored
Vermont policy specific
Security: Keep non-public procedures within state-controlled ChatVT or enterprise tenant; log queries.
Proof of value: Graded accuracy and completeness on a test-prompt set
Citation to source document per answer
Comparison to manual lookup time

Delivery

Role takeaway

Work: Build, evaluate and launch an AOT procedure knowledge repository with training.
Dependencies: Document owners across AOT teams
ADS platform support
Test-user availability
Ownership: AOT business intelligence unit with ADS support.
Skills and adoption: Monthly On Ramp for AI workshop plus scenario walkthroughs.
Governance checkpoints: Evaluation plan approved before build
Threshold review before launch
Periodic document freshness review
Acceptance criteria: Proposed: >=90% of graded test prompts rated complete and accurate
Proposed: every answer cites a source document
Proposed: document owners named for all 75 sources
Risks: Stale source documents produce confident wrong answers
Usage metric mistaken for value
Cost hedge unproven

Implementation considerations

Lighthouse Advisory interpretation across the operating dimensions a public-sector buyer must settle before this evidence becomes a design. Each note answers the question under its heading for this specific source.

Architecture and integration

What must connect, and where does the AI sit in the workflow?

Mixed estate: in-house ChatVT, Microsoft Enterprise AI Hub/Copilot Studio, and policy-permitted public tools on public data; knowledge repository implies retrieval over ~75 agency documents.

Governance

Who approves, reviews and stays accountable for outcomes?

Risk posture favors bounded AI steps within deterministic workflows over agentic automation; evaluation design is still a plan, not results.

Security and privacy

What data, permissions and controls need testing?

Public-tool use is restricted to public data; the article gives no detail on ChatVT data controls or logging.

Accessibility and workforce

Who is affected, and what skills or accommodations follow?

Monthly agency workshop and accountability framing ('if something goes wrong, they will have support') position training as the adoption lever.

Procurement

What should contracts, pricing and exit terms secure?

ChatVT framed as a hedge against per-token/license costs of commercial tools; no cost comparison published.

Operating model

Which teams own the service once it runs?

Agency BI unit selects tools and use cases under central ADS AI leadership.

Publication history

2026-10-09State Government · Issue 342 resources

Read preserved resource versions (JSON)

Stable resource ID: vermont-aot-chatvt-500-weekly-users-knowledge-repo-eval-aicg-20261008