PlanAI pilot separates rapid summarisation from weeks of preparation and quality assurance

Engage the consultation lead, information governance team and community-engagement staff around whether analysis capacity is delaying plan development. Ask how submissions arrive, how minority views are represented and what a corrected thematic report costs to produce. Offer a bounded comparison on a completed consultation, with the existing manual analysis retained as a reference. A credible hypothesis is less repetitive grouping while preserving accurate representation; it is not a promise of a 90% shorter consultation. Establish whether the buyer can fund preparation and review. Clarify that an assistant's thematic output does not replace the municipality's policy judgement or engagement obligations.

Build a test corpus spanning short, long, multilingual and overlapping-policy responses. Keep original identifiers through redaction, segmentation, tagging and summary generation. Prerequisites include an agreed policy taxonomy, lawful data access and reviewer-labelled examples. Evaluate omitted concerns, incorrect policy assignments and distortion of response volume, not just processing speed. Include adversarial instructions embedded in comments as untrusted data. Proposed validation should compare error rates and total analyst effort with the manual baseline, then repeat after configuration changes. Hosting and agent capabilities were not evaluated here; avoid autonomous updates to official consultation conclusions until the authority has validated a controlled approval boundary.

The consultation service manager should own completeness and fairness, supported by records staff, analysts and privacy reviewers. First establish a manual reference sample and an accessible alternative route; then rehearse redaction, correction and publication. Dependencies include policy clarity and sufficient review capacity. Train staff to inspect underlying responses when a summary is ambiguous. Proposed acceptance requires every sampled theme to be traceable, all high-impact omissions to be resolved, and the manual route to receive equivalent review. Track elapsed time from receipt through approved analysis. These are proposed checks. Risks include undercounting preparation, amplifying frequent views and unintentionally excluding residents who avoid AI.