New York audit documents the gap between AI policy and verifiable operating procedures

Qualify the assurance workload with central IT, agency program owners, procurement and internal audit. Ask whether a reviewer can trace policy requirements to executed tests, who settles classification disputes, and which supplier terms constrain evidence access. A bounded engagement could reconcile one agency's inventory and inspect a sample of approval and monitoring records. The credible value hypothesis is clearer accountability and less effort assembling defensible evidence. Do not claim that this older audit describes the customer's present posture or proves actual discrimination. Establish current remediation status independently and include the agency's explanation when documenting any disputed control gap.

Fit is evidence integration across inventory, configuration, contracts and test records. Prerequisites include named system owners, an agreed AI definition and access to supplier documentation. Build a trace from each material risk to its test, result and approval, including exceptions. Validate output accuracy and subgroup behavior using use-case-specific criteria; generic security reviews cannot establish these properties. Test whether a model or vendor change triggers renewed review. A proposed proof of value should reconstruct one complete approval history and repeat the relevant evaluation. Deployment may span cloud, on-premises and hybrid environments; the audit supports a governance approach, not selection of a hosting model.

Reconcile existing systems, document required procedures and assign recurring review to operational owners. Central IT supplies common definitions and templates, while agency leaders accept residual risks. Dependencies include procurement cooperation, specialist evaluation skills and access to legacy records. Train staff on how to identify AI features and report uncertain classifications.

Proposed acceptance criteria

every sampled system has a named owner, documented data terms, retained test evidence and an approved disposition for exceptions. Exercise a supplier change and verify that review occurs. Risks include stale inventories and treating a policy document as operational proof; report disputed findings and remediation evidence separately.