{"resourceId":"arxiv-decision-models-policy-gates-coding-agents-explicit-boundaries-20261101","versions":[{"version":"external-9d36e9d405839fc5e03656a045153595dd5de1c868cfe2679e92b65138bdccd4","resource":{"id":"arxiv-decision-models-policy-gates-coding-agents-explicit-boundaries-20261101","title":"Preprint: explicit policy boundaries with examples and counterexamples raise recall and cut false alarms when decision models gate coding-agent actions","organization":"James P. Beno (independent author), arXiv","sector":"AI agent runtime security and policy enforcement research","geography":"Not location-specific (synthetic dataset)","publishedAt":"arXiv preprint 2611.02155v1, submitted Nov. 1, 2026","publicationDate":"2026-11-01","eventDate":null,"sourceName":"arXiv","sourceLabel":"Decision Models as Policy Gates for Coding Agents: Explicit Policy Boundaries Improve Detection and Reduce False Alarms (Preprint, not peer reviewed)","sourceUrl":"https://arxiv.org/abs/2611.02155","evidenceClass":"independent-research","outcomeClass":"mixed","topics":["developers-agents","data-security","governance-procurement"],"finding":"Preprint, not peer reviewed. The author evaluated 22 hosted decision models and two prompted general models as classifiers for a 28-rule policy engine gating coding-agent tool calls, using a synthetic dataset of 2,643 tool calls labeled blind with a sealed holdout. Writing explicit boundaries as examples and counterexamples raised recall by 5.5 to 26.1 percentage points on all 23 available models and reduced false positives on 21. For the best-reported model, recall rose from 89.3% to 98.2% and unnecessary interruptions fell from 30.6% to 4.8% of 252 clean actions, at 120 ms median latency. Gains largely persisted on 16 scenarios written later. Limitations stated by the author: all cases are synthetic, only 28 of 67 rule templates were covered, the dataset skews small, and calibration and adversarial robustness were not established. It does not test Palo Alto Networks products.","sledRelevance":"Independent evidence relevant to Palo Alto's push for AI-generated and AI-enforced policy (Nov. 3 DLP post) and to the AI-agent controls in Prisma AIRS and Cortex. Interpretation: for agencies and universities piloting coding agents, the finding suggests policy wording matters as much as the model: rules written with concrete allowed and disallowed examples were caught more reliably and interrupted users less. Transferability caveat: synthetic, developer-oriented dataset from one author; not a public-sector sample.","evidence":"Opened the arXiv abstract page (v1, submitted Nov. 1, 2026, DOI pending registration) and read the HTML full text including method, results and limitations. Author releases dataset, rule texts and harness.","architectureImplications":"A runtime decision model sits between the agent and its tools, evaluating each proposed call against rules; latency and per-request cost were measured and appear compatible with inline use.","governanceImplications":"Interpretation: policy documents for AI agents should include concrete examples and counterexamples, and be versioned and tested like code against a holdout set.","securityPrivacyImplications":"Explicit boundaries improved detection of data destruction, secret leakage and mis-sent messages in synthetic cases; adversarial robustness and calibration were not tested.","caveats":"Preprint, not peer reviewed; single author who, per the paper, develops the policy engine and rules evaluated; synthetic data; partial rule coverage; no adversarial testing; no public-sector sample; Palo Alto products not tested.","streamIds":["palo-alto-networks"],"roles":{"sales":"Sales framing: this preprint gives a neutral way to discuss AI-agent guardrails with public-sector buyers without vendor hype. Agencies piloting coding agents worry about data deletion and secret leaks. The evidence suggests that how policies are written, with examples and counterexamples, changes detection and false alarms materially. A bounded engagement could be a policy-writing workshop for agent guardrails. It does not test Palo Alto products, so it should not be used to claim Prisma AIRS or Cortex performance.","engineering":"Engineering framing: the paper offers a test method engineers can reuse. Build a labeled set of the customer's own agent actions, including harmless look-alikes, seal a holdout, and measure recall, false positives and interruption rates for each rule. Compare prose rules with rules that carry examples. Results here come from synthetic developer tool calls, so validate on real agency workflows, and add adversarial cases because the author did not test robustness or calibration.","delivery":"Delivery framing: treat agent policy as a tested artifact. A delivery plan would collect representative agent actions, write rules with explicit allowed and blocked examples, measure them on a holdout, and review results with security and developer leads before enforcement. Proposed acceptance includes per-rule recall and interruption targets agreed in advance. Risks include overfitting rules to the test set, synthetic data that misses real workflows, and drift as agents and tools change."},"roleDetails":{"sales":{"customerProblem":"Agencies piloting coding agents need guardrails that catch harmful actions without interrupting developers constantly.","stakeholders":["CISOs","Application development leads","Enterprise architects","AI governance committees"],"discoveryQuestions":["How are AI-agent guardrail rules written today: prose or examples?","Are guardrail false positives causing users to bypass controls?","Is there a labeled test set of agent actions?"],"valueHypothesis":"Interpretation: rewriting agent policies with examples may improve detection and reduce interruptions, based on synthetic evidence.","potentialEngagement":"A guardrail policy-writing and testing workshop using the customer's agent actions.","unsupportedClaims":["Performance of any Palo Alto product","Robustness against adversarial attacks","Results on real public-sector workloads","Peer-reviewed confirmation"]},"engineering":{"fit":"Relevant to any SLED entity running coding or tool-using agents behind a policy gate.","architectureIntegration":"A runtime decision model inline between agent and tools, evaluating each call against rules.","prerequisites":["Representative agent tool-call samples","Written rule set","Sealed holdout set"],"constraints":["Synthetic dataset only","28 of 67 rule templates covered","Adversarial robustness and calibration untested"],"security":"Explicit boundaries improved detection of destructive and leaking actions in tests; injected content was included but robustness not established.","proofOfValue":["Measure per-rule recall and false positives on a customer holdout","Compare prose versus example-based rules","Add adversarial injection cases"]},"delivery":{"work":"Build a labeled action set, rewrite rules with examples, test on a holdout and stage enforcement.","dependencies":["Agent platform logs","Developer participation in labeling","Security review"],"ownership":"The application security lead owns rules; the AI governance committee approves enforcement.","skillsAdoption":"Teams need skills in writing testable policy examples and reading precision and recall metrics.","governanceCheckpoints":["Holdout sealed before tuning","Results reviewed before enforcement","Rules re-tested after agent or tool changes"],"acceptanceCriteria":["Proposed: recall target per high-risk rule met on holdout","Proposed: unnecessary interruptions below an agreed share of clean actions","Proposed: adversarial cases added to test suite"],"risks":["Overfitting to the test set","Synthetic data missing real workflows","Rule drift as tools change"]}},"retrievedAt":"2026-11-04T04:06:18Z","enrichedAt":"2026-11-04T04:09:09Z","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Fewer unnecessary interruptions lowers approval fatigue for developers and reviewers, a common cause of control bypass.","procurementImplications":"Interpretation: buyers evaluating AI-agent guardrail products can ask vendors for recall and false-positive results per rule on a holdout, which this paper models.","operatingModelImplications":"Suggests a practice of writing, testing and refining agent policies with labeled examples rather than prose-only rules.","sourceVerification":{"openedUrl":"https://arxiv.org/abs/2611.02155","referenceExcerpt":"explicit boundaries written as examples and counterexamples raised recall by 5.5-26.1 percentage points on all 23 available models and reduced false positives on 21.","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}