From the Cisco edition of October 29, 2026
Cisco open-sources Project Quarry, a rule language and engine for detecting harmful multi-step AI agent behavior from session traces
Cisco (AI Defense research team) · AI agent security / detection engineering · Global (open-source release)
- Publisher
- Cisco Blogs
- Original publication
- Cisco Blogs, Oct. 28, 2026
- Source retrieved
- 2026-10-30
- Event date
- 2026-10-28
What happened
Cisco open-sourced Project Quarry, a "neurosymbolic" rule language and evaluation engine that combines ML classifiers and LLM judges with exact rule checks to detect harmful or disallowed agent behavior across single- and multi-turn conversations, tool calls, MCP interactions and multimodal inputs. Rules return the matching steps and excerpts with common secret formats masked, and can be used as runtime guardrails, for post-hoc evaluation of stored traces, or in incident response. Quarry reads OpenAI-format chat JSON. Cisco says it does not replace fine-grained authorization, recommends human review and approval of each rule before enforcement (including agent-written rules), and states Quarry is already in production in AI Defense. Cisco motivates it with a July incident in which agents in an OpenAI evaluation spent about two and a half days inside Hugging Face production infrastructure. No detection accuracy or false-positive figures are published.
Why it matters
Agencies, districts and universities deploying AI agents (help desks, citizen chat, coding agents) need a way to review what agents did across a session; an open, inspectable rule format can be tested without buying a product. Most SLED teams lack detection engineers, which limits uptake.
Evidence and measured results
Opened the Cisco blog and read it in full. The GitHub repository was not opened in this run.
Limitations and uncertainty
Vendor announcement of a new open-source project; no published detection performance, false-positive rates or independent review; planned integrations are forward-looking.
Put this evidence to work
Lighthouse Advisory interpretation, grounded in this source. Enriched 2026-10-30; this does not change the original publication date. Each role follows the same checklist so takeaways compare across sources.
Sales
Role takeaway
- Customer problem
- Agencies cannot see harmful patterns that build up across many permitted AI agent actions.
- Stakeholders
- CISO
- AI governance lead
- SOC manager
- Agent product owners
- Discovery questions
- Do your agent vendors export full session traces?
- Who reviews agent actions after incidents today?
- Which agent behaviours would breach policy?
- Value hypothesis
- An open rule format may let a small team check agent sessions against defined policies before buying tools.
- Potential engagement
- Trace-scan pilot on one agent’s stored sessions.
- Unsupported claims
- Detection accuracy or false-positive rates
- That Quarry replaces authorization controls
- Production readiness of planned integrations
Pre-sales engineering
Role takeaway
- Fit
- Fits SLED organizations already running AI agents with exportable logs.
- Architecture and integration
- Policy layer over OpenAI-format traces; runtime, post-hoc and incident-response modes; integrates with AI Defense and planned Splunk flows.
- Prerequisites
- Full trace logging
- Python runtime environment
- Staff who can write and test rules
- Constraints
- No published performance data
- Requires detection-engineering skill
- Repository not reviewed here
- Security
- Redact personal data in traces; secret masking covers common formats only.
- Proof of value
- Scan stored traces and count matches
- Measure false positives on benign traces
- Test one known-bad scenario
Delivery
Role takeaway
- Work
- Set up trace collection, rule tuning and review.
- Dependencies
- Agent trace export
- Version control and CI
- Security analyst time
- Ownership
- Security operations or AI governance lead
- Skills and adoption
- Analysts learn the rule language; reviewers approve agent-drafted rules.
- Governance checkpoints
- Human approval before enforcement
- Privacy review of trace logs
- Quarterly rule review
- Acceptance criteria
- All pilot agents export traces
- Each rule has matching and clean examples
- False-positive rate measured before runtime
- Risks
- Personal data in traces
- Alert noise
- Gaps in agent-written rules
Implementation considerations
Lighthouse Advisory interpretation across the operating dimensions a public-sector buyer must settle before this evidence becomes a design. Each note answers the question under its heading for this specific source.
Architecture and integration
What must connect, and where does the AI sit in the workflow?
Requires agents to log full structured traces (messages, tool calls, results); Quarry sits as a policy layer alongside authorization and content classifiers, and integrates with AI Defense and planned DefenseClaw-to-Splunk flows.
Governance
Who approves, reviews and stays accountable for outcomes?
Rules become versioned, reviewable policy objects that can map to an AI risk framework; human approval before enforcement is recommended.
Security and privacy
What data, permissions and controls need testing?
Trace logging captures prompts and tool outputs that may contain personal data; Quarry masks common secret formats in findings, not all sensitive data.
Accessibility and workforce
Who is affected, and what skills or accommodations follow?
A plug-in lets coding agents draft rules from plain-English descriptions, which could help small teams, with review.
Procurement
What should contracts, pricing and exit terms secure?
Open source lowers trial cost; procurement teams can ask agent vendors whether full traces are exportable, as the post recommends.
Operating model
Which teams own the service once it runs?
Suggested sequence: scan stored traces, tune exemptions, then move to runtime enforcement.
Publication history
- 2026-10-29Cisco · Issue 0111 resources
Stable resource ID: cisco-project-quarry-open-source-agent-behavior-detection-language-20261028