{"resourceId":"register-cisco-quarry-limits-not-substitute-for-oversight-20261028","versions":[{"version":"external-dd3e2f57743a7c5293dc44ad0ba1713212c517dac7355db6a4353eadc1c23a90","resource":{"id":"register-cisco-quarry-limits-not-substitute-for-oversight-20261028","title":"The Register: Cisco says Project Quarry rules are not a substitute for human oversight and were not run against the OpenAI–Hugging Face incident data","organization":"The Register (interviewing Cisco AI threat research)","sector":"AI agent security relevant to SLED agencies and universities deploying agents","geography":"Global","publishedAt":"October 28, 2026","publicationDate":"2026-10-28","eventDate":null,"sourceName":"The Register","sourceLabel":"Independent technology reporting","sourceUrl":"https://www.theregister.com/security/2026/10/28/cisco-open-sources-tool-for-detecting-ai-agents-on-hacking-sprees/5305867","evidenceClass":"independent-reporting","outcomeClass":"emerging","topics":["data-security","developers-agents","governance-procurement"],"finding":"Reporting on Cisco’s open-source Project Quarry (already archived from Cisco’s blog), The Register adds stated limits from Cisco’s researchers: they have not run Quarry against data describing the OpenAI–Hugging Face intrusion and only suggest where rules would likely have flagged steps; rules are not a substitute for chain-of-thought monitors or other human oversight; and Cisco’s head of AI threat research says, “Do we have all the answers? No.”","sledRelevance":"Agencies and universities piloting agents (including through Cisco MCP servers) should treat Quarry as an early, community detection layer, not a validated control. The added limitations are what is new versus the archived Cisco announcement.","evidence":"Interview-based reporting with a vendor spokesperson; hypothetical walk-through of an incident; no test results, false-positive rates or deployments.","architectureImplications":"Quarry evaluates full agent sessions; organizations need complete, retained agent traces (tool calls, results, MCP interactions) for it to work.","governanceImplications":"Policies should require human oversight alongside rule-based detection; open rule sharing needs review before adoption.","securityPrivacyImplications":"Agent traces collected for detection can contain credentials and personal data and must be protected.","caveats":"The Register relays Cisco’s statements; it did not test Quarry. Hypothetical incident mapping is not evidence of detection.","streamIds":["cisco"],"roles":{"sales":"Use this to set expectations honestly: Cisco’s own researchers say Quarry is not a substitute for human oversight and was not tested against the incident it is often illustrated with. For SLED customers asking how to monitor agents, offer Quarry as a no-cost experiment within a broader oversight plan. Avoid any claim that it would have stopped a known attack. The value is a shared rule language, which is early and unproven in production.","engineering":"Quarry needs complete agent session traces, including tool calls and MCP interactions, so first confirm the customer can capture and retain them. Run the starter rule pack against benign and synthetic malicious traces to measure false positives before any runtime blocking. The interview makes clear there is no incident-based validation, so proof-of-value must be built locally. Treat collected traces as sensitive data under the agency’s retention and privacy rules.","delivery":"Deliver a time-boxed evaluation: trace capture for one agent workflow, starter rules applied in alert-only mode, and a findings review with the SOC. The SOC owns rule tuning; the AI governance lead owns oversight policy. Proposed acceptance criteria cover trace completeness and false-positive rate. Risks are rule maintenance burden on small teams and over-reliance on rules in place of human review, which the source itself warns against."},"roleDetails":{"sales":{"customerProblem":"Agencies deploying AI agents lack ways to detect harmful multi-step agent behavior and risk over-trusting new tools.","stakeholders":["SOC manager","AI governance lead","Application owners of agent pilots"],"discoveryQuestions":["Do you capture full agent session traces today?","Who reviews agent actions now?","Would alert-only evaluation be acceptable before blocking?"],"valueHypothesis":"Interpretation: a low-cost Quarry trial can show whether rule-based session detection adds signal for one agent workflow.","potentialEngagement":"A two-to-four-week alert-only Quarry evaluation on one agent workflow.","unsupportedClaims":["That Quarry would have detected the Hugging Face incident","Production-grade accuracy","Replacement of human oversight"]},"engineering":{"fit":"Teams with agent pilots that can export session traces.","architectureIntegration":"Feed agent and MCP traces to the Quarry scanner/runtime; send findings to the SIEM.","prerequisites":["Agent trace capture","Python runtime per repository requirements","Test datasets of benign and malicious traces"],"constraints":["No published accuracy data","Rules require ongoing maintenance","Trace data may be regulated"],"security":"Protect trace stores; redact credentials before analysis where possible.","proofOfValue":["Measure false positives on benign traces","Confirm detection on synthetic multi-step exfiltration traces","Verify findings preserve evidence from both linked actions"]},"delivery":{"work":"Set up trace capture, deploy scanner in alert-only mode, tune rules, report.","dependencies":["Agent platform logging","SOC analyst time","Test data"],"ownership":"SOC, with AI governance oversight.","skillsAdoption":"Train analysts on Quarry rule syntax and on interpreting session-level findings.","governanceCheckpoints":["Trace data privacy review","Alert-only to blocking decision gate","Rule-change approval process"],"acceptanceCriteria":["Proposed: ≥95% of agent sessions captured with tool calls","Proposed: false-positive rate documented on benign set","Proposed: synthetic attack sequences flagged"],"risks":["Over-reliance on rules","Maintenance burden","Exposure of sensitive trace data"]}},"retrievedAt":"2026-10-31T03:08:06Z","enrichedAt":"2026-10-31T03:12:07Z","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Writing and tuning rules requires detection-engineering skills scarce in SLED teams.","procurementImplications":"Open-source availability lowers cost of trial but shifts support burden to the adopter.","operatingModelImplications":"Detection of agent misuse becomes a SOC function needing agent-trace pipelines and rule maintenance.","sourceVerification":{"openedUrl":"https://www.theregister.com/security/2026/10/28/cisco-open-sources-tool-for-detecting-ai-agents-on-hacking-sprees/5305867","referenceExcerpt":"The Cisco team is quick to say that rules along are not a substitute for chain-of-thought monitors, or other forms of human oversight","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}