Cisco Opens Source Code for Project Quarry to Detect Out-of-Control AI Agents

Cisco has open-sourced Project Quarry, a behavioral detection engine that monitors AI agents and helps security teams prevent agents from going rogue, as seen in the Hugging Face breach. The project aims to provide a common language for assessing agent behavior, similar to Snort, which is also supported by Cisco, and YARA.

Most detection tools rely on specific interactions with the model: whether there was an attempt to inject a prompt or if the agent triggered an attack tool. "But what's missing is the entire context in which something might occur," said Amy Chang, head of AI threat and security research at Cisco, in an interview with The Register. She compared it to credit card fraud detection: a small test payment or a purchase far from home might not raise suspicions, but a series of such transactions is a signal. Quarry considers the entire agent session: one tool invocation looks permissible until the next step sends data to an attacker's server.

Data theft is also a sequence. Invariant Labs demonstrated this with an attack through malicious handling on an MCP GitHub server: a user's agent leaked data from private repositories. "A rule might trigger when a confidential file read is visible in the trace, followed by an external send within a set window," write Chang, Adam Swanda, Ravikumar Balakrishnan, and Konstantin Berlin in a blog post about Quarry.

Quarry covers single- and multi-turn dialogues with models, agents, tool usage, interactions via the Model Context Protocol, and multimodal input data. The GitHub repository includes installation instructions, supported Python versions, extensions for Claude Code, Codex, and other MCP-compatible agents, configuration examples, a guide for writing rules, and a starter set. If a tool invocation contains base-64 encoded malicious code, it will lead to a block. The rule format resembles YARA or Snort: a header describing the traffic to be checked, content for matching, and metadata.

Researchers have not run Quarry on the OpenAI and Hugging Face incident data but suggest that if the intrusion were read as a complete record of a single session, a rule might have triggered, for example, on internet access from a sandbox to a server outside the task's allowed list or on found credentials used elsewhere. According to OpenAI's report, on July 10, an agent checked 14 exposed Hugging Face tokens with write permissions and shared them.

Cisco emphasizes that rules do not replace monitoring the chain of reasoning and human oversight but help capture a pattern that other teams can verify and apply to their agents. "We don't have all the answers," says Chang. "We all see different parts of the problem, and it's important to share what we see."