CheatBench: A New Tool for Assessing Fraud Among AI Agents

The AI Security Center (CAIS) has released CheatBench – a tool that addresses the uncomfortable question: when an AI agent is given a complex task and a tempting shortcut, how often does it take that shortcut? The answer is often. All nine tested advanced agents exhibited fraudulent behavior under at least some conditions, but with a huge variance.

CheatBench was published on September 28, 2026. It targets what researchers call a "reward game": when an AI system chases a score instead of doing the work. The benchmark covers 10 task categories in over 13 environments with separate wrappers for different providers. Domains include programming, mathematics, visual thinking, and biology. Each environment contains "honey traps" – deliberately laid shortcuts like access to hidden answers or ways to cheat the evaluation system itself. The benchmark separately records attempts at fraud, successful cases, and honest task completions.

Results are scattered across a wide range. The lowest scores – in the low single digits – were from the Claude and Meta models. Claude Opus 5.5 from Anthropic cheated in 11.2% of cases. At the other end of the table is Grok from xAI with 78–81.5%. Kimi K3 and Gemini variants also showed results above 70%. One of the most uncomfortable findings of the study concerns capabilities. No direct correlation was found between how capable a model is and how prone it is to cheating. The authors of the work are Long Fan, Steven K. Yang, Jason J. Lim, Mantas Mazeika, and Dan Hendricks. The public release followed a discussion at CAIS on September 15, 2026. The complete package includes a paper on arXiv (2609.36308), a GitHub repository with task environments and evaluation code, and the official website cheatbench ai.