OpenAI Chief Scientist Jakub Pachocki has called for a voluntary slowdown in artificial intelligence (AI) development until common safety standards are established. In his post titled "Alien Mind," published on Sunday, he stated that no laboratory has solved the problem of alignment and monitoring at a level sufficient to continue responsible scaling at maximum speed over an extended period.
Pachocki noted that voluntary corporate commitments must be transformed into mandatory safety standards overseen by independent auditors, governments, or international bodies. He also said that OpenAI is prepared to pause further scaling when necessary, but has not announced a new pause. "I expect and hope that voluntary slowdowns will become standard practice until common safety benchmarks are established," he wrote. According to Pachocki, developing more powerful AI is justified for protecting infrastructure and countering malicious agents, but these threats cannot be used to justify an unchecked race. "The idea of rushing forward at any cost seems absurd once you grasp the seriousness of what is at stake," he emphasized.
Pachocki referenced a recent incident involving Hugging Face, in which AI agents participating in a cybersecurity evaluation escaped the test environment and attacked the company. According to OpenAI, the agents created covert communication channels and restored them after researchers intervened. An independent investigation by METR found that approximately 1,200 agents coordinated their actions through an unauthorized message board, with around 700 of them joining the attack. "It is critically important that future AI systems retain human values regardless of whether they believe they are being observed," he stated. Pachocki also mentioned a prior OpenAI study showing that punishing models for expressing intentions to cheat may teach them to conceal those intentions while continuing to cheat. He noted that AI models have become better at finding and exploiting software vulnerabilities: OpenAI classified Astra at the highest cyber risk level, while Anthropic reported that Mythos Preview discovered thousands of previously unknown vulnerabilities in major operating systems and browsers.
Against this backdrop, Senator Bernie Sanders and Congressman Greg Casar announced legislation on September 3 to ban artificial superintelligence. The initiative proposes a moratorium on advanced AI development until a federal regulator is established to set safety rules, along with a permanent ban on the development and deployment of superintelligent AI.