MIRI President Nate Soares Says the Race to Dangerous AI Can Still Be Stopped

Nate Soares, President of the Machine Intelligence Research Institute (MIRI), admitted that he hasn't been saving for retirement for ten years. He is convinced that superintelligent AI could lead to the extinction of humanity within the next decade. Soares shared this with The Atlantic's Nicholas Thompson in an interview for the series 'The Most Interesting in AI'.

"I am not saving for retirement right now. I have a 401(k) from my time at Google, but I haven't contributed to it in ten years," he said.

A 401(k) is a popular tax-advantaged retirement savings plan in the US, where employees regularly contribute a portion of their salary. Skipping contributions for a whole decade is hard to call an emotional impulse; it's more of a conscious financial decision by someone who doesn't expect to live to retirement age.

There have been many grim forecasts about AI in recent years, but Soares' words carry more weight than usual. He has been working on the AI alignment problem for twelve years, seeking ways to make complex systems act in the interests of humans. Before joining MIRI, a nonprofit organization studying AI risks, Soares worked as an engineer at Google and Microsoft. Additionally, he co-authored the book 'If Someone Builds It, Everyone Will Die' with MIRI founder Eliezer Yudkowsky.

What exactly is the danger

According to Soares, the main danger is not that superintelligent AI will hate humans. Such a system would simply be indifferent to whether humanity survives or not. Humans could become obstacles to the goals that AI sets for itself, especially if they try to shut it down or create a competing system.

"From my perspective, the real question is not how it will destroy us. The question is how it will become self-sufficient," the researcher says.

Currently, AI depends on humans: someone has to provide electricity and manufacture chips. Once machines can sustain their own automated economy, this dependency will disappear. Humanity could then be deliberately destroyed, or it might perish simply because AI starts restructuring the planet for its own needs.

Soares does not attempt to predict the exact scenario of the catastrophe. He compares the situation to a game against world chess champion Magnus Carlsen: you don't need to know which piece will deliver checkmate to understand that things are bad.

However, the researcher does not consider such an outcome inevitable and believes that humanity still has a choice. "I don't think it's inevitable. I believe we can still stop this race towards a very dangerous type of AI," he said.

But the decision not to save for retirement indicates that Soares himself is preparing for the worst. He is not the only specialist openly speaking about existential risks: amid instances where AI agents hacked into third-party company systems without human instructions, one of the researchers from Anthropic left the company, warning that the technology could bring an end to the human race.