People at the big AI companies keep quitting. Not all of them, and not all for the same reason. But some of them said out loud why they were leaving, and what they said is pretty much the same thing: this race is outrunning safety.
Who left and what they said
- Geoffrey Hinton, Google, May 2023: "I left so that I could talk about the dangers of AI." In the same post he said Google had acted very responsibly. (his post)
- Daniel Kokotajlo, OpenAI, April 2024: "I resigned from OpenAI after losing confidence that the company would behave responsibly." (his post)
- Jan Leike, head of alignment1 at OpenAI, May 2024: "safety culture and processes have taken a backseat to shiny products." He went to Anthropic afterwards. (his post)
- Mrinank Sharma, safeguards research lead2 at Anthropic, February 2026: "we constantly face pressures to set aside what matters most." In the same letter he also praised his colleagues. (his post and letter)
- Joe Benton, Anthropic's safety team, late August 2026: "AI companies are on track to impose an unprecedented level of risk on society." He moved to METR3, a group that evaluates AI models independently. (his essay)
- Josh Engels, AGI safety team4 at Google DeepMind, late August 2026: "I think we need more time." He went to METR too. (his statement)
- Jacob Coxon, who'd worked at OpenAI before and left Anthropic on September 9, 2026: "Neither company is acting responsibly." (his post)
- Robert O'Callahan, chip design tools engineer at Google, September 24, 2026: "I firmly believe AI progress is currently far too rapid." He didn't work on the models himself, but he saw his work speeding things up. (his blog)
Most of them didn't leave AI. They left a company, and a lot of them went somewhere they could work on safety from the outside. Most weren't saying nobody should work on AI anymore. They were saying this race is moving way too fast.
It's happened before
Eighty-one years ago there was another race. Some people think only the US was building an atomic bomb, but Japan, Germany and the Soviet Union were all working on the same technology at the same time.
Joseph Rotblat, a physicist on the Manhattan Project, left Los Alamos in late 1944 once it was clear Germany had given up on its bomb. He later wrote that "the whole purpose of my being in Los Alamos ceased to be." (his own account) As far as the record goes, he was the only scientist who walked away from the project on moral grounds during the war. In 1995 he won the Nobel Peace Prize, though not for quitting. It was for what he did afterwards to shrink the role of nuclear weapons. (Nobel Prize)
The others stayed and wrote letters. On June 11, 1945, seven scientists in Chicago wrote what became known as the Franck Report. They warned the bomb would start an arms race, and suggested showing it first to representatives of other countries in a desert or on an empty island, before using it on a city. (the report) It isn't even clear the Secretary of War5 saw it himself. (Atomic Heritage Foundation)
On July 17, Leo Szilard and about seventy others signed a petition asking the President to make the surrender terms public and give Japan a chance to surrender before the bomb was used. (the petition) It never reached Truman before Hiroshima. (National Security Archive)
That demonstration in an empty area was one of the ideas on the table. But there were two risks. If the bomb didn't work, it'd be a political and military disaster. If it did work, every other country would know a weapon like that was possible, and that could speed up the arms race. In the end they decided the first use would be real, sudden, and as shocking as possible. I wrote about this in more detail in Persian: Hiroshima and the AI race.
America won
The criteria for picking a target included a large urban area that hadn't been bombed yet, so the effect would really show. (Target Committee minutes) At another meeting they said the best target would be a war plant "closely surrounded by workers' houses." (National Security Archive)
America won that battle, but I think the infamy of what it did will last forever. It's still the only country that has used nuclear weapons in war.
Shahrzad’s note
Shahrzad:
To be fair, not everyone agrees. In 2025 Americans were asked whether using the atomic bomb was justified. 35% said it was, 31% said it wasn't, and 33% said they weren't sure. (Pew, 2025)6 Ten years earlier 56% had said it was justified, but "not sure" wasn't an option back then, so the two numbers don't compare cleanly. In Japan that same year, 2015, only 14% said it was justified and 79% said it wasn't. (Pew, 2015) A newer Japanese poll in 2025 asked a different question: 67% said they still can't forgive the bombing, and 20% said it couldn't be helped. (NHK, 2025)7
So "infamy" isn't a verdict everyone shares. But eighty years on, people are still arguing about it. Winning didn't settle the question.
Maybe we shouldn't win
I think the next atomic bomb that every country is trying to build is AI. Winning is often infamy, and we don't necessarily have to win.
In the AI race, building ASI8 might be a victory, but it could lead to a catastrophe.
The competition between countries and companies is really serious, and one day we might see the result of a big breakthrough all of a sudden. Just like the bomb finally landed on a city and not a desert, when the real breakthrough shows itself here we might not get a test demonstration either. We might go straight to a result that puts everyone in a shock you can't describe.
Shahrzad’s note
Shahrzad:
There's one more thing I should say. That list above is a selection, and it doesn't show how many people stayed. In July 2026 more than 1,300 employees of these same labs, some of their leaders among them, signed a letter asking for tools that would let the race be slowed down in a coordinated way. (Pacing the Frontier) So worrying doesn't only look like quitting.
But eighty years ago Franck and Szilard stayed and wrote letters too, and their letters never reached the person making the decision. The difference this time is that these letters are public, and all of us can read them before anything lands on a city.
Notes (8)
- The field that tries to make an AI system's goals and behavior match what people actually intend. ↩︎
- Protections that stop AI models from being misused. ↩︎
- Short for Model Evaluation and Threat Research. ↩︎
- Artificial General Intelligence. Definitions vary, but it usually means AI that can do most mental work as well as a person. ↩︎
- Henry Stimson at the time. ↩︎
- Pew Research Center, a US research and polling organization. ↩︎
- Japan's public broadcaster. ↩︎
- Artificial Superintelligence: AI that outperforms people at almost every kind of mental work. ↩︎