The “Safest” AI Company Is Reportedly Linked to a Strike That Killed 120 Iran Schoolgirls
The safest AI company in the world, according to independent experts who evaluate these things, is also the one answering questions about whether its technology helped target a girls’ school in Iran with missiles.
That is the contradiction at the heart of the Future of Life Institute’s Summer 2026 AI Safety Index, released this week. Anthropic tops the rankings again, leading five of six domains on transparency, safety frameworks, and governance. It is also the company the review panel singled out for “questionable military engagements” — including a reported link, first surfaced by the Washington Post, to the Minab school strike in Iran that killed around 120 girls on the first day of the war. Pressed by Bloomberg on his company’s possible role, CEO Dario Amodei conceded he didn’t know how the Anthropic system had been used, while maintaining that even such a use “doesn’t even violate our red lines,” because a human retained the final decision.
If the best student in the class is in that position, consider the rest of the room. Seven independent AI and governance experts graded nine companies from the US, China, and Europe across 37 indicators. Anthropic’s chart-topping grade is a C+. OpenAI and Google DeepMind earn Cs, Meta a D+, and three companies — xAI, DeepSeek, and Mistral — fail outright with Fs.
“AI companies are sprinting toward a cliff,” MIT professor and FLI chair Max Tegmark told me. “Despite acknowledging the great [existential] risks of artificial superintelligence, they continue racing to build it. Voluntary commitments were never going to be enough.”
Start with the military reversal, the starkest finding. From 2024 to 2026, Anthropic, OpenAI, Google DeepMind, and Meta — all of which once banned military applications — reversed course and embraced defense partnerships. “All of these companies used to say that they weren’t going to do AI for killing people,” Tegmark said. “And then, one after the other, they dropped this stuff. Every single one.”
Anthropic’s case is the most tangled. In February it refused the Pentagon’s demand for unrestricted use of its models, forfeiting the contract. Yet its red lines prohibit only fully autonomous weapons and mass surveillance — not AI-assisted targeting.
Sabina Nong, the FLI researcher who led the Index’s evidence-gathering, sees the flaw plainly: the safeguards are drawn around use cases, not outcomes. “Even if Claude actually made a mistake there,” she told me from the report’s launch in Seoul, “it’s not against the red lines.”
The same pattern — commitments that soften on contact with competition — runs through the industry’s safety frameworks. Tegmark recalls how the pledges were sold: “They were like, ‘No, no, no — a promise is a promise.’ But sure enough, they’ve all broken promises now.”
Nong walked me through the fine print: a few versions back, Anthropic’s responsible scaling policy “actually promised to pause unilaterally once certain safeguard thresholds are crossed. In the newest RSP (Responsible Scaling Policy), they’re only going to consider such pauses if their competitors do the same.”
OpenAI attached similar conditions; Google DeepMind and Meta, which had both pledged unilateral pauses, “voided those commitments entirely.” Even the labs’ recent public calls for a coordinated pause are, per the report, so heavily conditioned — no one willing to move first, no defined trigger, no adjudicator — that reviewers dismissed them as rhetoric that has “undermined safety frameworks across the board.”
Nong’s diagnosis is unsparing: “Corporate self-governance is just in itself inadequate. They’re just going to take back their promises more and more.”
Nowhere does that show more than in the report’s most damning row: Existential Safety, where no company scores above a C-. The CEOs of OpenAI, Google DeepMind, Anthropic, and xAI all signed a 2023 statement warning AI could cause human extinction — yet, Tegmark said, “there’s a jarring tension between the fact that they’re saying they want to build superintelligence, and it can happen soon, and the fact that none of them presented any convincing plan for how they can keep humans in charge.”
Nong put a finer point on it: “The leading companies are chasing recursive self-improvement at an unprecedented rate — AI agents that are not only able to read and write code, they’re running the code and delegating tasks to other agents. But no company has any concrete idea about how to make sure these models will stay under human control. It used to be a hypothetical scenario. Now it’s getting more and more real.”
Buried in the full report, the panel dismantles the industry’s favorite technical answers — interpretability and chain-of-thought monitoring are inadequate because “detection is not prevention” — and notes that not one of the nine companies runs human uplift trials measuring whether its models make real-world harm easier. The stakes are no longer hypothetical: Anthropic’s Mythos system autonomously found thousands of zero-day vulnerabilities and wrote working exploits for as little as $50 a run, prompting the Commerce Department to suspend its successor within days of release.
One last pattern jumps out: the top four are American companies with closed models, while open-weight Chinese labs and Mistral cluster at the bottom. But both my interviewees cautioned against the easy reading — the bottom rungs pair xAI, an American closed lab, with DeepSeek, a Chinese open one, at nearly identical scores. “It’s a very interesting question whether the top three are good at safety because they’re from America, or because they’re the richest companies who’ve been at this the longest,” Tegmark said. “I’m not going to speculate.” His takeaway, with the bottom three spanning three continents: “Lack of AI safety is a truly global problem.”
Tegmark reserves sympathy for the people inside the labs: “They feel really trapped in this race. If they stop cold, they’re just going to get their lunch eaten by their competitors. The only way this can ultimately work is if there are binding safety standards on everyone.”
The panel’s parting demand is simpler still: make the promise to pause a genuine and unilateral red line — not one contingent on what competitors do and push for comprehensive AI safety legislation through government as soon as possible.
*The full Summer 2026 AI Safety Index is available at futureoflife.org/index.*
