Anthropic CEO Fiercely Hits Back in Lengthy Post: 4 Months of Secret Mythos Development, Voluntarily Cutting Lucrative Revenue Stream for AI Safety

36Kr

Source published: Site published:

Original article · فارسی

In the early hours of the morning in Silicon Valley, tensions are high.

Just moments ago, Dario Amodei, head of Anthropic, broke his usual calm on X.

He personally published a long post of nearly 1,000 words, directly answering one of the public's biggest questions about Anthropic:

Is the company's talk about AI safety just an excuse to make as much money as possible?

In his response, Dario revealed something for the first time:

Anthropic had completed training an advanced model in the Mythos class and kept it from the general public for about four months before making it available.

He also listed the money his company had deliberately given up over the past year because of its safety decisions.

In an industry where model rankings can change every few weeks, four months is a long time. Many new models can appear during that period.

By choosing to wait, Anthropic likely gave up a large part of the advantage it could have enjoyed for those months.

"Anthropic Only Wants to Win the Superintelligence Race"

The person who pushed Dario to break his usual calm was Nathan Calvin, a well-known investigator in AI safety.

A few hours earlier, Calvin had published a harsh essay of about 1,000 words that quickly caused a stir across the AI industry.

He listed what he called Anthropic's "nine sins" over the past year, explaining each criticism in detail.

These included spending 500 billion US dollars on a computing power order, secretly building an in-house chip team, having Claude handle 26% of internal research and development tasks as part of its own improvement process, and quietly changing its safety promises.

Calvin's main message was blunt: Stop pretending. You just want to build superintelligence before everyone else.

The argument began after a draft federal law on AI safety leaked over the weekend.

After reading it, Calvin and several policy researchers were angry. They believed Anthropic had secretly helped push the proposal. Although the draft appeared to support regulation, they thought it contained hidden problems.

In their view, it would not only override stricter laws in different states but also set an extremely high bar for the federal government to stop a dangerous AI system.

Anthony Cimino, Anthropic's head of federal affairs, had planned to post a response over the weekend to calm the situation. Instead, his response made Calvin even angrier.

Calvin then asked a series of challenging questions, starting with one: What kind of company is Anthropic, really?

According to its executives' open letter, Anthropic is a company focused on safety that is always prepared to slow down or stop its work for the good of humanity.

Not long ago, Dario had even warned publicly that AI systems could take over the entire internet within the next 6 to 12 months. He called on the whole industry to slow down.

Calvin made a sarcastic comparison: If OpenAI had done these nine things, Anthropic employees would already have condemned it. They would have treated them as clear proof that their rival cared more about money than human lives.

Dario Shows the Numbers Late at Night

How Much Revenue Did We Give Up Voluntarily for Safety?

Faced with accusations that challenged the company's basic claim to be ethical, Dario chose to respond directly.

He began by thanking Calvin for keeping the company under scrutiny. After these polite opening words, he got straight to the point.

He then listed nine decisions that had cost his company money and would likely upset many Silicon Valley investors.

We Had an Advanced Model but Kept It Hidden for Four Months

This was the most surprising revelation in the entire post.

Dario publicly admitted for the first time that Anthropic had already trained a high-level model in the Mythos class in February this year.

But Anthropic chose not to release it to everyone.

At the time, the cybersecurity protections for a model of this level were not yet fully developed. So the company made it available only to a very small group of partners, specifically so they could use it to improve and test cybersecurity defenses.

The model was not officially made available to the public until June this year, after the safety protections had been put in place.

Anthropic gave up a full four months of potential advantage in an AI race where every second matters.

Dario said this four-month period gave many businesses time to strengthen their own security systems before AI models with powerful cyberattack abilities became widely available.

On June 9 this year, Anthropic released Fable 5 for ordinary users. It belongs to this advanced model class and has strict safety protections.

Mythos 5 was released at the same event. It was first made available to a small number of partners working on cybersecurity and critical infrastructure, with some of its capabilities less restricted.

This means access was limited even during the period when the model was kept from the public. When it became available in June, different users still faced different limits.

Of course, Dario did not provide exact figures for how much revenue the company lost by waiting or how many risks it prevented.

But this decision shows at least one thing: When an AI model is released has itself become part of AI safety.

"Zero Tolerance" for Biological Risks

Safety decisions involving biology are especially important to Dario, who has a background in the field.

He said Anthropic had seen worrying results in tests of its models' biological capabilities. The company had also stopped attempts to misuse AI in ways that could support the creation of biological weapons. As a result, it applies strict protections to high-risk uses.

The problem is that filters cannot always tell what a person intends to do.

If a filter allows a dangerous request through, this is called a false negative. If it blocks legitimate research, this is called a false positive. To reduce the first kind of mistake, Anthropic has accepted many more of the second kind.

This frustrates researchers doing legitimate work. Competitors have even used having "fewer restrictions" as a selling point.

Users also pay a price for these protections.

Anthropic previously confirmed in an official statement that when Fable 5 detects a request that triggers its biological safety filter, the request is passed to a less capable model to handle.

An update in August reduced these changes to less capable models for biology-related requests by about 85% in the company's tests. However, restrictions on research that could be used for both helpful and harmful purposes remain in place.

Another area is government regulation.

Dario said Anthropic had supported laws such as California's SB 53. It had also backed proposals that were stricter than the version eventually passed. In June this year, he publicly argued that the government should have the power to stop AI models from being released.

In effect, this means supporting outside authorities having the power to switch off the company's own products.

Finally, there is the 5% shareholding plan. Just a few days ago, Anthropic announced plans to put 5% of the company's shares into a public-interest fund, subject to shareholder approval. Once the company's value passes a specified starting point, the board may choose to add another 2% for every 50% increase in the company's valuation.

AI Is Moving Faster and Faster. Who Will Hit the Brakes?

Why would a company that says safety comes first continue to spend heavily on computing power and speed up research and development?

Dario explained this through an idea he called the "race to the top," meaning that companies compete to meet higher standards.

He believes a company at the leading edge of technology can encourage other companies to raise their standards by showing that it is possible to develop advanced AI responsibly.

Calvin's concern is simple, too: If a company feels it must win every round of the competition, how much is it really willing to give up for safety?

This question becomes even more important as AI starts helping to develop the next generation of AI. Improvements may happen faster and faster, leaving people with less time to find problems and fix security weaknesses.

This time, Anthropic was willing to wait four months.

But next time, when the model is more powerful and the orders are bigger, will the company still be willing to wait?

References:

[https://x.com/DarioAmodei/status/2112732230124818891?s=20](https://x.com/DarioAmodei/status/2112732230124818891?s=20)

[https://x.com/_NathanCalvin/status/2112295219765137570?s=20](https://x.com/_NathanCalvin/status/2112295219765137570?s=20)

This article comes from the WeChat Official Account "Xinzhiyuan". It was written by ASI Revelation, edited by Taozi, and published with permission from 36Kr.