Anthropic executive warns open-source AI models carry more safety risk

Quartz

Source published: Site published:

Original article · فارسی

Guillaume Princen, Anthropic's head of international and its top sales executive outside the U.S., argued Wednesday that open-weight AI models carry greater safety and cybersecurity risk than closed systems like those his company offers.

"We need to be really careful with open-source on safety," Princen told Bloomberg TV at the International AI Summit in Dublin. "I can't talk for open models. But by definition, there's more risk with them."

Open-weight AI models release their underlying parameters to the public, meaning anyone can download, adapt, and operate them on hardware they control. Proponents contend these models offer a cost advantage over the proprietary systems sold by Anthropic and OpenAI, and align more naturally with countries' desires to control their own AI infrastructure, according to Bloomberg.

Princen's remarks go further than those of Anthropic's chief executive. Dario Amodei wrote in July that Anthropic has never advocated banning open-weight models and that those without dangerous capabilities are a public good, according to The Next Web. Where Amodei's position turns on whether a specific model is dangerous, Princen's applies to the category as a whole.

Princen traced his concerns back to Anthropic’s own efforts to secure its products. Following notable AI security breaches, the company introduced protective measures aimed at keeping its models from being weaponized, according to Bloomberg. Anthropic has also called for slowing AI development until international standards and consistent evaluation methods are in place, according to Bloomberg.

The comments came as Anthropic faces growing competition from open-weight models in regions wary of relying on American AI technology. French startup Mistral AI has put forward a new model positioned as a sovereign option for Europe, with open weights expected to be released later this month. Comparable releases from developers in China have been drawing growing international interest, according to Bloomberg.

Earlier this week, Anthropic warned that small AI models could soon power a novel type of botnet that could spread autonomously through internet-connected devices such as smart ovens, according to Bloomberg.

That concern over open models extends to questions of cost competitiveness. When addressing the cost concerns of enterprise clients, Princen argued that committing to higher-end models is a bet that will prove worthwhile as time goes on. “Always be investing on the frontier,” he said. “Because six months, a year, two years from now, the cost of that use case will also be going down.” He said Anthropic’s Claude Opus 5.5 model is 40% cheaper than a previous version, a figure that reflects typical costs; its list price fell 20%, with the rest coming from lower token usage, according to The Next Web. Princen, who joined Anthropic from Stripe Inc. in 2025, declined to comment on the company’s IPO plans or model training costs.

Alibaba Chairman Joe Tsai made the opposite case last week, urging Europe to embrace open-source AI as the region’s best path to technological independence from the U.S. and China.