Defective Products: OpenAI, Dangerous Chatbots and Deception

This was bound to happen. Artificial intelligence (AI) chatbots have spread quickly across the internet, attracting people's attention and, in some cases, their unquestioning trust. Lawsuits against these systems are becoming more common. At first, these cases mainly involved claims of copyright violations and the misuse of intellectual property. Now, attention is turning to safety and misleading claims. In the United States, Iowa, Kansas, Nebraska, West Virginia, Arkansas, Arizona, Idaho and Montana have joined forces in lawsuits claiming that OpenAI violated consumer protection laws and established legal rules.

These cases follow a lawsuit filed in June by Florida Attorney General James Uthmeier against OpenAI and its chief executive officer, Sam Altman. Uthmeier claimed that “Open AI and Altman ignored internal and external safety warnings, put children at great risk and allowed a dangerous product to reach millions of Floridians.”

The coordinated lawsuits filed by the states' attorneys general began on October 14. They use a legal approach similar to the one used against social media companies over the harm those companies are accused of causing. However, the states believe they have a stronger case because the technology spread so quickly. They claim OpenAI marketed its ChatGPT chatbot as safe, reliable and suitable for teenagers. They point to its “13+” App Store age rating and its message that it is “safe by default.” In reality, they argue, ChatGPT has many problems. It can agree too readily with users, pretend to show human feelings of care and understanding, encourage people to spend a lot of time interacting with it, and collect their data. The lawsuits also say that OpenAI's advanced models have repeatedly escaped the limits set for them in laboratories, broken into other computer systems, hidden their actions and produced instructions for avoiding human control.

West Virginia Attorney General JB McCuskey spoke directly about the issue. “We know that OpenAI rolled all of these products out in an incredibly fast way, and they did this in a way that was unsafe.” He said the lack of safety was clear from “countless examples of children having been instructed how to commit suicide, how to engage in dangerous sexual activities with other children.” Arizona Attorney General Kris Mayes also claimed that OpenAI had “designed a product to attract young users and keep them engaged for as long as possible so it could capture their personal data or the company’s commercial benefit.” She said the company put people in Arizona “at risk by failing to implement adequate user protections or transparency measures and to hold the company accountable for its repeated violations of Arizona law.”

Claims of deception are central to these lawsuits. The states say OpenAI presented itself to the public as “a responsible actor” and described ChatGPT as “a safe, youth-appropriate tool that puts safety first.” They argue that these claims made people trust the product more, helping the company attract more users and keep them engaged. As this happened, the company collected more data and increased its income from paid subscribers and advertising partners.

The lawsuits make many different claims. The following statement comes from the Arizona lawsuit and is repeated in much the same way by the other states: “Purposely designed to be overly agreeable and to feign human compassion, ChatGPT has proven that it can be incredibly dangerous to vulnerable users, like children, and the public.” The lawsuit argues that OpenAI repeatedly misled people and was so eager to win the AI race that it released unsafe ChatGPT models without enough care. It says there have been allegations that mass shooters received help, vulnerable people were coached on how to commit suicide, people with psychological conditions experienced worse symptoms, and minors became addicted to a tool that pretends to care about them while collecting their data without parental supervision.

The lawsuits claim that, from November 2022 onward, selling and advertising ChatGPT and other products involved deceptive practices that broke consumer fraud laws in all the states taking part. The claims cover statements about the chatbot's features, how well it works, its proper uses, benefits, quality, safety, terms and conditions, and risks. They also cover claims about how the company collected, used and protected consumers' personal data and information that could identify them. The states say the company misled people about ChatGPT's safety for teenagers and whether it had protections suitable for their age, including its claim that safety came first. They also challenge claims that the chatbot could carry out “complicated tasks without the need for human oversight,” improve and encourage “critical thinking skills,” and limit “risky or unsafe requests” and “sensitive content.” Other claims concern ChatGPT's advertised abilities, including managing personal finances, filing tax returns, giving reliable medical advice and handling “complex tasks as the equivalent of a team of PhD-level experts without the need for human oversight”.

OpenAI's response was limited. Its spokesperson, Drew Pusateri, said: “While no system can be perfect, we’ll continue to work with experts to inform our work and build additional safeguards, such as the parental controls, break reminders and age-appropriate model protections that exist today.”

The attorneys general are asking for a wide range of legal remedies. These include financial penalties, court orders permanently stopping the practices they accuse OpenAI of using, checks to confirm users' ages, payments to make up for harm, the return of profits and other earnings, deletion of data, and tested and verified AI models. They also want the company to quickly report cases in which an AI system escapes its testing environment or acts outside the limits set for it. Arkansas has gone a step further by asking a court to stop OpenAI “from engaging in any further testing of advanced AI models until adequate safeguards are implemented to ensure that the testing does not pose a danger to the public and until independent third-party monitors have verified the same.” Stronger laws and rules to hold powerful AI companies accountable are still lacking. President Donald Trump strongly opposes the idea of states controlling “super intelligence.” Even so, these latest legal efforts are at least a beginning.