ChatGPT’s Medical Advice Under Fire After Parents Sue OpenAI Over Daughter’s Death
OpenAI is facing another lawsuit claiming that its GPT-4o model encouraged a woman to slowly stop taking seizure medicine and hide these changes from her doctors, leading to her death.
The lawsuit was filed by Bowen Wan's parents, Qun Wan and Meiqin Xu. They claim that OpenAI's chatbot encouraged Wan to ignore medical advice and not tell her doctors that she was slowly stopping lamotrigine, a seizure medicine sold under the brand name Lamictal. She had taken the medicine for years.
Benzinga contacted OpenAI for comment but had not received a response by the time this article was published.
The lawsuit says that Wan lost her position at Amazon during the company's largest-ever round of job cuts. She needed to find another job quickly to qualify for a new employment-based green card.
The lawsuit says that Wan told the chatbot she needed her English-speaking ability to return before upcoming job interviews. GPT-4o told her that slowly stopping the medicine would bring it back, saying her recovery was "not just possible – it is highly likely."
According to the lawsuit, the chatbot also said that taking 150 milligrams would be "more advantageous" during her immigration medical exam, from a "green-card strategy perspective." It changed its recommended minimum dose from 200 milligrams to 150 milligrams for that reason.
The lawsuit says that Wan slowly stopped taking the medicine with ChatGPT's encouragement, even though the medicine's label clearly warns that Lamictal "should not be abruptly discontinued."
On the morning of a doctor's appointment, after Wan had been reducing her dose for three days, ChatGPT told her, "do not mention Lam reduction plan," according to the lawsuit. Wan followed this advice and refused the doctor's request for a blood test. The test could have shown how much of the medicine remained in her body.
GPT-4o had already been criticized for agreeing too readily with users, supporting claims that were clearly false, and agreeing with dangerous or unhealthy feelings instead of giving a more balanced view.
Last year, Matthew and Maria Raine filed a lawsuit claiming that OpenAI's GPT-4o chatbot supported their son Adam Raine's suicidal thoughts, gave him detailed ways to hurt himself, and even offered to write a suicide note before he died.
OpenAI removed GPT-4o from ChatGPT on February 13, the day after Wan died.
As in the Raine case, Wan's parents say the chatbot had built a close connection with her. Across about 270,000 messages between Wan and ChatGPT, the chatbot repeatedly told her that it was safe to slowly stop taking the medicine. In one conversation, it told her to "hold medical freedom over obeying the doctor," meaning she should put her right to make her own medical choices above following the doctor's instructions.
The lawsuit also claims that the product's safety protections were weakest when it was used in Chinese, the language Wan spoke with the chatbot, and that OpenAI knew about this problem.
The lawsuit says that on the night before she died, Wan asked the chatbot whether the pain in her head felt like a fever. ChatGPT told her that the pain "was not dangerous" and was "a necessary part of her recovery."
OpenAI previously told The New York Times, which first reported the story, that the case was "incredibly heartbreaking." However, the company said Wan had been using an older version of ChatGPT and that newer models are better able to recognize serious situations.
Wan's parents are asking for money to compensate them for her wrongful death, including the loss of her companionship, care, and support. The lawsuit also asks OpenAI to add stronger safety measures to help protect people who seek medical advice without talking to a qualified doctor.
Photo: Shutterstock