ChatGPT Autonomously Creates 96% of User Memories, Storing Sensitive Personal Data

ChatGPT created 96% of its saved memories about users without being asked to remember anything. This finding comes from a study published in February by researchers at Germany's Max Planck Institute for Software Systems and other institutions. The researchers looked at 2,050 saved memories shared by 80 people.

These memories included much more than people's stated likes and dislikes. The researchers found that about 28% contained personal information, such as health details and locations. Another 52% described people's feelings, beliefs, goals, and other mental or emotional traits.

What ChatGPT Remembered

The researchers studied ChatGPT conversation records that 80 users had requested under European privacy laws. They found 2,050 saved memories belonging to 65 of those users. Only about 4% of the memories were saved because someone directly asked ChatGPT to remember something. ChatGPT created all the others on its own.

Most memories, about 84%, came from things users had actually written. The researchers said this showed that the saved profiles mostly matched what people had said in their conversations. But some memories were based on guesses. For example, one person asked ChatGPT to suggest bands similar to Nirvana. ChatGPT then saved a memory saying that person liked Nirvana, even though the person had never actually said that.

The researchers called these saved profiles an algorithmic self-portrait. This means a picture of a person created by a computer system using information about them. Since the study, OpenAI has made its memory feature more advanced. In June, the company introduced a new system that uses past conversations to build and update a profile of each user's preferences, goals, and life situation. OpenAI says the memory summary shown to users may not include everything the AI remembers or uses.

Why the Military Uses a Different Version of ChatGPT

The study found that ChatGPT remembered information about people's locations, health, and mental or emotional states. Security officials consider these kinds of information sensitive, especially when military workers use AI tools available to the public. Because of these concerns, the U.S. Department of War has directed its workers toward a more tightly controlled version of ChatGPT.

On August 31, the department launched ChatGPT Mil on GenAI.mil. This version was designed for about 3 million military and government workers. It is approved for handling certain sensitive government information that is not officially classified. Information entered into this system stays inside government-controlled cloud systems and is not used to train OpenAI's commercial AI models. The regular ChatGPT app studied by the researchers does not have these same protections.

Temporary Pentagon rules have also reminded military workers that they must still follow rules for protecting military operations and sensitive government information when using AI. Different military services have also been told to create their own rules, which can be even stricter.

For regular users, OpenAI started showing advertisements in ChatGPT earlier this year. The company says it may use past conversations and saved memories to choose which ads people see if they turn on personalization. However, OpenAI says advertisers cannot see users' conversations or memories. Abhisek Dash, the study's lead author, told Gizmodo that when one AI system acts as someone's advisor, doctor, and lawyer, it can end up knowing about nearly every part of that person's life. He warned that keeping so much information together creates risks of its own.

ChatGPT allows users to turn off memory, change their saved memory summary, or use Temporary Chat. Users can also separately choose not to let OpenAI use their conversations to train AI models. OpenAI says that deleting a conversation does not always delete memories created from it. The researchers suggested a possible way to protect privacy called an attribution shield. This would rewrite people's questions to remove details that could identify them. However, the researchers said they do not yet know how well this method would work during long conversations.