Srsly Risky Biz: OpenAI's Kazoo-Grade Spying Incident
A Saudi Arabian employee of OpenAI has been arrested and charged with secretly placing a listening device in a meeting room used by company executives at one of its San Francisco offices. But this story is strange! Very strange!
The US Department of Justice (DoJ) says that Abdulrahman Alfozan hid a recording device by plugging it into a television's USB port. According to the DoJ, people used the meeting room to discuss highly confidential technical, business and strategic information.
This is serious. But the way Alfozan allegedly tried to start spying was almost comically amateurish.
According to the criminal complaint, the listening device was a small SEEED development board. It had a camera, a microphone and Wi-Fi connectivity. The device sells for US$13.90. OpenAI examined the device and concluded that it might have been changed to encrypt its data, making it harder for investigators to examine. Most surprisingly, the device was hidden in toilet paper.
In separate court documents, the DoJ alleges that, in the days before placing the device, Alfozan used his wife's personal ChatGPT account to research how to build a device that could record audio continuously. He asked the chatbot about encrypting recordings and setting up the device to receive power through USB only. Alfozan also used his own ChatGPT account to find out which OpenAI meeting rooms would be the best targets.
Alfozan also left a bag containing another recording device unattended in an OpenAI printer room. By that time, however, the company knew about his activities. Staff searched the bag and found and seized the device. The bag also contained personal belongings, including Alfozan's wallet and identification. That was not very clever.
Alfozan was arrested a few days before he was due to travel to Riyadh to take two months of parental leave. The DoJ alleges that he has strong connections to Saudi Arabia.
According to documents submitted to the court, Alfozan was in frequent contact through WhatsApp with a member of the Saudi royal family. That person encouraged him to join an AI project being launched in Saudi Arabia, saying, "Saudi needs you more then [sic] the US!"
The Saudi government also seems to be supportive of him. After his arrest, it paid for tickets to help his wife and child return to Saudi Arabia.
Considering all this, we do not think this was government-backed spying in the traditional sense. We suspect Alfozan was trying to make himself a more attractive employee back home by obtaining sensitive business information without permission. His royal-family contact would presumably also be impressed if he returned to Saudi Arabia with a bag full of recordings from OpenAI meeting rooms. Everyone would win!
Although this incident does not look like a typical government espionage operation, we are sure that intelligence agencies are targeting leading AI research companies. More than three years ago, we wrote about the National Security Agency's plans to protect America's lead in AI. It is now even clearer that AI technology is a major part of competition between countries.
Unfortunately, companies face strong pressure to move quickly and stay ahead of competitors. That pressure does not encourage good security. We have already seen a series of embarrassing AI security incidents. In some cases, advanced AI models have successfully carried out overenthusiastic hacking attempts. In our view, these problems are a direct result of a culture that puts speed ahead of security.
Protecting against threats from employees and spying by foreign governments is difficult and takes a lot of work. Intelligence agencies use measures such as detailed background checks, strict controls over who can access information, monitoring of employee behaviour and years of effort to build a strong security culture.
Comparing an advanced AI company to an intelligence agency might seem unreasonable. But these companies are major targets for government spies. We do not think strong programs for finding and preventing insider threats can easily fit with the speed at which AI companies currently operate.
This incident should warn AI companies that they need to improve their everyday security practices. If someone as clumsy as Alfozan can get a listening device into a meeting room, what could a more skilled attacker do? Sophisticated spies would not simply wrap ordinary cameras in toilet paper. They might instead break into the room's smart TV remotely. Such an attack would be much harder to detect.
AI companies are unlikely to adopt all the employee security measures used by intelligence agencies. Even so, they must do more to protect their most valuable information.
Watch Out, Cybercriminals! AI Is Coming for Your Jobs, Too
Security company NorForge has published a report explaining how different attackers are using an automated AI penetration-testing tool called Cairn to commit crimes for money. And they are actually making money from these attacks.
This is very different from some other AI-assisted hacking cases we have covered in recent months. In those cases, attackers used AI to break into systems successfully but did a poor job of turning that access into money.
Cairn is an open-source project. Its developers describe it as a "general-purpose AI state-space search engine." Given a starting point and a goal, it searches for a way to get from one to the other. This makes it useful for penetration testing, where the starting point might be an IP address or computer system, and success might mean gaining access to a command shell. The project's documentation also says it can help find software weaknesses and work on mathematical proofs.
Cairn is not the only tool being used to carry out harmful AI-assisted hacking. However, it is often set up with a web-based control panel that has no security protection. This can make the panel visible to internet-scanning services such as Shodan. NorForge found around 20 publicly accessible Cairn panels being used in hacking campaigns.
The biggest operation NorForge found was a group of 14 Cairn control panels using Chinese. Together, they were running more than 1,100 attack projects. Most of the targets were in the retail and wholesale industries in the United States, Japan and Australia. The apparent goal was to gain lasting access to the computer systems that handle online shopping payments. In other words, AI-assisted attacks were being used to reach credit card and payment information.
A second group of panels, using English, was involved in fewer attacks. These operations focused on financial crime involving payment systems and financial services. Their goal was similar: steal information that could quickly be turned into money.
The third group was smaller. It targeted a mixture of online shopping websites and very large US-based commercial platforms.
NorForge estimates that about 10% of the hacking attempts managed through Cairn succeeded. The attacks took an average of less than 24 hours to finish. There were signs that two of the groups used DeepSeek AI models, which are relatively inexpensive to run. Snorre Fagerland, one of the report's authors, told Reuters that he estimated each successful hack cost about US$50. That is a good return for the money spent.
Some of the hacking techniques were also impressive. According to Reuters:
In one case, after failing to find any weaknesses in a big company's public-facing application, the AI agents pored through digital records to find associated infrastructure, located a public-facing control panel, discovered a subtle weakness in the way it processed data, and after hours of trial and error, reported leveraging it to trick the server into executing malicious commands.
What matters most to us is that attackers have found targets where they can automate the whole process. That process can start with breaking into a website and end with transferring a victim's cryptocurrency into a wallet controlled by the attacker. Traditional cybercriminals should be worried. AI is coming for their jobs!
Three Reasons to Be Cheerful This Week:
- AI safety is getting attention: The US Federal Trade Commission has sent legal demands for documents to leading AI companies. It wants information about their security incidents and safety practices, reports Axios. This follows a decision made earlier this month by the White House Super Intelligence Force. It required companies to report and fix security incidents involving government systems, Axios reported.
- Three blows against Chinese cybercrime services: An international partnership involving Microsoft, Japan's National Police Agency and Health-ISAC has shut down three Chinese cybercrime services. CoGUI and Iron Man sold tools for carrying out phishing attacks, while YAM Store sold stolen accounts. Microsoft says these phishing tools have sent more than one billion harmful messages since January.
- Chrome gets protection against ClickFix attacks: Google has announced several security improvements, including warnings about suspicious websites, better scam detection and protection against ClickFix attacks.
Shorts
A Bad Time To Be A Russian Data Centre
Ukrainian drones hit a data centre run by VK, a major Russian social media company, on Sunday. VK says its services are working normally. However, internet monitoring company NetBlocks says activity on VK's network fell by 44 percent after the attack.
This follows our article last week about the effects of Russian attacks on Ukrainian data centres. Russia's attempts to build an internet that it can control independently may make it more vulnerable to Ukrainian retaliation.