How an OpenAI model caught a spy inside its HQ

Chat GPT just discovered and arrested a spy in OpenAI's own HQ and that's the least craziest thing I'm gonna say on this entire episode, because this whole story is crazy. It starts off with an unknown individual planting an audio listening spyware device in the back of a television wrapped in toilet paper. Why it's wrapped in toilet paper, I have no idea, but the important part is the conference room, why do we care? Well, a bunch of really important executive.Executives and business partners at OpenAI meet to discuss proprietary information, secret product plans, so you can imagine if that information got into the wrong hands of a person, a company, or maybe even another country, that could probably become a bit of a problem. So OpenAI handed over the case to the FBI and their own internal model, who ended up ratting on the individual who had planted that device. Kash Patel, the head of the.FBI said this is the bureau's utmost priority at the moment, that AI has finally become an important or asset of national security and risk. Imagine if James Bond, 007, had access to Fable 5.1 back in the day, he'd, he'd be a bit of a beast. But also imagine if his enemies also had access to this model. That's what we're gonna uncover in today's episode of In the Loop, brought to you by Q Qualcomm.So the story starts off two weeks ago, September 30th, when an OpenAI employee discovered a device plugged into the back of a television in an executive conference room. And it was wrapped in toilet paper for God knows what reason, but upon recovering this device, it was handed over to the security team at OpenAir. They started looking at it and they were thinking, hmm, this looks like a USB stick, uh, but it has this weird circuitry that's kind of attached to it, so it seems like a bit of a hack job. What on earth is this device? And so this.Started to examine it and they asked their own internal AI model to uncover what this device might be used to. And they discovered three particular things. One, it had very heavy cryptography designed around its entire thing. So accessing the information that this device had was near impossible. Number 2, the device was designed to be continuously recording audio, that it would also be triggered to start recording audio when someone would start speaking. So, it seems very intentional that this.The device was to try and spy on conversations that were happening in that particular room. And then number 3, any kind of data that got extracted from this device that could be transferred to, you know, your computer to view it or, or review it or listen to it was under heavy encryption as well, which meant that the data transfer layer had been very meticulously designed. And so OpenAI kind of like looked at this and thought, this seems like an attempt to spy on us, so we should probably hand this case over to the FBI, give them access to this internal model or work with.At least, to try and figure out what the hell is going on. So OpenAI continued their internal investigations whilst they gave it to the FBI and very soon, using their own internal model of, I don't know, GPT 6.5, GPT 7, call it whatever their latest model is, the one that solved all the math stuff, definitely go check that episode I made. That very same model named someone. It traced back the entire circuitry design of this device and who was most likely to have planted the device and uncovered a name.So OpenAir had an idea of who it might have been, but let the FBI continue on their investigation. But while they were doing this, they also decided to plant a camera in almost the exact same position where the USB stick was. So in the event that someone who planted it came back to look at it, they would be able to capture the suspect red-handed. And that's exactly what happened a few days later, when the suspect returned, had a look behind.The TV and the camera captured exactly who the culprit was. And on the same day that this culprit was captured on this camera, security team discovered a bag left behind by the culprit by accident in the printing room. They went through it and they discovered a second device that looked very familiar to the one that they found behind the TV. So they just took the bag, put it in lost and found, but they took the device. They were trying to capture this individual red-handed. They also wanted to figure out what.On earth was going on what this individual might be using this information for, so they, again, were keeping hush and quiet, and they handed over the device and the information that they had to the FBI. Now this individual discovered that his bag had gone missing and went to lost on property to try and find it, and he found the bag, so he felt pretty good about it, except that he noticed that the second device was missing, and he also noticed that the device behind the TV when he went to go check it, was also missing. So he reported.that there had been a theft of his own property, but by that time, the FBI and the open AI team was already on him and issued an arrest or warrant for his arrest the exact next day. Upon arresting this guy, they discovered that the suspect, his wife, and another family member of the suspect had a one-way ticket to Saudi. So he was going to fly out with his family and likely never return because.Presumably he thought he'd been busted. So the guy got arrested and he has been internally investigated and being put through court or whatever all that stuff is by the FBI and Kash Patel, who's the leader of the FBI and his team has been handling it since then, but OpenAir has come out with an official statement, kind of giving a high level idea of what's been going on, but never named any names. Who has been naming names is pretty much every other media publication out there.Which gets to the next section, who the hell did this and why did they do this? The man's name is Abdul Rahman Al Fozan. He's a 35-year-old Saudi national who's lived in the US since 2020, but he's also worked at OpenAI for over a year on the product team, so he's like a product slash engineer, he's worked on multiple product releases at OpenAI, likely a core.Employee, been there for a while as the evaluations have increased, so likely his equity is worth quite a bit. This is a person that people actually know at the company. If you look at his LinkedIn history, he's worked on things like AI agents, which is important for the next bit of this story. What was he doing? How did he create this device, and why was he trying to use this device in that secret room? Let me take you back 6 weeks prior to this device being discovered. This.Abdul Rahman queried an internal personal agent at OpenAI. Guess what he asked that agent? Hey, um, where do high-powered execs at OpenAI have meetings? Also, business partners, where do these execs have meetings with those business partners? Can you tell me the exact rooms? This guy was literally asking an internal version of chat GPT which the security team and.Any open air employee or anyone with a brain cell could get access to and see who was querying it. That's the device that he, or rather, that's the agent that he used to try and target and locate an executive room. So he finds this room, he finds the perfect room, it seats 6 people or more. They only have discussions about proprietary research, designs, product releases, things that you don't want getting out to the public, but there's a problem. He can't.The room, he can't get access to the room, so that's only bookable by execs and their secretaries. So he tries to then devise a plan as to how he can be a listen in on the conversations there, and he comes up with a bright idea. I need to create an audio device that can spy and listen in to the conversations that are happening in there, but I don't know how to build that device. You know what might be useful? Let me ask AI.So he goes to his wife, who has a chatbot account and he goes on to ask it a series of very specific prompts, which is, I want to create an audio device that continuously listens to the surrounding conversation around it, but also when it stores this data, I want it to be encrypted such that only I have access to this data, and in the event that someone gets a handle on this device and is able to extract the data, I still want it.To encrypt the data so that no one can can get access to this thing except me. But I also need to design this device so that it plugs into a television in a room. But I don't want the television to detect the device cause that would give it away. So could you create the device such that it only appears to the TV as a power device, which means that it's charging the device, but it doesn't know what the device is being used for. So, a series of back and forth prompts over the next like 4 weeks from that initial.6 week kind of like starter, ask my agent which room should I plant this device in. He starts to plan this device and presumably he ends up building it. This is the thing that is wrapped around toilet paper. Investigators looked into, you know, what it took to make this device. It cost all but $24. So a $24 device costs this guy presumably a large chunk of his life. I looked into the prison sentence, it's like up to 5 years for the particular case, but before I get into that.The idea here is a $24 device basically caused this guy and the entire company and this entire kind of spy mission, a whole world of pain. OK, so then we get to September 28th, 2 days before the device is discovered. Abdul Rahman enters the open AI Office HQ building. He walks up to his desk, he chills.He's working, he's going about his day, seemingly, keeps glancing over to the executive conference room, where he keeps wandering around scouting the area. He enters the building at 2:18 p.m. and he's there till about 6:30 p.m., so whatever, a regular working day in his eyes. But he's caught on CCTV because of course investigators are now onto him and they can see.Him in hindsight, scouting the entire area where he then pops into the room when presumably someone exits and he's able to hold the door open. This is something straight out of a spy movie. This is ridiculous. Also, why the toilet paper? I don't understand. OK, so then fast forward two days later, September 30th, and it's the story I just told you. Someone discovers the device at the back of the television, presumably because the guy was caught on CCTV being shady around.The office or the conference room in the first place, everything comes down around him, he's caught, he's arrested, and they're currently working on what his sentence might be for his particular perjury.Now, before I get into what the ramifications of this is, I think it's important to spend a short moment how OpenAI's internal model discovered the guy, because it got access to the device and it basically reverse engineered the entire thing and it was able to uncover two things. Number 1, how it was particularly designed and in accordance with the CCTV footage, which OpenAir also fed it, they could then figure out who the person likely was.And then when they had a suspect, which was this Abdul Rahman guy, they started to look at his activity secretly without, you know, warning him, and they discovered his conversation with the internal personal agent that he had 6 weeks prior, basically asking,Hey, where do all the execs have super important information and high priority product release information so that I can plant a listening device. I'm not a spy, but spy 101, don't talk to the tools that your company's own if you're trying to pull off this kind of a heist, it's crazy. But then they took it an extra step further and this.This is the part where I'm, I'm still trying to figure out how they did it, but they were able to discover that he was having a conversation on another personal chatbot, and he then used this to, you know, prompt engineer what that $24 device would look like. He built it himself or he got someone to build it for him, and then it led to him planting it in the back of the room. So the guy used AI to plan.And build and execute the entire thing, and then AI that he used ended up becoming the exact point of reason for his downfall, discovery, and arrest. It's just this insane loop where you kind of use the AI to do the bad thing, but you can use AI to do the good thing as well, so it's kind of like, it's not a wholesome story, but it, it does give me more confidence when, you know, we talk about fear mongering around AI cyberattacks and AI's gonna wreak a lot of havoc.Air can be used for a lot of good as well. So it's just this insane story, and I have a lot of takes on this, so let's get into them on today's episode of In the Loop brought to you by Q Qualcomm.But the analogy that I would use here is he effectively walked into the building and what would be the analogy here, he asked the office receptionist where the most secretive conversations were happening amongst execs, found out the information from this receptionist, and then went to a colleague and asked them, hey, how do I build a listening device that I could plant in this room? He basically like gave the cards over to.To people or employees or agents of the building, and he's honestly stupid to have not thought that people wouldn't investigate the tools that he would use, that he wouldn't leave a digital trace. And, and I guess that's like one of the larger points about this whole thing, putting the, the politics and the crazy espionage theories aside. Everyone's digital footprint when they use AI has become massively enlarged. Remember, when you're talking to your chatbot, it may seem like it's quite a personal thing, but all your data is being tracked by.OpenAI Anthropic or whoever whoever the model lab is, unless you're using open models hosted at home on your own. Um, that's not necessarily a bad thing. It's a good thing in this case that they were able to trace it back, if this was an open model, probably not. The second thing is around the sentence, so I think right now, rumors or reports are saying at least that they have him on attempt.To overhear or record or something, I forget the term that they were using, it's, I think it's like oral conversations, something very generic like that. Nothing that would indicate espionage, spying, or a breach of national security, which would presumably result in a much larger sentence. But right now the max sentence that.He can have for recording of oral conversations is 5 years, which seems honestly surprising to me, uh, but he should get the charge for maybe like spy stuff. But again, details are unraveling. FBI is on it. Kash Patel, head of the FBI is fully committing resources to try and figure out what the hell this is, which brings me to my third take. AI has officially become.An issue of national security. Now, I've said this many times on previous episodes on this channel, and it's crazy to think that this is happening now, and there's real life examples of this happening. I think of a conversation that Jacob Coxon, who is the crazy safety and alignment.Researcher, remember the one that we spoke about on an episode previously where he left OpenAI, joined Anthropic, then left after 3 weeks because he did not have any faith that either company were taking model alignment seriously. He went on a podcast last week and he said that there's this clip of him that went viral.He said that there's a 100%, or sorry, 90% chance that there are spies at the Frontier Labs. What are the odds of him having that conversation, going live, and then literally two weeks later, OAAir uncovers this spy Abdul Rahman, who had been planted there for presumably a year. It's becoming all too real, if that makes sense, and it's one thing.Kind of talking about it to you guys, talking about it on a camera, hypothesizing what might happen. It's another thing, it actually happening, and then it's a third thing, us being aware of it, it being all revealed to us. But can you imagine how many other attempts are currently underway, if you're anthropic and you're seeing this news, if you're Google, if you're SpaceXSI or whatever Frontier Lab.You're gonna be now on high alert that someone is trying to access or listen to your conversations. Now, with all intents and purposes, this could have been a really bad situation for OpenAI. This guy could have attempted to do something even worse, not listen to conversations, but try and exfiltrate model weights or model blueprints that they could then use to build a.Competitive model or use it for another model for more nefarious purposes at the state level or at the company or entity or even the individual level. Like there's myriad different ways that you could use this thing. Thankfully, OpenAir's investigation has revealed that there's likely not a chance that he's been able to pull this off and personally, if the guy's wrapping his device in toilet paper, it doesn't sound as sophisticated, but you know,You can imagine that there are high level organizations probably trying to attempt this right now, maybe even still at OpenAA or at Anthropic or any of these other labs. The fourth take is AI is officially a national security threat and how we protect those models, how we keep them secure, needs to become a national priority. That's what Kash Patella said. But right now, it's under.Lock and key for small, well, small private companies such as Open Air and Anthropic who employ their own security team and who kind of take matters into their own hands, and they're not necessarily a public company. They don't have public investor interests, they're only beholden to themselves and their current shareholders. So I can only imagine that the government wants to get involved. They are probably placed on high alert now, and Kash Patel's probably thinking, I wonder how I can plant.An FBI agent or several FBI agents just in the day to day 9 to 5 at OpenAI's office, I'm almost certain that they're having this conversation. And then it kind of got me thinking about Dario's suggestion of pacing the frontier, uh, about a month ago, where he suggests that we want external parties to get involved and kind of evaluate and monitor the alignment side of AI. I'm noticing a trend, essentially, I guess is what I'm trying to say.Where external parties need to start getting involved or will likely get start getting involved with these private companies. Traditionally, it's been at the whim or direction of Sam Altman and Daria Murray, but now they may not have a choice and I wouldn't be surprised if in the next few weeks.Trump, the administration or whoever announces that there'll be some form of light nationalization of these companies because as these models get more powerful, and Anthropic and open air are both rumored to be housing much, much more powerful models than we currently have access to internally, which they can't release because.The ramifications of that could be quite dangerous. As a government, as a nation state, if that got into the wrong hands of your adversaries such as China or whoever that might be, you could end up having a massive problem, just from a pure security standpoint. So the labs are national security sites now. And then the other thing I was thinking about is Apple. Apple is currently suing the hell out of OpenAI, accusing them of stealing or reaching out to hundreds of Apple employees.Trying to exfiltrate their own blueprints and device architecture, that's it, for the AI hardware device. Seems like OpenAir's got their own problems, and if the sophistication levels of their own attacks that they're dealing with is a toilet paper wrapped device plugged into a television, it seems highly improbable that OpenAir has the sophistication to be able to pull off a similar attack on Apple, although I hope they're not doing that. So that's really the episode.That is an insane story that I read late last night. I woke up this morning and I thought I have to make an episode about this. If you're wondering where I still am, I am still in Kyoto, Japan, and yes, it is amazing. Yes, if you ever come here, get a kimono, but I still need to keep putting these episodes out guys. It's, it's fun and it sounds like all the crazy stuff is happening whilst I'm on this crazy time zone right now. I am currently 13 hours ahead of the world or New York as I call it.And um, you know, news breaks overnight as I'm asleep. So thank you so much for listening to this episode. I hope you enjoyed it. I hope there was some valuable and useful takes. I, I would say if you had to leave this episode with one top takeaway, it's that, you know, objectively, the DNA or surface area that you have online, on digital surfaces, whether it's your phone, whether it's your computer, whether it's any kind of agent or software package that you use.Is now being tracked, especially if you use chat GBT or cloud, and that's not necessarily a bad thing as we've seen in this instance, and so I'm happy to,Finally talk about a crazy dangerous thing that actually had a really good ending, because at the end of the day, I do think optimistically that, you know, when it comes to cyber defense or cybersecurity, using AI can actually end up putting us in a much better world than we exist in today and all the doom it takes about AI agents running rogue will definitely happen, but we'll have a way to figure it out. So if you enjoyed this episode or if you disagree with any of my takes, let me know, leave us a comment.Uh, please share these episodes with a friend, whether you listen to it on Spotify, Apple, or wherever you listen to it. We are anywhere and everywhere, you know, give us a rating, leave us a comment. I read all of them, I try and reply to all of them. Give me feedback. Um, I know this is a solo show now, but, you know, we're trying to figure it out, me and producer Luke. Um, so if you have any feedback, it's been great. I've been, you know, really enjoying the feedback that I've received so far. And also follow me on X. I kind of post a lot of my thoughts even when I'm, you know, in Japan with my family.Um, and so, you know, you can hear more news there, more current, up to date, and some of my crazy takes, I guess. You can also DM me there as well. But yeah, thank you so much for listening. And I will see you on the next episode of In the Loop, brought to you by Qualcomm. Thank you.