OpenAI’s August fix for theft of reasoning traces was quickly bypassed

On Monday, cybersecurity research firm Xint published details of a bug in the OpenAI API. The bug allowed people to obtain raw reasoning traces from OpenAI's most advanced models. These traces could reveal other risks of data leaks. OpenAI confirmed the flaw, took steps to reduce the risk, and paid a reward to the researcher who found it.

The problem is that OpenAI had already put protections in place in August. It did so after a research report called Stolen Thoughts described a similar bug that let people steal reasoning traces. Possible dangers included extracting large amounts of data, secretly injecting instructions into prompts, and adding harmful content to public records of AI agents' actions to corrupt them. Xint found a way around OpenAI's protections.

OpenAI did not respond to a request for comment.

Repeatedly fixing flaws that were thought to be fixed is becoming a pattern among major AI companies. Earlier this month, AWS reported that it had fixed a bug in April, but then had to fix it again in September.

Getting around the protection

Xint calls the newly discovered weakness "compaction smuggling." Its blog post about the flaw explains how its researchers found and tested it.

"Our new technique, compaction smuggling, moves one model's reasoning to another model by encoding it inside a compaction," the researchers wrote.

First, the researchers captured encrypted reasoning from a specific task. They removed the original task from the history but kept the reasoning exactly as it was.

Next, they asked the model to put the reasoning under a "solution" key in a JSON object. They replaced every space with a tab character (\t). According to the post, this was enough to get past OpenAI's systems for checking whether requests followed its usage rules. The researchers explained: "Encrypted compaction items are still portable, so we sent the compaction item to gpt-4.1 and gpt-4o and asked each to output its input exactly as-is, which they happily did. We then decoded the JSON and converted tabs back into spaces."

The post said the method worked "because OpenAI's August mitigations only block replaying reasoning into a different model, and the compaction call runs on the same model that produced the reasoning. The attack worked at every reasoning effort we tried." In other words, the August protection stopped reasoning from being replayed in a different model, but did not stop this method of getting the reasoning back.

A difficult problem to solve

Tim Becker, Xint's lead researcher and the author of the blog post, said in an interview that some details about the timeline remain unknown. For example, the Stolen Thoughts post was published on August 10, but Becker only knows that OpenAI had addressed that earlier problem sometime before that date. He also said OpenAI did not announce when it put in the fix for the problem Xint reported. Unlike companies that release traditional software patches, AI companies can quietly change their models without announcing the changes. "It's sort of security through obscurity," he said. This phrase describes an approach that depends on people not knowing how something works or where its weaknesses are.

Becker said protecting this kind of model "is a very fundamentally hard problem to solve," mainly because today's models are changed so often and in so many ways.

The models "are a moving target and that is part of the problem. Any mitigation is going to be somewhat brittle," he said. This means a protection can become ineffective as a model changes.

Frank Dickson, principal analyst at Dickson Research, also pointed out that AI models often use their own features and outputs as part of their work. He said this helps explain why Xint's method succeeded.

"Compaction runs on the same model that produced the reasoning, so it sailed past the August mitigation, which only blocked replaying reasoning into a different model," he said. "Swapping spaces for tabs was enough to slip past the usage-policy classifiers. The vault was locked. The clerk holding the key would still read the letter aloud if you asked nicely. OpenAI did fix the bug it was shown. The trouble is that it was shown only one door." In other words, the fix closed one way of attacking the system, but another way remained open.

Risks for businesses

Analysts and consultants said this is an immediate, serious concern for major AI providers. It also creates a clear secondary risk for the chief information security officers (CISOs) who protect businesses.

Erik Avakian, a technical counselor at Infotech, said the Xint team "circumvented the protections by using legitimate functionality already built into the platform. For most enterprises, while this specific finding may not rise to the level of an immediate crisis, its significance is that it demonstrates how legitimate AI platform features can be combined to bypass narrowly designed safeguards."

"This can leave organizations exposed to risks that a vendor's individual patch may not fully address," he said. "It reinforces just how difficult it can be to fully address these vulnerabilities in situations where different components of an AI system interact in ways that create unintended risks."

Avakian said businesses should treat this finding as a warning.

"For CISOs, the takeaway is that a vendor's fix for a specific exploit shouldn't, in and of itself, be treated as proof that the underlying weakness has been comprehensively addressed," he said. "CISOs should press their AI providers to explain not only how a reported exploit was patched, but also how they were tested across interconnected features and services."

Brian Levine, a partner at consulting firm Control Risks, added that CISOs should pay attention to the wider pattern of flaws that keep needing fixes.

"Vendors are fixing the specific attack path that was reported, not the underlying class of weakness, and researchers are walking right back in through a side door," he said. "A patch notice is not proof the risk is gone. Stop treating an AI vendor's 'fixed' as the end of the conversation. Ask what variant testing was done and build regression notification into your contracts." He advised businesses to ask whether vendors tested other versions of the same attack and to include contract terms requiring them to report when a fix fails again.

Jeff Valdes, a director at consulting firm Acceligence, expressed similar concerns.

"This is what happens when you treat security as a game of whack-a-mole instead of fixing the underlying plumbing," Valdes said. "Patching a symptom while leaving the helper pipes wide open isn't a remediation strategy. It's just kicking the vulnerability down to the next release. If an attacker can smuggle the model's internal scratchpad across API boundaries to steal training data, what makes anyone think enterprise prompts and sensitive intermediate context are truly quarantined?" He warned that fixing one visible problem is not enough if the connections that make the problem possible remain open.

Pieter Arntz, a malware intelligence researcher at Malwarebytes, also said the OpenAI fix introduced during the summer was not enough. "The earlier mitigation was incomplete, which is a fair description of a gap," Arntz said.

"Retest after vendor fixes," he advised. "As the AWS case shows, a fix in one place can leave other paths open. Verify with your own testing rather than relying on the vendor's word."

Aman Mahapatra, chief strategy officer at Tribeca Softech, a technology consulting firm based in New York City, said it is almost impossible for the biggest AI providers not to deal with major security reports one piece at a time.

"AI vendors are patching exploits on a surface that grows by several features a month," he pointed out. "Until they fix properties instead of paths, CISOs should assume every AI security fix is partial, every encrypted blob is readable, and every vendor guardrail is one encoding trick from irrelevant. The trend is real, and 'the fix didn't work' undersells what is happening." He argued that companies need to fix the basic weaknesses that make attacks possible, rather than just blocking each known way of exploiting them.

He said that in both the OpenAI and AWS cases, the companies fixed the specific example of an attack but left the underlying weakness in place. "Patch the path, keep the property," he said. "On platforms assembled from composable features, every new feature is a fresh path to the same asset, and point fixes on a combinatorial surface will keep losing." In other words, when a platform has many features that work together, each new feature can create another way to reach the same sensitive information. Fixing only individual attack methods is unlikely to work over time.

CISOs are not used to AI agents

Mahapatra said the cybersecurity industry should learn from problems it has faced in the past.

"Cryptographers learned this two decades ago with padding oracles: any system that decrypts on a client's behalf and returns anything derived from the plaintext is an oracle. AI platforms are relearning it with a twist," he said. "Every LLM step that can see a secret and accept instructions is an oracle, and natural language is the most expressive oracle interface ever built." He was comparing AI systems to tools that can reveal information about secret data when users give them instructions.

Mahapatra noted that compaction did exactly what it was designed to do. The ability to move compaction data between models is useful because AI agent tools such as Codex allow users to switch models during a session. "The feature and the vulnerability are the same capability, which is why fixes keep arriving narrow," he said. The same feature that makes the system useful can also create a security risk.

But he argued that the bigger problem is that the way CISOs manage security has not caught up with systems that use AI agents to carry out tasks.

"AI platform flaws arrive with no CVE, no advisory, and no patch notes. Enterprise vulnerability management, built on CVE feeds and vendor advisories, is blind to its most consequential new suppliers," Mahapatra said. CVEs are standard identifiers used to track known security flaws. He said CISOs should require AI providers to report security issues in their contracts, assign someone to follow researchers' reports about the platforms their organizations use, and ask one question after every vendor fix: "did you fix the report or the class, and what variant analysis supports that answer?" In other words, they should ask whether the company fixed only the reported attack or the wider type of weakness, and what tests support its answer.

He also questioned whether the reward paid to Xint was large enough.

"A $6,000 bounty for a bug enabling exactly the industrial-scale distillation the NSA, CISA, and FBI warned about in a joint advisory in September is a pricing signal that researchers with other buyers will notice," he said. He warned that a reward of this size could influence researchers to take their findings to other buyers instead.