OpenAI's rogue AI incident raises ad fraud fears as autonomous media buying gains ground

Kartik Sharma, founder of Rankinllm.ai, has a warning for advertisers who are preparing to let artificial intelligence make media-buying decisions: AI agents do not need to break down to cause financial harm. Loose spending limits, false advertising data, or harmful instructions could be enough.

“An agent doesn’t wake up one morning and decide to blow the budget. It works within the limits you give it. So if it spends too much or places ads in unsafe spaces, either the limits were too loose or someone got into the system,” Sharma told Storyboard18.

His warning comes as the advertising industry moves closer to letting AI systems buy and manage advertising on their own. At the same time, recent cybersecurity incidents have shown how difficult it can be to control AI systems that work with little human supervision.

On October 15, India's Computer Emergency Response Team (CERT-In) issued a warning about cases in which AI agents being researched and tested went beyond their authorised limits and accessed Indian information and communication technology systems. The agency also found that cybercriminals were using open-source AI agent tools to investigate systems, exploit weaknesses, and carry out attacks aimed at making money.

According to CERT-In, the affected organisations did not discover the unauthorised access themselves. It was found through information shared by AI model providers, investigations of digital evidence, and active searches for security threats.

The warning follows an incident in July involving OpenAI and Hugging Face, in which AI agents carrying out internal cybersecurity tests bypassed safety measures, gained internet access they were not allowed to have, and broke into parts of OpenAI's research systems and Hugging Face's systems.

OpenAI's investigation found that the agents had taken actions outside the limits of their assigned tasks while trying to complete cybersecurity tests. The company later improved its safety measures and shared more technical details in August.

Neither incident involved advertising campaigns. However, industry leaders believe the incidents show risks that could have serious financial effects when AI agents are allowed to choose advertising space, adjust spending, and eventually commit advertisers' money.

The timing matters. On September 22, IAB Tech Lab introduced AAMP 3.0, a framework designed to create common rules for how AI agents working for advertisers and publishers share advertising proposals, assess advertising space, and move towards buying agreements.

Its OpenProposal specification is open for public comments until October 22. This brings the industry closer to a system where AI agents could take part in media transactions that human planners and buyers have traditionally handled.

Could AI agents spend advertising budgets without permission?

Sharma says one of the biggest weaknesses is prompt injection. This happens when harmful instructions hidden in outside information change how an AI agent behaves.

An agent checking publishers or handling campaign information could come across false or altered data. This could lead it to make decisions that go against what the advertiser wanted.

“Security teams already rank prompt injection as the number one risk in AI applications, and an agent that can be fed a bad instruction can do real damage very quickly,” Sharma said.

However, harmful results do not always require someone to interfere with the system.

Akshay Mathur, founder and CEO of Unpromptd, said it is important to understand the difference between automation and full independence. Today, most advertising platforms still rely on people to set budgets, campaign goals, and targeting rules. Algorithms then improve results within those limits.

But as AI agents gain more freedom to act, they might focus on individual performance measures while ignoring wider business goals.

Mathur gave a made-up example of a retail campaign during a festive season. An AI agent notices that one platform is getting more conversions, meaning more people are taking the desired action, and keeps moving money towards that platform.

Each decision might seem reasonable on its own. But over time, other channels could receive too little funding, weakening the advertiser's overall marketing plan.

“The real risk isn’t necessarily that AI will disobey instructions. It’s that it may follow them too efficiently, without understanding the commercial context behind a marketing decision,” Mathur said.

Industry executive predicts 3–4 major AI-related ad fraud cases within a year

Mohit Ghate, co-founder of Wit & Chai, believes the advertising industry is underestimating how hard it could be to supervise thousands of AI-generated campaigns and groups of ads.

“See, the scale at which AI can produce creative options, ad sets, campaigns, etc. should scare all advertisers. It will not be humanly possible to look at 3000 ad sets, and AI will need only 5 of them to be bad actors to make it work,” Ghate said.

He argued that the industry has focused too much on protecting brands from unsafe content while overlooking the wider risk of losing control over how campaigns are run.

“The problem everyone is looking at is brand safety, while the real problem is lack of control,” he said.

Ghate predicts that the advertising industry could see at least three to four major cases of ad fraud involving AI within the next year. He believes these cases could involve advertisers with good intentions and AI systems acting on instructions that are not clearly defined.

He is concerned that an advertiser might give an AI agent a vague goal, such as getting as many conversions as possible or making a campaign more efficient, without clearly explaining what kinds of advertising space, traffic quality, or spending behaviour are acceptable.

The agent might then try to achieve that goal in ways that lead to unwanted business results.

This prediction is uncertain, but it reflects a wider concern among industry leaders that letting AI make decisions on its own could make existing weaknesses in digital advertising worse.

Sharma pointed to India's existing problem with invalid advertising traffic.

According to Pixalate, about 42% of India's mobile app programmatic advertising traffic measured in the second quarter of 2025 was invalid. This was the highest rate among the five Asia-Pacific markets included in its analysis.

“An agent buys on the signals it’s given, and if those signals are fake, it will still spend on them. That’s the part people tend to underestimate,” Sharma said.

Fraudsters could manipulate AI agents before an ad is even shown

Mathur believes the next stage of advertising fraud could involve tricking AI systems that make buying decisions, rather than simply creating fake ad views.

For example, a dishonest publisher could create a convincing business profile using made-up information about its audience, exaggerated estimates of how many people it reaches, and altered performance figures.

If an AI agent trusts this information while choosing potential advertising partners, the publisher could influence where the advertiser's money goes before any ad is shown. This risk could grow as AI agents start visiting publisher websites, using third-party measurement tools, and accessing outside datasets on their own. Mathur also warned that harmful instructions hidden in this information could change how an agent behaves.

However, he acknowledged that AI could also improve fraud detection by examining huge amounts of advertising data and spotting suspicious patterns faster than people working manually.

Nikhil Kumar, former chief growth and marketing officer at Affle, compared these risks with long-standing problems in programmatic advertising. In this type of advertising, automated systems sometimes place ads next to unsuitable content.

He gave a hypothetical example of an advertisement for fairness cream appearing beside a news report about an investigation into racial discrimination.

An automated platform might spot a matching keyword but fail to understand the wider meaning of the article or the possible harm to the brand's reputation.

Kumar warned that with AI agents, similar mistakes could trigger a series of actions across several systems before campaign managers notice the problem.

Who pays when an AI agent makes a costly media-buying decision?

As AI systems move closer to buying advertising without human approval, another question remains unresolved: who is financially responsible when something goes wrong?

Imagine an AI agent agrees to buy an expensive premium advertising placement without getting the advertiser's approval. Would the advertiser have to pay because it authorised the system? Would the agency running the campaign be responsible? Or could the demand-side platform or AI technology provider be held liable?

Sharma said there is no single answer for every case. Responsibility largely depends on the contracts between the parties.

“Usually the advertiser goes back to whoever it signed with, the agency or the platform vendor. That vendor might then pass it on to the AI provider if its own agreement has back-to-back liability built in,” Sharma explained.

He recommended clearly setting out spending powers and financial responsibility before AI agents start running campaigns on their own.

Mathur also argued that responsibility should depend on how much control each party has handed over or taken on.

If an agency sets up and supervises an AI agent, it must take responsibility for the rules and checks that govern its use. In the same way, a technology platform that fails to enforce agreed spending limits should not automatically avoid responsibility.

He expects contracts to include more rules written specifically for AI, covering approval limits, spending powers, liability, and how disputes will be settled.

Ghate believes responsibility should be shared across the advertising industry, although agencies and technology providers may face more scrutiny at first as disagreements arise.

Is human oversight becoming a formality?

Although advertisers increasingly use algorithms to improve campaigns, Mathur warned against assuming that the industry has already moved to fully independent AI advertising.

Most major platforms still require advertisers and agencies to set budgets, campaign goals, and operating limits.

However, another problem could arise when people approve AI recommendations without properly checking them. A campaign manager handling hundreds of AI-generated suggestions might technically have the final say, but that safeguard becomes useless if the manager routinely approves them without review.

Sharma recommended setting strict spending limits, deciding which advertising channels agents can use, limiting the tests they can run independently, and creating ways for people to take back control whenever needed.

Kumar also recommended constant monitoring, approved lists of advertising space, checks to make sure ads appear in suitable contexts, records that show how decisions were made, and immediate action when spending looks unusual.

This challenge is especially important as industry standards begin to define how AI agents involved in buying and selling advertising work together.

IAB Tech Lab's AAMP 3.0 is designed to make the process of proposing and assessing advertising opportunities work more smoothly across different systems. But common technical standards alone cannot settle questions about who pays for losses, how to deal with false information, or how much power AI agents should have.

For advertisers, the OpenAI–Hugging Face incident and CERT-In's latest warning show what can happen when AI agents act beyond the limits set for them.

There is no evidence from these incidents that an AI agent buying advertising has already caused a similar security breach or a major unauthorised advertising expense. But as AI agents move from recommending campaign decisions to potentially carrying them out, unclear instructions could cause more than technical problems. They could also put advertising budgets at risk.

The question for advertisers is therefore not just whether AI can buy advertising more efficiently. It is also who controls its decisions, who notices mistakes, and who ultimately pays when something goes wrong.