Stolen AI Keys Ran Up a 740,000 Dollar Bill - Rotation Is the Real Fix

HackerNoon

Source published: Site published:

Original article · فارسی

The attackers wanted the keys, not the data

On November 10, Railway published its final report about the October attack on its platform. The attackers were able to unlock environment variables in up to 1.85 percent of workspaces. That could mean as many as about 100,000 workspaces. They used stolen keys to run up an estimated $740,000 in unauthorized AI costs. They also reset account tokens connected to 1,852 users. These included credentials for AI agents that let customers use their own keys, as well as GitHub OAuth tokens. Customer databases were not accessed. Nobody logged into a Railway account using the stolen credentials. The attackers were not after customer data. They wanted the keys.

This difference is more important than the breach itself. When a database is stolen, private information is exposed. When a key is stolen, someone can keep spending money. A stolen database record does nothing on its own. But a stolen API key keeps working until someone cancels it, and every request made with it costs the victim money. Railway says the attackers wrote database searches specifically to find AI provider keys. The company believes the keys were used with services that pass along or resell AI access, possibly to help train models that copy the behavior of other models.

I spend my working life dealing with this kind of problem from the other side. I build systems that watch API traffic almost in real time, handling around 800,000 transactions every second, with tools that spot unusual activity. This attack lasted about 26 hours. The bill reached $740,000. There was a sign of the attack somewhere during that time.

How a build system escape turned into stolen credentials

The attack began at 07:28 UTC on October 20. The attacker used a previously unknown weakness to partly escape the file restrictions around a build process. This let the attacker read files on the host virtual machine. This was the first failure, and it was a failure of protection. A build runs code that cannot automatically be trusted. It runs whatever a customer has uploaded. If a build can read files on its host machine, the isolation system is broken. Everything that happens after that is an attempt to limit the damage.

The second failure involved trust settings. At 03:26 UTC on October 21, a mistake in the OpenID Connect trust configuration exposed a temporary cloud credential. The attacker used it to collect internal credentials and a private SSH key. Then the attacker entered a machine that managed customer workloads.

From there, the attacker read stored records from 353 builds. These records included the variables and secret values given to those builds. Railway says its logs can show which environments were exposed, but cannot show exactly which individual variables inside them were unlocked. That is why 100,000 is the highest possible number, not a confirmed count.

If your records cannot tell you exactly which secret was stolen, you must assume that every secret in the affected area was stolen. Railway told all customers to cancel and replace any sensitive variable stored on the platform before October 21.

One of the most worrying details is the software used in the attack. Railway describes the incident as a race between AI agents, with the attackers using models that had their safety limits removed through an agent system similar to ARTEX. Engineers noticed unusual requests to the registry API at 13:58 UTC on October 20. This was several hours after the first attack. The attackers were a group of AI agents, not a person typing commands by hand. The attackers were completely removed from the system at 09:24 UTC on October 21. But the difference in speed between attackers and defenders is now a lasting problem. AI agents make it possible for attackers to work at a much larger scale.

Defenders must use systems that can work at the same scale. Otherwise, they will lose simply because they are too slow.

The bill itself is a warning sign

Here is what should change the way teams think about API keys. The $740,000 bill is not just money lost. It is also a warning that the security system noticed too late. In a well-monitored system, a stolen key should give itself away through unusual activity. Spending suddenly increases. The pattern of requests changes. A key that normally makes a few hundred requests each day from one region suddenly makes millions of requests from computer systems the customer has never used.

Every key should have a normal activity pattern. This should include its usual number of requests, the amount of text or other data processed in each request, the models it normally uses, the networks it connects from, and the hours when it is usually active. The system should alert someone whenever activity moves far away from this normal pattern. The alert should also tell the team what action to take, not just show a chart. Detection must happen automatically because the time between a key being stolen and an attacker using it can now be measured in minutes. The attackers began at 07:28 UTC, but engineers did not spot unusual registry requests until 13:58 UTC. That gave the attackers six and a half hours to move around freely.

The refunds also show why this matters. AWS Bedrock and Google Gemini have started refunding fraudulent charges. OpenAI is reviewing claims one by one. Railway has also changed its earlier position from October 26, when it told customers it could not refund spending made with stolen keys. It now says it will pay for charges that AI providers refuse to refund.

A refund policy does not protect a system. Real protection comes from spending limits, alerts, and an emergency switch that can disable a key. These controls must already be in place before an attack happens, not added afterward.

The same kind of theft is happening everywhere

Railway was not the only company affected. OpenCode, Warp, and Fireworks AI reported similar thefts between October 6 and 23. Wiz has described this wider pattern of stolen credentials using the name Black October. Three platforms had three different security failures, but the attackers wanted the same thing.

The agent layer is the software that connects an AI model to a real working environment. This layer collects many important secrets in one place. Every agent needs keys to do its work. That means every platform that runs AI agents also becomes a place where valuable keys are stored.

What I would change on Monday

I would make four changes, starting with the most important. First, limit what every key can do. A key that can use any model, connect from any network, and spend unlimited money is like a blank check. Limit the models it can use, limit the networks it can connect from, and set a firm spending limit that starts low. A spending limit puts a maximum on the possible loss. That is especially important when it may take hours to notice an attack.

Second, replace keys regularly instead of waiting until something looks wrong. Railway had to reset 1,852 connected tokens after the attack. Regular key replacement would have reduced the value of stolen keys even before anyone discovered the breach. A key that stops working every 30 days is less useful to a thief than one that never expires.

Third, watch how quickly money is being spent, not just whether requests fail. Most API monitoring systems look for problems such as 5xx errors, timeouts, and rejected requests caused by rate limits. But a stolen key does not have to fail. It can work perfectly while making huge numbers of requests. The important alert is one that goes off when a key spends ten times its normal daily amount in just one hour. Fourth, treat the build system as something that must never be trusted automatically.

A build runs code that cannot be trusted, so its isolation must be checked all the time rather than simply assumed to be working. Railway's escape happened because of a weakness nobody had discovered before. This means a normal security checklist would not have caught it. Monitoring what happens while the system runs can help. If a build tries to access files outside its allowed area, the system should raise an alert within seconds.

The main lesson is that AI keys have become some of the most valuable credentials a company owns, but many companies protect them as if they were unimportant. These keys are stored in environment variables, often never expire, work from anywhere, and can create bills for every token used. Every team that stores an AI provider key should read the report and ask one question: If our keys were stolen tonight, how would we find out, and how quickly could we disable them? If the answer is that someone would check a dashboard the next morning, that is far too slow. Attackers are now targeting the keys. Companies must protect them accordingly.