OpenAI releases Decisions API, entering race for fast decision models

OpenAI released its Decisions API to all customers on Thursday. The company is entering a market for fast, low-cost AI that sorts information into categories and makes decisions. TypeSafe, another company in this market, was recently valued at $7.6 billion.

TypeSafe's Jev model helped create this new kind of AI tool. These tools use deep learning to act like very fast decision-makers. They are called System One models. When given several choices, they quickly pick one and give a score showing how confident they are in that choice. Software developers have started using these models instead of large language models, or LLMs, for many tasks. This helps their software work faster and costs much less.

Nikunj Handa, who leads API products at OpenAI, said his coworkers saw Jev and immediately understood how useful it could be. "We have like a million use cases for this," Handa said. His team worked quickly to create a competing product. Instead of building a special decision model from scratch and adjusting it for the task, they changed ChatGPT Luna so it could give answers faster and cost less to run.

Handa said the biggest improvement with the Decisions API is its speed. Tasks that once took one to three seconds can now take only 100, 120, or 180 milliseconds. A millisecond is one-thousandth of a second. OpenAI first announced the API at its developer event on September 29. When the company released it to everyone, it also added support for voice and images. OpenAI also arranged its computing resources in different parts of the world to help people get faster responses wherever they live.

Some of the first customers are Box, Datadog, Ramp, and Speak. Engineers at these companies started using the Decisions API instead of OpenAI's regular large language models so their systems could handle more tasks in less time. Andrew Hsu, the chief technology officer at Speak, a company that makes AI language tutors, said the API helped Speak reduce the time between actions from 1.4 seconds to 0.13 seconds. OpenAI also used the same technology to make its Dots voice assistant respond more quickly.

There is still no clear, widely accepted way to decide which decision model is best. Different jobs need different things. Some need smarter answers, while others need lower costs or faster responses. It also matters how well a model's confidence scores match how often it is actually correct. The best balance depends on what the model is being used for.

Diogo Almeida, the CEO of TypeSafe, does not fully trust tests that compare AI models. He encourages customers to try models on their own tasks instead. Even so, TypeSafe published its own rankings last week when it released the newest version of Jev. This version can also work with images and includes smart-if statements, which help it make decisions based on conditions.

Two tests used to compare these models are the AIM System One Index and the Artificial Analysis Decision Index. Liquid's D1 Ultra and Base Lab's Elephant-1 have the highest scores on these tests. The newest Jev model ranks seventh on the first test and third on the second. OpenAI's Decision Model ranks eighth and 14th. For now, people using OpenRouter ask for TypeSafe's models more often than other decision models.

One of the most entertaining tests is called JevMan. In this test, decision models play the classic video game Pac-Man. The newest Jev model is currently in first place. Felix Wunderlich, the chief growth officer at Opper AI, the company that created the test, explained why the game is useful. Pac-Man forces a model to make quick, important decisions again and again. Whenever the character reaches a place where it can turn, the model must choose which way to go in just a few hundred milliseconds.

OpenAI's model has an advantage because it can work with different kinds of information, including images. Handa said Luna is especially strong at sorting images into categories and has shown better results when measuring how well its confidence scores match its accuracy. Customers also want the Decisions API to have more flexible ways to produce images.

Handa believes the API will become even faster. He asked whether OpenAI could reduce response times to about 60 milliseconds. If that happens, developers might be able to make a separate API request for every single frame of a video.

As decision models become faster and cheaper, people may start using them much more often. This follows an idea named after the economist Jevons. He showed that when people find a more efficient way to use a resource, they may end up using that resource more widely rather than less. Handa believes decision models will lead to a huge number of new uses. Many of these possibilities may have been there all along, but people did not notice them until fast decision models made them practical.