GPT-6 Astra Outperforms Claude Sonnet 5.5 on Science Tasks While Using Far Fewer Tokens
GPT-6 Astra gets lower scores than Claude Sonnet 5.5 on the broad intelligence rankings from OpenAI and Anthropic. But it keeps doing better on the test that looks most like real scientific research. It also uses only a small part of the tokens Sonnet needs to finish the same kind of work.
On Terminal-Bench-Science, a test designed to copy the steps used in real scientific research, OpenAI's GPT-6 Astra scores 63%, while Claude Sonnet 5.5 scores 53%, according to Artificial Analysis. This is a clear difference on a test that matters to people using AI for research and development (R&D). But the score is not the main thing Anthropic should worry about. The bigger issue is how many tokens each model uses to get its result.
When Sonnet 5.5 uses its highest reasoning setting, it produces about 193,000 output tokens to finish a typical task on this test. Astra needs about 27,000. That is a difference of almost seven times, not the 22 times some people have claimed. Even so, this difference can change the cost of using the models. Astra costs about $3.26 per task, compared with $7.60 for Sonnet, because it uses far fewer tokens. This is true even though Anthropic charges much less for each of Sonnet's tokens than OpenAI charges for Astra's. Sonnet's cheaper tokens do not make up for using seven times as many.
For the past year, people have often judged AI models by their places on leaderboards. The model at the top of the Artificial Analysis Intelligence Index gets the headlines, and Sonnet 5.5 is still ahead, with 56 points compared with Astra's 53. But a company founder who runs thousands of research questions each day may care less about an overall score and more about the bill at the end of the month. For these users, how many tokens a model uses is becoming just as important as what it can do.
OpenAI's pricing seems designed to take advantage of this change. Astra is expensive if you look at the cost of each token. OpenAI charges $10 for every million input tokens and $50 for every million output tokens. That is five times Sonnet 5.5's listed price. But a model that uses 27,000 tokens instead of 193,000 can still cost less overall. It does not need cheaper tokens; it needs to use fewer of them. This gives OpenAI a different kind of advantage from the one Anthropic has usually competed on, and cutting prices alone may not be enough to catch up.
OpenAI pushed this idea further with GPT-6.1 Sol, which was released on September 29. It costs $2 per million input tokens and $10 per million output tokens, or one-fifth of Astra's prices. On Terminal-Bench-Science, Sol costs about $5.47 to complete a task. Astra costs $23.80, while Claude Opus 5.5, Anthropic's larger model, costs $23.21. These figures come from benchmark data covered by DataCamp and ITBrief when Sol was released. Sol does not score higher than Astra, but it comes close while costing only one-fifth as much. This kind of trade-off matters to a startup that runs thousands of automatic research tasks each day.
This does not mean Sonnet 5.5 is a bad model. It still scores higher on Artificial Analysis's broad intelligence index. Since the Claude 3 era, Anthropic has also promoted the depth of its models' reasoning as a key strength. But using more tokens has a cost, even if a task does not take much longer in real time. According to Artificial Analysis's own benchmark report, Sonnet's 193,000 output tokens per task is the highest amount the company has measured from any leading AI model so far. This is not a small difference. It means the model produces much more text while working through each task than almost any other model on the market.
What this means for anyone building on these models
If you run an AI infrastructure company or need to choose a standard model for a research and development system, the main question used to be, "Which model gets the highest score?" Now it is becoming, "Which model costs the least to give a correct answer?" These questions can have different answers. A lab might need thousands of research tasks completed each day, such as checking possible drugs, simulating materials, or running scientific tools that use a lot of code. For that lab, a sevenfold difference in token use could make a much bigger difference to its cloud bill than a 10-point gap on a general benchmark.
OpenAI has also made this a question of safety, not just cost. Astra is the first model OpenAI has given the "Critical" cybersecurity label under its own Preparedness Framework. OpenAI said it found that Astra could find previously unknown security weaknesses on its own and create working attacks to exploit them, with very little help from people. The model's ability to work efficiently on tasks that attack computer systems is the same kind of efficiency it shows in scientific work, although the goal is different.
Anthropic is still working to compete. Sonnet 5.5 really does score higher on the broad intelligence test. If Anthropic can reduce the number of tokens Sonnet needs for each task, it could keep its reasoning advantage while making the model cheaper to use. But for people who pay for every API request, Astra's lower token use may matter more right now than its score on a benchmark. The cost of finishing each task, rather than a model's place on a leaderboard, may decide which AI model becomes the standard choice for future automated research.