DeepSeek Opens DeepInfer, Presses OpenAI, Anthropic
DeepSeek has revealed the software it uses to run its AI servers. On October 20, 2026, the Chinese AI company released DeepInfer, its own system for running AI models, as free, open-source software under the Apache 2.0 license. The news gained wide attention after Gizmodo reported that DeepSeek was increasing the pressure on OpenAI and Anthropic. The release gives outside developers working software that DeepSeek uses to run its models for many users at once. It is more than a research paper or a collection of model files.
The first release has clear limits. DeepInfer currently supports only one model, DeepSeek-V4.1-Flash, and is software for running models on local or private computer systems, not a complete cloud service managed for customers. Even so, the move is important. A company based in Hangzhou, officially named Hangzhou DeepSeek Artificial Intelligence Basic Technology Research Co. Ltd., is sharing the kind of software that OpenAI and Anthropic have kept inside their own data centers. This matters to people following the global race to build AI systems and the technology that runs them.
What DeepSeek Released on October 20
DeepInfer is the name DeepSeek gives to its own system for running AI models. According to its official GitHub repository and related documents, the company published the software on October 20, 2026. Anyone can download, examine, and change it under the Apache 2.0 license. This license is one of the more flexible open-source licenses. It allows people to use the software commercially, change it, and share it without requiring them to release their own products as open-source software.
DeepSeek also released a smaller companion project called deepseek-recipe. This tool converts text into tokens and converts tokens back into text. Tokens are the pieces of text an AI model processes. The tool follows the same processing rules as DeepSeek's official API, so developers running DeepInfer on their own systems can expect token handling to match the company's hosted service. DeepSeek says it will add support for future models to the DeepInfer repository over time. This suggests the release is the start of an ongoing effort, rather than a one-time announcement.
SiliconANGLE's report on the release describes DeepInfer as the system behind DeepSeek's production services. This means the software comes from the same line of development as the code the company uses to handle real customer requests. It is not simply a demonstration project. Software built to serve real users must work under real-world conditions, which makes this release different from many open-source AI tools.
Inside DeepInfer: What the Framework Does
The documents released with DeepInfer describe features designed to run AI models at a large scale, rather than just test them on a single graphics processing unit (GPU). DeepInfer can spread model calculations across several chips and computers. It supports speculative decoding, a method that can speed up the creation of tokens. It also supports a technique DeepSeek calls “SWA bounded replay” and structured outputs, which help applications receive predictable, organized data instead of unrestricted text.
The system is designed to work with hardware from both Nvidia and Huawei Ascend. The New Stack's analysis of the release notes that DeepSeek's published code lists Nvidia's SM100 data-center GPUs, which belong to the Blackwell generation, and Huawei's Ascend 950 accelerators as supported hardware. DeepSeek gives both platforms a place in its documentation. For a Chinese company, this support for two chip makers is useful as well as meaningful. US export restrictions have made access to the newest Nvidia chips more difficult in China.
DeepSeek describes the project's purpose directly in its repository's README file. The company says, “DeepInfer is the high-performance inference framework that powers DeepSeek’s large-scale production services,” and explains that the open-source release provides offline model running with the main features needed to serve users at scale. A separate question-and-answer document about deployment in the same repository begins with a similar statement: “Today we are open-sourcing DeepInfer, our inference framework for large-scale cloud services.” These statements read like engineering documents explaining how the system works, rather than advertising written mainly for outside customers.
Why DeepSeek Is Opening Its Software Now
DeepSeek has previously explained why it did not release this code. According to The New Stack, the company said that “the engine is tightly coupled with our internal infrastructure, including cluster management tools, making it impractical for public deployment without significant modifications.” In other words, the software depended heavily on DeepSeek's own systems for managing groups of computers and could not easily be used elsewhere. Releasing it now appears to have required engineering work to remove or separate those internal dependencies so outside developers could run it.
DeepSeek says it made these changes to lower costs and make AI easier to access, rather than to challenge a particular competitor. In its announcement, the company said: “We’re sharing our production stack to help the community lower compute costs, reduce energy use, and make AI more affordable.” This focus on computing costs and energy use fits with how DeepSeek has presented its earlier releases of open model weights. The company has generally described those releases as ways to improve efficiency, rather than direct attacks on OpenAI or Anthropic.
Even without calling other companies competitors, DeepSeek's release could affect the market. Organizations that want to run DeepSeek-V4.1-Flash on their own hardware can now use documented software based on the system DeepSeek itself relies on. They can keep data within their own systems and security rules instead of sending it to an outside API. This offers more than simply downloading open model weights and hoping a community-built system, such as vLLM or another inference engine, can run the model as efficiently as possible.
DeepSeek-V4.1-Flash: The Model Behind the Release
DeepInfer's first version supports just one model: DeepSeek-V4.1-Flash. DeepSeek describes V4.1-Flash in its own product information as “smarter, faster, more efficient”. The model has also received independent coverage on this site. That includes a report on the HLE benchmark score of 42.6 for the V4.1-Flash-1017 checkpoint and another report on its DeepSWE score of 77.0 and the launch of a public search API.
Choosing V4.1-Flash instead of a larger flagship model appears to be a deliberate decision. Models with the Flash name are generally designed to respond faster and cost less for each token. This matches DeepInfer's stated goal of making it cheaper to run AI services at scale. It also gives developers a model they can try with the new software right away, instead of making them wait for support for a larger model that would need much more hardware to run on their own systems.
What Is Missing From the First Release
DeepSeek has not claimed that DeepInfer is a complete cloud platform. Documents released with the software, as described in reports about the launch, clearly state that some parts needed for a full production service are missing. These include KV-cache management, the ability to separate different stages of model processing across machines, sometimes called prefill/decode disaggregation, and load balancing across replicas. Replicas are separate running copies of a model service. These three features help turn a fast system for running models into a reliable cloud service that can adjust to changes in demand and handle sudden increases in traffic.
In practice, DeepInfer is currently closer to a high-performance core for serving AI models than a ready-made replacement for a managed cloud service. A company wanting to use it at a large scale will probably need to add its own caching system, manage traffic across copies of the service, and find a way to separate the initial processing stage, which requires heavy computation, from the token-generation stage, which uses a lot of memory. DeepSeek has indicated that it will add more features in future updates. For now, however, running DeepInfer yourself requires real systems-engineering work, not just a simple installation.
How This Compares With OpenAI and Anthropic
Neither OpenAI nor Anthropic has released an equivalent open-source version of the production software it uses to run its models. The reports available so far also do not show that either company has made a specific public statement about DeepInfer. We can, however, compare how the three companies let outside developers use their models. The table below shows these differences using only information the companies have publicly confirmed about how their systems are offered. It does not guess at technical details they have not disclosed.
| Dimension | DeepSeek (DeepInfer) | OpenAI | Anthropic |
|---|---|---|---|
| Serving code access | Open-sourced production inference framework (Apache 2.0) | Not open-sourced; API-only access | Not open-sourced; API-only access |
| Primary access model | Self-host or use DeepSeek’s hosted API | Managed API and ChatGPT products | Managed API and Claude products |
| Disclosed hardware support | Nvidia SM100-class GPUs and Huawei Ascend 950 NPUs | Not publicly itemized by accelerator model | Not publicly itemized by accelerator model |
| Model covered by this release | DeepSeek-V4.1-Flash only | Not applicable | Not applicable |
| Companion tooling released | deepseek-recipe (token encode/decode parity with the official API) | Not applicable | Not applicable |
The table shows the main difference without needing to invent numbers: DeepSeek is the only one of these three companies that has made its production software for running models available for others to inspect and run themselves. It is not yet clear whether this will lead customers to leave OpenAI's or Anthropic's managed products. The reports available so far do not answer that question, and this article does not claim that such a shift has happened.
The Open-Source Landscape for Running AI Models
DeepInfer is entering an area where several open-source tools already compete to run large language models efficiently. Its release should be understood in that wider context, rather than viewed on its own. Our earlier coverage of Modular's MAX system running the Kimi K2.5 model 5.5 times faster than vLLM shows how the software used to run models has become a competitive field in its own right, separate from the competition to build the models themselves.
| Framework | License | Primary hardware focus | Notable characteristic |
|---|---|---|---|
| DeepInfer (DeepSeek) | Apache 2.0 | Nvidia SM100-class GPUs, Huawei Ascend 950 NPUs | Built for one specific model family; mirrors DeepSeek’s own production stack |
| vLLM | Apache 2.0 | Primarily Nvidia GPUs, broader hardware support community-maintained | Widely adopted general-purpose serving engine for many open model families |
| TensorRT-LLM | Apache 2.0 (Nvidia-maintained) | Nvidia GPUs | Deep integration with Nvidia’s own compiler and kernel stack |
| SGLang | Apache 2.0 | Primarily Nvidia GPUs | Structured generation and scheduling optimizations for agentic workloads |
What sets DeepInfer apart is not proven speed. At the time of publication, the available reports do not provide independent benchmark tests comparing it with these other tools. The main difference is where it comes from. DeepInfer is not a community project created by outside developers trying to work out how to run a model efficiently. It is an internal tool released by the company that created the model, and it supports Huawei Ascend hardware from the start. This last point is especially important because people continue to question how AI companies in China can obtain advanced computing power.
Background: DeepSeek's Strategy With Open Model Weights
DeepSeek became known internationally for releasing open model weights that challenged the cost assumptions of the US AI industry. DeepInfer takes that strategy one step further into the software stack. Earlier releases gave developers the weights, which are the numerical values that help determine what a model knows and how it responds. DeepInfer gives them the software that runs those weights and affects how efficiently a model works in real-world use.
This is a meaningful next step. Having a model's weights does not guarantee that it will run efficiently. Many organizations have downloaded open model weights only to discover that running them well, at an acceptable speed and cost, requires advanced technical skills and infrastructure that their teams may not have. By publishing the software it uses internally, DeepSeek is trying to make it easier for anyone prepared to work with DeepSeek-V4.1-Flash to overcome this problem. This goes beyond publishing a model description and a collection of weight files on a hosting website.
Why Huawei Ascend Support Is Worth Watching
One of the most important details in this release is that DeepInfer supports Huawei Ascend 950 accelerators alongside Nvidia hardware from the very beginning. US export restrictions on advanced Nvidia chips being sold to China have been a lasting feature of the AI hardware market. As a result, Chinese AI companies have strong reasons to show that home-grown alternatives, such as Huawei's Ascend chips, can run advanced models competitively.
By including Ascend support in the first public version instead of adding it later, DeepSeek is sending a message to China's domestic market and to international buyers considering different hardware options: the framework is not built around just one chip maker. This also matters to companies outside China, especially in places where buying hardware from more than one supplier has become an important purchasing goal, regardless of how the AI competition between the US and China develops.
What This Means for Companies Buying AI Technology
For engineering teams deciding where to run AI workloads, DeepInfer adds a genuinely new choice rather than just another model to test. A company already evaluating DeepSeek-V4.1-Flash, perhaps because of the price differences described in our coverage of the 20x price gap between Claude, ChatGPT, and Gemini, now has a documented way to run that specific model on its own systems using software published by DeepSeek. It no longer has to depend only on DeepSeek's hosted API or on a serving system developed by the open-source community.
However, the missing features are important. A company without a skilled infrastructure team is unlikely to build a production-ready service using DeepInfer alone because it lacks KV-cache management and load balancing. The most likely early users are cloud providers, large companies with dedicated teams for building and running platforms, and AI infrastructure startups. These startups could turn the open-source software into packaged, supported products, much as companies have done with other open-source tools for running AI models.
Regulatory and Geopolitical Background
DeepInfer's release comes amid existing government restrictions. The US government has already restricted DeepSeek's tools. Earlier this year, the Department of Homeland Security banned DeepSeek's AI tools from federal systems, as described in our report on the DHS ban citing a 2018 federal procurement law. This does not change how DeepInfer works technically, but it affects who can legally use it. Government agencies and contractors covered by that restriction cannot choose to run DeepSeek's models themselves simply because the software is open-source.
For the wider commercial market, however, the Apache 2.0 license removes most legal barriers to adoption in places not covered by such restrictions. Open model weights and open AI infrastructure are becoming more common across the industry. Our separate analysis of the open-source AI bet reaching 86 percent, driven by releases from Beam and Mistral, has followed this broader trend. DeepInfer is part of a wider pattern in which AI companies compete partly by making more of their technology openly available.
What the Confirmed Reports Do Not Show
It is important to be clear about what has and has not been confirmed since DeepInfer was released. As of publication, none of the reports cited here provides a specific percentage showing how much money DeepInfer saves. They do not document a confirmed customer leaving an OpenAI or Anthropic product because of this release. They also do not provide an independent, direct comparison of DeepInfer's speed or response time with the internal systems OpenAI or Anthropic use to serve their models. Readers should be careful with any specific figures making these claims until a named news organization or one of the companies confirms them publicly.
What has been confirmed is more limited, but still important. DeepSeek has released usable software based on its production systems for a real model that is currently available. The software uses a genuinely flexible open-source license and supports hardware from more than one chip maker from the beginning. These are concrete facts, regardless of what happens next in the competition.
Five Predictions for What Happens Next
- Other Chinese AI companies may release more open-source infrastructure. DeepSeek has repeatedly set an example that other companies in China have tried to follow. Releasing software for running models is a logical next step for AI labs that want to remain competitive in openness.
- DeepInfer may gain KV-cache management and load balancing across model copies in an upcoming update. DeepSeek has explicitly said it will add support for future models and more framework components to the same repository. These two missing features are among the clearest obstacles to wider use in production.
- Specialist infrastructure startups may build products around DeepInfer for businesses. This could follow the pattern seen with other open-source serving tools, where companies sell technical support, performance tuning, and managed hosting built on top of free software.
- OpenAI and Anthropic may continue not to comment publicly on DeepInfer itself. This would fit their general pattern of not responding publicly to every release by a competitor. Their silence would not necessarily mean the release has no importance to them internally.
- Future DeepSeek models may receive DeepInfer support soon after their release. If this happens, the time between announcing a new model and making an efficient version available for people to run themselves could become shorter.
The Bigger Picture for the AI Infrastructure Market
DeepInfer shows that competition between AI companies is no longer just about benchmark scores or the number of parameters in a model. The infrastructure used to run AI, including scheduling software, specialized computing instructions called kernels, and model-serving code, determines how cheaply and reliably a model can operate. This infrastructure has become another major area of competition. Similar issues appear elsewhere in the industry, from the supply of computing chips to deals such as the $11.6 billion Akamai-Anthropic edge cloud deal, as well as continuing debates over tests that measure how efficiently AI models run.
For now, DeepInfer is an offline system that supports one model and two hardware platforms, and it still lacks important features. But it sets a precedent. The company behind one of the world's most closely watched open-weight model families is now willing to open up the software used to run that model. Whether this will pressure OpenAI or Anthropic to make a particular public response remains unknown. The available reports do not indicate that either company has announced plans to respond.
Frequently Asked Questions
What is DeepInfer?
DeepInfer is the system DeepSeek says it uses internally to run its AI services. The company released the software as open-source code on October 20, 2026, under the Apache 2.0 license.
Which models does DeepInfer support?
The first release supports only DeepSeek-V4.1-Flash. DeepSeek says it will add code for running future models to the same repository over time.
What license does DeepInfer use?
It uses Apache 2.0, an open-source license that allows people and businesses to use, change, and share the software.
Does DeepInfer work with Nvidia GPUs?
Yes. The release documents list support for Nvidia's SM100-class data-center GPUs from the Blackwell generation, as well as Huawei Ascend 950 accelerators.
Can any company use DeepInfer for production services right away?
Not with DeepInfer alone. DeepSeek's documents say the current release lacks KV-cache management, the ability to separate processing stages across different machines, and load balancing across model copies. These features are generally needed to build a complete production service that can grow to handle more users.
What is deepseek-recipe?
It is a companion project released with DeepInfer. It converts text into tokens and tokens back into text, using the same processing rules as DeepSeek's official API. This helps models running on a company's own systems behave consistently with DeepSeek's hosted service.
Have OpenAI or Anthropic responded to the DeepInfer release?
As of publication, no specific public response from either company has been reported. Current coverage discusses DeepInfer as a possible source of competitive pressure but does not report a direct reaction from either company.
Why did DeepSeek wait to release this software?
DeepSeek previously explained that the system was closely connected to its internal infrastructure and tools for managing groups of computers. This made it difficult for others to use without major changes. The release suggests that the company completed enough of that work to make the software available to the public.