DeepSeek’s Sparse Attention model halves AI API costs

DeepSeek has introduced an experimental model called V3.2-exp, aiming to reduce the high costs of running AI systems in long-context scenarios. The standout feature of the model is DeepSeek Sparse Attention, a mechanism that makes transformer operations more efficient by carefully selecting which parts of the context to focus on. The system uses a “lightning indexer” to identify the most relevant sections of text and a “fine-grained token selection system” to choose specific tokens within those sections. This process allows the model to handle large amounts of context while using less server power.

 

In preliminary testing, DeepSeek reported that the approach could lower the cost of a simple API call by up to half in long-context tasks. While more studies are needed to confirm these results, the model is open-weight and available on Hugging Face, meaning outside researchers will soon be able to verify the claims. The release also came with a supporting academic paper published on GitHub.

 

The breakthrough addresses one of AI’s ongoing challenges: inference costs, which are the server expenses tied to running trained models as opposed to training them. By making transformers more efficient, DeepSeek is contributing to efforts across the industry to reduce operational costs while maintaining strong performance.

 

Based in China, DeepSeek has gained attention for its unconventional approach to AI research. Earlier this year, the company launched the R1 model, which used reinforcement learning to cut training expenses. That debut was expected by some to reshape AI training, though it did not lead to widespread change. Since then, DeepSeek has been quieter on the global stage, but V3.2-exp marks a return with a more practical focus.

 

While Sparse Attention may not generate the same buzz as R1, it has the potential to offer U.S. and global AI providers valuable methods for lowering expenses in long-context operations. By making AI models more efficient to run, the approach could ease one of the biggest financial barriers in scaling advanced AI systems.