Download
DeepSeek-V3.2-Exp is an experimental release of the DeepSeek model family, intended as a stepping stone toward the next generation architecture. The key innovation in this version is DeepSeek Sparse Attention (DSA), a sparse attention mechanism that aims to optimize training and inference efficiency in long-context settings without degrading output quality. According to the authors, they aligned the training setup of V3.2-Exp with V3.1-Terminus so that benchmark results remain largely comparable, even though the internal attention mechanism changes. In public evaluations across a variety of reasoning, code, and question-answering benchmarks (e.g. MMLU, LiveCodeBench, AIME, Codeforces, etc.), V3.2-Exp shows performance very close to or in some cases matching that of V3.1-Terminus. The repository includes tools and kernels to support the new sparse architecture—for instance, CUDA kernels, logit indexers, and open-source modules like FlashMLA and DeepGEMM are invoked for performance.
Features
- Adaptive sparse attention scheduling that dynamically adjusts sparsity patterns based on input sequence length
- Mixed dense + sparse attention fallback mode for hybrid use cases
- Memory-efficient checkpointing for ultra long contexts (e.g. >1M tokens)
- Performance profiling and visualization dashboard to analyze attention behavior
- Plugin interface to swap different sparse kernel backends (e.g. FlashMLA, DeepGEMM)
- Support for federated fine-tuning of the sparse model on decentralized data
Project Samples
