What Is DeepSeek Actually Migrating?

From “the model runs” to “the full compute path executes efficiently”

Abstract: If you understand migration as “load the weights, run generation,” you cannot explain why this batch of open-source components exists. A single inference first batches, then computes layer by layer: within each layer, the indexer first picks the context positions and sparse attention reads KV by that index; then routing, dispatch, grouped GEMM, and combine complete the expert computation. Components exchange data under shared alignment and layout agreements. Beyond API compatibility, the scaling-factor arrangement, in-kernel pipelines, and synchronization still have to be adapted for Ascend. This piece first settles the two ledgers (37B activated per token, 671B resident for the whole service), then draws the line between what can be reused and what must be redone.

On September 30, 2026, DeepSeek open-sourced a batch of inference components for Huawei Ascend:

• DeepGEMM-Ascend, matrix-multiplication kernels;
• DeepEP-Ascend, expert-parallel communication;
• FlashMLA, sparse attention kernels;
• TileKernels, auxiliary kernels;
• DeepJIT, kernel compilation and loading.

All of these previously had NVIDIA versions. With this release, the major stages of an inference software stack have public implementations on both platforms for the first time. The coverage has boundaries: FlashMLA’s Ascend support currently covers only the sparse attention kernels, and its fused kernel and dense attention are still marked CUDA-only (FlashMLA README). But the main body is in place.

A question follows: DeepSeek’s models already run on Ascend, so why open-source a full set of matrix multiplication, attention, communication, and compilation runtimes?

The answer lies in the difference between two things: the model defines what must be computed; the software stack decides how that computation lands on the hardware. “It can produce text” only proves that a compute path exists. Whether that path runs fast is a different question, and deployment cost is decided by exactly that. This piece lays the whole system out and gives a judgment framework for “what can be reused and what must be redone.” The following six parts cover data layouts and matrix shapes, expert-parallel communication, the attention kernel, correctness verification, economics, and ecosystem sustainability.

---

1. First, Set Up the System: What One Inference Goes Through

An inference service batches first: multiple requests’ inputs are stitched into one batch and computed layer by layer. One easily overlooked fact: prefill and decode do completely different work.

• Prefill reads an entire input at once. Each request contributes a long stretch of contiguous positions, and the cost of reading weights is amortized across many positions. It also produces the KV cache, the intermediate record of “already-processed context” that later steps reuse.
• Decode generates one token per request per step. Few new positions participate, yet the whole context’s KV is read again.

The two stages differ sharply in data reuse and read/write ratios. Actual batch size further depends on concurrency, input lengths, and splitting strategy; it cannot be derived from “one token per request per step.” Section 4 gives official results for the same indexer kernel at two query scales; the shape effect is worth a close look.

Inside a layer, take a typical MoE layer: the attention segment decides “which context each position looks at”; the MoE segment decides “which experts compute each token.” Four layers of the stack each hold one part:

• the inference framework handles batching, scheduling, and the KV cache;
• kernel libraries execute each computation;
• the communication library moves tokens between devices;
• the compile runtime turns kernel source into loadable device binaries.

This release covers the main stages of the latter three.

String the two intra-layer paths together, and the five repositories fall into place:

• Attention path: the indexer calls DeepGEMM’s MQA logits kernel to score every context position for each query; position selection then produces the index table. FlashMLA reads KV sparsely by that index and computes attention. The index-table format and the KV layout form one contract shared across both repositories.
• MoE path: TileKernels’ router picks experts for each token; DeepEP’s dispatch sends inputs to the experts’ devices; DeepGEMM’s grouped GEMM computes (routing weights applied once here); DeepEP’s combine merges the weighted results back into the original requests.

After batching, a request flows through the five repositories mainly along these two paths. “Mainly,” because the framework’s scheduling and other kernels are also on the path; this piece only expands what this release covers. On compilation, DeepJIT compiles, caches, and loads the hand-written kernels of DeepGEMM and DeepEP; TileKernels takes the TileLang route (DeepGEMM also has TileLang implementations). Section 5 covers how the two toolchains divide the work.

One more piece of background belongs on this map: DeepSeek models were not previously unable to run on Ascend. V3.2-Exp’s release materials already provide an SGLang NPU image entry. What was released this time is a different thing: native high-performance kernels for Ascend 950 and their public interfaces became openly readable implementations that migration teams can study and reuse directly. Which new kernels each framework version actually adopted is not stated in public materials. Having an image entry also does not mean having a verified new runtime stack; framework-level integration and acceptance remain the user’s work. This distinction runs through the whole series.

---

2. Resource Structure: The Per-Token Ledger, and the Whole Service’s Ledger

To understand MoE deployment, first settle two ledgers. Take DeepSeek-V3: 671B total parameters, 37B activated per token (DeepSeek-V3 repo). V3 is cited only to explain resource structure; the latest FlashMLA targets V4.1.

A rough estimate involving only parameter count and storage precision: 671 × 10⁹ parameters × 2 bytes ≈ 1.34 TB. This assumes all parameters stored at two bytes; it excludes the KV cache, runtime workspace, and communication buffers, and it does not equal any official quantized checkpoint’s size. The official V3 weights ship mainly in FP8, so actual checkpoints are smaller; the two-byte figure is a conservative upper bound. Its job is just to translate “total parameter count” into the first number a deployment faces.

Already at this step, the difference between the two ledgers is visible:

The parameter count each token participates in, and the parameter count the whole service must hold, are two different ledgers. 37B measures how many parameters one token’s forward pass touches; it cannot by itself determine execution cost, because attention compute also scales with context length. 1.34 TB, under the two-byte assumption, is the storage volume of all weights and decides how much memory the fleet must jointly hold.

In a deployment with expert weights resident across devices, even a forward pass touching few experts must keep answering four questions:

1. Placement: where do all experts live;
2. Routing: where does the current token go;
3. Load observation and rebalancing: which devices carry more expert computation;
4. Merge and synchronization: how results return to their requests.

Later parts each answer one of these.

Write the four questions’ objects as a budget:

M_total = M_weights + M_KV + M_workspace + M_comm + M_reserve

This is an accounting frame, not any framework’s exact formula. Its use is to translate “some kernel got faster” into “which budget line it moved, what it squeezed or released.” In memory-constrained deployments, time-saving and memory-saving optimizations are worth different amounts. Every later part reuses this frame.

---

3. One MoE Layer’s Full Round Trip: Where Interfaces Meet, Where Fusion Sits

The two ledgers decide what the service must hold. This section walks the MoE path to see how the five repositories’ interfaces connect; it is the deepest cross-repository coupling in the piece.

It starts at routing. TileKernels provides top-k expert selection and scoring kernels, outputting each token’s expert ids and weights. It also bundles quantization (per-token/per-block FP8/FP4 conversion, fused SwiGLU quantization) and kernels such as mHC and Engram, all written in the multi-backend DSL TileLang with runtime backend selection (TileKernels README).

Then dispatch. Routing results go to DeepEP-Ascend’s dispatch, which groups each token’s input by expert and sends it out. A few interface constraints look like “details” but decide the engineering shape:

• Communication is not ad hoc; it is organized around a long-lived EPBuffer. One buffer per expert-parallel group, reused across layers and microbatches; all ranks must agree on token capacity; before resizing, reuse, or destruction, all outstanding operations must have completed.
• Dispatch’s alignment parameter must match the grouped GEMM’s token-alignment granularity. The communication library organizes sent rows and zero padding by that granularity so the downstream grouped GEMM can consume the received data directly.

The communication library and the compute library are not loosely coupled byte pipes; both implement against the same alignment agreement. Pre-send expansion and zero padding, expert-side device counting, and similar options are fixed steps on this path; Part 3 expands them (DeepEP README).

On the expert side, computation. DeepGEMM-Ascend’s grouped GEMM organizes matrix multiplication by the rows each expert receives; one expert network is two grouped GEMMs sandwiching a SwiGLU. Per the README’s standard example, expert computation applies the received routing weights once and outputs BF16. DeepEP’s combine then merges results back: it merges already-weighted results, not another multiplication by the weights.

The README’s standard usage is itself an interleaved schedule: initiate dispatch, compute whatever is independent during the wait (the shared expert in the example), then wait to collect the inputs; after expert computation, issue combine with the shared expert’s output folded in as bias. Communication and computation are not “first one, then the other”; they are a set of interleaveable operations managed explicitly by the handle (the record of the dispatch structure) and by wait timing.

Fusion is another choice. Section 1’s map lists communication and computation in separate components by function; these stages can also be merged into one operator. The clearest example is DeepGEMM-Ascend’s MegaMoE, which puts EP dispatch, two grouped GEMMs, SwiGLU, and combine into a single operator. Official benchmarks under EP8, top-6 routing plus one shared expert give latency, compute throughput, memory bandwidth, and communication bandwidth for the whole path; at the 384-expert, hidden 7168, 16384-token tier, compute throughput is 846 TFLOPS and communication bandwidth 103.3 GB/s (DeepGEMM README).

The README’s account of fusion: these stages are organized and scheduled inside one operator. The data dependencies, communication buffers, and synchronization between stages still exist, only handled inside the operator rather than stitched by the upper layer.

For the migrating side this is a trade-off: call dispatch, expert computation, and combine as separate segments, each replaceable and schedulable on its own; or adopt the fused operator and hand the whole path to one call.

---

4. The Attention Layer’s Two Roads: Who Computes the Index

Attention computation also spans repositories. DeepSeek Sparse Attention (DSA) splits attention in two: index computation finds the context positions worth attending to, and the sparse attention kernel computes attention only on the selected positions (DeepSeek-V3.2-Exp). The V3.2-Exp release notes are explicit: the high-performance index scoring kernels (including paged versions) live in DeepGEMM; the sparse attention kernel lives in FlashMLA.

So DeepGEMM serves more than MoE. Its MQA logits kernel is the Lightning Indexer’s scoring kernel. The README states the kernel is FIX-pipe bound: the bottleneck is the path moving results out, not matrix computation itself, with the FIX pipeline at 99% utilization (the README’s characterization of the kernel as a whole; no per-tier breakdown). The benchmarks give two query scales (FP8):

Watch how the two rows move: shrinking the query scale 16-fold (4096 to 256) lowers compute throughput by only about 17%, while the statistical bandwidth rises about 7.1-fold. Same kernel, different shapes; both columns move together. Per the README, both shapes are FIX-pipe bound; the bottleneck class does not switch with the stage.

FlashMLA consumes the index table: it reads KV sparsely by index and accumulates output on-chip with online softmax. Why its Ascend implementation reaches 410/360 TFLOPS (95%/83% of peak) is Part 4’s topic. Here only the system-level meaning matters: attention optimization spans two repositories plus a layout contract; index formats, KV layouts, and quantization scaling-factor arrangements must line up between producer and consumer. Much of migration’s interface work happens between repositories, not inside them.

---

5. The Compile Runtime: One Common Base, but Not the Only One

Beneath the five repositories sits a structure that is easy to misread.

DeepJIT is the shared JIT compile base of DeepGEMM and DeepEP: it compiles, caches, loads, and launches kernels at runtime, with CUDA and Ascend sharing one interface. On the Ascend side, Bisheng compiles, ld.lld links, and ACL loads. Cache keys cover source, tracked headers, compiler version, effective options, plus caller-provided dependency signatures; cache directories can be shared across users, processes, and nodes. The two libraries have separate environment-variable prefixes, falling back to the global prefix when unset (details in Part 5, on correctness). On top of the common base, each library keeps its own compile policy (DeepJIT README).

But concluding “the compile runtime is now unified” would be wrong. TileKernels’ kernels all go through TileLang with runtime backend selection; even DeepGEMM’s own mHC prenorm kernel is written in TileLang. Two compile routes coexist: DeepJIT JIT-compiles hand-written kernel source; TileLang generates multiple backends from a DSL.

For the migrating side, toolchains cannot be treated uniformly; each component is configured, cached, and versioned along its own route. Part 5 (correctness verification) further covers how cache-key design affects “the code you think you are running.”

---

6. Interface Alignment Is Not Capability Alignment

“API-aligned with the NVIDIA version” is this batch’s shared selling point. What is aligned is the interface shape, not the capability list. DeepEP-Ascend’s support status reads as a table (DeepEP README):

DeepGEMM-Ascend is likewise “same API shape, rearranged inside.” Signatures carry over, but the scaling-factor layout differs on Ascend: UE8M0 packed pairwise into int16, stored MN-major. The repo provides a dedicated layout-conversion tool, plus concurrency limiting by AI Core count, a determinism switch, and tests cross-validated against aclnn reference implementations for the corresponding GEMM interfaces (DeepGEMM README). FlashMLA’s Ascend coverage, as stated at the opening, is sparse attention only.

The environment differs too. DeepEP’s published numbers were collected on a manually configured PoC HDK, not a public release. Per the README, the commercial version is expected from the vendor around mid-October. Until it ships, between “the code is readable” and “the results are reproducible” stands one configuration environment. Part 7 folds this into the ecosystem judgment.

Putting these facts together draws a clear line:

• Reusable: interface shapes, buffer abstractions, algorithm designs, engineering knowledge;
• To be redone or awaited: per-backend kernel implementations, data layouts, pipeline arrangements, synchronization semantics, and the gaps on the list where “the API is there, the kernel is not.”

---

7. What This Release Is Worth: Judgment and Boundaries

The overall judgment can now be given.

The direct value: the major stages of the inference path now have studyable public implementations and official results on both platforms. Dense GEMM peaks at 99.8% of the hardware ceiling, sparse attention at 95%/83%, EP communication ships five-tier bandwidths, and even the fused path has whole-path benchmarks in MegaMoE (sources and conditions in the official benchmark tables of Parts 2 through 4). For deployers, official benchmarks give every stage a checkable reference after deployment: beyond “does it run,” “how well it runs.” For optimizers, every link has a public implementation to take apart.

But operators are not a service. Five repositories do not constitute a service, and per-component benchmarks cannot replace acceptance of a real framework combination. The interface list still has explicit gaps: load-balancing communication kernels unimplemented, graph capture unsupported, PP/Engram experimental, the results environment not yet public. These gaps define the real workload of “replacement”: upper-layer calls and knowledge migrate; data layouts (e.g., the scaling-factor arrangement), in-kernel data movement and compute pipelines, and cross-repository synchronization semantics must be redone for Ascend or awaited.

Back to the title’s question: migration is not replacing functions one by one; it is rearranging the relations among computation, data, and resources. The model defines one computation graph; how it executes efficiently differs across hardware, in three ways:

• which computations to merge: MegaMoE puts the whole MoE path in one operator;
• which interface contracts to share across repositories: alignment granularity constrains communication and computation together; index and KV layouts span two repositories;
• which waits are worth hiding with overlap: dispatch interleaves with the shared expert.

On the engineering checklist, three things:

1. Reuse upper-layer interfaces and calling knowledge directly;
2. Redo layouts, pipelines, and synchronization per Ascend semantics, item by item, or await capability;
3. Validate the assembled whole path yourself in a real framework.

The following six parts expand these layers one by one.

---

Sources and computation notes: This piece is based on the five projects’ public repositories and documents, fact-checked as of October 7, 2026. Results, interfaces, and capability lists cited in the text come from the corresponding official READMEs; the DSA two-way split and the SGLang image entry come from V3.2-Exp release materials; DeepSeek-V3’s parameters (671B/37B) come from the official repository. The 1.34 TB estimate (671 × 10⁹ × 2 bytes) and the M_total decomposition are this piece’s own derivation and analytical frame: assuming all parameters at two bytes, covering only main-model weights, not representing any official quantized checkpoint’s size; the official weights ship mainly in FP8, so the two-byte figure is a conservative upper bound. The commercial HDK timeline (around mid-October 2026) is relayed from the README; the vendor’s actual release prevails. No hardware measurements were made. This is Part 1 of the DeepSeek Ascend open-source series.