The Efficiency Limit of Model Memory: From MLA to CSA+HCA

The Evolution of LLM Memory Architecture: What V4's Choices Reveal About the Next Decade of Model Memory

The Economics of Inference Trilogy · DeepSeek V4 Flash output costs ¥2/M tokens, Claude Sonnet 5 output costs ¥107/M. The 53× gap is not a subsidy—it's physics. This series traces from pricing back to architectural choices to engineering scheduling, dissecting what actually determines the physical cost of a token.

The DeepSeek V4 technical report (55 pages) discloses two key choices along the "memory" dimension:

These numbers are not isolated engineering metrics—they are the physical expression of "how a model remembers." V4's two models made very different choices along the memory architecture evolution curve, and their pricing (Flash ¥2/M, Pro ¥6/M) precisely tracks the physical cost of those choices.

The methodology of this article: first, accurately understand the three-axis classification framework and main conclusions proposed by Zhoubian, Zhang, Kharlamov & Tang in Memory for Large Language Models (arXiv:2607.25380, submitted 2026-07-28, 20 pages, 4 figures), then test the framework's predictive power against real-world production data—V4 abandoning MLA for CSA+HCA, Kimi K3's KDA in production deployment, Qwen3-Next's Gated DeltaNet in production deployment. The goal is not to apply a taxonomy mechanically, but to see where the paper's theoretical framework aligns with and where it is surpassed by the production realities of late 2026.

I. The Paper's Framework: Three-Axis Classification and Main Conclusions

1.1 A Faithful Reading of the Three-Axis Coordinate System

The core contribution of Zhoubian, Zhang, Kharlamov & Tang (2026) in Memory for Large Language Models is the establishment of a comparable three-axis coordinate system within the chaotic "memory" literature, along with a formalization of four core memory operation mechanisms—write, route, state transition, and merge granularity.

The three axes:

The paper further analyzes hybrid memory architectures (designs spanning multiple cells), system-level efficiency trade-offs (computational/storage costs of different memory types), and multi-dimensional evaluation methods. The paper's implicit judgment is that current mainstream LLMs cluster in the "implicit / offline / short-term" cell, but the evolutionary trend points toward migration to "explicit / online / long-term"—the paper outlines multiple possible migration paths.

1.2 Testing the Framework with Production Data

V4's position in the framework:

Both V4 models fall in "implicit / offline / short-term"—consistent with the paper's assessment of mainstream frontier models. Claude, GPT, Qwen, and others occupy the same cell. V4's innovation is not "jumping to a new cell," but pushing efficiency to the extreme within this cell.

But production reality is already challenging the boundaries of the paper's framework:

1. V4's abandonment of MLA is a major architectural pivot the paper did not cover. MLA (Multi-head Latent Attention) was the core KV compression scheme of DeepSeek V2–V3, an efficiency optimization within the "implicit / offline / short-term" cell. V4 completely replacing MLA with CSA+HCA (see Section II) means one compression approach was falsified while hierarchical compression + sparse selection prevailed—a bifurcation that occurred only around the time of the paper's publication.

2. Linear attention hybrid architectures have crossed the production threshold—the paper may underestimate the velocity of this path. Kimi K3's KDA (2026-07) and Qwen3-Next's Gated DeltaNet (2025-09) are both in production-grade deployment (see Section VI), whereas the paper treats hybrid architectures more as a "trend."

3. DeepSeek's conservatism toward "migration" exceeds the paper's expectations. The paper outlines multiple migration paths, but V4 skips even the closest-to-production Titans-style inference-time updates entirely—Engram (explicit / online / long-term) is deferred to V5. The fact that DeepSeek, the most aggressive architectural innovator, chooses to "exhaust the existing paradigm" suggests that the engineering barriers to migration are greater than theoretical analysis indicates.

II. V4's Three Axes of Improvement Within Implicit Memory

Within the "implicit / offline / short-term" cell, V4 does three things, each with direct impact on pricing:

2.1 MoE Sparse Activation: Conditional Memory in Parameter Space

MoE is essentially conditional memory in parameter space—each expert is a persistent parameterized memory block, and the router performs "context-dependent addressing."

V4's choices:

• Flash: 284B total parameters, 13B activated per token (4.6%)
• Pro: 1.6T total parameters, 49B activated per token (3.1%)
• Pro has higher sparsity (3.1% vs. 4.6%), meaning a larger knowledge reserve while keeping per-token inference cost manageable

V4's MoE fine-tuning: the affinity score activation function is changed to Sqrt(Softplus(·)), and the first few dense FFN layers are replaced with Hash routing. These are engineering optimizations that do not alter the memory paradigm.

Impact on pricing: Flash 13B activation → single-request decode at 257.7 tok/s (H100); Pro 49B activation → 68.4 tok/s. The pricing ratio of 3× tracks the activated parameter ratio of 3.75×.

2.2 CSA + HCA Hybrid Attention: Hierarchical Compression of KV Cache

This is V4's core innovation in memory architecture. Not "a different way of remembering," but hierarchical compression of the same memory mechanism (KV cache).

An architectural decision worth emphasizing: V4 abandoned MLA. From V2 through V3, DeepSeek continuously used MLA (Multi-head Latent Attention) as its attention compression scheme—MLA achieves several-fold efficiency gains by compressing KV through low-rank projection and was one of V2/V3's core technologies. V4 completely replaces MLA with a hybrid CSA+HCA+MQA architecture. This is a major architectural pivot. UCLA PhD Yifeng Liu stated explicitly in a LatePost interview (2026-05): "V4 abandoned the MLA used from V2 through V3." This means DeepSeek concluded that MLA's compression potential had reached its ceiling, and that hierarchical compression + sparse selection (CSA+HCA) is the next step.

CSA (Compressed Sparse Attention)—fine-grained local dependencies:

• CSA first compresses every m tokens' KV into 1 entry (the technical report does not disclose the exact value of m; based on compression ratios, m ≈ 8–16), then uses Lightning Indexer for top-k sparse selection
• Lightning Indexer generates index queries in a low-rank manner, selecting top-k compressed KV entries
• Shared KV-MQA further reduces computational cost
• Handles the lower layer—preserving fine-grained local dependencies

HCA (Heavily Compressed Attention)—aggressive long-range compression:

• Every m' (m' ≫ m) tokens compressed into 1 entry
• Maintains dense attention (no sparse selection)
• Handles the upper layer—heavily compressing distant context

The two layers interleaved compress KV cache to 7% (Flash) / 10% (Pro) of V3.2. Note that Flash's compression is more aggressive—7% vs. 10%—because Flash's CSA parameter m is smaller and HCA's compression ratio is larger.

Why do Flash and Pro have different KV compression ratios? Because the two models are independently pretrained—they are not in a distillation relationship. Flash's design goal is "extreme inference efficiency" (suited for high-concurrency API), while Pro's is "extreme inference quality" (suited for complex reasoning). The KV compression difference is a direct manifestation of the quality-cost trade-off.

2.3 MTP (Multi-Token Prediction): Accelerator for Speculative Sampling

MTP is not itself a memory mechanism—it is a V3-era engineering design that predicts multiple tokens during training and is used for speculative sampling at deployment. But its effect (1.8× speedup) directly impacts aggregate throughput and thus the pricing floor.

III. The Paths V4 Didn't Choose: Memory Directions Explicitly Abandoned

Understanding what V4 chose is important; understanding what it didn't choose is equally important.

Engram (Conditional Memory Module): Deferred to V5

In January 2026, DeepSeek jointly published the Engram paper with Peking University—hash-addressed sparse memory slots that explicitly separate memory sparsity from expert sparsity. But the technical report explicitly states Engram did not make it into V4; it is named in "future directions."

Why wasn't it included? Three possible engineering reasons:

1. Engram's "in-model lookup" overlaps functionally with RAG's "external retrieval," and the latter is more engineering-mature
2. The hash-addressed new component adds training and inference complexity; the 55-page V4 report already contains enough architectural changes
3. It has not been validated at 1.6T scale—the Engram paper's experiments are far smaller than Pro

Implications for V5: If Engram enters production in V5, it will be the first mainstream frontier model to break the "implicit / offline" framework—transitioning from "read-only memory" to "writable memory." This is a paradigm leap, not an incremental optimization.

Titans / TTT: Not Even Mentioned

Inference-time memory updates (Titans' "surprise-driven gradient updates," TTT-E2E's "end-to-end optimization at test time") are entirely absent from the V4 report. This is not an omission—it is a deliberate exclusion.

Why? Inference-time parameter writes directly conflict with the core optimizations of modern inference systems:

• Different requests within a batch need to update different parameters → batching efficiency collapses
• Update computation is unpredictable → unstable latency
• Long-term stability unknown → production risk

Judgment: In the foreseeable future (2027–2028), inference-time memory updates will not enter mainstream frontier APIs. DeepSeek, the most aggressive efficiency optimizer, didn't choose this path, indicating that the engineering barriers are far greater than academic papers suggest.

Recurrent State Memory (Mamba/RWKV): An Alternative Path, Not a Fusion Path

V4 does not employ any recurrent state layers. Pure attention + MoE sparsification is V4's choice. Kimi K3's KDA (Kimi Delta Attention, a hybrid linear attention mechanism—combining linear attention with the Delta rule to achieve 12.5× KV cache compression and 6.3× decoding speedup) takes another path—mixing linear state compression with attention. KDA has entered production-grade deployment in Kimi K3 (2.8T parameters, released 2026-07-16).

These two paths are not mutually exclusive—AI21's Jamba is already exploring a three-layer MoE + Attention + Mamba hybrid. But DeepSeek chose to "push attention to the extreme" on V4 rather than "hybridize two memory paradigms."

IV. The Three-Layer Memory System: V4's Specific Position

Mapping the paper's three-axis classification onto V4's actual deployment:

Signal path across the three layers (using V4 Flash as example):

1. Architecture-level: CSA+HCA compresses KV cache to 7% of V3.2 → only ~25 KB KV cache per request (4K context)
2. Infrastructure-level: minimal KV cache → HBM accommodates more concurrent requests → batch ceiling rises significantly
3. Higher batch ceiling → aggregate throughput ~36,000 tok/s/GPU (batch=100, including MTP)
4. Throughput determines pricing floor → Flash ¥2/M maintains 83% gross margin at batch ~80 (per OpenCode CEO)
5. Agent-level: Harness framework generates many repeated prefixes (system prompts, tool definitions) → 95%+ KV cache hit rate (DualPath paper Agent scenario: 98.7% measured) → further reducing physical cost

Core insight: The three layers are not independently designed—V4's architectural choice (CSA+HCA) directly determines the infrastructure-level scheduling strategy, which in turn directly determines pricing. This causal chain is the analytical foundation of the companion article "The Physical Floor of Inference Pricing."

V. Evolutionary Trend Assessment

Trend 1: The V4 Path—"Pushing Implicit Memory to the Extreme"—Has 2–3 Years of Runway

V4 Flash's KV cache is already compressed to 7% of V3.2. Remaining technical headroom for further compression:

• More aggressive CSA sparse selection (currently top-k; future may be top-1 or dynamic k)
• Larger HCA compression ratios (currently m' ≫ m; future m' may reach hundreds)
• The FlashMemory-DeepSeek-V4 paper (2026-06) has already demonstrated the possibility of further compressing KV cache by 10×—Lookahead Sparse Attention

MoE sparse activation still has room: Flash's 4.6% activation ratio is already very low, but moving toward finer-grained expert partitioning (V3 uses 256-select-8; future could be 1024-select-4) can further reduce activated parameters.

Judgment: The "implicit / offline / short-term" cell has not reached its efficiency ceiling. Next-generation models do not need to switch memory paradigms—they just need to keep compressing within this cell.

Trend 2: The Implicit→Explicit Migration Is Accelerating, But Not Through V4

V4's position is clear: the apex stage of implicit memory. It is the most efficient model on this path—but also a signal that the path is nearing its end. With 7% KV cache and 4.6% activation ratio, there is limited room left.

Trend 3: Convergence of Architecture-Level and Infrastructure-Level—V4 Is the First Example

Currently, the two layers are designed independently. V4 is the first system to tightly couple them:

• Architecture-level: CSA+HCA compression makes KV cache so small that three-tier storage scheduling is nearly frictionless
• Infrastructure-level: DualPath dual-route loading + 3FS file system are specifically optimized for this minimal KV cache
• The synergy: 98.7% KV cache hit rate, batch 80+, 83% gross margin

Next-generation trend: model architects will design CSA/HCA parameters directly considering "the PCIe Gen5 transfer latency of compressed KV blocks"—this is already implicit in V4's design and will become explicit next generation.

VI. Maturity Assessment: What's Already in Your Model, What's Still on Paper

In Production

These technologies matured and entered production systems during 2024–2026. The engineering value of MoE sparse activation and attention compression has been independently validated by multiple teams, and inference engine ecosystem support (vLLM, SGLang) is in place. Beyond DeepSeek V4's CSA+HCA, Kimi K3's KDA and Qwen3-Next's Gated DeltaNet have also entered production-grade deployment in 2026-07 and 2025-09 respectively—linear attention hybrid architectures are no longer paper proposals.

In one sentence: When you call the V4 Flash or Pro API today, this combination of technologies is running. They are the physical foundation of ¥2/M and ¥6/M pricing.

Engineering Validation Stage (available implementations, not yet widely commercialized)

With KDA (Kimi K3) and Gated DeltaNet (Qwen3-Next) entering production-grade deployment in 2025–2026, linear attention hybrid architectures have crossed the research-to-production threshold. The remaining technical barriers in this stage are primarily in inference framework adaptation and company-level uncertainty.

Laboratory Stage (12–24+ months from production)

These technologies are furthest from production. The core barriers are fundamental conflicts with current inference system assumptions—inference-time memory updates (Titans/TTT) break batching efficiency, Engram's large-scale training stability is unverified, and Nested Learning requires distributed training infrastructure overhauls. Their path to production (2027–2028) depends on whether these engineering barriers can be resolved through independent system-level innovations.

VII. Anchor Predictions: Five Falsifiable Forecasts

Prediction 1: V4's CSA+HCA approach will be followed by at least 2 other vendors within 12 months

Underlying logic: CSA+HCA compresses KV cache to 7–10%, fundamentally altering inference economics. The technical report has been open-sourced for 4 months, providing sufficient time for reproduction. This efficiency gain is so large that vendors who don't follow will fall behind by an order of magnitude in pricing.

Falsification condition: By August 8, 2027, if fewer than 2 frontier models from vendors other than DeepSeek adopt similar hierarchical compressed attention (CSA or equivalent), this prediction is wrong.

Prediction 2: Engram enters production in V5 (or next-generation DeepSeek) with >60% probability

Underlying logic: DeepSeek explicitly lists Engram under "future directions" in the V4 report—this is not a generic literature citation but a product planning signal from their own research pipeline. The problem Engram solves (decoupling static knowledge from dynamic reasoning) becomes more urgent as context windows expand from 128K to 1M.

Falsification condition: Engram does not appear in the DeepSeek V5 model architecture. If V5 launches in 2027 without Engram, the probability should be revised down.

Prediction 3: Titans/TTT-style inference-time memory updates will not enter any mainstream frontier API before 2028

Underlying logic: DeepSeek is the most aggressive architectural innovator (CSA+HCA, Muon optimizer, mHC all shipped in V4), yet it completely skips inference-time updates. This indicates that engineering barriers are far greater than academic papers suggest—inference-time parameter writes directly conflict with three core requirements: batching efficiency, deterministic latency, and long-term stability.

Falsification condition: Any mainstream API deploys inference-time parameter updates in production before June 30, 2028.

Prediction 4: Flash's architectural path (<15B activation + extreme KV compression + high batch) will become the default form factor for "API-class models"

Underlying logic: V4 Flash's official release, after post-training optimization, surpassed V4-Pro preview—proving that the 13B activation + post-training path can achieve flagship-level capability in Agent/Coding scenarios. If the capability gap can be bridged by post-training, there is no reason to deploy a heavier model for API scenarios.

Falsification condition: By end of 2027, if ≥3 of the top 5 APIs (by revenue) use models with >50B activated parameters, this prediction is wrong.

Prediction 5: The first frontier-class model with input price ≤ $0.10/M tokens will necessarily combine MoE sparse activation + CSA/HCA-level KV compression + three-tier KV storage

Underlying logic: V4 Flash has already compressed KV cache to 7% and reduced activated parameters to 13B. One step further (the 10× compression demonstrated by the FlashMemory paper) + finer-grained MoE + mature three-tier storage → physical cost floor reaches $0.05–0.08/M. Dense architectures physically cannot reach this price point.

Falsification condition: Any vendor achieves frontier quality + input price ≤ $0.10/M tokens using a dense architecture.

References

• DeepSeek V4 Technical Report (2026-04, Towards Highly Efficient Million-Token Context Intelligence, 55 pages)
• Sining Zhoubian, Dan Zhang, Evgeny Kharlamov, Jie Tang. Memory for Large Language Models. arXiv:2607.25380, submitted 2026-07-28 (20 pages, 4 figures). Jie Tang: Tsinghua University; Kharlamov: Bosch AI. https://arxiv.org/abs/2607.25380
• DeepSeek DualPath Paper (2026-02, joint with Tsinghua University and Peking University)
• DeepSeek Engram Paper (2026-01, joint with Peking University)
• FlashMemory-DeepSeek-V4 Paper (2026-06)
• Kimi K3 Technical Blog (kimi.com/blog/kimi-k3, 2026-07-16) — source for KDA production deployment data
• Qwen3-Next-80B-A3B Release (Alibaba Cloud Tongyi Team, 2025-09-12) — source for Gated DeltaNet production deployment data
• LatePost interview with UCLA PhD Yifeng Liu (2026-05) — V4's abandonment of MLA architectural decision
• AI21 Labs Acquisition Coverage: Calcalist / Globes / The Information (2026-01~04) — Nvidia acquisition fell through, Nebius in talks
• Titans: Learning to Memorize at Test Time
• vLLM PagedAttention Paper (Kwon et al., 2023)
• SGLang RadixAttention Paper (Zheng et al., 2023)

---

Disclaimer: V4 architecture data is from the technical report (open-sourced 2026-04). Flash/Pro pricing is from the DeepSeek official website (2026-08-08). Maturity assessment is based on publicly available papers and technical reports and does not constitute investment advice. Data current as of August 8, 2026.