CMX and Distributed Memory: Which Tier Should Context Live In

CMX and Distributed Memory: Which Tier Should Context Live On

When to Expand DRAM, When to Add Shared Flash, and When to Make the Two Work Together

Technical Analysis | Sources verified as of October 26, 2026

Abstract Long-context inference is turning KV cache management from a single-node GPU memory problem into a cluster capacity planning problem. Taking the comparison between NVIDIA's CMX shared flash tier and distributed DRAM caching as its main thread, this article proposes a three-part test. First, whether a restore can beat recomputation within the remaining deadline. Second, whether the new hits can cover the cost of moving data and queueing. Third, whether the saved compute can actually be realized, and how the 2026 memory price surge rewrites this capacity math. The four worked examples in this article can all be recomputed; 34 sources are cited.

Reading Notes This article uses three kinds of material: public product and code facts, test results published by others, and hypothetical models used for derivation. Any figure labeled "illustrative" or "hypothetical" does not represent CMX measurements, procurement quotes, or a performance commitment by any product. This notice is not repeated at each occurrence below.

---

Long-context inference is pushing a problem that used to belong to single-node GPU memory management into the territory of cluster resource planning: for KV cache that has already been computed, how long should it be kept, where should it live, and at what cost should it be fetched back? NVIDIA's answer with CMX is to add a shared flash tier between compute nodes and traditional shared storage, dedicated to serving inference context. But for systems that already use distributed DRAM caching, adding another storage tier does not automatically mean acceleration. It can just as easily bring new lookup, movement, queueing, and operations costs.

This question became urgent in the second half of 2026, for two reasons.

First, the STX reference architecture that CMX builds on entered the second half of 2026's delivery window together with the Vera Rubin platform [1]. Partner solutions are moving from announcement toward validation, at uneven paces: Dell's October 1 technical post still describes a Grace-based software path and validation targeting Vera ARM64, and notes that the dedicated BlueField-4 CMX hardware was not yet commercially available at that time [2].

Second, server DRAM is going through a price surge unseen in years. In the first quarter of 2026, conventional DRAM contract prices rose about 93%–98% quarter over quarter [3]. The premise itself (how much DRAM is economically affordable) is changing.

Judging the value of CMX requires answering three questions at once:

1. Can it restore useful state within the request's deadline?
2. Can the recomputation saved by larger capacity offset a slower data path?
3. Can the saved GPU work actually turn into budget savings or usable serving capacity?

This article is organized around these three questions and closes with multi-storage APIs, interface responsibilities, and how NVIDIA is driving its ecosystem forward.

1 First Put the Options in the Same Coordinate System

In discussions of KV acceleration, "memory options" and "storage options" are often described as two separate routes, but actual deployments look more like a continuous hierarchy:

• Active state on GPUs must meet the access requirements of the current computation;
• CPU memory stages recent state;
• Distributed memory frees reusable state from the lifecycle of any single machine;
• SSDs extend retention further to larger, colder working sets.

No matter how the underlying media change, what serving systems ultimately need is state that is compatible with the current model and request and can participate in computation directly.

Distributed DRAM should serve as the strong baseline. It already provides cross-node sharing, low restore latency, and high concurrency. Using "no cache, recompute from scratch every time" as the baseline only proves that cache reuse has value; it cannot answer whether CMX is worth buying relative to an existing memory cache.

The more reasonable question is: with the existing DRAM hot tier kept unchanged, how many of the expensive misses that would otherwise happen can an added flash tier recover? That is a different experiment from "replacing DRAM with flash." The former adds capacity and retention time; the latter changes the data path for hot requests. Their benefits and risks are different.

Distributed memory also splits into two kinds:

• Idle memory on existing servers: the short-term incremental cost can be low, but it is exposed to inference instance scaling, and you still bear NUMA access effects, pinned memory, NIC power, and the opportunity cost of competing with other host tasks.
• Dedicated memory nodes: more stable, but you carry server, NIC, switch port, power, and maintenance costs.

The same applies to shared flash. Reusing idle capacity of existing storage and building a dedicated system with DPUs, networking, and software services cannot be compared on SSD unit price alone.

The real option set therefore includes at least six items: keep existing DRAM, expand DRAM, add local SSD, add shared flash, tier DRAM with flash, and recomputation. When GPUs are idle, prefixes are short, or storage is congested, recomputing directly can be faster and cheaper than restoring from cache.

2 What CMX Changes Is the Retention Radius of Context

NVIDIA positions CMX as the G3.5 tier: a flash tier connected over Ethernet, sitting between local SSD (G3) and traditional shared storage (G4), implemented jointly by BlueField-4, DOCA Memos, NIXL/Dynamo, Grove, and Spectrum-X. NVIDIA's phrasing is "seamlessly extend GPU memory across the entire POD"; the mechanism is that the KV manager prestages KV blocks from CMX into host memory (G2) or HBM (G1) before decode [4].

In other words, what CMX extends is the capacity and sharing scope of reusable context. The GPU still reads data that has already been moved nearby; it does not access remote flash directly at the cost of HBM.

The most valuable change here is that compute resources and context resources can scale independently. A request no longer has to return to the node where the KV was originally produced, and an idle session no longer has to hold an expensive GPU just to keep its state. The price is that the shared tier must carry cross-node metadata lookups, network contention, tenant isolation, and failure recovery. These costs never fully show up in single-node peak bandwidth tests.

Whether CMX pays off depends first on whether the retention radius actually covers state that has value:

• If cached entries are almost all reused within a short window, the existing DRAM already covers the working set, and the added flash spends most of its time receiving data that will never be read again.
• If many sessions return only after tens of minutes or even hours, and insufficient memory capacity causes repeated recomputation, then adding a slower but sufficiently large tier can improve average response.

Real workloads do show this long tail. The Mooncake team analyzed production traces from Alibaba Cloud Bailian [5]:

Hot clusters and a cold tail coexist. LRU or fixed-TTL policies tend to evict "cold but still valuable" blocks too early.

As for NVIDIA's published "up to 5x," it refers to two metrics, TPS and energy efficiency, with traditional storage as the reference in both cases [4]; it must not be read as five times faster than an optimized distributed DRAM cache. For procurement decisions, the more useful focus is where the benefit comes from under the same request trace: how much from capacity, how much from networking and the data path, and how much from prefetching or routing improvements. Even if the full-stack solution works, you need to know which part is actually creating the value.

3 KV Capacity Must Be Derived from the Model and the State Structure

"How much KV does a million tokens need" has no universal answer. For an ordinary multi-layer attention model, uncompressed KV can be estimated as:

token count × number of layers × number of KV heads × head dimension × bytes per element × 2

The final 2 accounts for K and V. This formula has several common sources of error:

• With GQA, the number of KV heads is smaller than the number of query heads; plugging in the hidden dimension overestimates;
• MLA, cross-layer sharing, sliding windows, and quantization all change the state structure;
• The formula excludes page management, alignment, replicas, and other runtime state.

From a serving-system perspective, three quantities must also be distinguished: "logical context length," "physically retained KV bytes," and "the bytes actually moved to restore one request." Shared prefixes can be deduplicated, and blocks can be sharded by layer or by rank; but a restore may need layout conversion, and a large object of which only part is used may still have to be read out in full. Each type of planning looks at a different quantity:

• Capacity planning looks at physical footprint;
• Network planning looks at actual transfer;
• Latency evaluation looks at state available on the critical path.

An engineering estimate can proceed in two steps:

1. For the deduplicated logical state, multiply the token count by the actual bytes per token;
2. Then add TP sharding or replication, block padding, storage replicas, capacity headroom, staging, and metadata.

TP does not always divide all state evenly across cards; replication terms must be checked against the specific implementation. Summing context lengths across all sessions without subtracting shared prefixes also overestimates total capacity.

DeepSeek-V4.1-Flash is a concrete example. The paper distinguishes two kinds of footprint [6]:

• global KV cache: resident in HBM. With CSA2's cross-layer KV reuse and FP4 KV caching, it drops to 890 bytes/token, about one quarter of V4-Flash.
• persistent KV cache: resident on SSD or host memory. With a deployment optimization called SWA Bounded Replay, it drops to roughly 1/8 of V4-Flash.

Per Section 3.2 of the paper, the persistent cache is retained for at least 72 hours; SWA-related state goes into a distributed pool made up of roughly 10% of each machine's host DRAM, with a minute-scale TTL, and on a miss is rebuilt through bounded replay; the official explanation notes that the result is an approximation of the SWA KV Cache and cannot be treated as an exact cache substitute in arbitrary scenarios [6].

This shows that tiering decisions can happen within a single request: state that stays valuable long term and is expensive to rebuild goes to the high-capacity tier; short-term hot state stays in DRAM; parts that can be boundedly replayed are rebuilt on demand. Changes in model algorithms also move the boundary of storage investment:

• KV compression lets more of the working set stay in DRAM, easing the pressure to expand;
• KV compression also reduces the bytes fetched back from flash, making it easier for the shared tier to meet deadlines.

These two effects push in opposite directions. "KV got smaller" does not directly imply "storage no longer matters"; it has to be re-measured.

4 Public Benchmarks Reveal the Capacity Knee and Mark the Boundaries of Comparison

Mooncake's SSD offload experiment illustrates the capacity knee well. In a test broken down by turn, output length was fixed at 1 token to isolate prefill latency:

• First six turns: the memory pool is sufficient, and hit rates for both configurations stay above 80%.
• From the seventh turn: the configuration without SSD starts evicting KV; hit rate falls from 83% to 36%, and TTFT climbs from about 6 seconds to about 16 seconds.
• With SSD enabled: evicted blocks remain on NVMe; hit rate stays above 84% through the eighth turn, and TTFT is held to 9.4 seconds [5].

The lesson: once the working set exceeds memory capacity, a slower tier can still improve overall response by reducing misses.

But this is not a controlled comparison of "same memory, media swapped only." The test ran on a single DGX (8×A100-SXM4-40GB, dual RDMA NICs) and compared four configurations in total (including HiCache L1+L2, the Mooncake DRAM pool, and DRAM pool plus SSD); the turn-by-turn comparison focused on the two configurations with and without SSD [5]. So the difference should not be attributed entirely to SSD, and this TTFT result should not be extrapolated to long-output serving.

LMCache's Raw Block TP=4 validation provides evidence in the opposite direction [7][8]:

This is neither a CMX test nor a comparison against remote distributed DRAM; it is useful for observing the boundaries between different media and implementation paths. The original authors' explanation: when host memory is ample and the working set is favorable, the page cache can be very effective [7]. One row of these results also shows that within "direct I/O" itself, the gap between Raw Block and one O_DIRECT file configuration can reach several-fold; implementation details matter no less than the medium itself.

The production-side evidence is harder. In May 2026 the vLLM team reported measurements after connecting the Mooncake distributed KV pool to vLLM [9]. Replaying real Codex agent traces on 12 GB200s (1P1D):

• Cache hit rate rose from 1.7% (hitting only the system prompt) to 92.2%;
• Per-GPU throughput rose from 2,012 to 7,669 token/s (3.8x);
• P50 TTFT dropped from 82.93 seconds to 1.80 seconds (46x);
• End-to-end latency fell 8.6x.

A separate scaling experiment used a synthetic workload parameter-matched to Codex with round-robin routing, scaling from 12 to 60 GPUs; throughput grew near-linearly and hit rate stayed above 95% throughout [9].

The Mooncake papers point in the same direction. The arXiv version states that the architecture lets Kimi handle about 75% more requests under real traffic [10]. The FAST '25 version reports that while meeting SLOs, effective request capacity improves 59%–498% over baseline methods; the actual deployment handles 115% and 107% more requests on A800 and H800 clusters respectively; and the system runs on thousands of nodes, processing over 100 billion tokens per day [11].

The framing needs emphasis: these numbers prove the value of "pooling and reuse avoid recomputation," with "no distributed cache" as the baseline. They are still not a controlled comparison of flash versus distributed DRAM. In the public materials verified for this article, we did not find a CMX dataset sufficient to run that controlled comparison. That says something only about the scope of public material; it does not mean such tests do not exist.

The experiments are not in conflict: one reveals the cost of misses when capacity runs out, the other reveals the serving cost of different data paths. Together they suggest that a performance report must provide all of the following:

• The relationship between working set and capacity;
• Actual host memory usage;
• The hot/cold state of the cache;
• Output length and concurrency.

Without them, "flash beats memory" or "flash is worthless" may both be artifacts of experimental conditions.

5 Break a Single Restore into Its Real Critical Path

From cache hit to the first token, the work does not end with a single read. A restore usually passes through these steps:

1. Complete identity verification and prefix matching;
2. Obtain location metadata;
3. Reserve the target buffer, issue the remote read, and wait for the data to arrive;
4. Complete decompression, verification, or layout conversion;
5. Place the state where execution needs it;
6. Compute the new suffix that is not yet cached;
7. Enter decode.

A stall at any step can erase the advantage of the underlying medium.

A useful decomposition is:

TTFT = routing and queueing + prefix restore or recomputation + new-suffix prefill + first decode step + return overhead

This expression conveys dependencies; it does not say every term must be added serially. Streaming chunking, layer-by-layer loading, and prefetching can overlap part of the work, but how much overlap is possible depends on the inference engine and model layout. "The API is asynchronous" does not automatically cancel transfer time.

Illustrative example. Suppose a restore has 8 GiB of data passing through the same bottleneck, with 32 GiB/s of effective aggregate throughput available to it; the transfer-only lower bound is then 250 ms. The byte count and bandwidth here are already in the same statistical scope, so you must not divide by the card count again for TP. This lower bound does not yet include metadata, queueing, storage read latency, or GPU-side processing.

Two common distorted comparisons should be avoided:

• Using "latency of tens of microseconds" to represent this kind of large-object restore. That number usually refers only to small I/O or device access.
• Comparing nanosecond chip-level DRAM access against end-to-end SSD restore. Remote DRAM also traverses the network, registered buffers, and rank aggregation.

If storage reads, network sends, and copies can be steadily pipelined, the steady-state time for a large object is closer to "data volume ÷ throughput of the slowest stage," plus startup overhead and processing that cannot overlap; if strict serial dependencies exist, the corresponding stages must be summed. The real critical path must be verified with traces. You cannot sum the bandwidth-time of every segment and then generically subtract a lump of "asynchronous benefit" at the end.

Prefetching offers another path. While an agent calls an external tool, the server may already know which prefix the next turn will use, and so moves KV from flash to DRAM ahead of time. What the user experiences is "the conversation continues quickly as soon as the tool returns," not the full storage read time. But prefetches can also read the wrong thing, read too early, or crowd out genuinely hot objects by holding resources too long. All four of the following must be tracked together:

• Prefetch coverage rate;
• Effective use rate;
• Extra traffic;
• Occupancy duration.

Writes should be evaluated separately from reads. If every generation turn has to wait for a remote persistence acknowledgment, storage has entered the user's response path. For caches that can be recomputed, the common practice is to make state visible in memory first, then asynchronously demote it to flash, accepting partial state loss on failure. Whether that is allowed is part of the service contract and must be stated explicitly; a cache cannot be the sole store of business truth.

6 A Response Model You Can Recompute

Start with a deliberately simplified model to lay out the conditions. Definitions:

• hD: the fraction of all requests that hit in DRAM;
• hF: the fraction that miss in DRAM but hit in flash (unconditional fraction); the remaining requests recompute;
• D and F: the restore times of DRAM and flash, respectively;
• R: the recomputation time after a miss;
• O: the average marginal overhead introduced by the added tier.

D, F, and R each represent complete, mutually substitutable prefix paths. Upstream lookups and waits must be counted in the corresponding path or in O; they cannot be dropped. Assuming uniform cost within each branch:

E_prefix = hD × D + hF × F + (1 − hD − hF) × R + O

This quantity is "the average service time of a reusable prefix," not complete TTFT and not GPU busy time. It excludes, for now, the common new suffix, decode, the network return, and queueing caused by load variation, and it weights requests equally. Real systems should model prefix lengths and branch distributions separately.

Illustrative assumptions:

• Existing DRAM hit rate 80%, DRAM restore 20 ms, miss recomputation 2 seconds;
• Added flash recovers another 15% of requests, bringing total hit rate to 95%;
• Flash restore 300 ms, average extra overhead O of 20 ms;
• O is counted for every request of the hybrid option only; the other options carry none.

Three readings from this table:

• 416 → 181 ms: a tiered option can clearly beat a memory option that lacks capacity.
• 181 vs 119 ms: if the budget suffices to keep the same effective working set in DRAM, expanding DRAM remains faster in this model. The premise of this comparison is that both options have the same total usable capacity and retain the same cacheable working set.
• 466 ms: a storage hit is not necessarily better than a miss; a restore that is too slow can drag down the whole.

Break-even condition. With the DRAM hit rate unchanged, the average benefit of adding flash is:

hF × (R − F) − O

Plugging in this example, F below 2 seconds is not enough; it must satisfy:

F < 2 − 0.02 / 0.15 ≈ 1.867 seconds

to cover the average extra overhead. If the tiered buffering squeezes out existing memory, the loss from a lower DRAM hit rate must also be counted: hot hits that used to take 20 ms, pushed to a slow tier, can quickly eat the benefit of the added capacity.

hF also cannot be copied directly from "the second tier's hit rate." If the second tier hits 75% of DRAM misses, and DRAM misses are 20% of requests, then the unconditional hF is 15%.

The model also has a selection bias that is easy to overlook: what enters flash is often not a random sample of requests but the group with longer reuse intervals and larger prefixes, whose R and F can both exceed the average. In practice, conditional benefits should be computed per request class, and lookup and wait costs should be counted separately for flash misses. Timing only flash hits systematically overstates the tiering benefit.

7 Beyond Averages, Bandwidth and P99 Decide Deployability

A lower average prefix service time does not mean P99 improves. In the illustrative distribution above, 5% of requests still fall into the 2-second recompute branch; an 181 ms average does not imply "P99 below 200 ms."

Percentiles cannot be averaged weighted by hit rate; they must be computed from the full request distribution. If every hit branch is faster than the miss, a 5% miss rate puts the overall P99 at roughly the P80 point inside the miss branch, because 95% + 5% × 80% = 99%.

If each branch in this example is treated as fixed latency and queueing is ignored entirely, the hybrid option's prefix P99 is 2.02 seconds, higher than the original option's 2 seconds. This is only a toy model, but it is enough to show: improvement in the mean does not guarantee improvement in the tail.

Queueing is harder. Flash reads, network transfer, and GPU prefill are all shared resources. After a tier gains capacity, previously scattered recomputation may turn into concentrated restore traffic. A single request getting faster does not mean it stays fast under sustained load. Hit rate describes whether the data exists; bandwidth and queues describe whether it can be fetched in time. Both are required.

Steady-state bandwidth check:

Flash read bandwidth requirement ≈ request arrival rate q × fraction of requests that must read from flash pF × average bytes read per such request

Assume 100 req/s, of which 20% of requests each need to read 8 GiB; that requires roughly 160 GiB/s of effective read capability. This illustrative value excludes prefetching, repeated reads, replication, writes, and failure rebuilding. If the network cannot sustain such throughput over time, the 250 ms-per-request demonstration is meaningless.

Bandwidth must also be checked per topology. Cluster-wide bandwidth can be sufficient while a particular rack uplink, storage target, or popular prefix's replica count still queues locally. When multiple requests share one prefix, merged restores or hotspot replication reduce duplicate I/O; conversely, if every request first reads the same large object and then trims it, you get the waste of "deduplicated logically, not deduplicated on the network."

Batching affects the three kinds of options differently:

• Recomputation can exploit large batches for higher GPU throughput, but queueing may lengthen;
• Storage can amortize metadata cost across batched requests, but must wait to fill a batch;
• DRAM serves small batches faster, but is still limited by NICs, registered memory, and scheduling threads.

Comparisons should therefore report both low-load response and a capacity curve under stepwise load increase, marking which resource saturates first, instead of each side picking its most favorable concurrency point. Multi-rank restores wait for the slowest shard; batching waits for the last required block. However good the local average read latency, it cannot describe the tail at these convergence points.

Restore strategies must have deadlines. When the predicted restore completion is later than the request's remaining budget, or later than direct recomputation, the read should be skipped, shortened, or canceled. When necessary, a read can race recomputation, but "whoever finishes first wins" consumes double resources and must be capped. In an already overloaded system, unconstrained hedged requests (hedging) tend to amplify tail latency further.

8 Compare Costs Three Ways, Not Just Price per TB

Equal effective capacity. Three systems retain the same usable state; after deducting replicas, reserved space, metadata, alignment waste, and unusable capacity, compare restore latency, serving capability, and cost. This framing suits evaluating the efficiency of a medium and an architecture, but does not necessarily reflect real procurement choices, since an all-DRAM option of equal capacity may exceed the budget.

Equal budget. Given the same annual cost, purchase either pure DRAM or DRAM plus flash, then measure hit distributions, SLO attainment, and sustainable arrival rate on the same trace. This framing best reflects the trade between capacity and speed. The logical capacity, reliability, and operational boundaries of both systems still need to be disclosed; if a price advantage is bought with weaker data protection, that must be stated.

Equal SLO. Fix the model, request trace, peak arrival rate, and P99 target; find the smallest configuration at which each architecture reliably meets the target; then compare full TCO. This framing is usually closest to production decisions. If the flash option needs extra DRAM, more network ports, or more GPU concurrency to mask read latency, those costs must be charged back to the flash option.

TCO boundaries. Full TCO includes annualized device cost, servers and DPUs, networking, software support, power and cooling, racks, operations, spares, capacity redundancy, and failure impact. Incremental TCO compares only the costs the current decision actually adds or avoids. Two principles:

• Existing memory cannot be counted at zero cost in one option and at fresh purchase price in another;
• A new system cannot report bare-drive prices only, treating client memory and networking as free.

The 2026 memory prices make the expand-DRAM math harder. Per TrendForce:

• Conventional DRAM contract prices rose 45%–50% quarter over quarter in Q4 2025; the blended figure including HBM was 50%–55% [12];
• Q1 2026's actual increase was roughly 93%–98%, above the upwardly revised February forecast of 90%–95% [13][3];
• Q2 was forecast to rise another 58%–63% [3].

Flash is rising too: February's forecast was for Q1 NAND contract prices to rise 55%–60% [13]. But its per-GB base is far below DRAM's, and the absolute per-GB price gap between the two is still widening. One main supply-side cause is HBM crowding out wafer capacity. Per distributor Fusion Worldwide's analysis, HBM already accounts for about 23% of global DRAM wafer capacity (about 8% in 2024), and one HBM wafer consumes the capacity equivalent of 3–4 DDR5 wafers [14].

Illustrative example: cache tier cost

The following numbers are all teaching assumptions and do not form a measured correspondence with Section 6. All three options provide 10 TiB of usable logical KV capacity; blocks duplicated across tiers are counted once; replicas, reservations, and necessary staging are already included in the hardware budget. Straight-line amortization over 36 months with zero residual value, 730 hours per month, power at $0.12/kWh.

Giving the hybrid option a DPU is only a cost assumption; it does not imply the configuration or pricing of a real CMX deployment.

Monthly cache cost = hardware budget / 36 + software support + allocated power × 730 × 0.12 + extra drive-replacement reserve

How to read the table:

• Request volume assumption. The last column assumes 10 million requests per month; it is only cost allocation and does not mean the options achieve the same peak throughput or SLO.
• What "flash capacity tier" means. It refers to media dedicated to carrying cache capacity; the system still needs host memory, staging, and GPU HBM.
• Price point in time. Prices in the table are static assumptions and do not reflect the increases described above. If the DRAM capacity tier's server-and-media budget were raised by 50%, its monthly cost would rise to about $5,796.
• Scope of accounting. This is a cache-tier cost subtotal, not the full TCO of an inference platform. Shared GPUs, base host memory, and datacenter PUE increments, labor, financing, taxes, and other repairs are all excluded. The initial SSDs are already in depreciation; the drive-replacement reserve covers only additional replacements and cannot be billed twice for the same batch of drives.
• Equal-SLO acceptance. The extra GPUs needed for restores, failed retries, and support overhead must also be charged back to the whole system's cost, with valid requests meeting all service constraints as the denominator.

Illustrative example: incremental investment

Suppose the existing 2 TiB DRAM has a hardware budget of $52,000, software support of $200 per month, and power of 1.2 kW; then its monthly cost is $1,749.56. Its per-TiB cost is higher than the 10 TiB option, for the assumed reason that a small configuration carries more fixed server cost.

Upgrading to the hybrid option above costs $872.56 more per month. This is computed as the amortization difference between two complete configurations, effectively pricing the existing DRAM at replacement cost. If it is treated as sunk cost, the increment only needs to count the added flash, DPU, and networking, and the conclusion tilts further toward the hybrid. Readers should choose the framing that matches their own asset position.

The benefit side of the estimate:

1. Assume that of 10 million requests per month, 15% become new effective flash hits, each avoiding 1.6 GPU-seconds of prefill, while the restore itself consumes 0.1 GPU-seconds.
2. The theoretical compute released is 10,000,000 × 15% × (1.6 − 0.1) / 3,600 = 625 GPU-hours. This and the wall-clock latency of the prefix are two different measures.
3. At $2/GPU-hour, the theoretical value is $1,250 per month.
4. But a cluster already purchased or reserved does not automatically refund you because some compute sits idle. Assume only 60% genuinely reduces billing or defers planned purchases; the realized value is $750.
5. $750 is below the added cost; the net benefit is −$122.56 per month.

This model needs a realization ratio of about 69.8% to break even. If there is neither downsizing nor new demand, cash savings can be zero.

This result is closer to a real decision than "flash is cheaper per TB." The freed capacity may show up as peak headroom, a better SLA, or deferred expansion, but at an unchanged arrival rate it cannot be booked directly as newly sold tokens. The same compute saving also cannot be counted both as a deduction from the GPU bill and again as new revenue. First confirm that the whole system meets targets at the intended load; then discuss which expense can actually be canceled.

9 Weight Hit Rates by Value, and Gate Flash Writes with Admission

Request hit rate is the most intuitive metric, yet it may not reflect cache value. One request hits a short system prompt; another hits a session prefix of hundreds of thousands of tokens. Each counts once in "hit requests," but the compute saved can differ by orders of magnitude. At minimum, track all of the following together:

• Request hit rate;
• Token hit rate;
• Byte hit rate;
• GPU work avoided.

For chunked reuse, record the longest usable contiguous prefix rather than counting any single block hit as a whole-request hit.

Token weighting is not the endpoint either. Across models, lengths, and batches, prefill cost per token differs; the same token count with different compression and layout yields different restore bytes. A more practical cache value is:

expected future reuse count × avoidable compute cost per reuse − write, read, occupancy, and maintenance costs

The probabilities of expiry and model switching must also be considered.

Not every block evicted from DRAM deserves an SSD write. A one-off long request can occupy a lot of capacity and never return; a medium prefix reused at high frequency may have higher value density. A tiered system needs an admission policy that separates "where to evict to" from "whether it is worth keeping." Discarding state whose session has ended and whose return probability is low is often more economical than demoting everything.

Flash also needs write-amplification and endurance assessment. Overwriting the whole session every turn versus appending only newly completed immutable blocks can differ enormously in write volume. Device-level garbage collection, system-level replicas, object repacking, and failure rebuilding all add physical writes. Host writes, actual NAND writes, peak write bandwidth, and the expected replacement cycle belong in the same ledger, not just sequential read bandwidth.

Hot and cold should also adjust dynamically with load. The same session is hot during continuous user interaction, temporarily cold while a long-running tool executes, and highly predictable again just before the tool returns. A single fixed TTL conflates these three phases. The more sensible approach is to adjust retention tiers by reuse interval, restore cost, business priority, and prediction confidence, while setting explicable quotas so a few very long sessions cannot swallow all resources.

Capacity sizing can be derived from reuse distances in the trace. For a given block, count the cumulative bytes of distinct live blocks appearing between its two consecutive accesses; that approximates the capacity needed for a byte-wise LRU to hit it again. Aggregating these distances into a "hit rate versus capacity" curve shows the marginal value of further DRAM expansion and the long tail flash can cover.

Real policies also involve TTL, prefetching, tenant quotas, and variable-length blocks, and need simulation or replay for calibration. Drawing the curve first and then sizing capacity is far more controllable than buying some TBs first and hoping for a high hit rate.

10 A Safer Coexistence Architecture: A Stable Hot Tier, a Degradable Warm Tier

Most systems already using distributed memory do not have to start with replacement. The lower-risk approach: keep the DRAM hot tier and treat CMX or other shared flash as a bypassable warm tier.

Read path. A request first checks for valid prefixes in local and remote DRAM, then decides whether to read flash based on deadline, object size, and queue state; if there is no benefit, it recomputes directly. Metadata can be queried in parallel, but no DRAM hot hit should be made to wait extra for flash confirmation.

Write path. Organized by completed blocks:

1. New KV becomes visible in the hot tier first;
2. Stable blocks worth keeping are written to flash in the background;
3. Only after an object is fully written and verified is its readable location published.

During reads, leases, reference counts, or equivalent mechanisms protect against a block being evicted and reused mid-flight. Whether a read-back is promoted to DRAM depends on the probability of subsequent reuse, not on unconditionally writing every flash read into the hot tier and displacing hot data.

Routing. Cross-node routing can consult cache locations but must consider compute queueing at the same time. Sending a request to the node where "all the cache is" but which is busy is not necessarily faster than sending it to an idle node and reading from the shared tier. The scheduling objective should be predicted completion time, not maximum hit rate. One of the meanings of shared flash is precisely to reduce dependence on single-node affinity, making it easier to balance load across the compute pool.

Failure handling is especially critical. Cache loss can usually be recomputed, but state corruption or tenant mixing must never be treated as acceptable degradation. Large-scale simultaneous loss triggers a prefill storm; after storage recovers, a warm-up storm may follow, contending with foreground reads for the same links. Set separate budgets for foreground requests, background writes, warm-up, and rebuild, and prioritize restoring high-value hotspots rather than scanning everything and moving it back in one go.

Version rollouts have a similar effect. After model weights, tokenizer, or KV layout changes, old state may be unusable even if byte-complete. During rolling releases, keep multiple namespaces per version, warm the new version gradually, and limit the reclamation speed of old-version state. Treating an upgrade as a controlled cache cold start is far more reliable than discovering total misses after rollout.

11 Multi-Storage Access: What NIXL Unifies Is the Data Movement Interface

To understand CMX's API, first separate three layers of responsibility:

• Inference framework: understands tokens, the model, KV layout, and execution dependencies;
• Cache control layer: understands keys, locations, replicas, retention policy, and hit value;
• Transport and storage plugins: responsible for moving specified bytes safely to specified locations.

NIXL sits at the key position for data movement, but it is not a universal database that has already implemented the cache lifecycle for every backend [15].

Northbound interface. Users work around agents, registered memory, and transfer descriptors: prepare source and target descriptions, create requests, issue READ or WRITE, query completion status, and finally release the request and memory resources. Paths involving remote agents also require metadata exchange. The asynchronous design allows computation and transfer to overlap, but buffer lifetimes and dependencies still have to be managed correctly by upper layers [16].

Southbound plugins. A plugin must declare its name, version, supported memory types, and configuration parameters, and provide entry points for creating and destroying backend instances. The NIXL v1.5.0 plugin header states explicitly that plugins must use the same C++ standard as the core (currently C++20) and a compatible libstdc++ ABI [17]. The backend engine interface includes responsibilities such as registerMem, prepXfer, postXfer, checkXfer, and releaseReqH; optional methods such as queryMem and estimateXferCost return "not supported" by default [18]. That a method exists does not mean every backend gives it the same semantics.

Plugin integration also has several hard constraints:

• Plugin API versions are checked by strict equality; once the core version number is bumped, out-of-tree plugins must be recompiled [15];
• A backend that declares remote transfer support must also support notifications, or it cannot be created [19];
• GDS and GDS_MT cannot coexist in the same agent [19];
• Only one instance per backend type can be created, and the backend preference order for each memory type is determined by creation order [19];
• When the backend is selected automatically, the source code handles it as "the first matching candidate" [19];
• There is no ordering guarantee between transfer requests and no locks on memory regions; avoiding concurrent writes to the same location is the user's responsibility [15].

For storage vendors, implementing a few virtual functions is only the beginning; long-term maintenance also includes packaging, version matrices, driver dependencies, diagnostics, and upgrade compatibility. For users, "automatically selecting the best backend" in the current implementation is far from federated scheduling that compares cost and queues across multiple storage clusters in real time. When several storage systems coexist, cataloging, routing, failure domains, and capability selection still require higher-level design.

Data paths differ across plugins. Per each plugin's documentation, POSIX supports DRAM and files, OBJ supports DRAM and objects, and INFINIA also lists VRAM and objects [20]. NIXL's overall design document still says, as of this writing, "The currently supported backends in NIXL are UCX and NVIDIA Magnum IO GPUDirect Storage. Support for other file systems (including DFS), block and object storage are in development." [21]; the official overview lags the plugins' actual progress, so selection should rely on the plugins' own documentation and source code.

The OBJ plugin's new S3 acceleration engine (an S3-over-RDMA data path based on cuObject) declares VRAM segment support when the fast path is ready, while the default S3/CRT engine still supports CPU only [22]. LMCache's Raw Block backend also states explicitly that its I/O path is "CPU buffer to storage," not NVMe-to-GPU GPUDirect Storage [7]. From this, "using NIXL" does not imply "bypassing host memory," much less "CMX is deployed." Whether it is zero-copy must be checked segment by segment along the complete data path.

12 After the API Is Connected, You Still Need a Complete State Contract

A multi-storage system that can run for a long time must first define "the same KV block" precisely. A key must bind at least the following:

• Model weights or an immutable version number;
• Tokenizer and preprocessing rules;
• Token prefix;
• Positional encoding settings;
• Adapters;
• KV precision and layout version.

Multimodal state may also depend on image or audio preprocessing. Tenant isolation must enter the namespace or access control, preventing unauthorized cross-tenant sharing just because prompts match. Access or transport mechanisms such as S3, GDS, and NVMe-oF do not automatically provide interchangeability of model state: that the bytes can be transferred, and that the target inference engine can use them correctly, are two different contracts.

Blocks are best designed immutable. New data is written under a new identity and published after completion; the same key always corresponds to the same semantics, and repeated writes can be handled idempotently. Content hashes help locate objects, but cannot replace version descriptions, checksums, and access permissions. If keys are truncated due to interface limits, collision handling and full identity verification must also be defined; the short key itself is not proof of correctness.

The read contract must distinguish error types. Miss, expired, being written, corrupted, permission denied, backend timeout, and congestion each call for different follow-up actions:

• A miss can be recomputed;
• Corruption requires isolating the object and alerting;
• A permission error cannot simply be retried against another backend.

Batch reads also need partial-success semantics, so that one failed block does not force all already-transferred data to be resent.

The lifecycle must solve a classic race. Between the control layer's check that an object exists and the actual read, the storage layer may already have evicted it. Options include short leases, pin/unpin, versioned read tokens, or allowing failure with a safe fallback. Whichever is chosen, it must be clear who owns the source data and the target buffer during transfer, and when reuse is allowed after a timeout. contains plus get alone cannot eliminate this time window.

LMCache's CMX integration provides concrete observation points. DOCA_MEMOS is integrated as an object-style backend, keys must be adapted to the 128-bit constraint, and the relevant PR was merged on June 2, 2026 [23]. But in the existing NIXL storage code, several semantics need item-by-item verification [24]:

• The static path's contains depends on local mappings;
• The dynamic path may call query_memory after a local metadata miss, with behavior depending on configuration;
• The local accounting of object remove cannot be equated directly with remote physical deletion;
• Dynamic pin/unpin and other semantics still need item-by-item verification.

This legacy integration also requires DOCA_MEMOS to use CPU buffers, so it cannot be claimed to read and write the GPU directly on this path. The corresponding in-process official documentation is already marked deprecated; plugin support, test coverage, and deployment of the new multiprocess (MP) path need separate confirmation and cannot be extrapolated from the old connector's merged status [24].

These details do not negate the integration's progress; they show that "connector support" and "full lifecycle compatibility" are two different completion standards. When procuring or integrating, obtain a capability matrix covering at least:

• Object size and alignment requirements;
• CPU/GPU buffer restrictions;
• Completion visibility;
• Cancellation and timeout;
• Leases;
• Deletion, TTL, and quotas;
• Restart recovery;
• Statistics and error semantics.

This is the integration contract this article recommends; it does not mean CMX has publicly implemented every item. The open-source side is filling in the gaps item by item: LMCache's new multiprocess architecture has implemented supports_query, supports_delete, and atomic publication (atomic_publication) as backend capability bits. A backend configured with an eviction policy that does not support deletion errors out directly instead of failing silently [25]. Control-plane capabilities are turning from requirements in papers into configuration items that can be checked.

In NIXL v1.5.0's public plugin build manifest, we did not see DOCA_MEMOS listed alongside the regular backends [20]. That says only what that version's public manifest covers; it cannot be used to infer that the plugin does not exist. SGLang's relevant PR was still unmerged at verification time, and explicitly involves requirements such as huge pages, layout, and buffer alignment [26]. Before actual deployment, framework version, how the plugin is obtained, and the hardware combination must be confirmed; code on a development branch is not an out-of-the-box product guarantee.

13 How NVIDIA Drives the Ecosystem: Chaining Integration, Distribution, and Acceptance

From the available material, NVIDIA's approach has clear layers:

• Foundation: the STX reference architecture and BlueField set the direction for hardware and data paths;
• Middle layer: software interfaces such as NIXL lower the cost for inference frameworks to adopt different backends;
• Upper layer: framework connectors, official packages, and partners combine into deployable stacks;
• Acceptance: certification and production validation reduce procurement risk.

The STX launch announced a broad lineup of partners [1], but the partner list itself cannot prove that every product has the same implementation or delivery status.

Software distribution matters especially. DDN's INFINIA plugin PR was merged on June 5, 2026 [27]. DDN subsequently stated that the plugin has entered the official NIXL distribution and the Dynamo containers, and users still need to install the DDN client package separately [28]. This lowers the cost of maintaining a connector independently, but does not mean the full storage server side is thereby open-sourced. For vendors, entering the default distribution channel is often closer to sustained adoption than a single joint demo.

The distribution channel is also embracing open-source de facto standards. TensorRT-LLM merged a native Mooncake Store connector (KVCMv2) on October 10. Before that, vLLM, SGLang, and LMDeploy had each built in Mooncake integration in turn; FlexKV, developed by Tencent with NVIDIA and the community, also implements distributed KV reuse based on the Mooncake Transfer Engine [29]. In other words, while NVIDIA builds its own CMX/Dynamo data path, its own inference stack is also compatible with the open-source distributed memory pool. The two routes coexist at the software layer, rather than being an either-or choice.

Each partner's progress must be read separately:

• Dell: the October 1 material describes Grace-based CMX emulation and software validation targeting Vera ARM64, and notes that the BlueField-4 CMX hardware described in the post was not yet commercially available at that time [2].
• WEKA: distinguishes three states: the commercially available Augmented Memory Grid, validation on Grace/BlueField-3, and follow-up integration targeting BlueField-4 STX [30].
• VAST: the public HiCache guide takes another path: relayed through host DRAM, using the NIXL POSIX plugin and NFS over RDMA to connect to storage [31]. This proves conventional storage can participate in KV tiering through the shared interface, but it is not the same as using CMX-specific hardware or DOCA Memos.

These are all statements made at specific times about specific product scopes; neither "the CMX ecosystem is fully delivered" nor "everything is still unavailable" follows. The ecosystem is expanding, but product labels cannot replace verifying the actual data path.

On the broader accelerated storage interface front, NVIDIA announced in September that the cuObject client and server libraries are generally available (GA), and is advancing xio-sig and the SCADA Server SDK. SCADA's partner examples include prototypes, and the protocol interoperability work has its own evolution stages [32]. This shows NVIDIA wants to converge vendors' proprietary interfaces toward a more stable ecosystem entry point. But cuObject, SCADA, and DOCA Memos must not be conflated; supporting one does not automatically confer the full capabilities of another.

The value of certification is moving the discussion from peak numbers to sustained operation. The NVIDIA-Certified Storage documentation emphasizes real workloads, distributions from p50 to p99, failure recovery, and support response, and states explicitly that certification applies only to the specific product and configuration tested [33]. A general storage certification is therefore not a conformance proof for all CMX APIs, nor can it be inherited directly across product families.

For users, the most worthwhile evidence to demand is portability:

• Can the catalog and objects be exported?
• Who controls the state format?
• How much data must be re-warmed when switching backends?
• When something breaks, is the framework side or the storage side responsible?

A unified interface can reduce the number of adapters but cannot automatically remove these switching costs. A mature ecosystem should let performance competition happen on comparable contracts.

14 Acceptance Should Start from a Request Trace

The most valuable test input is not a set of hand-repeated prompts but a production trace that preserves arrival times, session gaps, prefix reuse, and length distributions. Anonymization must preserve reuse relationships, or the real caching problems degrade into meaningless random I/O. Model version, input/output lengths, TP configuration, precision, and serving targets must also be fixed, or the differences cannot be explained.

Round one: draw the curve for the existing distributed DRAM. Vary capacity from small to large, and observe how byte hit rate, avoided computation, TTFT distribution, and GPU utilization change. Look for two things: where marginal benefit starts to decline, and whether an obvious knee appears once the working set overflows. Without this curve, there is no way to judge whether the added flash fills a capacity gap or masks a hot-tier policy problem.

Round two: keep the hot tier unchanged and add shared flash. Count buffers, page cache, and prefetch occupancy in full, and report four scenarios separately:

• A true cold start;
• Metadata restored but data still cold;
• Steady state after warm-up;
• Cross-node migration.

Then re-test under the three framings of equal capacity, equal budget, and equal SLO, raising the arrival rate step by step, while keeping traffic bursts and large-object mixes in the picture.

Metrics cannot be TTFT only. They should also include:

• Inter-token latency and total completion time;
• Valid requests completed within SLO;
• Hit rate per tier and bytes actually read;
• Prefetch waste and write amplification;
• Cancellation ratio and recomputation volume;
• Queues on each link.

Only by chaining control-layer, transport-layer, and GPU execution times into a complete per-request trace can you know where to optimize.

Failure drills. Take down a storage target, restart the catalog service, create network congestion, expire some objects, switch model versions, and observe three things:

• Whether service degrades in a bounded way;
• Whether a global recompute or warm-up storm is triggered;
• Whether the system returns to a stable state after recovery.

Cache can be lost, but loss is not free of serving cost.

Rollout should start in bypass mode. First only record "if we had read flash, would there have been a benefit"; then enable it for a small set of requests with clear benefit and expand gradually; always keep a one-click fallback to "DRAM + recompute." The LMCache paper discusses how the crossover point between storage restore and recomputation shifts with bandwidth [34], but its long-context threshold under a specific configuration must not be taken as a universal rule. Thresholds should come from the current model, network, and load, not from a fixed token count.

15 Selection Conclusions: Divide the Work by Working Set and Deadline

• The hot working set fits economically in DRAM, and interactive P99 requirements are strict: optimize or expand distributed memory first, usually the more direct route.
• Insufficient capacity is already causing a large volume of expensive recomputation, and the reuse distribution has a clear long tail: DRAM plus shared flash is more worth validating.
• Requests rarely reuse, recomputation is cheap, or data restore has long been network-bound: continuing to rely on recomputation may be the more sensible choice.

The meaning of CMX is that it brings context capacity, the network data path, and inference scheduling into one infrastructure design. Its value cannot be proven by "flash replaces memory," nor by multiples against a no-cache baseline. What truly needs proving is: while holding the hot tier's response, the added capacity recovers valuable misses on time and carries more real demand at a cost advantage that can actually be realized.

For teams already running distributed KV memory pools, the endpoint most worth pursuing is often a degradable combination:

• DRAM carries hot state;
• Shared flash keeps warm state worth keeping;
• Recomputation handles the part that is cheap or cannot be restored.

The three are connected by a state contract, deadlines, and a cost model. Once you reach that point, the storage hierarchy genuinely serves the inference system, instead of being one more line item on a procurement list.

---

References

[1] NVIDIA. NVIDIA Launches BlueField-4 STX Storage Architecture With Broad Industry Adoption, 2026-03-16. https://investor.nvidia.com/news/press-release-details/2026/NVIDIA-Launches-BlueField-4-STX-Storage-Architecture-With-Broad-Industry-Adoption/default.aspx [2] Dell. Breaking the AI Inference Context Memory Barrier with Dell Storage, 2026-10-01. https://www.dell.com/en-us/infohub/articles/breaking-the-ai-inference-context-memory-barrier-with-dell-storage [3] TrendForce. Rapid Contract Price Surge Drives 1Q26 DRAM Industry Up 81% QoQ (conventional DRAM actually up about +93-98% QoQ; 2Q26 forecast +58-63%), 2026-06-01. https://www.trendforce.com/presscenter/news/20260601-13070.html [4] NVIDIA. Introducing NVIDIA BlueField-4-Powered CMX Context Memory Storage Platform for the Next Frontier of AI, 2026-03-16. https://developer.nvidia.com/blog/introducing-nvidia-bluefield-4-powered-inference-context-memory-storage-platform-for-the-next-frontier-of-ai [5] Mooncake Community. Scaling KV Cache Beyond Memory with Mooncake SSD Offloading (SSD offload experiment setup and turn-by-turn results in the article and its figures), 2026-07-15. https://kvcache.ai/blog/scaling-kv-cache-beyond-memory [6] DeepSeek-AI. DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression (§3.2), arXiv:2609.19969, 2026-09-17. https://arxiv.org/abs/2609.19969 [7] LMCache / Samsung. Raw Block in LMCache: Building a Fast, Recoverable NVMe Tier for KV Cache, 2026-09-22. https://blog.lmcache.ai/en/2026/09/22/raw-block-in-lmcache-building-a-fast-recoverable-nvme-tier-for-kv-cache [8] LMCache. PR #2948: [Feat][RawBlock] Add TP>1 support and compact batched retrieval path (TP4 end-to-end validation). https://github.com/LMCache/LMCache/pull/2948 [9] vLLM Team. Serving Agentic Workloads at Scale with vLLM x Mooncake, 2026-05-06. https://vllm.ai/blog/2026-05-06-mooncake-store [10] Qin et al. Mooncake: A KVCache-centric Disaggregated Architecture for LLM Serving, arXiv:2407.00079 (v5, revised 2026-10-08, abstract). https://arxiv.org/abs/2407.00079 [11] Qin et al. Mooncake: Trading More Storage for Less Computation (FAST '25). https://www.usenix.org/system/files/fast25-qin.pdf [12] TrendForce. Price Rally Drives 4Q25 DRAM Revenue Up 29.4%; Samsung Regains No. 1 Market Share, 2026-02-26. https://www.trendforce.com/presscenter/news/20260226-12937.html [13] TrendForce. Memory Price Outlook for 1Q26 Sharply Upgraded; QoQ Increases of All Product Categories to Hit Record Highs, 2026-02-02. https://www.trendforce.com/presscenter/news/20260202-12911.html [14] Fusion Worldwide (Czuczwa, A.). Why DDR5 Prices Have Nearly Quadrupled in 2026 (distributor research), 2026-07-23. https://info.fusionww.com/blog/why-ddr5-prices-have-nearly-quadrupled-in-2026 [15] NVIDIA NIXL. Backend Plugin Interface Overview (BackendGuide.md). https://github.com/ai-dynamo/nixl/blob/main/docs/BackendGuide.md [16] NVIDIA NIXL v1.5.0. Northbound C++ API (src/api/cpp). https://github.com/ai-dynamo/nixl/tree/v1.5.0/src/api/cpp [17] NVIDIA NIXL v1.5.0. backend_plugin.h. https://github.com/ai-dynamo/nixl/blob/v1.5.0/src/api/cpp/backend/backend_plugin.h [18] NVIDIA NIXL v1.5.0. backend_engine.h. https://github.com/ai-dynamo/nixl/blob/v1.5.0/src/api/cpp/backend/backend_engine.h [19] NVIDIA NIXL v1.5.0. nixl_agent.cpp (backend creation and matching logic). https://github.com/ai-dynamo/nixl/blob/v1.5.0/src/core/nixl_agent.cpp [20] NVIDIA NIXL v1.5.0. Plugin directory and build manifest (src/plugins, including each plugin's README). https://github.com/ai-dynamo/nixl/tree/v1.5.0/src/plugins [21] NVIDIA NIXL. NVIDIA Inference Xfer Library (nixl.md). https://github.com/ai-dynamo/nixl/blob/main/docs/nixl.md [22] NVIDIA NIXL. OBJ plugin README (S3 acceleration engine and memory type table). https://github.com/ai-dynamo/nixl/blob/main/src/plugins/obj/README.md [23] LMCache. PR #3486: DOCA_MEMOS Storage Backend, merged 2026-06-02. https://github.com/LMCache/LMCache/pull/3486 [24] LMCache. NIXL storage backend implementation (dev branch) and Legacy NIXL Backend documentation, verified 2026-10-25. https://github.com/LMCache/LMCache/blob/dev/lmcache/v1/storage_backend/nixl_storage_backend.py [25] LMCache. Multiprocess architecture backend capability bits (nixl_native_l2_adapter.py, dev branch). https://github.com/LMCache/LMCache/blob/dev/lmcache/v1/distributed/l2_adapters/nixl_native_l2_adapter.py [26] SGLang. PR #26073: DOCA Memos Support, unmerged at verification time on 2026-10-25. https://github.com/sgl-project/sglang/pull/26073 [27] NVIDIA NIXL. PR #1569: DDN Infinia Plugin, merged 2026-06-05. https://github.com/ai-dynamo/nixl/pull/1569 [28] DDN. DDN Becomes the First Storage Vendor Natively Integrated into NVIDIA KV Cache Management. https://www.ddn.com/blog/ddn-becomes-the-first-storage-vendor-natively-integrated-into-nvidia-kv-cache-management [29] kvcache-ai. Mooncake GitHub project activity (TensorRT-LLM KVCMv2 2026-10-10; FlexKV 2026-01-28). https://github.com/kvcache-ai/Mooncake [30] WEKA. WEKA Maximizes Token Output With Lower Cost Per Token on NVIDIA BlueField-4 STX, 2026-03-16. https://www.weka.io/news/neuralmesh-nvidia-stx [31] VAST Data. SGLang HiCache Using NIXL, updated 2026-10-22. https://kb.vastdata.com/docs/sglang-hicache-using-nixl-nvidia [32] NVIDIA. Expanding AI Storage Access with NVIDIA cuObject and the NVIDIA SCADA Server SDK, 2026-09-30. https://developer.nvidia.com/blog/expanding-ai-storage-access-with-nvidia-cuobject-and-the-nvidia-scada-server-sdk [33] NVIDIA. NVIDIA-Certified Storage. https://docs.nvidia.com/certification-programs/certified-storage/latest/nvidia-certified-storage.html [34] Cheng et al. LMCache: An Efficient KV Cache Layer for Enterprise-Scale LLM Inference, arXiv:2510.09665, 2025-10-08. https://arxiv.org/abs/2510.09665