Agent Storage Paradigm Reassessment: After FMS 2026, Projections Became Products

August 2024, Santa Clara. SK Hynix was presenting the production timeline for 12-layer HBM3E. August 2025, Kioxia walked away with Best of Show for a 245.76TB SSD. August 2026, Kioxia GP1 took Best of Show again — not for capacity, but for 10 million IOPS and 512-byte access granularity.

Three years, same summit, three completely different stories.

About a month ago in When Storage Becomes Agent Working Memory, we projected a four-stage framework for AI storage: weight pipeline stage → in-HBM management stage → cross-tier state management stage → Agent working memory stage. FMS 2026 hadn't happened yet; the analysis was based on the preview agenda.

Now FMS 2026 is over. NVIDIA open-sourced cuFile and released the SCADA architecture. ICMSP defines SSDs as first-class KV Cache memory nodes. Kioxia GP1 proved that SSDs can operate at memory granularity. OCP published the HBF standard. Samsung showcased a zHBM concept model. Projections are becoming products.

But not one product — multiple independent lines of work answering the same question. NVIDIA's ICMSP relies on BlueField-4 DPU for semantic translation. Moonshot AI's Mooncake uses a pure software layer for distributed cache scheduling. DeepSeek's DualPath achieved 98.7% KV Cache hit rate on a 660B production model. OCP's HBF takes the open standard route, connecting NAND directly into the processor package.

This article uses the actual product shapes from FMS 2026 to reassess every projection from a month ago. The core correction: the four stages are not sequential. Stage three (cross-tier state management) and stage four (Agent working memory) are happening simultaneously — because they run on the same hardware infrastructure. We also examine the concrete changes in Agent deployment across three dimensions: bus, media, and protocol.

I. Projection Retrospective: What the Framework Said About a Month Ago

A brief recap of the core reasoning chain.

Stage one (weight pipeline, 2023H2–2024) and stage two (in-HBM management, 2024H2–2025H1) remain valid. Storage's role was storing training data, loading weights, writing checkpoints. KV Cache was managed entirely inside GPU HBM. As context windows expanded, FP8 quantization and CPU offload were stopgaps; the storage industry was still on the sidelines.

Take Llama 2 70B (80 layers, 8 KV heads, head_dim=128, FP16): per-token KV Cache is ~320 KB; a 128K context single request is ~40 GB; 1M context exceeds 300 GB. A few concurrent requests blow past HBM capacity. KV Cache spillover was only a matter of time.

The inflection point came in 2025H2. Prefix Cache was the catalyst. When systems discover that multiple requests share the same prefix (system prompts, long documents, few-shot examples), KV Cache becomes a reusable asset across requests. This immediately raises three storage-system-level questions:

1. Naming: How do you identify a prefix KV Cache block?
2. Addressing: Is it in HBM, DRAM, or SSD? How do you move it?
3. Lifecycle: When do you evict? LRU or frequency-based?

At this moment, KV Cache transformed from a data structure into a storage object. The storage industry was finally pulled into the inference critical path.

The original framework predicted next entering stage four: Agent workloads imposing three new shocks on storage systems — state persistence, IO pattern inversion, memory-semantics extension. FMS 2026 showed that stages three and four are happening concurrently. The reason is simple: Agent inference is itself the largest consumer of KV Cache. ICMSP is simultaneously a cross-media KV Cache scheduling solution and the hardware foundation for Agent state persistence. The same DPU + SSD infrastructure solves both stages at once.

II. Bus Dimension: The Fundamental Shift in Agent Data Access Paths

This is the deepest-impact dimension. It's not "storage got faster" — Agent data access paths shifted from CPU-mediated to GPU + DPU-mediated. This change reshapes the architectural constraints of Agent deployment.

The Legacy Path: CPU as Broker for All Data

The Agent runtime runs on CPU. Every step of the Agent loop:

1. Observe environment → CPU reads RAG vector index from SSD (through the full IO stack: application → system call → VFS → filesystem → NVMe driver → SSD)
2. Assemble prompt → CPU assembles token sequence from DRAM
3. Call model → CPU copies prompt to GPU HBM via PCIe
4. GPU inference → generates output → copies back to CPU via PCIe
5. Update state → CPU writes to SSD or DRAM

CPU is in the path at every step. A single Agent loop involves 4–5 CPU interventions, each adding 10–50μs IO latency plus PCIe copy overhead. A 100-step task accumulates seconds of pure shuffling overhead.

Key constraints:

• Agent state lives in CPU-side DRAM (the Agent framework's Python process memory); GPU only receives data during inference via PCIe
• KV Cache lives in GPU HBM; on overflow, CPU moves it to host DRAM (CPU-managed offload)
• Multi-Agent communication goes through IPC or network messaging
• Multi-tenant isolation relies on OS process permissions

Post-FMS 2026: Three New Paths Open Simultaneously

Path A: GPU → DPU → SSD (SCADA + ICMSP)

NVIDIA's SCADA architecture lets the GPU issue storage I/O requests via the cuFile API, with BlueField-4 DPU handling actual NVMe queue management. The GPU sends the request, BF4 executes, and the CPU is absent from the data path entirely.

ICMSP adds a semantic layer on this path. A dedicated FTL (Flash Translation Layer) runs on the BF4, mapping KV Cache key-value addressing to physical SSD blocks. To the GPU, the interface is memory-style — "read attention head 0 layer 15's KV data." Downward, it still issues standard NVMe commands. The SSD hardware itself needs no customization.

Demonstrated performance (note: this was a FMS live concept demo using H100 GPUs; different from the Vera Rubin / BF4 production architecture discussed below): 3 H100 GPUs driving 44 PCIe Gen6 SSDs, achieving 230 million random read IOPS, with single-server storage throughput of 118 GB/s. SCADA throughput is 5.3× the CPU-triggered mode.[^1]

A Vera Rubin SuperPod's BF4 rack has 64 BlueField-4 units + 9600TB NVMe SSD. ICMSP organizes these SSDs into a pod-level KV Cache pool, using Spectrum-X Ethernet and NVMe-oF for RDMA data transfer.

Impact on Agent deployment: Agent KV Cache can reside on ICMSP-managed SSDs (9600TB per SuperPod), with the GPU reading on demand. Agent state is no longer bounded by CPU-side DRAM capacity. 9600TB means simultaneously maintaining KV Cache for hundreds of thousands of long-context sessions. The tradeoff is latency in the μs range (an order of magnitude slower than HBM), suitable for warm/cold KV Cache and persistent state.

But note: SCADA + ICMSP currently requires the full NVIDIA stack (BlueField-4 DPU + Spectrum-X network + Vera Rubin platform). This is a platform-lock-in solution, not an open standard. Does an open alternative exist? Mooncake implements similar distributed KV Cache management in pure software (DRAM + SSD multi-tier caching, no dedicated DPU needed), but no public benchmark compares its performance against ICMSP.

Path B: GPU → CXL Memory Pool

CXL (Compute Express Link)'s CXL.mem protocol allows the CPU to directly access memory on CXL devices via load/store instructions, with end-to-end latency around 200–300ns (measured on Intel Sapphire Rapids CXL 1.1 platforms; CXL 3.2 expected to be lower), sitting between local DRAM (~100ns) and NVMe SSD (10–100μs).

At FMS 2026, the CXL ecosystem looked close to deployable — Samsung CMM-D MD310 (CXL 3.2 / PCIe 6.0 / 256GB / 72 GB/s) in mass production, Marvell Structera providing sub-microsecond shared memory pools, Montage MXC CXL 3.2 controller entering pilot production. But between "ecosystem ready" and "large-scale production deployment in Agent scenarios" there remains structural friction: CPU vendors are half-hearted about CXL memory pooling (it erodes their platform lock-in); OS support for CXL memory pool hot-plug and page migration is still early; NVIDIA doesn't push CXL either (it promotes the GPU-direct-to-storage ICMSP/SCADA route). CXL's home turf may not be AI inference but traditional databases and virtualization.

Impact on Agent deployment: Agent hot state (the current session's working memory) can live in a CXL memory pool. Multiple Agent instances can share the same pool: a tool result written by one Agent is immediately readable by another (within the CXL coherence domain). This solves the state-sharing problem in multi-Agent collaboration — no need to exchange state via message passing (IPC/network), just direct reads through shared memory.

Current architectural limitation: CXL still requires going through the CPU's root complex — the GPU cannot directly initiate CXL requests. The path for GPU to use CXL memory is GPU→PCIe→CPU root complex→CXL device→CPU memory→GPU, adding an extra PCIe round trip. This means Path B still has CPU involvement (though it doesn't move data, only does protocol translation). Future CXL extensions may support GPU-initiated CXL requests (through the GPU's own root complex or UCIe bridging), but current hardware doesn't support this.

Path C: GPU ↔ In-Package Media (HBF / zHBM)

HBF (High Bandwidth Flash) attaches directly inside the GPU package via UCIe interface. The GPU's memory controller can directly address data on HBF. NAND latency is an order of magnitude higher than DRAM, but HBF's bandwidth advantage comes from extreme parallelism — massive NAND channels working simultaneously through TSV, trading throughput for latency. Suitable for KV Cache warm tier.

Samsung's zHBM takes a more aggressive route: wafer bonding stacks HBM vertically directly above the AI accelerator, with projected performance at 8× HBM5, but mass production isn't until 2029.

Impact on Agent deployment: Agent's high-frequency warm data (e.g., prefix cache) can sit on HBF: two orders of magnitude faster than SSD, 5–10× cheaper than HBM. This fills the previous gap where "HBM is too expensive but SSD is too slow." But HBF products won't ship until 2027H2 for Grade 1.

Three Paths Combined: Tiered Storage for Agent State

The three paths are not mutually exclusive. A complete Agent storage architecture could use all three simultaneously:

This tiering is fundamentally different from the traditional "CPU DRAM + SSD" two-layer structure. Agent deployment shifts from "cram state into Python process memory" to "data tiering scheduling across five media tiers."

Architectural Shifts in Agent Frameworks

Current Agent frameworks (LangGraph, CrewAI, AutoGen) are designed with a CPU-centric assumption: state lives in Python process memory, LLM calls are remote API requests, tool results are serialized to files or databases.

When the GPU can directly address SSDs (SCADA/ICMSP), CXL memory pools (shared state), and HBF (in-package warm data), the design constraints for Agent frameworks change:

State doesn't need serialization. Agent state can live in a DPU-managed storage tier, directly addressable by the GPU — no CPU serialize/deserialize needed. Current Agent frameworks encode state as JSON or pickle at every step, generating substantial CPU overhead and memory copies across a 100-step task. When state is directly addressable via cuFile, this overhead disappears.

Multi-Agent collaboration doesn't need message passing. Multiple Agents running in the same CXL coherence domain share state in memory — no IPC or network needed. A tool result written by one Agent is readable by another in the next clock cycle. This transforms multi-Agent systems from "distributed systems problems" into "shared-memory concurrency problems" — a completely different programming model.

Agents don't need to be pinned to one machine. ICMSP's KV Cache can be shared across nodes via Spectrum-X RDMA (BF4's ConnectX-9 provides RDMA connectivity). Agent state follows the inference task, not the machine. This means Agent migration shifts from "serialize state → network transfer → deserialize" to "switch RDMA pointers."

These changes mean Agent frameworks will evolve from the "CPU process + remote LLM API" model to a "GPU process + local inference + DPU-managed distributed state" model. This transition has barely begun; none of today's Agent frameworks are designed for it.

III. Media Dimension: What Agent Workloads Demand

IO Pattern Inversion: From Sequential Bandwidth to Random IOPS

This is the most underestimated of the three shock dimensions, and the one FMS 2026 validated most thoroughly.

Every step of Agent execution — observing the environment (RAG retrieval producing random reads), calling tools (producing small file writes), updating state (random R/W on state objects), saving snapshots (periodic medium writes) — generates fine-grained random IO.

NVIDIA's Storage-Next initiative, with 40+ vendor partners, specifically optimizes for 512-byte reads and writes — while the minimum management unit (physical page) of traditional SSDs has reached 8–16KB, and a single KV Cache or Agent state access may be only 512 bytes. One effective 512-byte read moves 16–32KB of physical data; actual read amplification can exceed 30×.

Kioxia GP1 winning Best of Show wasn't about capacity — 245.76TB QLC drives are no longer novel at FMS — but because it proved SSDs can work at memory access granularity: 512-byte access granularity, 10 million IOPS, 50 DWPD.

The NAND Physics Inference Chain

Why is Agent random fine-grained IO a serious problem for SSDs? We need to reason from NAND's physical properties.

NAND flash's minimum write unit is a page (typically 4–16KB); the minimum erase unit is a block (containing dozens to hundreds of pages, typically 1–4MB). You cannot overwrite in place. To modify data in a page, the entire block must first be erased and rewritten. This is the root cause of write amplification (WAF).

Under traditional AI workloads, writes are large and sequential (checkpoints of several GB at a time). The FTL can efficiently batch-write empty pages; background GC reclaims invalid blocks in bulk; WAF stays near 1. Latency is predictable.

Agent workloads break this equilibrium. 1,000 concurrent Agents, each updating a 512-byte state fragment every few hundred milliseconds:

1. Page waste: Each 512B write consumes a full 4–16KB page; the rest is wasted
2. GC thrashing: Scattered random writes quickly exhaust free pages, forcing GC to trigger more frequently
3. Latency spikes: GC needs to read valid data, migrate it to a new block, erase the old block — a single 50ms GC pause is a timeout for Agent interactions requiring p99 < 10ms
4. WAF degradation: Valid data is scattered across many blocks (due to random writes); GC migrates a higher proportion of valid data; WAF can degrade from 1.1 to 3–5 (projected values; no public Agent workload benchmark validation)
5. Lifespan consumption: WAF degradation means actual writes are 3–5× logical writes; SSD lifespan burns faster

The cascading effects unfold layer by layer:

• FTL: Must shift from "bulk sequential allocation" to "fine-grained random write WAF control" — finer-grained page mapping, write coalescing
• GC: Background bulk GC latency spikes are unacceptable in Agent scenarios. Must shift to foreground low-latency GC — more frequent but reclaiming fewer blocks per cycle, reducing peaks
• OP (Over-Provisioning): Typical enterprise SSD OP is 7%; Agent scenarios may need 15–25% to maintain stable latency
• DRAM Buffer: Shift from bulk read-ahead to small-file hot caching — identifying which Agent state fragments are hot

FMS 2026 status: GP1's 50 DWPD and 512-byte granularity indicate the controller layer is already preparing for this workload. But no vendor has demonstrated actual WAF benchmarks under Agent workloads. The inference chain is logically sound; empirical validation is missing.

The Media Spectrum

Looking at all Agent-relevant media together:

*HBF pricing is industry speculation; not yet in mass production.

Latency distributes continuously from nanoseconds to hundreds of microseconds. Different types of Agent data land at different positions on this spectrum — not all state needs HBM latency, and not all state can tolerate QLC's hundred-microsecond range. The scheduler's job is to select the most appropriate media tier based on each data type's latency budget.

IV. Protocol Dimension: Who Controls Agent Data Scheduling

The physical connections and media nodes exist. But who decides where a piece of data lives? This is the protocol layer's contest.

cuFile + SCADA: GPU Takes Over Storage Access

NVIDIA open-sourced the cuFile API and released the SCADA architecture. The traditional storage protocol stack is: application → system call → VFS → filesystem → block device layer → NVMe driver → SSD. SCADA shortens it to: GPU → cuFile → BF4 DPU → NVMe queue → SSD. The CPU is not in the path.

This isn't just performance optimization — it's an architectural power transfer. When the GPU directly manages NVMe queues (via DPU), traditional filesystem semantics (open/read/write/close) are no longer the only data access method. For KV Cache, which "doesn't need filenames, only key-value addressing," this removes an unnecessary layer of abstraction.

ICMSP FTL: Storage Understands Data Semantics

ICMSP's dedicated FTL brings KV Cache semantics into the storage layer. A traditional SSD's FTL doesn't know what's inside its blocks. ICMSP's FTL knows:

• Which data belongs to the same attention head
• Which data belongs to the same prefix cache namespace
• Which data can be safely evicted

Currently, ICMSP understands KV Cache semantics. But the architecture's logic points further: if the DPU-resident FTL can understand KV Cache semantics, it can also understand Agent state semantics.

Agent state is more complex than KV Cache. Tool call results have causal dependencies (step 47's decision depends on step 23's output); execution trajectories need versioning (roll back to step 23 for re-execution); multi-Agent messaging needs namespace isolation. Today's ICMSP FTL doesn't handle these — it only knows attention heads and prefix cache.

But the inference is straightforward: when the DPU FTL evolves from "KV-aware" to "Agent-aware," the storage system transforms from an "inference-specific memory node" into an "Agent working memory node." This hasn't happened yet, but the architectural foundation is laid.

FDP: Industry-Standard Data Placement

The NVMe working group's FDP (Flexible Data Placement) lets the host explicitly tag data placement — checkpoints to the SLC tier as burst buffer, hot data to TLC, cold data sinks to QLC.

FDP and cuFile operate at different layers — cuFile defines how the GPU initiates access; FDP defines how the SSD arranges placement. They're complementary. The real contest is this: NVIDIA wants cuFile to become the de facto standard for GPU-storage access (just as CUDA is for compute), while standards bodies want to keep storage interfaces vendor-neutral.

Mooncake / DualPath: The Software-Layer Route

Not every solution depends on a dedicated DPU. Moonshot AI's Mooncake uses a disaggregated architecture to organize KV Cache as a distributed caching resource — DRAM as the hot tier, SSD/NVMe as the cold tier, managed entirely in software. Mooncake Store supports multi-tier caching, allowing cold data to persist on higher-capacity, lower-cost flash tiers.

DeepSeek's DualPath achieved 98.7% KV Cache hit rate on a 660B production model, with throughput improvements of 1.87×/1.96×. This is a pure software solution requiring no specialized hardware (DPU/CXL).

Key judgment: ICMSP and Mooncake represent two competing routes. ICMSP puts intelligence in the DPU layer (requiring NVIDIA infrastructure); Mooncake puts intelligence in the software layer (running on commodity hardware). Which becomes mainstream depends on TCO — ICMSP has higher hardware costs but lower latency; Mooncake has lower hardware costs but latency is constrained by the general IO stack. No public head-to-head benchmark exists today.

Core Judgment

The protocol-layer competition points to one conclusion: the center of value in Agent storage architecture is shifting from the media layer to the scheduling layer. Media is substitutable — HBM, HBF, CXL DRAM, GP1 SSD; whoever is cheaper wins. But the scheduling interface — how the GPU initiates access, how the SSD understands data, where data is placed — determines who holds the most influence.

NVIDIA understands this clearly. cuFile + ICMSP + Storage-Next together form an attempt to become the de facto standard for Agent storage scheduling. It's the exact same strategy as CUDA for the compute layer.

V. What Agents Need Is Not a Traditional Filesystem

We made this call about a month ago. FMS 2026 didn't just validate it — it strengthened it.

Every step of Agent execution produces state changes — tool call results, intermediate reasoning artifacts, environment observation snapshots. A 100-step Agent run can produce 500+ state fragments, each ranging from a few hundred bytes to several KB. The gap between traditional filesystem design assumptions and Agent requirements:

ICMSP's FTL already proved the value of "making storage understand data semantics" at the KV Cache level — transforming from a generic block device into an attention-head-aware memory node. The next step is doing the same for Agent state: evolving from KV-aware FTL to Agent-aware FTL.

Current Agent frameworks (LangGraph's checkpointer, MemGPT's memory management, Letta's persistence layer) each handle state management in the software layer, with no semantic interface to the storage layer. When Agent state's causal dependencies, versioning, and namespace isolation can be directly managed by a DPU-resident FTL, Agent frameworks won't need to maintain these in Python process memory anymore.

VI. Three Open Questions

FMS 2026 resolved the hardware layer. Three questions remain open.

Question One: WAF Benchmarks Under Agent Workloads Are Missing

GP1 has 512-byte granularity and 50 DWPD, but no vendor has shown actual WAF under Agent workloads. The inference chain (512B writes → page waste → GC thrashing → WAF 3–5) is logically sound but lacks empirical validation.

The impact of this gap: enterprises procuring SSDs for Agent deployment can't accurately estimate lifespan and cost. If WAF really is 3–5, actual writes are 3–5× expected — a decisive impact on TCO calculations. We need the first large-scale Agent deployment's real-world data.

Question Two: DPU FTL from KV-Aware to Agent-Aware

ICMSP understands KV Cache semantics (attention head / prefix cache). Agent state is more complex — causal dependencies between tool call results, execution trajectory versioning, multi-Agent namespace isolation. These are currently handled by Agent frameworks in software, with no semantic interface to the storage layer.

When the DPU FTL can understand causal dependencies like "step 47's decision depends on step 23's tool output," the storage system can actively participate in Agent state management — prefetching state needed for step 48, automatically garbage-collecting historical versions no longer referenced. But this hasn't happened yet.

Question Three: The "CUDA Moment" for Agent Storage Hasn't Arrived

The hardware exists — five media tiers, three access paths, HBF in-package interconnect. ICMSP/SCADA defines the GPU→storage interface. But the scheduling decision of "which piece of data goes on which media" is still implemented independently by each inference engine (vLLM manages HBM-internal, Mooncake manages DRAM+SSD, ICMSP manages BF4+SSD). There's no unified abstraction layer.

This parallels early GPU computing — hardware existed, but CUDA hadn't shipped yet. The unified scheduling layer for Agent storage (if it arrives) would define the standard abstraction for "data type → media selection → access path," so Agent frameworks don't need to care where data physically lives. This abstraction doesn't exist today.

VII. Sober Assessment

FMS 2026 product signals let us re-tier the Agent storage landscape:

Already happened (product-level validation):

• Cross-media KV Cache management: ICMSP / Mooncake / DualPath — three independent routes validating the same direction
• GPU takes over storage scheduling: SCADA + cuFile open-sourced; Storage-Next initiative joined by 40+ vendors
• SSDs working at memory granularity: GP1 (512B / 10M IOPS / 50 DWPD) wins Best of Show
• New media filling the gap: HBF standard ($2–8/GB range), CXL 3.2 memory pools on the eve of mass deployment
• Agent state capacity no longer bounded by CPU DRAM: ICMSP 9600TB per SuperPod

Being told (products exist but lack independent validation):

• ICMSP's 82.7% TTFT reduction and 67% memory savings are vendor figures; await independent benchmarks
• CXL memory pools for large-scale Agent deployment. The supply chain has products but lacks structural drivers — CPU vendors are half-hearted, the software stack is immature, and NVIDIA pushes GPU-direct storage management over CXL. CXL's home turf may not be AI inference
• The evolution of Agent frameworks from "CPU process + remote API" to "GPU process + local inference + DPU state management"

Not yet happened:

• DPU FTL evolving from KV-aware to Agent-aware
• WAF benchmark under Agent workloads
• Unified scheduling layer for Agent storage (the "CUDA moment")
• The intersection of computational storage and Agent workloads

This direction is irreversible. As Agents move from concept to production deployment, storage systems are adapting to an entirely new mode of work. FMS 2026 delivered the first set of product-level answers: let SSDs understand KV Cache semantics (ICMSP FTL); let GPUs bypass the CPU to access storage directly (SCADA); let NAND work at memory granularity (GP1); let NAND connect directly into the processor package (HBF).

FMS 2024 was about "how to feed data into the GPU." FMS 2026 is about "how to let Agents remember what they're doing." ICMSP is NVIDIA's answer. Mooncake is Moonshot AI's answer. HBF is the OCP consortium's answer. Who will become the de facto standard isn't settled — but the problem itself has been claimed by the industry.

---

Primary Sources (by importance):

• FMS 2026 official agenda and keynote materials
• NVIDIA FMS 2026 keynote: cuFile open-source, SCADA architecture, ICMSP G3.5 layer, Storage-Next initiative
• Kioxia GP1 product page and Best of Show press release
• OCP HBF specification document (SK Hynix + Sandisk joint release, 2026-08)
• Samsung FMS 2026 roadmap: zHBM / zNAND-O / V10 BV-NAND / HBM4E
• Moonshot AI Mooncake technical blog / Mooncake Store documentation
• DeepSeek DualPath paper (2026-02), measured data from 660B production model
• Previous article: When Storage Becomes Agent Working Memory: FMS Three-Year Trend Migration (2026-07-18)
• Related: FMS 2026: The Storage Hierarchy Is Becoming Networked

Not investment advice. Data as of August 9, 2026.

[^1]: NVIDIA/Solidigm/AIC jointly published data; vendor-reported results awaiting independent benchmark validation. ICMSP architecture details and the 82.7%/67% figures originate from FMS 2026 technical presentation slides; SCADA performance data originates from the NVIDIA FMS 2026 keynote. No independent benchmark as of writing.