When Storage Becomes Agent Working Memory: FMS Three-Year Shift and Technical Roadmap Analysis

In August 2024, on the stage of the Santa Clara Convention Center, SK Hynix presented the production timeline for 12-layer HBM3E, while Kioxia demonstrated a broadband SSD with an optical interface. In August 2025, on the same stage, Kioxia won "Best of Show" with a 245.76TB SSD, and analysts warned that hyperscale data center capital expenditure was approaching 20% of total spend. In August 2026 (FMS 2026 is scheduled for August 4–7; the following analysis is based on the preview agenda), NVIDIA will discuss how to build the storage ecosystem for Agent workloads.

Three years, the same summit, three completely different stories.

This is not simple topic rotation. The Future of Memory and Storage (FMS), formerly the Flash Memory Summit—three years of agenda shifts reflect a fundamental transformation in the storage industry's understanding of AI. The core of this transformation runs deeper than media iteration: the role of storage systems in the AI industry chain is undergoing a four-stage transition.

Four Stages, Not One Trend

To understand the three-year evolution of FMS, one must first step outside the "one hot topic per year" narrative framework. In reality, AI storage technology has progressed through four overlapping stages, each driven by a different core tension:

Stage 1: Weight Pipeline Era (2023H2–2024)

Core tension: GPU compute capacity was growing far faster than data supply capability. Training cluster GPU utilization was bottlenecked by "feeding" speed.

Storage's role was straightforward: store training data, load model weights, periodically write checkpoints. Data flow was unidirectional, bulk, and sequential. The metrics that mattered were bandwidth (GB/s) and capacity (TB). HBM3e production pushed bandwidth past 1TB/s, QLC NAND and Zoned Storage pushed single-drive capacity past 128TB+, PCIe 5.0 delivered approximately 4 GB/s per lane, with x4–x16 SSD configurations reaching 16–63 GB/s.

During this stage, inference storage was simply not a topic. Inference engines managed KV Cache entirely within GPU memory. vLLM's PagedAttention treated it as "virtual memory in VRAM"—scheduling fragmentation elimination and concurrency improvement. The storage industry paid no attention to any of this, because KV Cache had not yet spilled to the storage layer.

Stage 2: In-GPU-Memory Management Era (2024H2–2025H1)

Core tension: Context windows expanded from 32K to 128K and even 1M tokens. Single-request KV Cache grew from a few GB to tens of GB. An 80–192GB HBM package was no longer sufficient.

The first response was software-level: FP8 KV Cache quantization and compression, trading precision for space. The second response was CPU offload: pushing inactive KV Cache to the CPU-side DRAM. These solutions remained within the "inference engine internals." The storage industry still stood on the sidelines.

But pressure was accumulating. Taking Llama 2 70B as an example (80 layers, 8 KV heads, head_dim=128, FP16), per-token KV Cache is approximately 320 KB; at 128K context a single request consumes about 40 GB, while 1M context exceeds 300 GB. A few concurrent requests would exhaust HBM. KV Cache spillover was only a matter of time.

Stage 3: Cross-Tier State Management Era (2025H1–2025H2)

This was the turning point.

When KV Cache inevitably spilled to SSD, a fundamental cognitive shift occurred: KV Cache transformed from "a data structure internal to the inference engine" into "a system object requiring cross-tier HBM→DRAM→SSD management."

The even more critical catalyst was Prefix Cache. When systems discovered that multiple requests shared identical prefixes (system prompts, long documents, few-shot examples), KV Cache became a reusable asset across requests. This immediately raised three storage-system-level questions:

1. Naming: How to identify a prefix KV Cache block so subsequent requests know it exists?
2. Addressing: Is it on HBM, DRAM, or SSD? How to move it to the right position before the GPU needs it?
3. Lifecycle: When to evict? LRU or request frequency? How to share across nodes?

These three questions are fundamentally storage system questions. At this moment, KV Cache transformed from a data structure into a storage object. The storage industry was finally pulled into the critical path of inference.

NVIDIA's actions confirm this judgment. At GTC 2025, alongside the Vera Rubin GPU launch, the company devoted significant attention to the CMX (Cache-Mix) architecture, extending KV Cache management across the full chain: BlueField DPU + NVMe SSD + Spectrum-X network. The Dynamo inference framework included built-in KV Cache routing. This was not a makeshift "use SSD when you run out" solution—it was a system-level architectural redesign.

At FMS 2025, KV Cache began appearing sporadically as an independent topic. CXL discussions deepened in parallel, shifting from 2024's "how to connect" to "how to manage and use," with the core scenario still being database memory expansion.

Stage 4: Agent Working Memory Era (2025H2–2026)

Agents are not single-turn inference.

An Agent loop works like this: observe environment → reason and decide → call tool → get result → update state → reason again → ... potentially lasting dozens of rounds, across multiple sessions.

This working pattern imposes three entirely new categories of pressure on storage systems. FMS 2026's 28 AI&ML keynotes (based on preview agenda; final numbers may change), including 5 Agent-related and 7 KV Cache-related sessions, are a concentrated manifestation of these three pressures.

Three Dimensions of Impact

Dimension 1: State Persistence — From Passive Container to Active State Management

Traditional inference is stateless: a request arrives, the model generates a response, done. Agents are stateful: every decision, every tool call result, every intermediate planning artifact must be persisted for cross-session recovery, backtracking support, and multi-Agent collaboration.

Storage systems therefore transform from "passive containers" (store whatever you're given) to "active state managers." An Agent's state is not a traditional "file"—it is closer to a continuously flowing state stream containing environment observations, tool outputs, execution progress, and planning paths.

This gives rise to a deeper proposition: Agents do not need traditional file systems.

The reasoning unfolds as follows. Each step of Agent execution produces state changes—tool call results, intermediate reasoning artifacts, environment observation snapshots. An Agent running 100 steps may produce 500+ state fragments, each ranging from a few hundred bytes to several KB. These fragments have causal dependencies (step 47's decision depends on step 23's tool output), require versioning (rollback to step 23 for re-execution), and require isolation (different Agent instances must not pollute each other's state).

The mismatch between traditional file system design assumptions and these requirements becomes clear when examined point by point:

Addressing: Agents do not care about file names or directory paths. They need to "find the result returned by the search tool at step 23." Path-based addressing means the Agent must maintain a "path → meaning" mapping—an extra burden. Content addressing (hash-based) is a natural fit: the hash of a state fragment serves as both unique identifier and integrity check.

Consistency: POSIX strong consistency requires all clients to see the same file system state at the same time. But in Agent scenarios, 100 concurrent Agents each update their own state without directly depending on each other—what they need is causal consistency (if Agent B reads Agent A's output, B must see A's latest version; but if B and A are independent, no synchronization is needed). Strong consistency introduces unnecessary coordination overhead.

Versioning: Each Agent state update is equivalent to a commit. Manual version control across a 100-step task means 100 explicit saves—impractical. Auto-versioning is needed: every write automatically creates a new version, with rollback to any version.

Isolation: Traditional file systems isolate by user permissions (UID/GID), at user-level granularity. Multi-tenant Agents need instance-level isolation—each Agent has a completely independent namespace and cannot accidentally overwrite another Agent's state.

Access granularity: Traditional files are read and written in KB–MB units. Agent state fragments are typically a few hundred bytes, far smaller than a file system block.

The more accurate description, then, is not "Agent file system" but "Agent state management layer." This layer might be built on top of a traditional file system (as a software abstraction—an opportunity for storage system vendors), or it might bypass the file system to manage block devices directly (an opportunity for SSD controller and firmware layers). The industry implications of these two paths are completely different.

Dimension 2: IO Pattern Inversion — From Sequential Bandwidth to Random IOPS

This is the most underestimated change among the three dimensions.

For the past few years, the performance narrative around AI storage has revolved around sequential bandwidth. Training data reads, model weight loading, checkpoint writes—all are large-block sequential IO. Storage vendors competed to showcase GB/s-level throughput numbers.

Agent workloads shatter this assumption. Each step of Agent execution—observing the environment (RAG retrieval producing random reads), calling tools (producing small-file writes), updating state (random read/writes of state objects), saving snapshots (periodic medium-sized writes)—generates fine-grained random IO. When multiple Agents execute concurrently, these operations compound into massive random IOPS pressure.

Traditional inference's demand on SSD was essentially limited to "loading weights at startup"—bandwidth mattered, IOPS did not. Agent workloads push the SSD into the position of "Agent workspace persistence layer," where IOPS and p99 tail latency become core metrics.

What does this mean for SSD controllers? The answer requires reasoning from NAND flash's physical characteristics.

NAND flash's minimum write unit is a page (typically 4–16KB), but its minimum erase unit is a block (containing dozens to hundreds of pages, typically 1–4MB). Overwrites are not possible—to modify data in a page, the entire block must first be erased and rewritten. This is the root cause of Write Amplification Factor (WAF).

Under traditional AI workloads, writes are large-block sequential (checkpoints of several GB at a time). The FTL can efficiently write to empty pages in batches, and background GC reclaims invalid blocks in bulk, keeping WAF close to 1. Latency is predictable because GC operates quietly in the background.

Agent workloads break this equilibrium. 1,000 concurrent Agents, each updating a 512-byte state fragment every few hundred milliseconds. The FTL faces:

1. Each 512B write consumes an entire 4KB page (since page is the minimum write unit)—the remaining 3.5KB is wasted
2. These scattered random writes quickly fill empty pages, forcing GC to trigger more frequently
3. GC must read valid data, migrate it to a new block, then erase the old block—this process consumes bandwidth and produces latency spikes
4. Valid data is scattered across many blocks (because writes are random), so GC migrates a high proportion of valid data, and WAF may deteriorate from 1.1 to 3–5
5. WAF deterioration means actual writes are 3–5× the logical write volume, accelerating SSD wear

The cascading implications unfold layer by layer:

• FTL: Must shift from "large-block sequential allocation" to "fine-grained random write WAF control"—finer-grained page mapping, write coalescing, and intelligent recycling strategies aware of Agent state lifecycle
• GC: Background batch GC's latency spikes are unacceptable in Agent scenarios. A 50ms GC pause, for an Agent interaction requiring p99 < 10ms, is a timeout. Must shift to low-latency foreground GC—more frequent but reclaiming fewer blocks per cycle, trading peak load for higher frequency
• OP (Over-Provisioning): Larger OP gives the FTL more room to maneuver, reducing GC frequency. Typical enterprise SSD OP is 7% (1.92TB capacity corresponds to ~28% physical space); Agent scenarios may require 15–25% to maintain stable latency
• DRAM Buffer: Shift from large-block prefetch (effective for training data loading) to small-file hot-spot caching—identifying which Agent state fragments are hot (frequently read/written) and keeping them in DRAM to reduce NAND access

Dimension 3: Memory Semantics Extension — CXL's Agent Moment?

To understand this problem, we first need to clarify two fundamentally different access models.

Memory Semantics: The CPU accesses data directly via load/store instructions with byte-level addressing. Hardware maintains cache coherence, and latency is in nanoseconds. Executing load rax, [0x7ffe] completes a read in a single instruction—no system call, no driver, no interrupt. Atomic operations (CAS, fetch-add) are guaranteed directly by the CPU cache coherence protocol (e.g., MESIF).

Storage Semantics: Data is organized in blocks (typically 4KB). Access requires traversing the full I/O stack: application calls read()/write() → system call → VFS → filesystem → block device layer → NVMe driver → SSD controller. Even an optimal 4KB read takes 10–50μs—100–500x slower than memory access. Writes require additional fsync() for persistence guarantees.

Where Does Agent Break?

Agent runtime state access follows a specific pattern: high frequency (multiple accesses per loop step), fine granularity (tens of bytes to a few KB per state fragment), random (different Agent instances access different states), and latency-sensitive (p99 tail latency directly impacts response time). This is fundamentally a memory-semantics workload.

But Agent state must also be persisted—process crash recovery, multi-Agent state sharing, and audit trails all require durable storage. The persistence layer (SSD) provides storage semantics. This creates a structural gap:

• Reading a 256-byte state fragment → SSD must read an entire 4KB block → 99% of data transfer is wasted
• Updating one field → read-modify-write the entire block → write amplification
• 1,000 Agents each updating their own state concurrently → 1,000 × 4KB I/O requests → effective data under 1MB, but I/O queue is saturated

Today's Stopgap: Layered Software Translation

Current practice relies on software translation: Agent runtimes keep hot state in DRAM, periodically checkpointing to SSD. This is essentially manual cache hierarchy management. The problems:

1. DRAM capacity is limited (typical servers: 256–512GB), while highly concurrent Agent working sets can easily exceed 1TB
2. Long checkpoint intervals → crash data loss; short intervals → checkpointing itself becomes the bottleneck
3. Every Agent framework implements its own state management layer—no unified abstraction, massive reinvention

Why CXL Is a Structural Solution

CXL (Compute Express Link)'s CXL.mem protocol allows the CPU to directly access memory on CXL devices using memory semantics (load/store), with end-to-end latency of approximately 170–250ns (including root complex overhead; protocol-layer transaction latency is approximately 20–40ns)—between local DRAM (~100ns) and NVMe SSD (10–100μs).

This means Agent state can reside on CXL-attached DRAM, and the CPU can read/write it using ordinary pointers—no I/O stack involved. Meanwhile, the CXL device can manage its own persistence strategy (e.g., background flush to SSD) transparently to the CPU.

But here lies a deeper question: Should storage devices directly support memory semantics?

Two routes are competing:

• Route A: CXL DRAM as a front tier. CXL-attached DRAM serves as the hot state layer; SSD serves as cold storage; a software management layer handles tiering. CXL handles memory, SSD handles storage—each does what it's good at. Advantage: clean architecture. Disadvantage: software-managed tiering adds non-trivial latency overhead.
• Route B: Storage devices expose a memory-semantics interface directly. SSDs present a byte-addressable interface to the CPU via CXL.mem, enabling direct load/store access to SSD-resident data. Intel Optane Persistent Memory (discontinued) and Kioxia XL-FLASH both attempted this path. The technology was proven viable, but media cost and endurance issues led to product failure.

Currently, Route A is closer to production deployment because it requires no new storage media—existing DRAM + existing SSD, bridged by CXL. Route B requires storage-class memory (SCM), and Optane's exit demonstrates that this media category has not yet achieved economic viability.

If Memory Semantics Were Supported, Which Operations Matter Most?

From the Agent workload perspective, the operations worth prioritizing are not the full POSIX suite, but a very narrow instruction set:

1. Byte-level load/store: Read/write by byte or word, not by block—solves the 99% waste in 4KB block reads
2. Atomic operations (atomic CAS / fetch-add): Lock-free synchronization for concurrent multi-Agent state updates
3. Persistence fences (x86 clwb + sfence): Explicitly tell hardware "these writes must reach persistent media," distinguishing volatile cache from durable data
4. Zero-copy transfer: Direct data movement between CXL memory and GPU without CPU mediation (feasible when the CXL coherence domain covers the GPU)

These four operations cover 90% of Agent state management needs. By contrast, the full POSIX file semantics (directories, permissions, links, truncate) are almost useless for Agent scenarios—Agents don't need chmod.

Three Prerequisites for CXL in Agent Scenarios:

1. CPU-side maturity: Full CXL 2.0 support requires Intel Diamond Rapids-class or newer platforms, and the root complex's protocol conversion overhead must be sufficiently low.
2. CXL memory device economics: CXL Type 3 devices (pure memory expansion cards) require sufficient suppliers and production volume to drive prices to acceptable levels.
3. CXL-aware Agent frameworks: Agent runtimes need to know which states should reside on CXL memory and which can spill to SSD. This "data placement policy" layer does not yet exist.

In recent years, CXL's limited deployments have been concentrated in core database memory expansion. Agent scenarios are technically an ideal application for CXL, but production deployment may still be 1–2 years away. If actual CXL + Agent benchmark data appears at FMS, it would be more worth tracking than any concept PPT.

Two Undercurrents of Compute-Storage Convergence

The three dimensions above are directly exposed in the FMS agenda. There are two additional undercurrents that have not yet explicitly intersected with Agent topics in the summit agenda but are worth watching.

Undercurrent 1: Computational Storage

Agent runtimes have a category of high-frequency operations: vector retrieval (similarity search in RAG), data filtering (selecting relevant fragments from large volumes of tool output), state aggregation (merging outputs from multiple Agents). The computation is modest, but the data movement is substantial—traditional architectures must move data from SSD to CPU memory, the CPU performs the computation, then the result is moved back.

What is the cost of this movement? An optimal NVMe read takes 10–50μs, while a single vector dot product computation takes only nanoseconds. If the dataset is a 1GB vector index (medium scale), moving it entirely to the CPU for retrieval consumes 10–50ms (limited by SSD bandwidth), while the actual computation may take less than 1ms. Data movement time is 10–50× computation time.

Computational storage (ARM/RISC-V or FPGA embedded in the SSD) follows a direct logic: since the data is already on the drive, why not perform these simple computations in-drive and return only the results? A 100-dimensional vector similarity comparison yields only a few bytes of result—no need to move 1GB of data.

Currently, the computational storage standard (NVme CS) is still in early stages, and its intersection with Agent scenarios is virtually zero. But the math above is straightforward: if the Agent's compute-to-data ratio is low enough (simple computation but large data volume), computational storage delivers order-of-magnitude gains.

Undercurrent 2: DPU/NIC Storage Offload

First, let us clarify what problem DPU solves in the storage path.

Agent multi-tenant deployment means: a single server runs dozens to hundreds of Agent instances, belonging to different users or business lines. These Agents share the same set of SSDs, but require:

1. Bandwidth isolation: Agent A's large checkpoint writes must not crowd out Agent B's real-time state read/write bandwidth
2. Encryption isolation: Agent A must not be able to read Agent B's state data
3. Access control: Agents may only access authorized storage regions

The CPU overhead of doing this in software: bandwidth isolation requires per-tenant IO queue scheduling (CPU cycles); encryption requires AES-256 encrypt/decrypt (CPU AES-NI instruction throughput is approximately 3–5 GB/s/core; multiple concurrent Agent encrypt/decrypt operations quickly saturate CPU cores); access control requires a policy check on every IO (microsecond-level, but cumulative). With a typical 8-channel NVMe SSD at full load, encryption/decryption alone may consume 2–4 CPU cores.

DPU (such as NVIDIA BlueField) comes with its own ARM cores, crypto engines, and network interfaces. Offloading IO scheduling, encryption/decryption, and access control to the DPU: the CPU issues an ordinary read request, and the DPU performs permission checks, decryption, and bandwidth rate-limiting on the data path as it returns from the SSD—the CPU receives clean, ready-to-use data. CPU cores are freed for inference and Agent logic.

As Agents move from single-node to distributed, the DPU's role expands further: cross-node KV Cache sharing requires RDMA + consistency protocols, and the DPU naturally has the network interface and programmability to handle this. NVIDIA's CMX architecture placing BlueField in the critical KV Cache management path is precisely the productization of this logic.

Who Is Pushing What

FMS 2026 features 28 AI&ML keynotes and 24 participating vendors. Each category of player has different motivations:

Agenda heat does not equal deployment heat. NVIDIA's CMX architecture and KV Cache management chain already have product-level validation—they are "happening now." Agent-native state management layers and CXL Agent memory pools are more "being narrated."

Sober Assessment

The three-year evolution of FMS reveals a genuine trend: the role of storage in the AI industry chain is shifting from passive infrastructure to active state management systems. Agent workflows are indeed reshaping storage architecture. State persistence, IO pattern inversion, and memory semantics extension—these three dimensions are not vendor-invented stories.

But they must be assessed in layers:

Already happening: KV Cache has transformed from an engine-internal data structure to a cross-tier storage object. The emergence of Prefix Cache is the watershed moment—it turned KV Cache into a storage asset requiring naming, addressing, and lifecycle management. NVIDIA's sustained investment from GTC through FMS demonstrates that this direction has passed product-level validation. FMS 2026 dedicating 7 keynotes to KV Cache is not empty talk.

Being narrated: Agent-native state management layers. Agent workloads do generate random IO, but there is a gap between "needing a new storage architecture" and "market-ready solutions being available." Among FMS's 28 keynotes, the number with actual production Agent deployment data likely does not exceed a handful.

Not yet happening: Large-scale CXL deployment in Agent scenarios. The technical logic holds, but requires simultaneous maturation of CPU platforms, device ecosystems, and software stacks. This judgment may not be verifiable until 2027–2028.

Analyst Jim Handy's warning at FMS 2025 about unsustainable hyperscale data center capital expenditure remains valid. If AI infrastructure investment enters a correction period in the second half of 2026, the Agent storage narrative may experience an awkward window of "PPT first, deployment later."

The direction is irreversible. As Agents move from concept to production deployment, storage systems must adapt to an entirely new working mode—not storing large files or reading large data blocks, but frequently, randomly, and with low latency reading and writing Agent runtime state. This will have sustained impact on SSD controllers, state management abstractions, and memory expansion architectures.

FMS 2024 was about "how to feed data into the GPU." FMS 2026 is about "how to let an Agent remember what it's doing."

This is not an upgraded version of the same problem. This is a new problem.

---

Primary Sources (by importance):

• FMS 2024 / 2025 / 2026 official agendas and keynote materials
• Jim Handy, "11 Key Changes Coming," FMS 2025
• Kioxia, "Optimizing AI Infrastructure Investments with Flash Memory," FMS 2025
• NVIDIA GTC 2026: Vera Rubin, CMX Architecture, BlueField-4, Dynamo
• Saniffer / CSDN, "FMS 2025 闪存峰会参会情况和技术趋势会后分析" (FMS 2025 Flash Memory Summit Attendance and Post-Show Technology Trend Analysis), 2025-08
• 半导体产业纵横 (Semiconductor Industry Panorama), "11 个关键词，看透存储趋势" (11 Keywords to Understand Storage Trends), 2026-01

For in-depth technical analysis, see the companion piece: "When AI Agents Reinvent the File System: From 'Everything Is a File' to 'Everything Is Context'"

Not investment advice. Data as of July 17, 2026. FMS 2026 has not yet convened; agenda analysis is based on preview information.