1. The Gap Between a Trillion Parameters and One Machine
Fitting DeepSeek V4-Pro into a single machine begins with arithmetic. The model totals 1.6 trillion parameters and activates only 49 billion per token, but inference has to read the entire weight file: under the quantization scheme given in the Huawei paper, the weights occupy 865 GB. One H200 carries 141 GB of GPU memory, and 865 divided by 141 comes to about 6.1, so a single card cannot hold even a sixth of the weights. Eight of them pool to 1.13 TB, only about 30 percent of headroom over the requirement, and that is on the premise that not one byte of the 1.13 TB is spent on anything else, with KV cache and activations budgeted separately.
Kimi K3 is tighter still: 2.8 trillion parameters, 104 billion activated, 1,560 GB of weights; an eight-card machine is out of the question. Even the 512 GB of unified memory in a maxed-out Mac Studio holds less than 60 percent of DSV4-Pro's weights.
This arithmetic gets harder to dodge in 2026, because open-model sizes have collectively crossed the trillion mark. DSV4-Pro, Kimi K3, and Qwen3.8-2.4T fill the front rows of the leaderboards, and all are sparsely activated mixture-of-experts (MoE) architectures: the model consists of many "expert" sub-networks, and a router sends each token to only a small handful of them. Three numbers therefore come apart: total parameter count determines how much the model "knows," activated parameter count determines the compute cost of each token, and the size of the weight file follows the total. A model can keep growing in knowledge without its per-token compute cost climbing in proportion.
The slope keeps steepening: Kimi K3 used native MXFP4 (4-bit microscaling float) quantization-aware training to become, in its maker's words, "the first open 3T-class model"; Qwen3.8 activates about 95B against 2.4T total parameters, a ratio that by the official figures has slipped to around 4 percent. The gap between compute demand and memory demand widens year by year, and memory becomes the bottleneck first.
The industry's mainstream answer is to split, taking the model apart across a cluster. Expert parallelism places different experts on different GPUs; prefill (reading in the entire prompt at once) and decode (generating token by token) are disaggregated, so the two phases run on machines that suit each; and a large enough request pool amortizes the cost of keeping the whole model resident. This path carries a premise that rarely gets said out loud: concurrent requests numerous enough to justify trading an entire cluster's memory bandwidth for throughput.
One class of workload fails that premise by nature. An enterprise agent workbench often has only single-digit active sessions; a sovereign inference deployment needs the model inside its own datacenter; a local evaluation setup wants it on tap. The three share low concurrency, many turns, and long contexts. Building a cluster for one to eight concurrent sessions fails the cost accounting, with the machines spending most of their time paying power and depreciation for concurrency that does not exist.
So the question gets concrete: if the goal is to pack a trillion-parameter model into one machine, how should that machine's memory be provisioned?
The paper Huawei filed to arXiv on September 14, "Trillion-Parameter MoE in a Box: Decoupling Memory Provisioning with High-Bandwidth Flash," answers exactly this provisioning question. What the paper does, stated before the conclusions:
- Subject: DSV4-Pro and Kimi K3, two open trillion-parameter MoE models; the workload portrait comes from operator analysis, a measured expert-routing trace (a record of which experts each token actually touched), and three multi-turn agentic serving traces;
- Method: the workloads replay on the LLMServingSim 2.0 simulator while two key provisioning dimensions are swept, state-tier bandwidth and the HBF bandwidth opened toward the host, with the pass line set at completion time within 1.10x of the reference configuration;
- Output: two conclusions you can write straight into a procurement list. Commodity memory modules are enough for the state tier; HBF needs to open only about a quarter of its internal bandwidth toward the host;
- Boundary: this is a design-space exploration answering "how a 2027 device should be provisioned"; measured performance waits for real silicon.
The paper's core conclusion fits in one sentence: once the weights move into High-Bandwidth Flash (HBF), the word "memory" splits into three independent budget lines, and each line can be provisioned separately at its cheapest point. What remains, for a trillion parameters in a single machine, is configuration arithmetic.
2. HBF: NAND Inside the Processor Package
To see why the three knobs come apart, first meet the new medium: HBF.
HBF's idea fits in one sentence: stack NAND flash the way HBM is stacked, and put it inside the processor package. Two terms here need unpacking. NAND flash is the base medium of SSDs and phone storage, dense and non-volatile, but slow to access. HBM (High Bandwidth Memory) is the highest-bandwidth memory beside the GPU today: multiple DRAM dies and a logic layer are stacked together and packaged side by side with the compute chip on a silicon interposer (a silicon substrate that carries interconnect wiring on behalf of several chips), with bandwidth counted in TB/s. A conventional SSD's bottleneck sits on the path: flash dies reach the CPU only through a PCIe bus and a controller chip, a dozen or so GB/s is the practical ceiling, and the potential of the NAND itself is throttled by that path.
HBF tears the path out: multiple NAND dies stack on a logic base die and connect directly to the processor over a UCIe (Universal Chiplet Interconnect Express) interface, and the sheer parallelism within and across dies pushes read bandwidth into the TB/s class.
Sandisk's first generation, by the official numbers, is 256 Gb per die and 512 GB per 16-die stack with 1.6 TB/s of read bandwidth, physical footprint and power profile tracking HBM4 closely; on the roadmap, the second generation aims at 1 TB stacks and 2 TB/s, the third at 1.5 TB and 3.2 TB/s. All of these figures come from vendor internal testing and simulation, with no third-party verification; this qualifier will recur.
The medium has a natural division of labor: fast to read, slow to write. Writes are slow, and they also consume flash lifetime, on the order of one hundred thousand program cycles by the SK hynix paper's accounting. It suits write-once, read-many data, and inference happens to contain one enormous read-only block: the model weights. A talk at Hot Chips 2026 gave the ratio (as relayed in the HBFSim paper): roughly 93 percent of a trillion-parameter model's bytes are MoE expert weights. Medium and workload meet here: HBF looks like a warehouse built for exactly this class of data.
Standardization has moved unusually fast. Sandisk published its first technical brief in July 2025; on February 25, 2026, it launched a dedicated workstream with SK hynix inside the Open Compute Project (OCP, the open-standards body for datacenter hardware); and on August 3, a little under half a year later, the first HBF high-level base die specification, v0.7.0, went up in the OCP document library.
The spec fixes two things: stacking in 8-high and 16-high form factors with capacity up to 512 GB, and bandwidth in three grades from Grade1 to Grade3, spanning roughly 0.4 to 3.0 TB/s. The interconnect is UCIe, and the spec attaches electrical characteristics, stacking-package reliability, and a software read/write guide. This is the first time the storage industry has connected NAND to a processor through an in-package standard interface. Per the vendor roadmap, HBF memory samples arrive in the second half of 2026, and the first HBF-bearing inference devices sample in early 2027.
Placing the memory media on one chart makes the layout obvious. HBM3e delivers 36 GB and 1.2 TB/s per stack, a bandwidth-to-capacity ratio (bandwidth divided by capacity, a measure of how much bandwidth each GB of capacity carries) of 33.3 s⁻¹. HBF delivers 512 GB per package with 1.6 TB/s of internal read bandwidth, 14 times HBM's per-stack capacity (512 against 36), while the bandwidth-to-capacity ratio drops below a tenth, about 3.1 versus 33.3 s⁻¹.
LPCAMM2 is a commodity memory-module form factor already on the market, 64 GB at 136.5 GB/s, a figure that works out exactly to LPDDR5X-8533 times a 128-bit bus (8533 MT/s × 16 bytes; MT/s, megatransfers per second). SOCAMM2 is the server-facing module form factor from the same family, with larger per-module capacity; at the 9600 MT/s roadmap point it works out to 153.6 GB/s. At the far right sits the NVMe SSD, with capacity to pile on and bandwidth stuck in the teens of GB/s. HBF fills a position no product previously occupied: read bandwidth close to HBM, with NAND's capacity and cost structure.
3. Splitting "Memory" into Three Knobs
A conventional inference node packs everything into one medium: weights, KV cache, activations, runtime state, all in HBM. The KV (key-value) cache is the intermediate result a model saves for the tokens it has already processed; long conversations rely on it to avoid recomputing, at the price of a footprint that keeps growing with context length. This single-medium setup has only one knob: capacity and bandwidth are bound to the physics of the same medium. A bigger model demands more HBM; more HBM brings more bandwidth along, whether or not the bandwidth is needed; once HBM tops out per card, you add cards, then machines, then network. Every step pays for bandwidth that goes unused.
The industry had already rendered this judgment: "Memory and interconnect have displaced compute as the chief constraint in inference hardware." The judgment carries authoritative backing: David Patterson and Xiaoyu Ma, in the May issue of IEEE Computer this year, ranked the research directions for inference hardware and put HBF first among the four, with a one-line definition: 10x capacity at HBM-class bandwidth. The Huawei paper steps past that consensus and re-describes the problem itself.
The paper's workload analysis shows the coupling can be taken apart, because the three classes of data scale in completely different ways inside the machine.
The first class is the weights. Their scale follows the model and is nearly independent of concurrency: by the paper's measurement, at a concurrency of 8 the weights account for 95.9 percent of resident data (DSV4-Pro) and 87.1 percent (Kimi K3). The terabyte-scale capacity demand comes almost entirely from the weights, which is precisely HBF's territory: capacity-hungry, read-heavy, write-light.
The second class is runtime state, mostly KV cache, which follows context length and concurrency. A single one-million-token sequence occupies 4.66 GB on DSV4-Pro and 28.99 GB on Kimi K3, whose attention is organized differently; going by the heavier of the two numbers, eight concurrent Kimi K3 sequences total about 232 GB, and with serving overhead on top, 256 GB is a reasonable capacity floor for the state tier. Next to the weights this is a small number, and DRAM is enough to hold it. Why it is enough, section 4 answers with numbers.
The third class is activated-expert traffic, which follows routing behavior. A measured routing trace from a SPEED-Bench code session supplies the key number: 16 effective tokens activate 51.6 experts on average, and 57.8 percent of the activated experts receive exactly 1 token. This number explains the bandwidth predicament of MoE decoding: a routing decision pulls some expert's weights out of storage to serve one or two tokens, reuse is minimal, and the decode phase is therefore acutely sensitive to the latency of weight delivery. How much of HBF's internal bandwidth opens toward the host, and how wide the package pins run, are decided by this traffic.
Memory provisioning thus goes from one knob to three: HBF capacity follows resident weights, DRAM bandwidth follows runtime state, and host transport follows activated-expert traffic. The paper's own phrasing in its conclusions: three resources, each following its own workload driver. Taking memory apart is not in itself a new idea; what is new is the quantification that follows: how far each of the three knobs must turn, and how independent they really are.
Before the numbers, the experimental setup in full. The workloads replay on the LLMServingSim 2.0 simulator, an event-driven simulator that drives a memory-system model with real traces; no real machine takes part anywhere in the loop. Compute is fixed at 450 TFLOP/s of FP8 (8-bit floating point, the working precision for current inference acceleration) throughput per die. The HBF device parameters come from the OCP specification and the FlashAccel paper's modeling. Where the three agentic traces originate, the paper does not say. In other words, every knee in what follows is a simulation result; its credibility depends on how close those three layers of modeling sit to reality, and the final verdict belongs to real hardware.
4. Two Answers: Commodity Modules, and One-Quarter Exposure
The paper's evaluation framework takes completion time as its yardstick, in three steps. First, run an entire multi-turn conversation to the end under a given memory configuration and record the completion time. Second, compare against a reference configuration with the state-tier bandwidth saturated, allowing 10 percent degradation, that is, completion time within 1.10x of the reference; this line is referred to below as the 1.10x contour. Third, under that pass line, sweep the two dimensions, DRAM bandwidth and per-package HBF exposure toward the host, and find the least-resource combination that clears it.
The result lands in an L shape: completion time degrades sharply only when either resource enters its starvation region. Stacking DRAM bandwidth to 9.6 TB/s cannot rescue an HBF opened to a single transport quantum; opening the HBF host interface all the way cannot rescue a 0.14 TB/s state tier. The paper's own wording is "largely orthogonal": once a resource leaves starvation, piling on more barely moves the other knee. The provisioning goal is the lower-left corner of the L, standing inside the 1.10x line with the least resources.
The first answer concerns the state tier: once the weights move out, commodity memory modules are enough. At the 256 GB capacity floor, holding the 1.10x target requires a DRAM bandwidth-to-capacity ratio of only 4.0 s⁻¹ (DSV4-Pro) and 1.4 s⁻¹ (Kimi K3), against HBM3e's 33.3 s⁻¹, a relaxation of 8 to 24 times. Multiplying 4.0 s⁻¹ back against the 256 GB floor, DSV4-Pro needs about 1.02 TB/s of state-tier bandwidth; in procurement terms, the paper's configuration is eight LPCAMM2 modules, 512 GB totaling 1.09 TB/s, just over the line. Kimi K3 asks for less, and four LPCAMM2 modules already suffice.
SOCAMM2, competing in the same arena, is the one that drops out: its bandwidth-to-capacity ratio is only 1.20 s⁻¹, not enough to carry DSV4-Pro's 4.0, and in the paper's simulation a two-module 128 GB configuration completes in about 1.5 times the reference time, clearly outside the 1.10x pass line. The state tier thereupon exits the advanced-packaging contest, turning from an in-package resource of the compute die into a board-level resource. A demotion in status, and a discount in price.
The second answer concerns HBF exposure toward the host: one quarter is enough. A package's internal read bandwidth is 1.6 TB/s, and intuition says open as much of it to the host as possible; the bandwidth is in there, so why not bring it out? The paper's numbers say you don't need to.
Host links come in 96 GB/s quanta, the minimum step of per-package exposure; the knee lands at 4 to 8 quanta per package: six HBF packages each exposing 4 quanta, 384 GB/s per package (4 × 96) and 2.30 TB/s in aggregate (6 × 384). The full-exposure reference used for comparison is 6.14 TB/s, the full-link aggregate of 64 quanta × 96 GB/s; 2.30 against 6.14 is 62.5 percent lower, and both models still hold the 1.10x line across all three traces. This configuration's exposure ratio is 0.24, computed as the per-package 384 GB/s against the 1.6 TB/s of internal bandwidth, a bit over a quarter; it and the 62.5 percent are ratios at two levels, one against per-package internal bandwidth, the other against the full-link aggregate.
The third knob, capacity, is not on the sweep axes; model size locks it directly. But how that capacity is spent across packages matters. Taking the same fully used 64-quantum host link, there are two routes: HBF×4 with each package fully opened at 16 quanta, or HBF×8 at 8 quanta per package (4×16×96 and 8×8×96 both equal 6,144 GB/s). On the decode-heaviest trace, the latter runs about ten percent faster. The difference sits inside the packages: more packages mean more parallelism across dies and planes (the units within flash that can read and write in parallel), a larger internal bandwidth pool, and the option of near-data compute, none of which a wider link can buy. Capacity is floored by model size; performance is bought with package count.
How much HBF-to-host bandwidth to provision is becoming a first-class question in HBF system design, and academia supplies independent corroboration: KAIST's DASH architecture opens two delivery paths for HBF-resident weights, one direct to the GPU and one relayed through the HBM base die, with routing assigned at expert granularity; on a representative workload its throughput is 1.94 times a relay-only design. Huawei tunes exposure, KAIST opens a second path, and both adjust the same variable: how the weights' route from storage to compute can be made cheapest.
Taken together, the two answers point to an engineering conclusion worth real money. The state tier goes to board-level LPCAMM2; HBF needs only partial exposure; per-package link width keeps falling as packages are added; and host-interface bandwidth density need not scale in proportion with HBF capacity. On that basis the paper notes that the UCIe-S standard packaging path (organic substrate) becomes a viable option, with no further dependence on UCIe-A-class advanced packaging (both are physical implementation grades of UCIe: S runs on the traditional organic substrate, A requires a silicon interposer or a redistribution layer). The machine's three most expensive items fall into place accordingly: NAND capacity at flash prices, state bandwidth at commodity-module prices, packaging at organic-substrate prices.
5. The Skeptics Marked the Same Land
HBF has serious skeptics, and their homework is solid.
GPU IP company OXMIQ ran a dedicated HBF applicability analysis at Hot Chips 2026, and the press boiled the conclusion down to one deflating sentence: HBF cannot replace HBM across the vast majority of workloads. The method puts a 72-GPU rack to a trillion-parameter model (FP4, 4-bit floating-point quantization) and compares three configurations at equal cost and power: all-HBM delivers 20.7 TB of memory and 1,584 TB/s of aggregate bandwidth; all-HBF balloons capacity 14-fold to 294.9 TB while bandwidth falls to 922 TB/s; a hybrid sits in between, with bandwidth floating between 279 and 1,418 TB/s depending on load.
OXMIQ's extrapolation shows that as concurrent users multiply and interactivity speeds up, the HBM rack's cost-per-token advantage starts to reverse the comparison, because at that point the system is stuck on bandwidth while HBF's capacity mostly idles. Cheap bytes do not equal cheap tokens.
Read on its own, the analysis puts HBF in an awkward spot. But plot the sweet zone OXMIQ drew onto the map, and it gets interesting: the applicability it assigns to HBF is low-concurrency, capacity-bound, small-scale local or enterprise-private deployment, with big racks explicitly excluded. And the appliance the Huawei paper defines sits at concurrency 1 to 8.
It is the same land. Skeptics and designers have marked the same region on the map, and the disagreement is only over what to call it: one side says "HBF's only opportunity," the other says "the vast territory beyond HBF's boundary."
Beyond the industry analysis, the academic debate is finer-grained: five papers, each with its own axis of criticism, taken in turn.
The heaviest fire comes from the paper with the question mark in its title, "HBF Sucks?" Its authors drove an extended TokenSim with four two-hour production traces from Alibaba Cloud's Bailian platform, dropped HBF into a Mooncake-style KV-offload stack (a tiered scheme that keeps hot data in memory and sinks cold data to a cheaper storage tier) as an SSD stand-in, and characterized the full stack. The verdict was a negative optimization: a faster device produced a slower system. Average end-to-end latency rises by a factor of 2 to 5.5, and the maximum goodput under the SLO (service-level objective, the performance floor a system promises to hold) falls by a factor of 1.1 to 2.7, with goodput meaning effective throughput, the rate of requests that satisfy the objective, consistently across both H100 and B200.
The paper offers a three-condition framework: far-tier acceleration pays only when "read I/O is the bottleneck, reads outnumber writes, and the bandwidth is sustainable" hold at the same time, and transient KV violates all three; writes outnumber reads on every trace, the two-tier hierarchy keeps the reuse near the host, and persistently write-heavy streams get dumped on the far tier.
Three corroborations follow: scaling the medium's read latency by 3.75 times changes end-to-end latency by less than 1 percent, which locates the problem in system structure rather than in the medium itself; a 3D-ICE thermal model shows the stack hitting its thermal limit well below peak bandwidth; and the TLC (triple-level-cell flash) tier exhausts its write lifetime sooner than the SSD pool it replaced. The conclusion leaves a door open: used selectively, with reuse-aware data placement, write budgeting, and thermal coordination, HBF earns its seat in inference serving.
The Oxford team's HMA paper adds another axis of criticism from the GPU microarchitecture side: HBF inherits 3D NAND's read latency, with page-open time (the preparatory step before a page can be read) roughly 1000 times DRAM's, and that starves the GPU scheduler outright; after the scheduler issues a read it waits the equivalent of a thousand normal round-trips while the pipeline idles. An all-HBF configuration even loses to CPU offloading by a factor of 2.05 in geometric-mean performance. Their fix is predictive migration: HBM and HBF sit side by side with the GPU inside the same package, and a hardware migration policy keeps high-latency accesses off the critical path, 2.79 times faster than all-HBF in geometric mean, pushing per-GPU deployable capacity in architectural simulation to 739 GiB, 9.2 times an 80 GB A100.
The systems study "Exploring HBF" attacks from the write path. Ten observations and five conclusions, the weightiest being this: writing intermediate data into HBF raises per-decode-iteration write traffic by as much as 100 times, and KV-cache writes bring non-trivial performance overhead and endurance challenges. For the capacity dividend to cash out, read bandwidth has to hold at HBM's level, especially under SLO constraints.
The affirmative side carries hard results too. FlashAccel from the Institute of Computing Technology, Chinese Academy of Sciences, builds the full co-design for integrating HBF into GPUs, delivering 2.49 times per-GPU throughput and 1.93 times energy efficiency against a pure-HBM GPU under a 100 ms latency constraint with a 6-stack configuration. Note the conditions: weights and KV both live in HBF, held up by specialized data layouts, a latency-hiding architecture, and a storage management layer as one integrated whole. Together with KAIST's DASH above, the affirmative camp's shared posture is that the gains exist, but must be bought with architectural work.
The five papers' axes of criticism differ, but on one thing the whole room agrees: model weights are write-once, read-many data and belong in HBF, the paper with the fiercest title included; what it attacks throughout is transient KV, the portion of the cache that is invalidated soon after being written and almost never read again. A second point of agreement matters just as much: as of September 2026, the HBF ecosystem has no public third-party silicon measurement, and every number above came out of a simulator. This argument still has to be fought again on real hardware.
What the argument is really about is the division of labor in the memory hierarchy, and both sides can hold. Cloud high-concurrency clusters remain HBM's territory, with the whole prefill-decode disaggregation and expert-parallelism apparatus still valid. What HBF opens is a tier with no prior product form: one machine, the whole model, low concurrency, long context. The tier did not exist before, because no medium could deliver terabyte-scale capacity and terabyte-per-second read bandwidth at the same time.
6. Who Needs This Machine: Workload Profile and Economics
Back to the concrete question: who buys.
The three agentic traces used in the paper's evaluation sketch a fairly typical portrait: multi-turn conversations of 29 to 52 rounds, input contexts from 18K to 261K tokens, cache-hit rates from 94 to 96.6 percent, and a pure-decode share of total makespan ranging anywhere from 11 to 50 percent. This is the shape of the agent workload: long system prompts hitting the cache again and again, each turn generating only at the tail of the conversation, few sessions but great depth per session. The paper also quantifies how load shape moves the configuration: the higher the pure-decode share, the farther right the host-transport knee shifts and the steeper the curve; prefill-heavy loads are more forgiving of state-tier bandwidth. Provisioning follows the workload portrait; there is no static lookup-table answer.
On the datacenter side, the storage vendor supplies the counterpart. SK hynix's H³ architecture places HBM and HBF on the same interposer (the silicon substrate carrying interconnects) attached to the GPU, read-only data into HBF and writable data in HBM, and the scenario it aims at is shared precomputed KV, that is, cache-augmented generation: system prompts and knowledge bases precomputed into read-only caches, a natural fit for flash's endurance profile. Weights on the single-machine side, shared caches on the datacenter side: the same constraint, "immutable data goes to HBF," grows into two instantiations across two deployment shapes.
The economics deserve an honest reckoning. Cost per token is the cloud cluster's home field; a low-concurrency appliance's throughput deficit is a given, and there is no point entering that contest. What it actually tallies are three things: single-instance total cost of ownership (one machine against a fleet of nodes plus network plus facilities), the sovereignty-and-compliance premium of data that never leaves the premises, and the fit between workload shape and the medium's sweet spot. The demand-side slope helps too: memory already comes close to half the system value of an AI server (a Micron figure we recorded in our on-site FMS 2026 coverage), and any architecture that knocks the memory bill down a notch earns a line on procurement lists.
For scale, take the vendor numbers: Sandisk claims HBF delivers 8 to 16 times HBM's capacity at similar cost. Patterson and Ma's survey supplies authoritative backing for the other side: HBF's 10x capacity can shrink the system directly, cutting power, total cost of ownership, carbon emissions, and network overhead in one move.
Read both sides' qualifiers clearly: the vendor numbers are all internal simulation, the survey gives direction, and the paper itself offers no BOM (bill of materials) or pricing analysis. What it draws is the performance boundary, where performance starts to collapse; the cost boundary is left for real hardware to answer. The paper's contribution is having marked in advance which cell to test when the hardware arrives.
7. Why Huawei: Industry Cycle and Research Positioning
One last layer of context: why this paper comes from Huawei.
Start with what is evidenced. The paper's eight authors all carry Huawei affiliations, it was filed in computer architecture, and its workloads are drawn directly from China's two flagship open models. A second Chinese institution shows up on the research map: FlashAccel comes from the Institute of Computing Technology, Chinese Academy of Sciences. A systems vendor and an academy institute entering HBF from the two ends, whole-node configuration and accelerator co-design, in the same year, alongside the SK hynix-Sandisk specification and devices: the participant profile of this supply chain is already clear.
On industry context, storage sits in a cycle of loose NAND supply while HBM capacity is locked up by strategic customers. Our FMS 2026 coverage recorded the numbers: the three original manufacturers' 2027 DRAM and HBM output is essentially booked out, while NAND's quotas for next year were still under negotiation at the time. HBM is a seller's market and NAND a buyer's market, and converting the surplus medium's density advantage into a substitute for the scarce medium's bandwidth is a direction that appeals to any systems vendor sitting on abundant NAND capacity. Huawei's own inference line (the Ascend series) shows no HBF product plans in public information; this paper is research positioned ahead of silicon, timed neatly against the early-2027 device-sampling window.
What must be labeled as speculation: whether Huawei is advancing an HBF-related appliance internally, or binding it to the Ascend line, the paper does not say, and neither does any reliable industry source. What can be said is this: under constrained HBM access, the first party to write out the system-configuration answer for "NAND hosting trillion-parameter weights" was a systems vendor.
8. Summary and Judgment
The configuration answer the Huawei September paper gives to the "single-node trillion-parameter appliance" compresses into one line: weights in HBF, state tier retired onto commodity LPCAMM2 modules, host transport exposing only a quarter of internal bandwidth, and the whole machine landing on organic-substrate packaging. Behind the answer stand operator analysis of two trillion-parameter models, a measured routing trace, and three agentic load replays; on the other side stand OXMIQ and five academic papers forming the debate map, with distinct criticism axes and a unanimous "weights belong in HBF."
Three judgments:
First, HBF's territory was drawn clean in 2026. Low concurrency, capacity-bound, local and private deployment is its sweet spot; cloud high-concurrency clusters remain HBM's domain. "HBF replaces HBM" is a false proposition and "HBF opens the single-node form" is a true one, and what the two camps marked on the map is in fact the same region.
Second, the key to a trillion parameters in one machine sits at the systems layer. Decoupling the three knobs lets the machine dodge three expensive options: HBM's capacity price, advanced packaging's bandwidth price, and the cluster's network-and-operations price. Victories of this kind, taking a problem apart into its cheapest solutions, will recur throughout the memory-wall story.
Third, the verification points come due densely within 18 months, and there is a ready-made scorecard. Patterson and Ma's survey left an open question: what the ratio of traditional memory to HBF in a system should be. The Huawei paper's 4.0 and 1.4 s⁻¹ are the first quantitative answer that question has received. Three things to watch: whether Sandisk's HBF samples arrive in the second half of 2026 as promised; whether the early-2027 inference devices' measured bandwidth and exposure support the partial-exposure conclusion; and whether Huawei or another systems vendor turns the appliance into a product. Any one of the three delivered turns the contour lines in the paper into a machine on a shelf; any one overturned hands this debate its first silicon data.
Declaration: This article is based on the paper "Trillion-Parameter MoE in a Box: Decoupling Memory Provisioning with High-Bandwidth Flash" (arXiv:2609.15636), filed to arXiv by Huawei on September 14, 2026, and cross-checked against the OCP "HBF High-Level Base Die Specification v0.7.0," official Sandisk and SK hynix materials, five HBF academic papers, and public coverage of Hot Chips 2026 and FMS 2026. It is not investment advice. Data are current as of September 16, 2026.
References
- Pengfei Xia et al., "Trillion-Parameter MoE in a Box: Decoupling Memory Provisioning with High-Bandwidth Flash," arXiv:2609.15636, 2026-09-14 (Huawei, primary paper)
- Open Compute Project, "High Bandwidth Flash (HBF) High-Level Base Die Specification v0.7.0," 2026-08-03 (Sandisk x SK hynix)
- Sandisk, "HBF Fact Sheet," 2025-07; official blog "Scaling Beyond the Wall"
- SK hynix Newsroom, "HBF at FMS 2026," 2026-08
- Minho Ha, Euiseok Kim, Hoshik Kim (SK hynix), "H³: Hybrid Architecture Using HBM and HBF for Cost-Efficient LLM Inference," IEEE CAL, DOI: 10.1109/LCA.2026.3660969
- Seeyeon Kim et al. (KAIST), "Beyond Capacity: Scalable MoE LLM Inference via HBF with Direct GPU and HBM Paths," arXiv:2608.14333
- FlashAccel (Institute of Computing Technology, Chinese Academy of Sciences), "Leveraging High-Bandwidth Flash for High-Throughput LLM Inference," arXiv:2607.10186 (cited as v2, 2026-08-22)
- Dowon Son et al., "Exploring High-Bandwidth Flash for Modern LLM Inference," IEEE CAL, DOI: 10.1109/LCA.2026.3705817 / arXiv:2608.13868
- "HBF Sucks? A Full-Stack Characterization of High-Bandwidth Flash for KV-Centric LLM Serving," arXiv:2608.11668 (cited as v4, 2026-09-14)
- Hakam Atassi, Noa Zilberman, Amro Awad (University of Oxford), "Hardware-Managed Heterogeneous HBM and Flash in LLM Inference Systems," IEEE CAL, 2026-08
- Xiaoyu Ma, David Patterson (Google DeepMind), "Challenges and Research Directions for LLM Inference Hardware," IEEE Computer 59(5), 2026-05, DOI: 10.1109/MC.2026.3652916
- Tom's Hardware / ServeTheHome coverage of the OXMIQ Labs Hot Chips 2026 presentation
- HuggingFace official model cards: deepseek-ai/DeepSeek-V4-Pro (MIT); moonshotai/Kimi-K3 (Kimi K3 License, native MXFP4); Qwen/Qwen3.8-2.4T-A95B (Qwen3.8-Max License)
Note: the 865 GB and 1,560 GB weight figures are the paper's numbers; Kimi K3's official model card states no volume, and the floor implied by 2.8T parameters at 4-bit is about 1.4 TB, consistent with both. Except where noted, performance figures are simulation or vendor-internal data; as of 2026-09 the HBF ecosystem has no public third-party silicon measurement.