
When large-model training hits its first hard limit, it is usually the interconnect, not the GPU. A Hot Interconnects 2026 simulation replaced a conventional "in-rack NVLink plus CLOS scale-out" fabric with 72 hypothetical 1-Pbps high-radix switches: job completion time fell 17.3% in a realistic deployment and 27.5% under pure network stress, and GPU utilization rose from 56–64% to 67–88%. This article restates the simulation setup and results against the conference slides, unpacks what the numbers mean, and traces the industry response and the hardware and software implications.
When large-model training hits its first hard limit, it is usually the interconnect, not the GPU, that gives out first. At IEEE Hot Interconnects in August 2026, Turbalance and Lawrence Berkeley National Laboratory (LBNL) published a simulation study that puts numbers on this claim. The model is the same 20T-parameter mixture-of-experts model with 2,304 experts, the hardware is the same 2,304 GPUs, and the only change is the fabric: a conventional in-rack NVLink plus two-tier CLOS structure is replaced by a flat, single-hop network of 72 hypothetical 1-Pbps high-radix switches. Job completion time falls 17.3% in a realistic deployment and 27.5% under pure network stress, and GPU utilization rises from 56–64% to 67–88%.
The study is a pure simulation; a 1-Pbps single-chip switch does not exist today. But it quantifies how much training time the network actually eats, at a precision one could act on. Industry has already moved in the same direction: in October 2026, the high-radix switch-chip company Eridu emerged from stealth with $230 million raised, building the kind of switch the paper models.
1. The Network Wall: Compute Grew 80x, Bandwidth Grew 14x
Taking 2017 as the baseline, GPU FP16 compute grew roughly 80x by 2026, while switch bandwidth grew only about 14x. The curves are baselined on public specifications for NVIDIA GPUs and the Broadcom Tomahawk switch line; the 2026 data points are projections. The paper calls the scissors gap between the two curves the "network wall," and states it bluntly: what limits large-model training now is the interconnect, not the GPU.
Mixture-of-experts (MoE) raises the network pressure. Every MoE training step sends tokens to their routed experts and gathers the results back; this dispatch/combine pattern is an All-to-All in communication terms. The slides put it as "expert routing turns every training step into an All-to-All."
To be more precise, it is not one All-to-All but many: each MoE layer's forward pass has a dispatch and a combine, and the backward pass walks two more gradient exchanges. A model with dozens of MoE layers performs hundreds of All-to-All operations in a single training step. All-to-All is also the most demanding communication pattern for the network: every card exchanges data with almost every other card, and the slowest flow determines when the collective finishes. It therefore tests three things at once: bisection bandwidth, hop count and tail latency.
Expert counts keep climbing: Kimi K3 in 2026 already routes 896 experts, and SemiAnalysis reports Moonshot recommends serving it on a scale-up domain of at least 64 accelerators. More experts spread across more cards means the All-to-All reaches across more of the fabric.

2. The Radix Wall: The 512-Lane Ceiling
Why can't the network keep up? The paper locates the constraint in switch-chip "radix," the number of lanes a single chip can drive out. The current state of the art is 512 × 200G lanes: Tomahawk 6, Silicon One G300 and Spectrum-6, shipping from the second half of 2026, all top out at 102.4 Tbps. Extrapolating the trend, next-generation ASICs gain only about 2x, landing at 204.8 or 230.4 Tbps.
NVIDIA's Spectrum-6 SN6800 bundles four chips in a chassis to reach 409.6 Tbps. The number looks like a 4x jump, but as the paper notes, "the fabric still sees the same 512 lane per-chip radix": aggregating inside a chassis does not change the radix.
Why radix matters more than total bandwidth (this article's derivation): in a single-tier multi-rail network, every switch must connect directly to all GPUs in the domain, so the size of a single-hop domain is set by "switch lanes ÷ lanes consumed per GPU."
| Switch chip | Per-GPU link | Single-hop domain cap |
|---|---|---|
| 512 × 200G (today) | 1 × 200G | 512 GPUs |
| 512 × 200G (today) | 1 × 400G (2 lanes) | 256 GPUs |
| 5120 × 200G (paper's assumption) | 1 × 400G (2 lanes) | 2560 GPUs |
This is what the paper means by 512 lanes capping the single-hop domain at "a few hundred XPUs." Total bandwidth can be bought by stacking chips; how many endpoints a single chip can reach cannot. To connect more accelerators, the only option is more tiers, paid for with more switches, more cables, more power and longer tail latency. Today's ten-thousand-GPU clusters are built exactly this way: multi-tier CLOS, thousands of switches, cross-rack paths starting at three hops.
The paper's 5,120 lanes are ten times 512, and the round number happens to cover the 2,304 GPUs: each GPU consumes 2 lanes, 2,304 × 2 = 4,608; 4,608 is itself exactly 9 × 512, so it would just fit, and the paper's 5,120 leaves roughly ten percent of headroom (this article's derivation).
3. The Setup: One 20T Model, Two Fabrics
3.1 The model: Maverick scaled 50x
The study projects to 2028–29. The model is derived from Llama 4 Maverick (~400B total parameters, 17B active, 128 experts) and scaled to roughly 20T total parameters and 165B active, with 2,304 experts per MoE layer and 16 selected per token (top-16).
Several ratios are worth noting (this article's derivation):
- Total parameters grew about 50x while active parameters grew about 10x — the model is sparser. Each token activates only 16/2304 ≈ 0.7% of experts, and active parameters are about 0.8% of the total.
- The expert count exactly equals the GPU count (2,304): at full expert parallelism, each card hosts one expert per layer.
- Top-16 means each token is replicated to 16 destinations at dispatch, so All-to-All volume scales with top-k.
The platform is 2,304 NVIDIA Rubin Ultra packages across 16 racks of 144.
3.2 The two fabrics
Both fabrics give each GPU the same scale-up bandwidth of 28.8 Tbps (3.6 TB/s):
- Baseline: in-rack NVLink (16 NVL144 racks, 576 NVSwitches in total, i.e. 36 per rack) plus a 6.4 Tbps two-tier CLOS scale-out (256 leaf and 96 spine switches, all 230.4T, 4 rails × 16 planes). 928 switches in total, 3-hop diameter.
- Petabit: 72 hypothetical 1-Pbps switches, a single ASIC with 5,120 × 200G lanes, ten times today's radix. Each GPU package runs 72 × 400G links, one to every switch, for the same 28.8 Tbps; the whole fabric is one hop across, and it replaces the NVSwitch tier outright.
The key to this setup is not "the same 28.8T" but the cross-rack segment (this article's derivation):
| Baseline | Petabit | |
|---|---|---|
| In-rack bandwidth per GPU | 28.8T (NVLink) | 28.8T |
| Cross-rack bandwidth per GPU | 6.4T (CLOS) | 28.8T |
| In-rack / cross-rack ratio | 4.5 : 1 | 1 : 1 |
| Cross-rack path | 3 hops (leaf—spine—leaf) | 1 hop |
| Switch count | 928 | 72 |
The baseline is a two-level bandwidth staircase: 28.8T inside the rack, 6.4T — less than a quarter of that — the moment you leave it. The Petabit fabric pulls all 2,304 GPUs onto the same 28.8T and the same single hop. In other words, the paper is not comparing "a faster switch" but "the removal of the rack boundary."
Port utilization of the Petabit fabric (this article's derivation): each switch terminates 2,304 GPUs × 2 lanes = 4,608 lanes, 90% of 5,120; the remaining 512 lanes have no stated purpose in the slides (scale-out or storage uplinks, for example). The 72 switches total about 73.7 Pbps of capacity, against roughly 66.4 Pbps of aggregate GPU injection bandwidth (2,304 × 28.8T).
3.3 The method: change only the network
The tool is Turbalance's TurbaSim discrete-event simulator, driven by Chakra execution traces, replaying the same run on both fabrics with identical compute time. The paper's words: "Compute time is identical — the entire improvement is network-driven." Compute is held equal and the network is isolated as the only variable, which is the cleanest property of the study.
Under the hood, TurbaSim offers three backends — packet, flow and analytical — and this paper uses the flow level. Instead of simulating queues packet by packet, it approximates each flow as a bandwidth-allocation problem, which is fast enough to replay a full training run across more than two thousand GPUs, at the cost of only approximating tail latency under incast congestion. The Chakra execution traces that drive it record how computation and communication interleave within a training step; replaying the same trace on both fabrics executes the compute as-is and re-times the communication against each topology. That is what "compute time is identical" means in practice.
The limits are also in the simulation. Flow-level simulation does not capture multi-tier congestion and tail latency with perfect fidelity, and the 1-Pbps switch itself is a modeled device. The paper answers "what if such a chip existed"; it is not an engineering commitment. The team's own stated next step is extending the simulation to a 5,040-package full domain, more than double this study's 2,304. That number uses a different lane accounting than this study: at one 200G lane per package, a single tier holds up to 5,120 packages, and 5,040 approaches that ceiling; this study's packages each consume two lanes (400G), which is why its single tier is 2,304 packages plus about ten percent headroom.

4. The Simulation Results, Unpacked
The study runs two scenarios, and the numbers should be read separately.
4.1 Scenario A: the stress test
Expert parallelism maxed at 2,304, making the whole machine one All-to-All domain, with dispatch/combine crossing the full fabric. Job completion time improves 27.5%; per-GPU average runtime drops from 32.66 to 23.66 seconds; GPU utilization rises from 64% to 88%; All-to-All accelerates 4.50x (ReduceScatter 4.35x); collectives' share of iteration time falls from 36% to 12%, with All-to-All alone going from about 27% to under 10%.
Converting the percentages to seconds, the numbers corroborate one another (this article's derivation):
| Baseline | Petabit | Change | |
|---|---|---|---|
| Per-GPU runtime | 32.66 s | 23.66 s | −9.0 s (−27.5%) |
| Compute (runtime × utilization) | ≈20.9 s | ≈20.8 s | essentially unchanged |
| Collectives (× share) | ≈11.8 s | ≈2.8 s | −9.0 s |
| of which All-to-All | ≈8.8 s | ≈2.0 s (at 4.5x) | −6.8 s |
| other collectives | ≈3.0 s | ≈0.8 s | −2.2 s |
Three things read off this table:
- "Compute time is identical" holds numerically. Both sides work out to about 20.8–20.9 seconds of compute; the 9 seconds saved come almost entirely from collectives.
- Runtime = compute + exposed communication. 64% + 36% and 88% + 12% each sum to 100%, suggesting the counters track communication not hidden by compute (conjecture). A system with more aggressive compute-communication overlap would lose less on the baseline fabric, and the gain would shrink accordingly.
- All-to-All is the dominant cost, but not all of it. About 6.8 of the 9 seconds saved come from All-to-All; the remaining ~2.2 come from ReduceScatter and other collectives.
4.2 Why exactly 4.50x
The All-to-All speedup is 4.50x — exactly the baseline's in-rack-to-cross-rack bandwidth ratio, 28.8T ÷ 6.4T = 4.5. This is unlikely to be a coincidence (this article's derivation):
- In Scenario A, 2,304 experts are spread evenly across 16 racks. With balanced routing, the chance that a token's chosen expert sits in its own rack is only about 143/2303 ≈ 6%; roughly 94% of dispatch traffic must cross racks.
- The baseline's All-to-All is then almost entirely limited by the 6.4T cross-rack injection bandwidth per GPU; on the Petabit fabric that ceiling becomes 28.8T, and communication time shrinks by the bandwidth ratio, 4.5x.
The speedup matching the bandwidth ratio exactly suggests that in this simulation All-to-All is bandwidth-bound, with hop count and latency playing a secondary role (conjecture). It also marks the reading boundary: if congestion and tail latency in a real network are worse than the flow-level simulation paints, the flat fabric's real gain could exceed 4.5x; conversely, if the baseline keeps more traffic in-rack through topology-aware expert placement, the gain shrinks.
4.3 Scenario B: the realistic deployment
Expert parallelism contained in 64-GPU groups, TP4/PP3/DP3 mixed, no cross-group All-to-All. Job completion time improves 17.3%; per-GPU average runtime drops from 48.04 to 39.72 seconds; utilization rises from 56% to 67%; the All-to-All speedup is the same 4.50x.
The parallel dimensions multiply out to TP4 × PP3 × DP3 × EP64 = 2,304, so EP64 is a dimension independent of the other three. A 64-GPU EP group would fit inside a 144-GPU rack, yet the All-to-All still accelerated 4.50x. The slides state why: TP stays on NVLink while EP, PP and DP all cross racks. With TP4 occupying adjacent GPUs, an EP group spans 4 × 64 = 256 GPU positions, more than a rack's 144 (the exact GPU mapping is not given in the paper; this article's derivation).
The slides do not break Scenario B into components, but the two headline numbers can be inverted (this article's derivation, baseline runtime normalized to 100):
| Baseline | Petabit | |
|---|---|---|
| Runtime | 100 | ≈82.7 |
| Compute | 56 | 56 |
| Non-compute | 44 | ≈26.7 |
| Non-compute reduction | — | ≈39% |
(Working backward from 17.3%, the new utilization is about 67.7%, consistent with the slides' 67% within rounding.)
Comparing the scenarios: Scenario A's non-compute time is cut by about 76% (11.8 s → 2.8 s), Scenario B's by only about 39%. A large share of Scenario B's remaining overhead is therefore not All-to-All, and a flat fabric cannot help with it. Likely sources include pipeline bubbles from PP3 and TP communication that already rides the 28.8T NVLink and is unchanged by the swap (conjecture).
4.4 The two scenarios together
- 17.3% and 27.5% mean different things. 17.3% is what today's parallelism strategies gain from swapping the fabric alone; 27.5% is the ceiling if parallelization is redesigned along with it.
- An easily missed detail: even on the baseline, Scenario A's utilization (64%) beats Scenario B's (56%); on the flat fabric the gap widens to 88% vs 67%. The two scenarios differ in more than EP, so the utilizations are not directly comparable — but it at least suggests that once the fabric flattens, "wide EP, fewer pipeline stages" may beat today's mainstream configurations. That is one reason the paper lists "rebalancing TP/EP/PP/DP" as future work.
- The essence of the gain: GPUs spend less time waiting; collectives' share of iteration time falls from about a third to about a tenth.
4.5 The economics and the knock-on benefits
The paper also runs the economics. At Colossus 1 scale (about 200,000 GPUs), lifting utilization from 64% to 88% is worth roughly $1.3 billion per year of effective compute output, at an implied $3 per effective GPU-hour. Check: 200,000 × 8,760 hours × 24 points × $3 ≈ $1.26 billion, consistent with the paper's figure. Note that this uses Scenario A's utilization lift and linearly extrapolates a 2,304-GPU simulation to 200,000 GPUs — read it as an order-of-magnitude estimate.
There is a knock-on benefit: with wider expert parallelism, each GPU hosts fewer expert parameters and HBM use drops. In Scenario B, each card hosts 36 experts' parameters per layer; at expert parallelism 2,304, one. Experts-per-GPU and All-to-All traffic are two ends of the same dial: when communication gets cheap, spreading experts out becomes worth paying for, and the freed memory can go elsewhere.
The switch count also falls from 928 to 72. The slides' own figure is "over 80% fewer units," which works out to about 80% against the 352 CLOS switches alone; including the 576 NVSwitches the flat fabric also replaces, the full-network comparison is 928 to 72, a 92% reduction. The "72 switches replace 352" line in Chinese coverage uses the narrower count and drops the NVSwitch tier. Fewer switches does not mean proportionally lower cost, though: a single 1P switch is far more complex, optically packaged and power-hungry than today's devices, and the paper offers no whole-network cost or power comparison.

5. What the Paper Shows, and What It Doesn't
Pulling the analysis together:
What the paper credibly shows:
- Holding compute equal, fabric structure alone accounts for a 17–28% difference in job completion time, with internally consistent numbers (compute conserved; speedup matching the bandwidth ratio).
- For MoE training, the baseline's core bottleneck is the bandwidth cliff at the rack boundary (28.8T → 6.4T), not a shortage of aggregate bandwidth.
- A flat fabric opens up the design space of parallelism; swapping the fabric alone captures only part of the gain.
What the paper doesn't answer, or answers only partially:
- How strong is the control group? The baseline's cross-rack bandwidth is 6.4T. Against a stronger control — a larger scale-up domain, NVL576-class designs, or higher scale-out bandwidth — the gap would narrow by an unknown amount.
- Congestion and tail latency. Flow-level simulation is weakest exactly where multi-tier networks suffer most: incast, congestion spread and long tails. The real gap could be larger, or smaller with careful congestion control.
- Load imbalance. The derivations assume balanced expert routing. Real MoE training has hot experts and uneven traffic, and the two fabrics may absorb it differently.
- Cost, power and reliability. Whole-network power for 72 1P switches, and the blast radius of a single switch failure (each one connects to all 2,304 GPUs), are not addressed.
- Scale extrapolation. 2,304 cards is the size of one flat domain; at 200,000 cards, flat domains still need a scale-out network between them, and how much of the gain survives there is outside the paper's scope.
6. The Industry Follows
The route did not stop at the paper. In October 2026, the high-radix switch-chip company Eridu emerged from stealth with $230 million raised to date: an oversubscribed Series A led by Socratic Partners, with John Doerr, Hudson River Trading and Capricorn participating, plus industrial capital from MediaTek, TDK Ventures and Bosch Ventures. The founding team comes from optical networking and silicon: CEO Drew Perkins co-founded Infinera and served as its first CTO, and the company sold to Nokia for $2.3 billion in 2025; before that he founded Lightera, an optical-switch company acquired by Ciena. Co-founder and CMO Mike Capuano is also an Infinera alumnus, and fellow co-founder Omar Hassen comes from networking-chip design at Broadcom and Marvell.
Eridu's product direction matches the paper's setup: a clean-sheet design with an order-of-magnitude advance in radix and throughput, pitched as "one switch replaces 30 lower-radix switches." For comparison, the paper's full-network switch ratio is 928:72, about 13x; the CLOS-only ratio is 352:72, about 5x. "30" is the most aggressive of the three numbers, so mind the denominator when quoting. Its AI Infra Summit sponsor material prints the Turbalance simulation results directly. One caveat for readers: the marketing renders "utilization up 24 points" as "38% higher utilization," which is the relative reading of 64% to 88%.
Turbalance's own business is simulation and orchestration software for AI workloads, and the paper doubles as a demonstration of that simulator. On the LBNL side, John Shalf is Department Head for Computer Science at Berkeley Lab and formerly served as Deputy Director for Hardware Technology in the DOE-led Exascale Computing Project (ECP); George Michelogiannakis co-authored the flat Dragonfly+ topology, and flat fabrics have been the pair's long-standing research direction. Around this line, the OCP Foundation and 650 Group co-host "network wall" webinars, and alternative routes raised money in the same window: nEye Systems, in optical circuit switching, closed a $58 million Series B; Salience Labs announced a 32-port all-optical OCS; and Lightmatter, at the same HotI session, claimed a 3x time-to-train improvement.
Academic argument, startups and industry organizations arriving within the same quarter is rare in networking, an industry that usually moves incrementally.
7. Hardware: Why CPO Is Likely
5,120 × 200G lanes are first of all a packaging problem. The 512 lanes of today's 102.4T chips already press against the density limit of pluggable optics; multiply the lane count by ten and the faceplate physically cannot hold that many ports. A petabit-class switch will therefore most likely use co-packaged optics (CPO) or near-package optics (NPO), integrating the optical engines with the switch die. This is why the market watches Eridu's optical pedigree, and why MediaTek, TDK and Bosch invested.
The engineering challenges are concrete. At about 1 pJ/bit for SerDes, serialization alone approaches 1 kW per switch; aggregate laser power and reliability engineering across 72 petabit switches is a new systems problem; and a large die with about 5,000 lanes needs redundant-lane design to offset yield losses. On an optimistic schedule, volume production lands in 2028–2030, roughly consistent with the paper's 2028–29 projection.
There are other routes. Optical circuit switching (OCS) does not aim for full any-to-any connectivity but reconfigures topology on demand; Google has run OCS in TPU clusters for years, with nEye and Salience on the startup side. Another route keeps expanding the scale-up domain, with NVL576, NVLink Fusion and UALink pushing single-hop domains from rack to row. Add photonic fabrics like Lightmatter, which change the medium itself, and four routes share the same underlying judgment: the network is the next decade's bottleneck.
8. Software: What Changes for Parallelism
A new fabric does not just speed up existing communication; it changes which parallelism strategies make sense. The paper lists "rebalancing TP/EP/PP/DP" as future work, and Section 4 explains why: about 60% of Scenario B's non-compute overhead survives the fabric swap, and claiming the rest requires reworking parallelism itself. When every GPU package is equidistant (one hop, uniform latency), tensor parallelism no longer has to stay inside the NVLink domain, expert-parallelism domains can open up, and pipeline depth can shrink, compressing bubbles. In Scenario B, TP staying on NVLink is today's conservative choice; on a flat fabric it may not remain so.
A class of current optimizations gets simpler. Topology-aware routing, and the joint optimization of expert placement with communication cost (communication libraries like DeepEP include such work), lose their object of awareness when the fabric is uniform everywhere. Other questions gain weight: load balancing across the 72 planes, and switch-level co-design of congestion control and buffer depth, which the paper names as next steps. Inference has a parallel payoff: KV-cache transfer between disaggregated prefill/decode clusters is also All-to-All-shaped traffic, listed as future work, and its industrial demand may be arriving earlier than on the training side.
Watchpoints
- The network wall now has isolated-variable evidence, but the magnitudes are bound by the simulator. Until the first physical validation, such as Eridu silicon or a hyperscaler prototype, 17% should be read as an order-of-magnitude estimate, not a performance promise.
- The choice of control group sets the size of the gap. This study's gain comes mostly from erasing the 28.8T → 6.4T cross-rack cliff. How much of that cliff larger scale-up domains or higher scale-out bandwidth would fill in is the key comparison for evaluating this route.
- The high-radix route's schedule follows optical packaging. CPO yield, laser power and cost curves determine the pace of this route.
- Three signals to track: the actual radix of Eridu's first product; whether Broadcom's and Cisco's next-generation ASICs break the "only about 2x" radix assumption; and the race between MoE expert counts (896 to 2,000+) and single-hop domain sizes.
A closing footnote: as the paper traveled through Chinese tech media, Colossus 1's scale was transcribed from 200,000 GPUs down to 20,000, while the "$1.3 billion per year" figure kept appearing unchanged, losing its denominator. Secondhand numbers deserve a trip back to the source before citing; every paper-related figure in this article was checked against the conference slides for that reason. Passages marked "this article's derivation" are inference from those numbers, not the paper's own conclusions.
Sources
- P. Berdesinski, L. Montigny (Turbalance), J. Shalf, G. Michelogiannakis (LBNL), Breaking Through the Network Wall: Petabit-Class Switching for Single-Hop MoE Training at Scale, IEEE Hot Interconnects 33 (HotI 2026, August 2026); figures from the official conference slides (hoti.org)
- Guangxin Ji ("光芯纪") WeChat account, "Turbalance: Breaking the Network Wall with High-Radix Switches" (November 2026, secondary; the Colossus transcription error is corrected in this article)
- Eridu emergence press release (Business Wire, October 2026) and AI Infra Summit 2026 sponsor material; LinkedIn company page and Mike Capuano's OCP EMEA session
- Berkeley Lab AMCR staff page (John Shalf's role and ECP tenure)
- nEye Systems Series B (GlobalData/Verdict); Salience Labs 32-port OCS announcement (Business Wire, March 2026); Lightmatter HotI 2026 session material
- SemiAnalysis on Kimi K3 serving (as cited in the paper's slides); Llama 4 Maverick parameters (Meta public materials)
Related reading: "When One NIC Splits into Three Devices," "The Evolution of AI Datacenter Networks," "Huawei's One Network: HC 2026 Networking Panorama"
