Hot Chips 2026 specialist · AI factory networking
Day 2's networking agenda held three talks: NVIDIA's Spectrum-X multi-plane and BlueField-4, and Broadcom's Thor Ultra. Individually, product launches; on one timeline, a change in kind: the network stops being a tool that connects compute and becomes the AI factory's core infrastructure and operating system. Open interconnects are the second united front of Hot Chips 2026. This piece uses the five networks as its organizing frame: NVIDIA's answer first, then every other networking announcement of the conference placed by position: optics on scale-out, the factory OS on scale-in, Broadcom's NIC on the open side.
1. Spectrum-X Multi-Plane: Five Dedicated Networks
NVIDIA's network design for the AI factory is five dedicated networks. Why five? An AI factory carries at least five classes of traffic: tensor-parallel communication between chips, training synchronization across racks, data and model movement across campuses, a node's IO in and out of the factory, and agents' context and KV access. Their latency sensitivities differ by orders of magnitude, their bandwidth profiles are distinct, and their failure-isolation requirements conflict. A single general-purpose fabric serving all five must arbitrate priority through a scheduler, and arbitration cost diverges with scale. Five networks replace arbitration with topology: each traffic class gets its own plane, its own hardware, its own failure domain.
Scale-up (chip-to-chip): NVLink. Tensor and pipeline parallelism demand terabyte-scale bandwidth at microsecond latency; Ethernet cannot deliver that today, so this plane stays on NVLink. The NVL72 rack forms a single 72-GPU domain from spine and switch trays. On openness, NVLink Fusion brings third-party XPUs into the same domain, at 3.6Tb/s per XPU and 72 per domain. This plane's requirements are the most extreme: bandwidth first, everything else yields.
Scale-out (cluster east-west): Spectrum-X Ethernet. Training gradients and data pipelines flow between racks. Carrying them is purpose-built Ethernet: 102.4T switches and 1.6T SuperNICs, adaptive routing and congestion control redesigned for AI traffic, officially 1.6× RDMA bandwidth and 3× lower jitter. Multi-plane lives on this layer, 8 planes by 4 rails. The measurements support the design: DeepSeek V3 training runs 1.9× faster than on OTS Ethernet in a multi-job environment, and NCCL operations in Nemotron Ultra pretraining run up to 14× faster.
Scale-across (between campuses): Spectrum-XGS. Multi-factory, multi-campus interconnection demands distance awareness: load balancing and congestion control that can cope with long links, scaling across cities, states, and continents, with the goal of minimizing cross-datacenter latency. The product is Spectrum-XGS with ConnectX-9 at 800G, officially 1.9× multi-site performance.
Scale-in (the node side): BlueField-4 DPU. Newly announced this year, this plane answers how a node enters the factory: the scale-in data path lands directly on BlueField-4, with the DPU carrying the node's IO orchestration, security, and telemetry. Officially, its quantified position is 18× AI-factory services bandwidth, 10× packet rate, 3× lower latency. It is the plane closest to compute — the host's first hop.
AI context (context traffic): the Vera BlueField-4 storage processor. The other new plane this year, serving the memory and context access of agentic workloads: KV footprints are large, reads dominate, sharing crosses nodes, and the traffic couples naturally with storage. NVIDIA chose the storage-side BlueField-4 to serve it close to the data, as its own plane, officially 2× storage acceleration. Agent workloads are written into network topology by name for the first time.
The division of labor is shown in fig1. The cost is equally clear: five operations stacks, five procurement paths, five failure domains. The payoff shows at 512K-GPU scale, where a general fabric's scheduling cost keeps rising while physical separation's isolation benefit grows in the same direction. The official cost figures give three numbers: 1.7× fewer scale-out switches than a multi-tier single-rail topology; when one plane fails the rest keep 90% of bandwidth, with fault detection at 2.68ms and recovery at 100ms, for 1.6× factory goodput. The 64× scale increase is where all of it leads: 512K Rubin GPUs organized as 8 planes by 4 rails, 1.6T of scale-out per GPU, 100T switches built from 512×200G ports, pushing addressable scale from 8K to 512K (fig2). Physical separation brings not just isolation but a failure domain that converges.
2. Optics on scale-out: CPO Volume Production
Optical interconnect sits on the switching layer of the scale-out plane — the first. The larger the factory, the more optical modules, and the economics of the optical layer set the scale-out cost curve. CPO went from demo to volume production at this conference: micro-ring modulators in mass production on a 3D-stacked silicon photonics engine (TSMC's COUPE process), with detachable fiber connectors entering the product form. Our earlier corning-glassbridge and cpo-vs-npo pieces compared routes — glass bridge versus silicon-photonics NPO coupling; this conference delivered production economics: micro-ring volume resolves the yield and cost curve, and 4× fewer lasers, 5× lower power, and 10× better MTBI change deployment economics directly.
Production lands in Spectrum-X's CPO switch modules, so the economics hit the switching layer. Four companies stand in the chain: TSMC, SPIL, Lumentum, and Foxconn. The talk itself gives the scale reference: optics in an AI factory draw roughly 10% of compute power, so every watt CPO saves is amplified on the factory's total power budget.
3. The Carrier of scale-in: BlueField-4 and the Factory OS
The second is the scale-in plane. BlueField-4 carries this network, but the chip's role goes beyond the data path. Its release name states its position: "BlueField-4 Spectrum-X scale-in network" — NIC and network plane ship together, and the scale-in data path lands on this chip. It is one of the seven chips of the Vera Rubin system, with Astra's 7Tb/s aggregation carrying its throughput. ServeTheHome's on-site comment deserves full quotation: "a completely different class of device from Thor Ultra." Between an 800GbE NIC and a factory OS lies an order-of-magnitude difference in system positioning.
The change in positioning follows a familiar thread: our June MRC analysis set the thesis of pushing intelligence from the switch to the NIC, and this year the line advances two steps. At the protocol layer, MRC evolves into MRC++, its data path now public. At the positioning layer, the DPU's positioning shifts from "offloader taking auxiliary tasks off the CPU" to "AI factory operating system," holding the factory's data-plane and IO orchestration rights.
Where orchestration sits, NVIDIA's own decomposition of agentic workloads gives the first-hand answer: prompt and orchestration on the CPU, context/observe/reason/act on the LLM, memory spanning CPU, DPU, CUDA, LLM, and network, security on CPU and DPU. Control-plane orchestration stays with the host; data-plane and IO orchestration go to the NIC. This corroborates the boundary drawn in the Intel piece: the network side's "factory OS" means the data plane, the Intel side's "orchestration layer" means the control plane; the two lines meet at the rack and each manages its side.
4. The Open Side: Thor Ultra on the scale-out NIC
The only non-NVIDIA announcement comes from Broadcom. Thor Ultra is positioned on the NIC side of the scale-out plane: the same plane, the open end of the ecosystem. Its backdrop is the Broadcom-OpenAI 10GW collaboration: self-designed accelerators with Ethernet networking, ramping from H2 2026 through 2029, the source of the 100,000+ chip single-cluster claim. In product lineage, it extends the route Tomahawk Ultra opened in July 2025, moving from switch silicon to the NIC.
The genuinely new material is MRC++'s data path, officially defined as enhanced RoCE (eRoCE). Its four capability groups each answer a pain point of the prior generation. The prior generation had no multipathing; eRoCE splits flows by per-QP/per-packet header entropy, with ECMP/E-ECMP/DLB on the switch side and packet spray across up to 8 planes on the NIC side. The prior generation's Go-Back-N retransmission wasted bandwidth; eRoCE moves to SACK/NACK selective retransmission with path-quality probes. RDMA responses may be placed out of order while Sends and Atomics deliver in order, with bitmap reordering at the receiver. Congestion control takes receiver-credit RCCC as the baseline, with P4-like engines carrying programmable algorithms — hard-to-tune becomes programmable. Two eights are worth setting side by side: MRC++ sprays across up to 8 planes, and Spectrum-X multi-plane runs exactly 8 planes. The alignment of NIC and switch on the same topology is the most direct evidence that multi-plane is reaching the open ecosystem beyond NVIDIA's own.
This 5nm NIC chip of 2.4 billion transistors draws 40-42W (50-55W for the board without optics) and provides PCIe Gen6 x16, 8×100G SerDes, and 64K+ queue pairs. Initial figures come from a Gen5 GPU platform (2 nodes, 8 GPUs each, 16×400G): all_gather at 381.9 GB/s, all_reduce at 383.9, reduce_scatter at 380.2, all above 96% of line rate; TCP at 791 Gbps unidirectional, RDMA writes at 781 Gbps. A direct comparison with ConnectX-8 is still missing, but the data path has moved from unpublished to published. Broadcom's own positioning: "QoS and virtualization enabling XPU/CPU as a service." The NIC layer's endpoint is compute as a service.
5. Summary and Judgments
Put the three talks back into the five-network frame and the picture is clear. NVIDIA shipped the whole frame itself: multi-plane switching, volume-production optics, the factory OS on the node side. Broadcom's Thor Ultra lands on the scale-out NIC, aligning to the 8-plane topology with MRC++. The one plane without an open-side announcement is scale-up: in this conference's networking agenda it remains NVLink's own domain, but the challenger has already appeared elsewhere: Microsoft's Ethernet-plus-ATL push, covered in our cloud-silicon piece, targets exactly that plane. How a 512K-GPU factory gets connected: the answer is filling in, piece by piece.
Judgment one: traffic layering determines network layering, and the cost and benefit of five networks flip with scale. Five networks mean five operations stacks, five procurement paths, five failure domains; the fragmentation critique carries weight. But a general fabric's scheduling cost rises with scale, and at 512K GPUs the isolation benefit of physical separation starts to outweigh five operations stacks. The 1.7× switch savings and the 90% bandwidth kept on failure are the quantitative evidence for that flip.
Judgment two: CPO volume production is the optical-interconnect watershed. The line from demo to deployment is production metrics. Optics already draw roughly 10% of compute power, and the power curve together with the cost curve sets how fast optics penetrates from the switch side toward the host side.
Judgment three: the network becomes an OS; BlueField-4's repositioning is a system-level signal. A share of orchestration rights is moving from host CPUs and switches to the NIC, and network devices are beginning to define how the factory runs. The network's move from connecting tool to core infrastructure ends as an operating system.
What to watch: a direct Thor Ultra vs ConnectX-8 comparison; MRC++ multi-plane measured at hyperscale; the CPO cost and power curves; scale-in and AI-context deployment materials; the 10GW ramp cadence in H2 2026.
Forward commitment: when MRC++ multi-plane is measured at scale and the CPO cost curve updates, we will come back and check the results.
Data as of 2026-08-28. Facts from Hot Chips 2026 vendor presentation materials and on-site coverage (ServeTheHome and others); single-source and transcribed figures are marked in the text.
Further reading: Hot Chips 2026 Overview · Four Routes Against the Memory Wall · Intel's Triple Launch · Cloud Silicon Generation Two