← Thinking Thinking

OCP 2026: Tahan and ESUN — Ethernet Scale-up Ships First, Open Interop Still Unproven

Ethernet scale-up ships first, open interop still unproven

2026-10-18Thinking34 min read
Site cover: OCP 2026: Tahan and ESUN — Ethernet Scale-up Ships First, Open Interop Still Unproven

OCP 2026: Tahan and ESUN

Ethernet scale-up ships first, open interop still unproven

Among the open scale-up developments at OCP 2026 there are three advances of different nature: Tahan is shipping today, serving Meta's MTIA 300 systems; a next-generation 72-accelerator rack has entered public preview; and ESUN, after its 1.0 base specification was published earlier in the year, continues toward later versions. They make the Ethernet route into tightly coupled accelerator interconnect more concrete — but they have not yet merged into one product that adopts ESUN and freely mixes accelerators. [1]

That distinction decides how this year's releases should be read. Tahan answers how Ethernet enters one concrete accelerator system today; the 72-accelerator design answers how next-generation switch silicon widens rack scale; ESUN answers what network requirements different implementations can build against in common. For a CTO planning supernodes, what matters is that the three are converging — not a nonexistent spec sheet assembled from the most advanced specification, the largest rack, and a shipping product's delivery status.

Tahan serves Meta MTIA on existing transports

Tahan uses Broadcom Tomahawk 5 switch silicon; four switches connect 16 MTIA 300 accelerators into a nonblocking fabric. Meta describes its status as "shipping today," and this article uses "shipping" strictly within that product context of serving Meta accelerators. The phrase says the product has entered shipment; it does not say any particular delivery is complete, nor that it is generally commercially available to all customers. Chip capacity and port accounting are detailed in the technical notes. [1]

Tahan did not wait for a brand-new end-to-end protocol stack. It reuses existing RoCE transport with the accompanying routing, flow control, and traffic management, managed by FBOSS. This is the route of extending an established Ethernet engineering base into scale-up: first build a determined system boundary around your own accelerator, then progressively optimize protocol overhead and network behavior. It reuses existing technical assets — it does not mean plugging an ordinary data-center switch into a GPU rack yields the same result.

Figure 1  Official Tahan design
Figure 1 Official Tahan design. The product is shipping and serves Meta MTIA; the current design uses RoCE transport. Source: Meta, October 12, 2026. [1]

For accelerator teams building their own silicon, this route has practical appeal. Switch silicon, a network OS, and operations tooling need not all be built from zero; hardware teams can spend more of their engineering budget on endpoints, backplanes, and communication software. For enterprises buying third-party accelerators, the other side must be seen clearly: Tahan working with MTIA proves one concrete implementation — it does not certify interoperability between any two vendors' accelerators. Network openness and compute-platform replaceability are two separate product goals, each to be achieved on its own.

This is also what changes in Tahan relative to the old "can Ethernet do scale-up at all" debate. The argument is no longer only about whether switching latency is low enough; it now involves which transport a real product adopts, how it attaches to accelerators, and which management capabilities carry over. Compared with abstractly claiming that open interconnect lowers cost, the sturdier evidence today is that a proprietary compute endpoint can build a deliverable system around commodity switching technology. Which dependencies that actually lowers, and which integration work it adds, still has to be computed along the system boundary.

The 72-accelerator preview shows next-step density, not current configuration

Meta also previewed its next-generation scale-up blade: a 102.4T Tomahawk6C-based design in which eight blades connect a rack of 72 accelerators. The first thing to watch here is the change of rack scale. Higher chip capacity creates the hardware conditions for a larger local communication domain; whether those conditions become model benefit depends on endpoint capability and communication patterns. The specific port mix is not the same as full chip capacity; the accounting is listed separately in the technical notes. [1]

Going from a 16-accelerator system to a 72-accelerator rack is more than a scale number. More devices can sit inside a low-latency communication domain, giving model sharding and collective communication a different option space; at the same time, backplane, cabling, power, and cooling must support much denser connectivity. Fitting in one rack does not mean every model should place all devices in one high-frequency synchronization group. The gain from a wider communication domain depends on which communications the model can migrate from the external network into the local domain, and on whether the added parallelism pays for the synchronization overhead.

Figure 2  Next-generation MTIA scale-up blade preview
Figure 2 Next-generation MTIA scale-up blade preview. The 72-accelerator rack and the future ESUN-based generation must be read separately. Source: Meta, October 12, 2026. [1]

The most easily missed distinction is tense. Meta's body text explicitly says a future generation of scale-up blades plans to adopt the ESUN data plane. Although the caption on the same page ties the new blades closely to ESUN, reporting should follow the body text's qualification about generations. Today's Tahan, the next-generation 72-accelerator design, and a future ESUN implementation form one evolution path — they are not capabilities one product already has simultaneously. Public materials are not yet sufficient to give mass-production dates, customer-purchasable configurations, or software adaptation lists for every link.

This generational ordering reveals a product method: hardware capacity can advance first, and protocols and open interfaces converge in later product generations. This avoids being locked to specification progress, but it raises version-management demands. System vendors must state clearly which modes the current interface supports and whether future capability requires new silicon, new boards, or only a software update. If buyers read a roadmap as a free-upgrade promise, the open ecosystem can instead leave behind a new lock-in cost.

ESUN writes what the network must deliver into shared requirements

ESUN 1.0 was not born at the October 2026 summit. OCP announced it on March 10, with the specification's effective date of February 9. What it provides is a base specification oriented to operator requirements, centered on the Fabric — the network accelerator packets traverse. During the summit Meta said ESUN 1.1 is close to completion, with switch profiles and reduced inter-frame gap among the directions; as of this article's information date it cannot be written as officially released. [2][3]

Figure 3  Progress of product versus specification
Figure 3 Progress of product versus specification. Product and specification advance separately. Tahan is shipping; the 72-accelerator rack is a preview; ESUN 1.0 is published, 1.1 still in progress. Compiled by the author from Meta and OCP materials, as of October 17, 2026. [1][2][3]

The meaning of a base specification is to break "Ethernet suitable for scale-up" into requirements that can be implemented and tested — including shrinking fixed per-packet overhead and giving congestion control and local recovery explicit rules. Large messages amortize fixed overhead easily; fine-grained messages amplify every handling step and wait into a computational dependency. Optimizing small messages is therefore about reducing the stalls that recur inside distributed computation, not merely about link utilization. Specific headers and flow-control mechanisms are in the technical notes. [3]

A tighter header does not convert directly into a model speedup. A complete packet still carries other fields, common-link overhead, and physical transmission cost, and endpoints still perform commit, memory access, and notification. If a model is bounded mainly by compute or memory bandwidth, a smaller network header may not move the bottleneck at all; another model may be sensitive only during brief synchronization phases. The specification provides mechanisms that lower cost; products must show how large that cost is under their real workloads.

These network requirements also help suppliers state responsibility boundaries: what the switching Fabric guarantees, what requires endpoint cooperation, and what must be left to job-level recovery. Compressing all of it into one phrase — "lossless network" — hides the differences between products.

Describing interconnect capability with "supports lossless Ethernet" has often been insufficient for comparing products. Now one asks further: which capabilities are mandatory and which optional; how do two ends with different capability sets interoperate; and how do performance and recovery change when optional mechanisms are switched off. The value of standardization, first of all, is giving these questions a common vocabulary — not guaranteeing that all vendors achieve identical results under identical load.

After the Fabric opens, endpoints and software still decide the system boundary

ESUN does not attempt to unify the execution model of all accelerators. Between a packet arriving at a port and a compute core safely consuming remote data lie memory access, DMA, completion notification, and execution-stream conventions. When a remote write counts as complete, how notifications order against data, what range of atomic operations is supported, and who retries after an exception are interfaces the endpoint and software must define. The network provides a path for these operations; it cannot dictate all the semantics on the accelerator's behalf.

Figure 4  The scope of ESUN's Fabric
Figure 4 The scope of ESUN's Fabric. An open network interface preserves accelerator implementation freedom; it does not unify endpoint semantics. Source: Meta, October 12, 2026. [1]

This distinction matters most for heterogeneous products. Two accelerators attaching to the same switching Fabric proves only a common physical and network foundation. If they disagree on completion, ordering, address protection, or error handling, upper-layer jobs still cannot treat them as one unified execution domain. Open scale-up is therefore more likely, in the near term, to lower the barrier on network supply choice first, and only gradually widen endpoint and software interoperability. That latter judgment is an industry-path analysis and must not be presented as a product outcome any organization has committed to.

UEC's Ultra Ethernet Specification 1.1, released October 12, is related but separate work. It adds scale-up-oriented optimizations and coordinates follow-on ESUN and endpoint work with OCP; it is not another name for ESUN 1.1. Writing "UE 1.1 released" as "ESUN 1.1 done" conflates two standards organizations, different scopes, and different progress. This article covers ESUN's Fabric boundary; the specifics of UE transport, congestion signaling, and programmable interfaces belong to the network-systems article in this series. [4]

Software also decides whether new hardware scale is actually useful. Collective-communication libraries must map communication groups, message sizes, and parallel strategy onto topology. Small messages may depend mainly on the number of communication phases; large messages on effective bandwidth; expert switching adds destination imbalance. Identical hardware interfaces do not guarantee that a collective algorithm tuned only for an older system fits a 72-accelerator rack. Network topology, rank placement, and model sharding must be validated as one product configuration.

This co-design should not mean piling manual parameters onto applications. A more sustainable approach lets the runtime know bandwidth tiers, shared bottlenecks, and available paths, adjusting communication mappings after hardware changes while keeping application semantics stable. Collective libraries such as NCCL still impose strict participation rules — data counts and types must match across ranks; a more open network does not remove those requirements. [5] More supply choice becomes a long-term gain only when it arrives with maintainable software adaptation.

The next competition is over who delivers a complete open system

The trend Tahan and ESUN advance together is to take scale-up — historically procured wholesale with the accelerator — and gradually separate it into endpoint, switching Fabric, and software, each an inspectable engineering object. This can widen participation for switch silicon and system vendors and give in-house accelerator teams more system-design choices. But opening interfaces does not reduce all the work; part of the integration, validation, and lifecycle responsibility transfers to the platform builder.

System selection should therefore start by pinning the currently deliverable version. For shipping products, examine the real endpoint, software, and topology combinations; for previews, interface freeze, prototype validation, and upgrade conditions; for specifications, implementation coverage and interop evidence. Building evidence separately for the three, then comparing effective throughput, tail latency, power, and failure recovery under the target workload, avoids pricing current products at future capability.

Validation must also move from links to jobs. Point-to-point tests show basic paths; collective communication reveals concurrency and topology effects; only real models show whether compute and communication overlap. Tests should add degraded paths, endpoint withdrawal, and mixed versions to see whether open interfaces remain usable in a partially unhealthy environment. What is needed is not an endless list of test items, but the evidence corresponding to each product promise — especially the new risks of a widened communication domain.

The larger the scale-up domain, the larger the blast of maintenance and failures on jobs. One degraded link may only reduce bandwidth; one endpoint reset may force an entire communication group to recover — they cannot share one metric, "how long until connectivity returned." What platforms must compare, under equal workload and reliability targets, is how much communication cost a wider local domain saves versus how much recovery, redundancy, and upgrade cost it adds.

The conclusion OCP 2026 supports is already more specific than "Ethernet can do scale-up too": today there is Tahan, a product built on mature transports; a hardware preview aimed at larger racks; and ESUN, which has begun defining common Fabric requirements. What to track next is when these threads converge inside one deliverable system, and whether endpoints and software offer enough portability. The competitiveness of open scale-up will ultimately be proven by interfaces that stay usable across product generations and by reproducible application gains.

Technical notes

Tahan chip and port accounting

Tahan uses a 51.2 Tbps Tomahawk 5. Meta's body text gives 64×800G total capacity, half of it serving accelerators through the backplane; the 32×800G in the official caption must not be read as the chip having only half its capacity. Four switches connecting 16 MTIA 300 is one specific system configuration and does not imply any accelerator can be attached directly. [1]

The current transport stack is RoCE, routed IPv6, PFC, and AI-tuned QoS, managed by FBOSS. It has not been described in official body text as a production system adopting the ESUN data plane. "Shipping today" is recorded here only as shipping — not expanded into general commercial availability or complete delivery. [1]

Interface configuration of the next-generation blade

The preview blade uses a 102.4T Tomahawk6C with an interface plan of 216×400G; eight blades connect a rack of 72 accelerators. 216×400G is 86.4 Tbps — not the full capacity of a 102.4T chip. Chip capacity, summed interface bandwidth, per-accelerator injection capability, and job-available bandwidth are different metrics and cannot substitute for one another. [1]

Meta's body text places the ESUN-data-plane design in a future generation; the shorthand of a caption on the same page cannot override that tense. Public materials give no mass-production dates, customer-purchasable configurations, or software upgrade conditions for every link. [1]

Headers, flow control, and recovery mechanisms

The ESUN 1.0 EH header is 4 bytes, retaining the information needed for congestion feedback, traffic classes, load spreading, and hop-by-hop lifetime. That length does not include all common overhead of a complete packet and does not convert into an application speedup. [3]

Flow-control requirements have concrete boundaries too. The specification takes PFC as the baseline and distinguishes endpoint and switch support requirements for credit-based flow control (CBFC) and link-level retry (LLR). The three mechanisms solve different problems: pauses control transmission as buffers approach limits; credits pace injection to receiver capability; link retry shortens recovery from local bit errors. They create no additional bandwidth, do not eliminate chronic congestion by themselves, and do not replace end-to-end recovery after equipment failure. [3]

"Supports lossless Ethernet" is not enough for an interoperability judgment. One should also record mandatory and optional capabilities, endpoint-versus-switch support differences, negotiation and fallback paths, and behavior with optional mechanisms disabled. Testing link recovery and job recovery after device reset separately confirms what each mechanism actually carries.

Sources and caveats

Information as of October 17, 2026. Product generations, specification versions, and delivery status are distinguished per official body text; mechanism impact, supply-chain judgment, and validation advice are the author's analysis, not vendor performance promises. Previews, specification releases, and open interfaces are not equated with completed cross-vendor system interoperability.

[1] Meta, OCP Summit 2026 Powering AI with Open Innovations in Scale-Up Scale-Out and Scale-Across Networks, October 12, 2026.
https://engineering.fb.com/2026/10/12/open-source/ocp-summit-2026-powering-ai-with-open-innovations-in-scale-up-scale-out-and-scale-across-networks/

[2] OCP, The OCP ESUN 1.0 Specification Has Been Released, March 10, 2026.
https://www.opencompute.org/index.php/blog/the-ocp-esun-10-specification-has-been-released

[3] OCP, ESUN Network Operator Requirements Base Specification Rev 1.0, effective February 9, 2026, sections 5–7.
https://www.opencompute.org/documents/ocp-esun-network-operator-requirements-base-specification-rev-1-0-final-pdf

[4] Ultra Ethernet Consortium, Ultra Ethernet Consortium Releases Specification 1.1 Advancing AI and HPC Networking, October 12, 2026.
https://ultraethernet.org/ultra-ethernet-consortium-releases-specification-1-1-advancing-ai-and-hpc-networking/

[5] NVIDIA, NCCL Collective Operations, official software documentation, retrieved October 17, 2026.
https://docs.nvidia.com/deeplearning/nccl/user-guide/docs/usage/collectives.html