Put 72 accelerators in a rack and ask them to behave as one machine, and the link between them stops being a network in the usual sense. It has to carry loads, stores and atomics at something close to memory timing. Three camps want that job: NVIDIA's NVLink, the UALink Consortium's open specifications, and a set of Ethernet efforts often lumped together as Scale-Up Ethernet.
Coverage tends to treat this as a bandwidth race. An architect has three harder questions: how much latency each option actually commits to, what a memory operation means on the wire, and who can change the rules. This post sets the three side by side, separates targets from shipping parts, and gives a small model you can redo with your own numbers.
Key takeaways
- The latency figures are not comparable. UALink's white paper sets a goal of under 1 µs round trip, SUE-T specifies under 2 µs for a 1,500-byte packet over one hop, and Broadcom claims under 400 ns between accelerators on its own switch. We found no latency figure in the NVIDIA pages we read.
- All three carry load, store and atomic operations, but they define them at different depths. UALink specifies the transactions. Broadcom's original SUE framework treats commands as opaque and leaves the operation set to the accelerator.
- Control runs from one company to a 12-member board to an open Ethernet forum, and the camps overlap. NVIDIA is a founder of ESUN, Broadcom was still a UALink member in July 2025, and AMD's first Helios racks run UALink over Ethernet.
- NVIDIA's Vera Rubin NVL72 page now lists NVLink 6 at 3 TB/s per GPU, where the same page once showed 3.6 TB/s, which is a useful reminder that published numbers move.
- Native UALink switch silicon is the main schedule question. The Consortium's chair expects 1.0 chips in labs in the second half of 2026 and in products during 2027.
Three different kinds of thing
The comparison is awkward because these are not three equivalents. NVLink is a product line. UALink is a full stack, from protocol down to a modified Ethernet physical layer. "Scale-Up Ethernet" is a collection of documents from different bodies, and each covers a slice:
| Layer | NVLink | UALink | Ethernet route |
|---|---|---|---|
| Accelerator protocol | NVIDIA | UALink Protocol Layer Interface, with originator and completer roles | Left to the accelerator. Broadcom's SUE says it does not define an XPU-level protocol for interoperation between different devices |
| Transport | NVIDIA | Transaction layer with 64-byte flits and a streaming address cache | SUE-T at OCP, written by Broadcom and Arista, using the Unified Local profile and reliable ordered delivery from UEC |
| Link and framing | NVIDIA | Data link layer with 640-byte flits, 32-bit CRC and link-level replay | ESUN 1.0 at OCP: a 4-byte header, Priority or Credit Based Flow Control, Link Level Retry |
| Physical layer | NVIDIA SerDes | IEEE 802.3 based, 212.5G signalling, with reduced FEC interleave to cut latency | IEEE 802.3 |
That bottom row is the heart of Ethernet's pitch, and UALink borrows it too. The two open options share cables, connectors and retimers far more than their rivalry suggests. They differ mainly in what sits above the PHY. The SerDes underneath is a contest of its own, as we covered in How DSP Is Killing the Analog in SerDes.
Latency: read the conditions first
| NVLink | UALink | Ethernet route | |
|---|---|---|---|
| Published figure | None in the NVIDIA pages we read, which describe it only as low latency | Goal of request-to-response under 1 µs. The Register reports a 100 to 150 ns switch target | SUE and SUE-T: under 2 µs round trip. Tomahawk Ultra: 250 ns switch, under 400 ns XPU to XPU including the switch |
| What kind of number | Not applicable | Design goal for a specification | Requirement for the spec. Vendor claim for the chip |
| Stated conditions | Not applicable | Cables under 4 m, 1 to 4 racks, up to 1,024 endpoints | SUE-T: 1,500-byte packet, single hop. SUE: up to 10 m to the switch |
| Silicon status | NVIDIA said in January 2026 that Vera Rubin was in full production | Native 1.0 chips in labs from the second half of 2026, per the Consortium's chair | Tomahawk Ultra announced as shipping in July 2025 |
Do not line these up as a ranking. Broadcom's sub-400 ns figure belongs to one chip, and the release text we read does not say whether it is one way or what packet size it assumes. If it is one way, the round trip is near 800 ns before any memory access at the far end. The 2 µs in SUE-T is a ceiling for a large packet, so it does not show that Ethernet is twice as slow as UALink's 1 µs goal. Neither UALink nor SUE-T states, in the text we read, how much of the far-end memory access time the figure includes.
The Register also pointed out that AMD's tunnelled UALink over Ethernet will not approach the 100 to 150 ns switch target, and that Broadcom's own claim of beating UALink on latency cannot be checked until hardware ships. So the honest position at the time of writing is that no public source we read allows a like-for-like measurement across all three.
Memory semantics: same words, different depth
All three will tell you they do load, store and atomics. The detail is where a design team's work appears:
| Question | NVLink | UALink 1.0 | SUE and SUE-T |
|---|---|---|---|
| Operations | NVIDIA decks from 2016 and 2018 list reads, writes and atomics to peer GPUs. Its 2025 blog calls NVLink a memory-semantic fabric | Direct read, write and atomic transactions, specified by the standard | SUE: generic commands whose opcodes are XPU-specific. SUE-T maps to UEC formats for load, store, atomic and fetching atomic |
| Coherence | Coherent over NVLink-C2C. We found no current statement for GPU-to-GPU traffic | Software coherency | Could be defined on top, but outside SUE's scope |
| Addressing | In the 2018 deck, the GPU translates and packets carry physical addresses | 10-bit accelerator IDs, with addresses compressed by a cache in the transaction layer | Translation and memory registration handled by the XPU, outside the transport |
| Ordering | Not stated in the current pages we read | Same-address ordering for requests; completions unordered | SUE: strict per source, destination and channel, or unordered across ports. SUE-T: ordered delivery with go-back-N |
| Largest unit | 256 bytes in the NVLink 1.0 deck | 256-byte payload in the white paper's example | 256 bytes per transaction in SUE-T, packed up to 16 commands in a packet |
| Recovery | Not stated in the current pages we read | Link-level replay on each 640-byte flit | Link Level Retry, plus transport retransmission. In SUE-T, retransmission is optional and a lost packet is then fatal to the connection |
The NVLink column rests on old decks, and we are assuming the model has carried forward. Check it against your NVIDIA documentation if your design depends on it.
Two rows deserve a second look. First, what an acknowledgement means. Broadcom's SUE says a one-sided transaction is acknowledged as having arrived, and not as having been placed in memory. SUE-T offers two modes: a Delivery ACK, and an Operational ACK that carries the XPU's response. Fence and completion logic in your runtime has to match the mode you pick. Second, the opaque command model. It lets a company with an existing on-chip protocol carry it over Ethernet through an AXI-style interface, which is attractive for a custom accelerator. The price is that two different vendors' accelerators will not talk to each other just because both use SUE. UALink aims to give you that common protocol, although its interoperability and compliance programmes were still described as planned in April 2026.
A worked example: what the round trip costs
A longer round trip does not slow a link's raw bandwidth, but it raises the amount of state you must hold to use that bandwidth. The model below uses illustrative assumptions: 9.6 Tb/s per direction per accelerator, 256 bytes per transaction, and three chosen round-trip times. None of these are measurements of any product. Replace them with your own.
| Round trip | 0.5 µs | 1 µs | 2 µs |
|---|---|---|---|
| Bandwidth per direction (9.6 Tb/s ÷ 8) | 1.2 TB/s | 1.2 TB/s | 1.2 TB/s |
| Data in flight (bandwidth × round trip) | 0.6 MB | 1.2 MB | 2.4 MB |
| Outstanding 256-byte reads to fill the pipe | 2,344 | 4,688 | 9,375 |
| Blocking reads per second, one thread (1 ÷ round trip) | 2,000,000 | 1,000,000 | 500,000 |
Doubling the round trip doubles the request trackers and response buffering you need at the same throughput. That is silicon area in every accelerator port. It also halves the rate of any code that waits for each load before issuing the next, however wide the link is. Bulk stores barely notice. Pointer chasing and fine-grained synchronisation notice straight away.
Header overhead is the other argument you will hear. Broadcom says its switch reduces headers from 46 bytes to as low as 10. Here is what that does for 256-byte stores, ignoring per-command control bytes, preamble and inter-frame gap, so these are upper bounds:
| Payload efficiency | 46-byte header | 10-byte header |
|---|---|---|
| One store per packet | 256 ÷ 302 = 84.8% | 256 ÷ 266 = 96.2% |
| Four stores per packet | 1,024 ÷ 1,070 = 95.7% | 1,024 ÷ 1,034 = 99.0% |
Packing recovers most of the loss, but only under load. SUE-T says transactions are not delayed for the sake of packing, so a lightly loaded queue sends one store per packet, and that is where the smaller header helps. For reference, UALink's white paper shows 20 of 21 flits as payload in a five-write example, which is about 95% at that layer alone. It counts write completions and leaves out lower-layer overhead, so it is not like for like with the table above. Treat it as a sign that the gap is small once traffic is busy, and test your own traffic mix.
Who controls each standard
| NVLink | UALink | ESUN and SUE-T | |
|---|---|---|---|
| Owner | NVIDIA | UALink Consortium, incorporated October 2024 | Open Compute Project workstreams, aligning with UEC and IEEE 802.3 |
| Who steers | NVIDIA alone | Board: Alibaba, AMD, Apple, Astera Labs, AWS, Cisco, Google, HPE, Intel, Meta, Microsoft, Synopsys | ESUN: 12 founders grown to over 175 companies, with Meta and Microsoft leading the core spec. SUE-T contributed by Broadcom and Arista |
| Can you read the spec? | No public specification that we found | Yes, public download | Yes, under OCP's Open Web Foundation agreements |
| Can you implement it alone? | Through NVLink Fusion, on NVIDIA's terms | That is the stated aim, under Consortium terms | That is the stated aim, under the OCP agreements |
NVLink Fusion, announced in May 2025, is NVIDIA's answer to the openness charge. Custom accelerators connect through a UCIe-to-NVLink bridge chiplet, and custom CPUs through NVLink-C2C IP. ServeTheHome reported at launch that it was "an either/or technology", with NVIDIA silicon needed on every node. The Register reported the same limit: an Intel CPU could not talk to an AMD GPU through it. Terms may have moved since. Ask NVIDIA directly, and take advice before relying on any of the licence terms above. This is not legal advice.
Openness also has limits. UALink's board includes AMD, and the author of the 1.0 white paper is listed with AMD. Broadcom authored the base that became SUE-T and sells switches built for it. Neither fact is a flaw, but a standard's history tells you whose hardware it suits first.
The "war" framing also hides how much the camps overlap. NVIDIA is a founding member of ESUN. Broadcom was a UALink member in July 2025, with no UALink switch on its roadmap. AMD's Helios brochure says the rack uses UALink over Ethernet for up to 72 GPUs, quoting 260 TB/s of scale-up bandwidth, and HPE's Helios switch uses Broadcom's Tomahawk 6. That brochure marks a number of its figures as projections, so read it as AMD's claim.
Bandwidth numbers can move
NVIDIA's Vera Rubin NVL72 page now lists 3 TB/s per GPU and 216 TB/s per rack. Press comparisons of archived copies say it earlier showed 3.6 TB/s and 260 TB/s. We did not find an explanation from NVIDIA that we could read directly, so we will not guess at the reason here. When we read them, NVIDIA's general NVLink and Fusion pages still carried the older numbers. If your plan compares racks, quote the product page and its date, and do not set AMD's 260 TB/s against NVIDIA's 216 TB/s as if both were measured the same way. Copper may not stay the only medium for long either, which we looked at in After CoWoS, Optics.
Which should you plan around?
Answer these five questions before you commit an accelerator or a rack design:
- Are you buying accelerators or building one? If you buy NVIDIA GPUs, the choice is largely made for you. The rest of this list is for custom silicon teams and those buying from other vendors.
- Do you need accelerators from different vendors in one pod? If yes, a shared protocol matters, which favours UALink. Opaque Ethernet commands will not provide it.
- Do you already have a proven on-chip protocol? If yes, carrying it over SUE-T through AXI may be far less work than adopting a new one.
- How much of your remote traffic is blocking loads? Mostly bulk stores means bandwidth and header cost lead. Mostly fine-grained loads means round trip leads, and you should ask each vendor for a measured figure at your packet size.
- When does your silicon need a switch to exist? A tape-out that assumes native UALink switches should hold a dated fallback, such as UALink over Ethernet.
Two or more answers pointing the same way make a working assumption. If they split, keep the protocol layer separable from the link so you can change course. UALink's new chiplet specification, which it describes as compliant with UCIe 3.0, and NVIDIA's bridge chiplet both show that this idea is taking hold. Our chiplet lifecycle on The Semiconductor Hub shows where these decisions fall in a programme.
FAQ
Is UALink over Ethernet the same as UALink?
No. As we read it, it keeps the UALink approach to memory traffic but uses Ethernet switches in place of native UALink ones, as in HPE's Helios design. We did not find a technical definition to read. Expect different switch latency from the native target, and ask for measured numbers.
Is ESUN a competitor to UEC?
Not in the sources we read. ESUN says it aligns with UEC and IEEE 802.3, and SUE-T can run over ESUN, over UEC's UFH or over UDP/IP. SUE-T refers to a draft of UEC 1.1, which it says was due for publication in mid-October 2026, so check that against the final text.
How large can each domain get?
UALink specifies up to 1,024 accelerators, and SUE-T scopes up to 16K, with a single hop preferred for latency. NVIDIA's shipping rack is 72, and its Fusion page plans domains of up to 1,152. Those are all limits or plans, and none of them is an observed deployment.
Sources
- Kalyanasundharam (AMD, UALink Consortium), "Introducing UALink 200G 1.0 Specification", white paper, April 2025.
- UALink Consortium, news release on four specifications, 7 April 2026.
- The Register, "No-Nvidia interconnect club delivers 2.0 spec before v1.0 silicon ships", 7 April 2026, for the chair's silicon timeline.
- Broadcom, "Scale-Up Ethernet Framework Specification", Scale-Ethernet-RM104, 26 September 2025.
- Schramm et al. (Broadcom, Arista), "SUE-T: Base Specification 1.0.0", Open Compute Project, effective 8 October 2026.
- Open Compute Project, "Introducing ESUN", 13 October 2025, "The OCP ESUN 1.0 Specification Has Been Released", 10 March 2026, and the ESUN project page.
- Keysight, "ESUN gives up IP", blog, 6 August 2026.
- Broadcom, Tomahawk Ultra release, 15 July 2025, as reproduced by Microwave Journal, and The Register, "With Tomahawk Ultra, Broadcom asks who needs UALink when there's Ethernet?", 15 July 2025.
- NVIDIA, NVLink and NVLink Fusion product pages, read 5 November 2026, and Vera Rubin NVL72 product page, read 7 November 2026.
- Tom's Hardware, report on Jensen Huang's CES keynote and Vera Rubin production, January 2026.
- TweakTown, report on the Vera Rubin specification change, 2026.
- DeLaere (NVIDIA), "Scaling AI Inference Performance and Flexibility with NVIDIA NVLink and NVLink Fusion", 21 August 2025.
- NVIDIA, Pascal GPU deck, Hot Chips 28, 2016, and NVSwitch and DGX-2 deck, Hot Chips 30, 2018.
- ServeTheHome and The Register, reports on NVLink Fusion, 19 May 2025.
- AMD, Helios rack-scale brochure (undated, footnotes from 2025), and The Register, "HPE backs AMD's Helios AI rack with Juniper's scale-up switch", 2 December 2025.
The worked example uses illustrative assumptions. Vendor, Consortium and specification figures are the authors' own claims and targets, as published on the dates shown, and none are independent measurements. This post is not legal advice.
Your comments will be moderated before it appears here.