TDEB Top Bar

Trending
Semiconductor Industry News — The Digital Electronics Blog
Follow Us

TDEB Masthead

The Semiconductor Hub

Semiconductors.  Chips and Systems.  Innovation.

Indexed by leading Search Engines, AI and referenced by Engineers worldwide 500+ Articles 10M+ Pageviews 30+ Reports 50+ Citations

What Ultra Ethernet Fixes in AI Back-End Networks, and What It Leaves Alone

Murugavel Ganesan
by
0
Share Facebook X LinkedIn WhatsApp Email
Illustration of data streams spreading evenly across many parallel paths in an AI cluster network fabric.

Put eight 400 Gb/s flows on eight equal uplinks, let a hash choose the path for each, and there is a 99.76% chance that at least two share a link. In the illustrative model below, a transfer that should take 2 ms takes about 5.19 ms on average, because the busiest link sets the finish time. Spray the same packets across all eight links and even an unlucky link finishes in about 2.04 ms.

That gap is the main thing Ultra Ethernet goes after. This post goes through what the Ultra Ethernet Consortium (UEC) changed for congestion and load balancing, using specification v1.1 of 12 October 2026, and then what stays as it was, including the problems no load balancer can solve.

Key takeaways

  • Ultra Ethernet is mainly a new transport. The specification says it does not require changes to the network layer or to the Ethernet PHY and link layers, and that it can run over existing switches.
  • Spraying still uses ordinary ECMP hashing in the switch. The sender changes an entropy value in each packet, normally the UDP source port, and the switch hashes it as usual.
  • Network signal congestion control (NSCC) is mandatory. Receiver credit control, packet trimming and the newer in-band telemetry are optional.
  • Spraying removes collisions in the fabric. It does nothing for incast on the last link, where eight senders in our example take 16 ms however well they are balanced.
  • Published work is not unanimous. One preprint argues that single-path transport can get close to spraying for collective traffic.

Why AI traffic defeats flow hashing

The specification separates a front-end network, which links compute nodes to the outside world, from a back-end scale-out network, which carries training traffic within a cluster. The back-end is where this post sits.

ECMP hashes header fields so that every packet of a flow takes the same path. That suits a cloud network, where many small flows average out. Alibaba's SIGCOMM 2024 paper on its HPN network gives the contrast: general cloud computing produces millions of flows, each typically using less than 20% of the NIC, while training produces very few flows in periodic bursts that can reach the full 400 Gb/s of the NIC. The paper adds that a flow in a three-tier network is hashed at three layers using the same five-tuple, which can make the imbalance worse.

Meta reported the same problem from its own clusters in an August 2024 engineering post. ECMP performed poorly on training because of low flow entropy, and Meta moved first to path pinning, then to hashing on the RoCE destination queue pair (QP) as well, with traffic split across more QPs. That raised AllReduce performance by up to 40% against baseline ECMP in its tests, yet Meta still named the probabilistic nature of hashing as a persistent downside.

Timing makes it worse. UEC's 2023 overview points out that the last message to arrive holds back every GPU, so one overloaded link slows the whole job.

An illustrative example: flow collisions under ECMP

Every number in this section is an illustrative assumption, not a measurement from any network. A switch has eight 400 Gb/s uplinks. Eight flows of 100 MB each leave at the same moment, each to a different destination. The hash places each flow on an uplink independently and uniformly. Protocol overhead is ignored.

With no sharing, each flow takes 800 Mb divided by 400 Gb/s, which is 2 ms. If k flows share a link, they split it evenly and all finish at k × 2 ms. The job waits for the busiest link.

Most flows on any one uplinkProbabilityJob finish time
1 (no collision)0.24%2 ms
249.84%4 ms
340.94%6 ms
48.00%8 ms
5 or more0.98%10 ms or more

The chance of no collision is 8! divided by 88, or 40,320 over 16,777,216, which is 0.24%. The average of the busiest link is 2.60 flows, so the average finish is about 5.19 ms, roughly 2.6 times the clean figure. On average, 2.75 of the eight uplinks (8 × (7/8)8) carry nothing at all while the job waits.

Now split each flow into four sub-flows of 25 MB, which is the idea behind QP scaling. There are 32 sub-flows, each adding 0.5 ms of work to its link. The busiest link averages 7.05 sub-flows, so it needs about 3.52 ms. That is better, but still about 1.76 times the clean figure.

Finally, spray. At 4,000 bytes of payload, each flow is 25,000 packets, so there are 200,000 in all and each uplink expects 25,000. If each packet picks a link independently, the standard deviation per link is the square root of 200,000 × 1/8 × 7/8, about 148 packets. A link three standard deviations high carries 1.78% more than average and finishes in about 2.04 ms. Real hashes are less tidy and this leaves out queues and retransmission, but the size of the gap is the lesson.

What packet spraying actually changes

The switch has little new to do. UET packets carry an entropy value (EV) in the UDP source port, which sits inside the five-tuple that switches already hash. Destination port 4793 identifies UET. The specification assumes that two packets with the same EV follow the same path, and that switches do not modify it. Path selection therefore moves to the sender, using a mechanism already in the switch.

Three modes are allowed:

  • Single path. Used only by reliable ordered delivery (ROD). One EV is held for a period, and the specification recommends changing it no more often than every 10 round trips by default if the flow never goes idle.
  • Oblivious spraying. The sender walks pseudo-randomly through a space of typically 256 EVs and keeps no record of which are congested. The specification says this works best with packet trimming and discourages it without.
  • Path-aware spraying. The receiver reports which EVs saw congestion, and the sender avoids reusing them for at least one round trip. If most paths are congested, skipping them no longer helps, and the specification suggests 50% as a reasonable default for that point.

Spraying cannot work if the receiver insists on order, so delivery order had to change as well. In reliable unordered delivery (RUD), packets can arrive in any sequence, and that is where the multipath recommendation applies. ROD still exists and uses go-back-N recovery, the method UEC's overview criticises in RoCE: one lost packet forces retransmission of up to N packets.

UET does not mandate a load balancing algorithm, which the specification says is to leave room for vendor differentiation. It describes two example approaches. One is REPS, published at EuroSys 2026, in which a sender reuses EVs from ACKs that show no congestion. It needs under 25 bytes of per-connection state and nothing from the switch beyond ECMP hashing and ECN. The authors, several of whom are also listed as UEC contributors, report simulated results of up to 6 times better than ECMP and 1.25 times better than oblivious spraying in symmetric networks. The other example is a per-EV bitmap. Version 1.1 also adds programmable congestion management, so operators can define their own algorithms in software.

Congestion: five causes, different tools

The specification lists five causes of congestion in a data centre network, and spraying addresses only some of them.

CauseMain UET responseLimit
Hot links, from hash collisions or failuresSpraying and path-aware EV choiceNon-sprayed traffic and hash unevenness remain
Most paths congested, as in an under-provisioned coreNSCC reduces the send windowSlows senders, and cannot add capacity
Outcast at the sourceLimits on data in flightThe specification leaves NSCC's handling to implementers
Incast at the destinationNSCC, and optionally receiver credits (RCCC)The last link cannot be rerouted around
Memory access at the destinationDestination flow control within NSCCFabric balancing cannot help, on our reading

NSCC and RCCC

NSCC, which every implementation must have, keeps one congestion window across all sprayed paths. It reads two signals, ECN marks and queueing delay, and treats ECN as the leading one because modern switches mark at dequeue. No mark means the window grows, faster when delay is low. A mark with low delay means no change. A mark with high delay means a cut in proportion to the excess queue. On loss, a quick adapt step sets the window directly from measured goodput.

NSCC deliberately ignores ECN marks that come without delay, since those often mean imperfect balancing and the load balancer should get the first chance to react.

RCCC handles incast from the other end, with the receiver issuing credits to senders. The specification recommends running NSCC alongside it, with ECN turned off on the last hop so that last-hop congestion does not confuse the load balancer. Its own text notes that core congestion from oversubscription needs an extra mechanism.

Packet trimming

Spraying makes loss hard to spot. Single-path transports infer loss from out-of-order arrivals, but sprayed packets are out of order all the time. Trimming solves this: instead of dropping a packet that fails buffer admission, a switch cuts off the payload and forwards the header in a separate traffic class. The receiver sends a NACK, and the retransmission will, with high probability, take a different path. Trimming is optional for switches, but endpoints must accept trimmed packets. The UEC authors' 2025 overview notes that it does not reveal corruption drops, only congestion ones, so other detection is still needed.

What stays unchanged

  • Physical layer. UE uses IEEE 802.3 100G and 200G per-lane signalling. Better optics, including co-packaged optics for AI clusters, are a separate story from the transport.
  • Link layer. All UE link features are optional, including link level retry, credit-based flow control, idle reduction and the Unified Forwarding Header. Ethernet's own layers 1 and 2 keep working as defined.
  • Network layer. Fabrics use IPv4 or IPv6 forwarding and existing routing protocols, or UFH forwarding under the Unified Local profile.
  • Switch forwarding. ECMP stays, and the specification says the forwarding paradigm needs no changes. Trimming, Congestion Signaling (CSIG) and in-network collectives are optional, and those do need NOS extensions.
  • Application frameworks. Running TensorFlow, PyTorch or JAX unchanged is a stated goal, with the collective communication library sitting underneath. Libfabric v2.0 and libibverbs are both supported interfaces.
  • Topology assumptions. Congestion management targets folded Clos networks, and the specification sets no performance objectives for other topologies.
  • The arithmetic of the last link. If all eight flows in our example went to one 400 Gb/s receiver, 6,400 Mb would take 16 ms whichever balancer was used.

Three caveats sit alongside that list. First, the specification expects switches to mark ECN at dequeue rather than enqueue, using instantaneous queue depth. It calls dequeue marking common in current switches, but says NSCC will not perform as desired without it, so check your silicon. Second, the transport itself has to live somewhere, so on our reading the real change falls on the endpoint: the NIC or the accelerator's network interface. Third, the design aims at best-effort networks. UET can run on lossless ones, but the congestion control section says that use case is specifically not covered in the initial design.

UE is also a different kind of Ethernet extension from Time-Sensitive Networking. TSN schedules traffic to bound its latency, while UET aims for fast overall completion and accepts short-term unfairness on the way there.

What is still being argued

Other routes to the same goal exist, and three published ones are worth reading.

  • Change the topology. Alibaba's HPN uses a two-tier, dual-plane design to avoid hash polarisation, and reports end-to-end training performance more than 14.9% better after one job moved over. That figure reflects the whole architecture, not hashing alone.
  • Keep the single path, and be smarter. The Ethereal preprint (arXiv, February 2025; we found no conference listing) argues that single-path transport, with flows split and assigned deliberately, can approach ideal spraying because collective traffic is so predictable. In its authors' simulations, completion times fall by up to 30% against end-host packet spraying and up to 40% against REPS and ECMP. These studies use different setups, so their numbers should not be set side by side.
  • Run RoCE differently. Meta reported over a year at 400G with DCQCN switched off, using PFC and receiver-driven admission in the collective library, with stable performance. That is one operator's result on its own traffic.

So the gain depends on your traffic and your fabric. REPS's own evaluation shows little difference between balancers in cases where congestion control dominates, and large differences on permutation patterns. This post makes no claim about shipping products or market share, because these sources do not establish them.

Questions for chip and system teams

This list is our reading of the specification, and it does not replace the normative text.

  1. Which profile are you building? UET now defines four: Unified Local, AI Base, AI Full and HPC. Version 1.1 adds Unified Local for scale-up networks, with its own ordered delivery mode.
  2. How good is your hash on the UDP source port? A simple modular counter for EVs is permitted, but it relies on the switch hash for its randomness.
  3. How much per-path state can the NIC hold? This decides between oblivious spraying, REPS-style recycling and a bitmap.
  4. Will your fabric trim? Without trimming, oblivious spraying is discouraged and path-aware spraying matters more.

These decisions belong early in a networking chip programme. Our SoC lifecycle on The Semiconductor Hub shows where they fall.

FAQ

Do I need new switches to run Ultra Ethernet?

The specification says UE operates over existing Ethernet switches, with ECN marking the only required in-network telemetry, and it expects that marking at dequeue. Trimming, CSIG and in-network collectives need switch support, and they are optional.

Does it replace RoCE?

That was the stated aim of UEC's members in the 2023 overview. Version 1.1 also adds a verbs interface mapping.

Sources

The worked example uses illustrative numbers and a simplified model. Results from papers are the authors' own, in their own test conditions.

Murugavel Ganesan
About the Author
Murugavel Ganesan
Product Director | SerDes · IVN · Edge AI · Compute | Ex Intel · Infineon

A semiconductor product leader working across product architecture, definition, strategy, and silicon and software execution, from concept through volume production. He has worked with teams across Asia, Europe and the USA, including at Intel, Infineon and ArrayComm.

His work covers ASICs, SoCs, FPGAs and chiplets for wireless, wireline, automotive, IoT and AI, including PCIe, CXL and DDR interfaces alongside RF and mixed-signal design.

Share Facebook X LinkedIn WhatsApp Email

Post a Comment

0 Comments

Your comments will be moderated before it appears here.

Post a Comment (0)

#buttons=(Ok, Go it!) #days=(20)

Our website uses cookies to enhance your experience. Learn more
Ok, Go it!