DeepEP's pitch is an O(top-k) all-to-all that should beat a dense / allgather alternative. This section shows — with measured charts — how that pitch holds up across five independent scaling axes on AWS EFA: node count, concurrency, world size, expert count, and the multi-comm knob. Every chart is calibrated to the same campaign: 2× p5en.48xlarge, 16 EFA NICs/node, Qwen3-30B-A3B-FP8 EP16 unless noted. Color key throughout: DeepEP all-to-all · dense / alternative.
The one-line finding, stated up front: on EFA the throughput gap never closes by scaling. Intra-node it is a tie; cross-node the alternative wins (TRT) or ties (vLLM); more nodes hold the tie; more experts make DeepEP slower; more concurrency changes nothing. The charts below are the evidence, axis by axis.
When the whole expert-parallel group fits in one node, the
cross-rank all-to-all is served entirely over NVLink (intra-node, kIsScaleupNVLink).
No EFA NIC is touched, so the CPU-proxy round-trip that handicaps DeepEP on EFA simply does not exist.
DeepEP and the alternative converge to a tie (≈0.98×, NVLink parity).
Single-node is a tie by construction — there is no fabric decision to make. The DeepEP-vs-alternative question only becomes real once EP spills past one node and the all-to-all has to cross EFA. Everything below starts at that boundary.
Schematic. measured single-node EP8 shows DeepEP and the alternative within NVLink parity — no EFA fabric on the data path.
At EP16 the group spans two nodes and the all-to-all must cross EFA for the first time. This is the campaign's calibration point. On TRT-LLM the dense allgather path is fast, so DeepEP runs consistently at ~0.67× its aggregate tok/s — the same ratio across three workload shapes.
measured TRT-LLM 0.21, DeepEP (EP_NUM_COMMS=8) vs dense allgather, EP16 on 2× p5en EFA, AIPerf 0.10.0. The ratio is flat ~0.67× across conc4, conc32, and a long-prompt ISL2048 shape — DeepEP loses by a constant factor regardless of the workload. (EXP-AB-TRTLLM-DEEPEP-VS-DENSE-2026-06-23.)
A natural hope: maybe DeepEP catches up under heavy load, as the all-to-all volume grows. It does not. On vLLM (DeepEP-V2 vs allgather_reducescatter) the DeepEP/non-DeepEP ratio stays flat ~1.0 from conc32 to conc256 — both arms scale cleanly, the ratio simply refuses to climb.
measured vLLM DeepEP-V2 vs allgather_reducescatter, EP16. Decode ratio 0.97 / 1.03 / 0.94 / 0.96; prefill ratio 1.17 / 0.95 / 0.94 / 1.10 — both hover on the tie line with no trend. Absolute throughput scales cleanly in both arms (vLLM decode 248→2435 tok/s, DeepEP 241→2325) — so the flatness is real scaling, not saturation. (docs/research/2026-06-19-MAKE-DEEPEP-BENEFICIAL-CLOSEOUT.md.)
Scaling the world size from 16 to 32 ranks (2 → 4 nodes) halves the per-GPU dispatch budget — per-GPU SO drops 33 → 22 GB/s as bytes spread over more peers. But the DeepEP/non-DeepEP serving ratio holds a dead tie ~1.00–1.02×, and the underlying transport SO barely moves (36 → 35 GB/s). Scaling out does not unlock a DeepEP advantage.
measured vLLM deepep_v2 vs allgather_reducescatter at EP32 (4× p5en). Per-GPU dispatch SO falls 33→22 GB/s purely from sharding the same bytes over twice the peers, yet the serving ratio is a dead tie (1.00–1.02×) and transport SO holds 36→35. The DeepEP advantage does not emerge at larger world size. (EP32 4-node A/B; transport SO held re-confirmed EXP-G-series.)
This is the axis where DeepEP scales the wrong way on EFA. Doubling experts from 128 to 256 splits the same payload into more, smaller cross-node messages. On an issue-bound CPU-proxy path (limited by per-op post rate, not bytes), smaller messages are pure overhead: dispatch SO collapses 36 → 13 GB/s (0.36×).
measured DeepEP-V2 dispatch SO, decode regime. 128 experts → 36 GB/s; 256 experts → 13 GB/s. More experts = more, smaller messages = worse for the issue-bound proxy. The MoE design dimension that should help (more experts → finer routing) actively hurts DeepEP's transport on EFA. (MEASURED-DATA.md transport table.)
There is a knob that moves the transport number a lot. Because a single NCCL-GIN
proxy thread is the issue-bound bottleneck, running K independent comms (EP_NUM_COMMS=K)
gives K parallel proxy threads. Transport dispatch SO climbs K=1 (~8) → K=8 (33 GB/s) — a
4.3× recovery, peaking at K=8. This is the single most important way to run
DeepEP well on EFA.
EP_NUM_COMMS=8 only recovers DeepEP from 2× slower (142 tok/s,
single-comm) up to parity (296 tok/s) with the allgather alternative (304 tok/s) — it does
not make DeepEP beat the alternative. Turn it on (it is the right way to run DeepEP), but do not
expect it to flip the verdict.
measured transport SO per EP_NUM_COMMS; serving recovery 142→296 vs allgather 304. (project_g2_multicomm_k2_beats_8gbs_ceiling; EXP-AB-DEEPEP-MULTICOMM-VS-DENSE-2026-06-23.)
Five axes, one conclusion: scaling DeepEP on EFA does not convert its byte-level advantage into a serving-throughput win. The verdict per axis:
TIE. All-to-all rides NVLink, no EFA — DeepEP ≈ alternative. No fabric decision to make.
Alt wins (TRT 0.67×) / ties (vLLM 0.97×). The cross-node EFA tax appears and DeepEP pays it.
Dead tie (1.00–1.02×). More nodes hold the tie; transport SO held 36→35. No advantage emerges at scale.
WORSE (0.36×). 128→256 experts drops SO 36→13 — smaller messages starve the issue-bound proxy.
NO CHANGE. Ratio flat ~1.0 from conc32→256. Load does not let DeepEP catch up.
4.3× transport, but serving only 142→296 = parity. The right way to run it — not a way to win.
Why scaling can't close the gap (the mechanism): DeepEP's advantage is in
all-to-all bytes (O(top-k)), and at ~2.2% NIC utilization the EFA dispatch path is
issue-bound — limited by per-op post rate through the CPU proxy, not by wire bandwidth. Every
scaling axis either leaves that per-step proxy tax untouched (concurrency, world size) or makes it
worse (more experts). The only lever that would convert bytes into tok/s is GPU-initiated transport
(GDAKI / IBGDA), which removes the host round-trip — and that is not available in any shipping EFA stack today. Update 2026-07-17: a native kernel-posted path (CQ-poll, no hardware counters) is now measured in §10 — it beats the proxy on dispatch/combine latency but remains a research path, not a serving recipe; this section's verdict is unchanged.
Use DeepEP on EFA for capacity (fitting a too-big MoE) and run it with EP_NUM_COMMS=8;
do not expect to scale your way to a throughput win.
Next: §4 Microbenchmarks → the per-layer time budget behind these numbers — where the issue-bound tax actually lives.