§4 · Microbenchmarks + real inference numbers

This is the evidence section. Every number below is measured on the efa-gda campaign — 2× p5en.48xlarge, AWS EFA, Qwen3-30B-A3B-FP8 EP16 — and cites the experiment record it came from. The arc is one finding, measured two ways: at the transport layer DeepEP's O(top-k) volume reduction is real (and UCCL is ~4× faster still); at the serving layer none of it converts — DeepEP ties or loses on tokens/second. The wire is 98% idle; the path is issue-bound, not bandwidth-bound.

33–35

SO GB/s prefill · shipped ceiling (multi-comm)

10–12

SO GB/s decode · shipped ceiling

113

GB/s node injection ceiling — silicon is not the limit

~2.2%

EFA NIC utilization — issue-bound, not wire-bound

1 · Transport-layer microbenchmarks measured

DeepEP-V2 test_ep all-to-all bandwidth, split by path and regime. SO = scale-out (cross-node, over EFA SRD). SU = scale-up (intra-node, over NVLink). Prefill carries 4096 tokens/rank; decode carries 128. The shipped numbers use the EP_NUM_COMMS=8 multi-comm knob (K=8 NCCL-GIN progress threads behind the ElasticBuffer wall); the raw floor is the single-comm path before that knob.

DeepEP-V2 dispatch / combine bandwidth, GB/s. SU is NVLink intra-node; SO is cross-node over EFA.
path	regime	SO (cross-node EFA)	SU (NVLink)	raw floor (single-comm)	source
dispatch	prefill (4096 tok)	33–35	110–117	1–7	EXP-G40 · EXP-AB-MULTICOMM-2026-06-23
dispatch	decode (128 tok)	10–12	24–25	1–7	EXP-LOOP2-BASELINE-2026-06-23
combine	prefill (4096 tok)	27–28	88.7	1–7	EXP-AB-MULTICOMM-2026-06-23
combine	decode (128 tok)	10–12	24–25	1–7	EXP-LOOP2-BASELINE-2026-06-23

The 3.3× cross-node penalty

SO/SU ≈ 0.30 on dispatch — the same payload moves 3.3× slower cross-node (EFA) than intra-node (NVLink). That gap is the whole problem this site is about. It is not the wire: the same 16-NIC node injects 113 GB/s when measured directly (EXP-G40). The penalty lives in the per-op post rate through the CPU proxy + libfabric, not in EFA's bandwidth.

Multi-comm is the lever, and it tops out

EP_NUM_COMMS stripes K independent NCCL-GIN progress threads: K=1 ≈ 8 → K=2 = 12–13 (1.7×) → K=4 = 21 → K=8 = 33 SO (4.3×, peak). Both prefill and decode peak at K=8; past that, per-thread issue efficiency is the ceiling. This is the recommended way to run DeepEP on EFA. EXP-AB-MULTICOMM-2026-06-23

Cross-node SO bars (orange) sit far below the NVLink SU bars (blue) and far, far below the 113 GB/s node injection ceiling (green dashed). The bottleneck is the plugin/proxy issue path — not EFA silicon.

2 · Real inference numbers measured

The transport bytes don't tell you what a user sees. These are end-to-end serving A/B runs — DeepEP vs the same framework's dense / allgather path, identical model + build + nodes — measured with AIPerf 0.10.0, backend-asserted (the framework provably instantiates ElasticBuffer on EFA, not a silent fallback). Aggregate tokens/second, per-user tokens/second, and time-to-first-token.

Serving A/B — DeepEP vs same-framework dense. agg = aggregate tok/s across all users. Ratio <1 means DeepEP is slower.
framework	DeepEP backend vs dense	regime	DeepEP agg	dense agg	DeepEP / dense	source
TRT-LLM 0.21	ElasticBuffer GIN-V2 vs allgather	conc4 / ISL120 / EP16	64.24	106.36	0.60×	EXP-AB-TRTLLM-DEEPEP-VS-DENSE-2026-06-23
TRT-LLM 0.21	DeepEP + EP_NUM_COMMS=8 vs dense	conc4 / ISL120 / EP16	68.7	105.66	0.65×	EXP-AB-TRTLLM-DEEPEP-VS-DENSE-2026-06-23
TRT-LLM 0.21	DeepEP-mc8 vs dense	conc32 / ISL120 / EP16	545.04	794.56	0.69×	EXP-AB-TRTLLM-DEEPEP-VS-DENSE-2026-06-23
TRT-LLM 0.21	DeepEP-mc8 vs dense	conc16 / ISL2048 / EP16	284.22	424.67	0.67×	EXP-AB-TRTLLM-DEEPEP-VS-DENSE-2026-06-23
vLLM	deepep_v2 (EPNC8) vs allgather_reducescatter	conc32 / decode / EP16	296	304	0.97× (TIE)	EXP-AB-DEEPEP-MULTICOMM-VS-DENSE-2026-06-23
vLLM	deepep_v2 (EPNC8) vs allgather_reducescatter	conc64 / prefill / EP16	18201	22797	0.80×	EXP-AB-DEEPEP-MULTICOMM-VS-DENSE-2026-06-23
vLLM	deepep_v2 vs allgather_reducescatter	EP32 prefill (4-node)	—	—	1.02× (TIE)	EXP-AB-DEEPEP-MULTICOMM-VS-DENSE-2026-06-23
vLLM	deepep_v2 vs allgather_reducescatter	EP32 decode	—	—	1.00× (TIE)	EXP-AB-DEEPEP-MULTICOMM-VS-DENSE-2026-06-23

The per-user signature — a fixed CPU-proxy tax. DeepEP per-user throughput is flat at 19.4–19.7 tok/s (σ=0.13) while dense is flat at 26.9–28.1 tok/s (σ=0.51), unchanged across conc4→32 and ISL120→2048. The difference is a constant +14.7 ms/token proxy round-trip. That it is constant across every regime is the evidence it is a fixed per-decode-step cost, not noise — and a fixed per-step cost cannot be amortized by batching. EXP-AIPERF-SGLANG-AB-2026-06-20

TTFT degrades as input length grows measured

Time-to-first-token is where the per-step proxy tax shows up most sharply: the dispatch happens on every prefill step too, and longer inputs mean more steps before the first token. DeepEP's TTFT is progressively worse than dense as ISL grows.

measured TTFT ratios 1.05× / 1.49× / 3.14× across the short→mid→ISL2048 regimes. Source: EXP-AB-TRTLLM-DEEPEP-VS-DENSE-2026-06-23. Lower is better; the green dashed line is parity with the dense path.

3 · DeepEP vs UCCL vs pplx measured

Three expert-parallel transports on the same EFA hardware. UCCL uses a raw-ibverbs path; pplx is the pplx-garden reference; DeepEP-V2 uses the CPU-proxy NCCL-GIN path. Measured two ways: a transport-layer microbenchmark (µs per op) and a serving A/B (tokens/second through a real framework).

⚠ Caveat — the cross-bench µs are NOT apples-to-apples. The three transports are timing different op shapes: DeepEP measures a unified-dispatch; UCCL measures a low-latency send + recv; pplx measures a round-trip. They are the same order of magnitude and useful for a rough rank, but the microseconds are not subtractable across rows. Treat "~4× faster" as a coarse ratio, not a precise delta.

Transport microbenchmark — dispatch + combine latency, µs. Different op shapes per row (see caveat above).
transport	path	dispatch µs	combine µs	op shape (why not subtractable)	source
DeepEP-V2	CPU-proxy NCCL-GIN	186	264	unified-dispatch (one fused op)	EXP-UCCL-SGLANG-LIVE-2026-06-19
UCCL	raw-ibverbs	47	84	low-latency send + recv (split)	EXP-UCCL-SGLANG-LIVE-2026-06-19
pplx (pplx-garden)	reference kernels	222	362	round-trip (matches published; no serve adapter)	EXP-3c · EXP-3d-2026-06-19

Serving A/B — DeepEP-V2 vs UCCL, same framework (SGLang), ≥2-rep, decode variance <1%.
concurrency	DeepEP-V2 agg tok/s	UCCL agg tok/s	DeepEP / UCCL	source
32	263.9	254.2	1.038×	EXP-UCCL-SGLANG-LIVE-2026-06-19
256	2173.8	2105.4	1.032×	EXP-UCCL-SGLANG-LIVE-2026-06-19

Under standardized AIPerf, DeepEP is 1.03× aggregate over UCCL and ~3% faster on ITL / per-user — a statistical TIE, despite UCCL's ~4× transport-microbench edge. The all-to-all is not the serving critical path.

4 · Transport speed ≠ serving speed measured

This is the central finding of the campaign, in one picture. UCCL is ~4× faster than DeepEP at the transport layer. Run them through the same serving stack (SGLang, same model, same nodes) and that 4× edge collapses to ~1.0× — a dead tie on tokens/second. A faster wire does not make a faster serve when the wire is 98% idle and the bottleneck is somewhere else entirely.

Bars normalized to DeepEP = 1.0 within each group. measured Transport: dispatch 186 µs (DeepEP) vs 47 µs (UCCL). Serving: 263.9 vs 254.2 agg tok/s (conc32). Source: EXP-UCCL-SGLANG-LIVE-2026-06-19. The 4× transport advantage does not survive the trip to tokens/second — the all-to-all is not the serving critical path on EFA.

5 · Methodology + honesty measured

Setup

Hardware: 2× p5en.48xlarge, AWS EFA (16 NICs/node, 25 GB/s/NIC effective, 113 GB/s node injection measured).
Model: Qwen3-30B-A3B-FP8, EP16 (128 experts, top-8) unless a row says otherwise.
Tool: AIPerf 0.10.0 for serving; DeepEP-V2 test_ep for transport.
Transport: NCCL_GIN_TYPE=2, deep_ep.ElasticBuffer, EP_NUM_COMMS=8 (CPU-proxy NCCL-GIN). SRD: no atomics, no ordering, CQ-local, 1-SGE write.
Backend-asserted: every serving cell verifies the framework instantiates ElasticBuffer on EFA — not a silent dense fallback that would look like DeepEP but isn't.
Isolated serves: one backend per serve; no co-tenant interference on the measured pods.

What is and isn't statistically tight

≥2-rep, var <1%

The DeepEP-vs-UCCL decode sweep (conc32 + conc256) is replicated, decode variance under 1%.

n=1 (honest caveat)

Most serving A/B cells are single-run. The evidence of reliability is the cross-regime ratio tightness: per-user tok/s holds σ=0.13 (DeepEP) / σ=0.51 (dense) across conc4→32 and ISL120→2048. A fixed ratio that survives every regime is the signal that the +14.7 ms/token tax is structural, not noise.
Cross-transport µs (§3) are different op shapes — same order of magnitude, not subtractable.

Bottom line. DeepEP's transport advantage is real in bytes (1.45× over UCCL-published prefill; O(top-k) volume reduction) and never reaches tokens/second on EFA. The CPU-proxy posts each cross-node dispatch through the host; at ~2.2% NIC utilization the path is issue-bound, so cutting bytes doesn't cut the per-step latency that gates decode. Only GPU-initiated transport (GDAKI/IBGDA) would remove the host round-trip — and that is not available on EFA SRD today.

Next: §5 Alternatives →

All figures measured on the efa-gda campaign (2× p5en.48xlarge, AWS EFA, Qwen3-30B-A3B-FP8 EP16). Sourced per row in docs/experiments/EXP-* and deepep-calculator/MEASURED-DATA.md. Sanitized — no IPs, ECR account IDs, or hostnames.