§9 · Reproduce everything from PUBLIC roots — six gates, one repo measured

Everything this guide claims about DeepEP-V2 on EFA is now reproducible by anyone with 2× p5en and public internet: official NGC bases, deepseek-ai/DeepEP, PyPI wheels, aws/aws-ofi-nccl, and a set of committed patch files. No private image, fork, or registry anywhere in the chain. The July 2026 re-run also produced one new headline the June campaign only had on a fork build: the multicomm lever doubles vLLM throughput on the pure public stack.

The repo: github.com/antonai-work/deepep-v2-efa-repro — GUIDE (what to run, in order), README (full recipe + pins), RESULTS (every number on one page), patches/ (each verified apply-clean on its public root), bench/ + training/ + trtllm/ + llmd/ lanes, and results/ with the as-run AIPerf exports and serve-log evidence behind every row below.

The six gates (all PASS, 2026-07-06/07 re-run)

2× p5en.48xlarge (H200, 16 EFA NICs/node), Qwen3-30B-A3B-FP8, EP16 · AIPerf 0.10.0 ISL120/OSL128 conc 4/8/32
#	gate	what it proves	measured
1	micro D+C (test_ep, cross-node)	the transport itself	443.6 µs p50 (contract ≤740; dispatch 185.4 + combine 258.2)
2	vLLM 0.24.0 + AIPerf	unmodified PyPI wheel serves MoE	c32 144.6 agg tok/s · 0 err (first PyPI release with `DeepEPV2All2AllManager`)
3	SGLang 0.5.11 + seam patch + AIPerf	second engine (PR #24443 lineage)	c32 255.2 · 0 err (1.01× the June campaign baseline)
4	TRT-LLM 0.21.0 + native seam + AIPerf	third engine, no shim (one-file patch)	c32 563–587 · 0 err · `AlltoallMethodType.DeepEP` asserted
5	Megatron-LM train step	training path (= open PR #4632)	loss 26.41→24.61 · grad_norm finite · EFA TX 1.096 GB
6	NeMo-RL rollout	RL rollout (transitive via Megatron)	rollout PASS 9.45 s world=16 · loss 26.41→24.59

Every serve is backend-asserted in-log: ElasticBuffer ctor ×16 + "Selected provider is efa, fabric is efa-direct" ×16 (pods expose no hw_counters sysfs; provider-line substitute recorded). llm-d has no gate by design — it is a router, not a DeepEP consumer (its EFA story is NIXL KV-transfer); its routing lane is validated separately in the repo.

NEW measured result: EP_NUM_COMMS=8 doubles vLLM on the public stack

2.02×

agg tok/s at conc4 (19.5 → 39.4)

1.89×

agg at conc32 (144.6 → 273.0)

½ ITL

203 → 100 ms/token p50

0 err

232/232 requests, 3 cells

vLLM 0.24.0 EP16, identical serve except EP_NUM_COMMS · measured 2026-07-07
arm	c4	c8	c32	per-user c4	TTFT p50 c4	ITL p50
EPNC=1 (pure public)	19.5	38.8	144.6	4.9	475 ms	~203 ms
EPNC=8 (multicomm patch)	39.4	76.5	273.0	10.0	196 ms	~100 ms

Why it works: the CPU-proxy path is issue-bound (§2's proxy tax — the NIC sits ~98% idle). Striping the all-to-all over K independent NCCL comms (EP_NUM_COMMS=8) multiplies the in-flight issue streams, attacking the actual bottleneck instead of the wire. This is the first lever in this guide that converts to serving throughput on EFA — it doubles vLLM but does not change the §6 verdict: 273 tok/s still sits near SGLang's 255 and below TRT's dense arm; the ceiling is still the proxy, now at half the tax. The lever is one committed patch (deepep-multicomm-overlay.patch) + a _C rebuild + one env var — inert when EP_NUM_COMMS is unset/1.

The cross-framework serving table (same workload, each framework's proven config)

agg output tok/s · AIPerf ISL120/OSL128 · all cells 0 errors · not a single-variable A/B (engines differ)
arm	c4	c8	c32	per-user	ITL p50
vLLM EPNC=1	19.5	38.8	144.6	~4.9	203 ms
vLLM EPNC=8	39.4	76.5	273.0	~9.9	100 ms
SGLang 0.5.11	34.5	67.2	255.2	~8.6	117 ms
TRT-LLM shim	66.5	147.5	586.7	~19.3	52 ms
TRT-LLM native seam	67.6	142.4	563.3	~18.6	53 ms

TRT-LLM remains the fastest engine on this workload AND still loses to its own dense arm (~106 agg at c4) — §6's verdict stands. Native seam ≡ shim within run noise (−4% at c32): the clean integration costs nothing.

What changed since the June campaign (§4–§6 context)

claim as of June 2026	status July 2026
vLLM DeepEP-V2 needs a fork (`tlrmchlsmth/vllm`)	CLOSED — PyPI `vllm==0.24.0` carries the manager natively (PR #41183 merged)
TRT-LLM serves DeepEP only via a private 1,500-line sitecustomize shim	CLOSED — one-file public seam patch on `deep_ep_utils.py`; shim retired (kept as legacy); upstream PR drafted
multicomm 2.09× measured only on the fork build	REPRODUCED on the pure public stack: 1.89–2.02×
Megatron/NeMo-RL patches private	PUBLIC — Megatron seam = open PR #4632 (head fetchable via `pull/4632/head`); NeMo-RL env fix MERGED (#2585)
DeepEP QP-cap trio (PR #612)	still OPEN upstream; committed as patches, apply-clean on `b306af06`
SGLang ElasticBuffer support (PR #24443)	still OPEN upstream; seam patch committed, apply-clean on the 0.5.11 wheel
serving verdict: DeepEP ≤ dense on EFA	UNCHANGED — capacity/fit + prefill regimes remain the use case; EPNC=8 halves the gap but does not flip it

The gotchas that cost the most hours (so you don't pay them)

Serving

"Gin barrier timeout tag:N" is a symptom — one rank died elsewhere (import error, JIT); find the first dead worker, don't debug the transport.
Any pip install can silently replace nvidia-nccl-cu13/nvshmem-cu13 → deep_ep._C undefined symbols. Re-pin with --force-reinstall --no-deps after EVERY install.
gdrdrv kernel 2.4 + libgdrapi 2.5 skew: transport bench passes, serving combine faults CUDA 719 — build the plugin from aws/aws-ofi-nccl@9c44d34 + the two committed scripts.
ElasticBuffer serve timeouts ≥600 s (frameworks pass none; the 100 s default loses to first-dispatch JIT skew across 16 ranks).

Benching

Run test_ep via torchrun, not mp.spawn (spawn overflows NCCL's 256-node XML cap on p5en) — and MASTER_PORT must equal the rdzv port.
num_max_tokens_per_rank must be uniform across ranks — it's JIT-substituted into the kernel template; divergent values deadlock the barrier.
Reap by explicit PID from nvidia-smi, fresh dist port per attempt — leftover GPU contexts corrupt the next measurement (a 22 h-old pod measured 4× slower than a fresh one).
A number without the in-log backend assertion (ElasticBuffer + efa provider) is not evidence.

July 2026 public-stack re-run. Hardware: 2× p5en.48xlarge, H200, 16 EFA NICs/node, gdrdrv kernel 2.4. Transport: deep_ep.ElasticBuffer over NCCL-GIN type-2 CPU-proxy (never GDAKI/IBGDA — EFA-dead). Every number traces to a committed AIPerf JSON or serve log in antonai-work/deepep-v2-efa-repro (RESULTS.md is the one-page index). Sanitized — no internal IPs, registry IDs, or hostnames.