§8 · DeepEP V1 — the NVSHMEM backend on EFA measured

The rest of this guide covers DeepEP-V2 (the NCCL-GIN / ElasticBuffer path). This section covers the other DeepEP backend — V1, the legacy NVSHMEM backend (deep_ep.Buffer, low_latency_dispatch), built on source-compiled NVSHMEM linked against EFA's libfabric. It is what vLLM drives via --all2all-backend deepep_high_throughput / deepep_low_latency. We ran a fresh head-to-head on p5en / H200 / 16 EFA NICs: our NVSHMEM 3.6.5 fork vs the AWS 3.7.0 build.

V1 vs V2 — two different DeepEP backends

Both are CPU-proxy paths on EFA. Neither uses GPU-initiated RDMA — EFA exposes no IBGDA. The "proxy tax" §2 describes applies to V1 too.

The V1 campaign scoreboard — four frameworks, one transport

All four capstones are complete. The same NVSHMEM/EFA transport validated across serving (vLLM + SGLang), training (Megatron), and full RL (NeMo-RL GRPO):

The headline: ours vs AWS on H200 (same nodes, same DeepEP tuner)

Why AWS V1 is ~2× faster (same architecture, NOT IBGDA)

Decode / low-latency on H200 — the handle-cache bug + the fix

vLLM's deepep_low_latency backend calls Buffer.low_latency_dispatch/combine. On our NVSHMEM 3.6.5 build this segfaulted on H200 (signal 11 @ 0x28). Root cause (verified at source + runtime, not an SRD-ordering or NIC-count issue — reproduced at NIC_PER_PE=1):

The real fix is NVSHMEM 3.7.0: its changelog explicitly closes this — "Fixed a bug in remote transports during memory registration … memory handle management cache" + a NVSHMEM_CUMEM_GRANULARITY static-heap override. The AWS V1 build already runs full LL D+C on H200 (~0.82 ms) because its patch deletes the device-side rkey cache, so all handle resolution happens CPU-side in 3.7.0's fixed transport — the device never executes the buggy offset >> log2_granularity.

The capstone: does V1 actually serve faster? (vLLM EP16, H200, measured)

The microbench above says AWS V1 moves dispatch bytes ~2× faster. The real question is whether that converts to serving throughput. We built a single NGC-from-scratch image (NVSHMEM 3.7.0 + DeepEP 567632d + the EFA shim + vLLM from source) and served Qwen3-30B-A3B-FP8 with Expert Parallelism (EP16, DP16×TP1) across 2× p5en/H200, driving both DeepEP-V1 backends. Repro = one Dockerfile + one serve.yaml + one run_test.sh: github.com/dmvevents/deepep-efa-ab-h200/tree/master/capstone.

REVISED: the eager confound, resolved (2026-07-08) measured

The original matrix above ran both backends --enforce-eager because NGC triton lacks triton_key. We retested with -O.level=0 -O.cudagraph_mode=FULL (level-0 compilation never touches inductor, so no triton_key; CUDA graphs capture fully). LL improves 21-38% TPOT; HT is unchanged (within 1.4% — it was never eager-limited).

Evidence: dmvevents/deepep-efa-ab-h200 commit 55c322a, capstone/results/{cudagraph-retest,dense-ab}/. Retest 2026-07-08. Dense arm run under eager (no cudagraph, matching the V2 campaign's dense-arm methodology).

Capstone-2: the same stack under SGLang — and the verdict INVERTS measured

Same image lineage, same NVSHMEM 3.7.0 + EFA shim, same model and nodes — but served by SGLang 0.5.5.post3 (--moe-a2a-backend deepep, TP16 + DP-attention 16). We swept all three --deepep-mode values across the same 4 regimes. On vLLM, "low-latency" was 29–75% worse. On SGLang, low_latency wins every regime:

auto mode == normal within noise (auto routes prefill chunks through the normal path). Raw JSONL + logs: capstone/sglang/results/ in the repro repo.

Capstone-3: training (Megatron-LM) measured

Does V1 train? The vLLM/SGLang capstones validated serving; this validates training. Stock Megatron-LM @23dd639c is a native DeepEP-V1 consumer: megatron/core/transformer/moe/fused_a2a.py imports deep_ep.Buffer directly with zero patches (the V2 campaign's Megatron PR #4632 exists to steer this file away from V1 — so for V1, upstream-as-is is the recipe). We ran the V2 lane's Gate-5 "Shape Y" driver (Qwen3-30B-A3B-style MoE: 128 experts, top-8, hidden 2048) on 2× p5en/H200 with the same seeds and base commit → direct A/B of Buffer/NVSHMEM-proxy vs ElasticBuffer/NCCL-GIN as a training transport.

Losses agree to ~3 decimals (bf16 reduction-order noise); grad-norms match digit-for-digit. NVSHMEM v3.7.0 banner, NVSHMEM_REMOTE_TRANSPORT=libfabric, provider efa. Evidence: github.com/dmvevents/deepep-efa-ab-h200/tree/master/capstone/megatron.

Capstone-4: the full RL loop (NeMo-RL GRPO) measured

Does V1 train with reinforcement learning? The vLLM/SGLang capstones validated inference serving; Capstone-3 validated supervised training. This validates the complete GRPO loop (rollout generation → reward → training → weight refit) — the stack V2's Gate-6 explicitly skipped as "too heavyweight". Stock NeMo-RL @46be4e8 drove 3 complete GRPO steps on Qwen3-30B-A3B across 2× p5en/H200, using vLLM colocated rollout generation (32 MATH completions per step, mean ~927 tokens), reward calculation, GRPO policy training through stock Megatron fused_a2a.py (deep_ep.Buffer → NVSHMEM 3.7.0 → EFA), and ZMQ weight refit back into vLLM (3.4 s per step).

Timing split (step 3): generation 62%, policy training 8%, refit 4%. Rollouts are real — 32 completions/step, logged with rewards/advantages/logprobs. Source: github.com/dmvevents/deepep-efa-ab-h200/tree/master/capstone/nemo-rl (commit bc930e7).

How vLLM consumes V1 (the inference mapping)

Verified in vLLM distributed/device_communicators/all2all.py: DeepEPHTAll2AllManager → deep_ep.Buffer(...); DeepEPLLAll2AllManager → Buffer.get_low_latency_rdma_size_hint + low_latency_dispatch.

The verdict for V1