DeepEP-on-EFA Calculator measured-calibrated

Should you run a Mixture-of-Experts model with DeepEP expert-parallel all-to-all, or a dense / allgather alternative, on AWS EFA (p5en, CPU-proxy NCCL-GIN)? This tool works from the measured behaviour of DeepEP-V2 on EFA — not vendor spec sheets — and tells you, for your model + workload, whether the expert-parallel all-to-all helps or hurts your serving throughput. Every output carries a measured / derived / assumed provenance tag.

§1 · Model + expert config

§2 · Topology + workload + targets

§3 · The three verdicts

■
Use DENSE / allgather

EP all-to-all is not the bottleneck at your scale. On EFA's CPU-proxy path the per-decode-step proxy round-trip costs more than the expert-volume it saves. Default for EP≤16 serving.

◆
Use DeepEP for CAPACITY

Model doesn't fit / KV-cache starved without sharding experts. Use DeepEP (or any EP) to fit — accept the all-to-all toll because the alternative is OOM, not a slower-but-working serve.

▲
DeepEP wins on throughput

Only when GPU-initiated transport (GDAKI/IBGDA) removes the per-step proxy tax — not viable on EFA today (SRD has no atomics/ordering; the kernel-posted-WQE dispatch path is upstream-gated).

§4 · What this tool is — and is not

IS
  • Calibrated to a measured fair A/B: DeepEP-V2 (EP_NUM_COMMS=8 multi-comm) vs dense allgather, same model/build/nodes, EP16, 2× p5en EFA, AIPerf 0.10.0.
  • A decision aid for EFA / CPU-proxy NCCL-GIN (NCCL_GIN_TYPE=2, ElasticBuffer).
  • Honest about the fixed per-decode-step proxy tax that load-invariantly handicaps DeepEP here.
IS NOT
  • A claim about InfiniBand / NVLink-only / MNNVL clusters — there DeepEP + IBGDA can win; this is EFA.
  • A correctness statement — both arms produce identical coherent output; this is about throughput/$.
  • A ≥2-rep statistical study — headline cells are n=1; cross-regime ratio tightness (σ small) is the evidence. See provenance tags.

§5 · Why — the mechanism (measured)

The CPU-proxy posts each cross-node expert dispatch via the host (D2H ring → CPU proxy → libfabric fi_writedata → EFA SRD). At ~2.2% NIC utilization the path is issue-bound (per-op post rate), not wire-bound — so reducing all-to-all bytes (DeepEP's O(top-k) advantage) doesn't reduce the per-step latency that gates decode tok/s. GPU-initiated transport (GDAKI) would post the WQE from the kernel and remove the host round-trip — that is the only path measured to change the verdict, and it is not available on EFA SRD today.