{"id":"4437ab6d-c8a3-4dab-81d6-f1a8d5aa07b4","arxiv_id":"2607.05116","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"low","formal_verification":"none","parameter_count":3,"one_line_summary":"Communication-aware expert placement plus device-level pruning yields 1.23–1.86× MoE inference throughput and better accuracy at equal speedup than load-balance or sequential baselines.","lead":"CAP places and prunes Mixture-of-Experts so tokens visit fewer GPUs and nodes, cutting communication while keeping accuracy. It can raise inference throughput 1.23–1.86× on real multi-GPU and multi-node hardware versus common baselines.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"No significant objection identified","rationale":"The paper’s strongest claim is an empirical systems result: CAP’s joint placement+pruning pipeline yields 1.23–1.86\times higher SLO-compliant throughput than EPLB and sequential placement, and better accuracy at equal speedup under lossy pruning. That claim rests on (1) co-activation structure being exploitable, (2) a spectrum of placements that trade communication for balance, and (3) device-level pruning that removes expensive destinations. All three are measured across three single-node regimes and two multi-node bandwidths, two models, and multiple p values. The stability of the offline grouping is the softest modeling premise, yet Section VIII-F already quantifies it and shows the resulting placements remain superior to Default. Because the paper itself supplies the sensitivity data the reader flags, and because no contradictory control or arithmetic gap appears, the CONDITIONAL verdict (accept once artifacts/sensitivity are tightened) needs no further downward adjustment. The concrete test above would still be worth running for completeness, but a negative result is not expected from the existing tables.","tokens_in":16951,"tokens_out":516,"duration_ms":5270,"concrete_test":"Re-run the end-to-end SLO-attainment sweep of Fig. 11 on Node A and Cluster B using a grouping profiled solely on HumanEval (or Arxiv) rather than LMSYS; if the reported 1.23–1.86\times throughput multipliers fall below 1.1\times while accuracy under the same p remains comparable, the stability premise would become load-bearing. Otherwise the claim stands.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The reader's weakest assumption (offline co-activation stability) is real but secondary. The paper already supplies direct evidence for it: in-workload variation <2.9% communication / <5.8% imbalance across 16 LMSYS windows (Fig. 15) and cross-domain Device-per-Token / Max/Avg load that remain better than Default (Tables III–IV). The central throughput claim (1.23–1.86\times over EPLB and sequential vLLM placement) is independently supported by hardware-regime ablations (Fig. 11–13), the placement-spectrum Pareto argument, and the pruning accuracy-vs-speedup comparison (Fig. 14 + Table I). No internal inconsistency or missing control that would overturn those numbers is apparent from the manuscript.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"The paper presents CAP, a three-component framework for distributed MoE inference that jointly optimizes expert placement and dynamic pruning for communication volume, load balance, and accuracy. Co-activation-driven placement (Algorithm 1, Eqs. 2–3) groups frequently co-activated experts to reduce device/node visits per token; a subsequent adjustment procedure generates a spectrum of placements trading communication against load imbalance (Eqs. 4–5); and device-level communication-aware pruning (Eq. 6) removes low-contribution routing destinations. A four-step pipeline (Section VII) first collects pruned routing traces, then places, adjusts, and profiles. On Qwen3-30B-A3B and DeepSeek-V2-Lite across three single-node regimes (comm.-constrained 3090s, balanced A100s, comm.-rich H100s) and two multi-node clusters, CAP reports 1.23×–1.86× throughput gains over sequential (vLLM-style) placement and DeepSeek EPLB while preserving higher accuracy at matched speedup under pruning.","tokens_in":17174,"tokens_out":1018,"duration_ms":8589,"significance":"If the reported gains hold under broader deployment, the work is a useful systems contribution for modern large-expert-count MoEs, where communication—not only load balance—has become first-order. The explicit communication–load spectrum and hardware-aware selection are more complete than pure load-balance placers, and the device-level pruning formulation correctly targets routing destinations rather than individual experts. Strengths include clear algorithmic statements (Alg. 1, Eqs. 2–6), multi-regime evaluation that isolates the spectrum (Figs. 12–13), pruning accuracy–speedup comparison (Fig. 14 + Table I), and direct stability/transferability measurements (Fig. 15, Tables III–IV). The offline overhead claim (≤6 min) is practical. These elements make the central throughput and accuracy claims falsifiable and useful to practitioners.","major_comments":[{"comment":"Section VIII-B / Fig. 11: the headline 1.23×–1.86× throughput numbers are reported without variance, confidence intervals, or repeated runs under the same RPS/SLO regime. Because the claim is the paper’s primary result and the gains vary sharply by hardware class, at least summary statistics (or error bars) over multiple request-stream seeds are needed to establish that the ranking over Default and EPLB is stable.","section":null},{"comment":"Section VIII-F and Tables III–IV: the weakest load-bearing assumption is that a single offline co-activation grouping remains adequate. The paper supplies supportive evidence (in-workload variation <2.9 % communication / <5.8 % imbalance; cross-domain Device-per-Token still better than Default), yet the evaluation uses only three source windows and three datasets. A short sensitivity experiment that forces re-profiling after a domain shift (or quantifies degradation when the grouping is stale) would strengthen the claim that frequent regrouping is unnecessary.","section":null},{"comment":"Section V / Eq. (5): λ is introduced as the continuous trade-off weight, but the implemented Adjustment Approach generates a discrete spectrum by iterative swaps and then selects by profiling. The manuscript never states how many swaps, which experts are eligible, or how the spectrum is guaranteed to approximate the Pareto front of Eq. (5). Clarifying the algorithmic correspondence (or acknowledging that the spectrum is a heuristic proxy) is required for reproducibility of the “optimal placement” step.","section":null}],"minor_comments":[{"comment":"Figure 6 and the surrounding text use “Comm. & Comp.” and “comm.-comp.” inconsistently; expand once and keep a single abbreviation.","section":null},{"comment":"Eq. (3) defines the inter-node cost c, yet the concrete values used for Node A/B/C and Clusters A/B are only partially stated later; a short table of c and bandwidth ratios would help.","section":null},{"comment":"Table I reports accuracy only for Qwen3 under three p values; a parallel row or column for DeepSeek-V2-Lite would make the “preserves better accuracy” claim fully symmetric with the throughput plots.","section":null},{"comment":"Algorithm 1 line 12–13 falls back to uniform random selection when score = 0; a one-sentence note on how often this occurs in practice would be useful.","section":null},{"comment":"Typos / style: “Communicationcomputation” in the abstract, “F .” spacing before “Stability”, and occasional missing spaces around × symbols.","section":null}],"recommendation":"minor_revision","confidential_remarks":"The work is solid systems engineering rather than a deep theoretical advance; the contribution is real but incremental relative to the densest systems venues. Fit for a solid journal or conference track is good once the variance and spectrum-algorithm clarifications are added. No integrity or novelty-disclosure concerns."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The useful takeaway is simple: once MoEs have dozens of experts per GPU, placement and pruning stop being pure load-balance problems and start controlling all-to-all volume. CAP is the first clean pipeline I have seen that treats that fact end-to-end.\n\nWhat is new is the three-stage sequence. They build a co-activation graph, seed a communication-oriented partition (Alg. 1), then walk a spectrum of expert swaps that trade communication for balance, and finally prune at device granularity with an explicit cost term (Eq. 6). The spectrum idea is the part that actually lands: on the 3090 node the communication-minimal placement wins, on the balanced A100 a mid-spectrum point wins, and on NVLink H100 the load-balanced end wins. That matches the hardware story and is better than either pure EPLB or sequential vLLM placement. Throughput gains of 1.23–1.86× and the accuracy-vs-speedup comparison (same 33 % latency cut at a milder pruning threshold) are consistent with the claimed mechanisms.\n\nSoft spots are ordinary for this venue. The partitioner is greedy; stronger graph-partition baselines would have been nice but are not load-bearing. Pruning threshold p and inter-node cost c are free parameters, though they are exposed as user knobs rather than hidden. Stability of the offline co-activation stats is the weakest modeling assumption, yet the paper already shows <3 % communication variation across LMSYS windows and decent cross-domain transfer, so it is secondary rather than fatal. No code release is the practical annoyance.\n\nThis is for people who ship multi-GPU MoE inference. The math is elementary, the evaluation is multi-regime and multi-model, and the citation pattern is fair. I would send it to peer review; a referee can ask for the missing variance bars and a code artifact without the central claim collapsing. Worth reading if you care about MoE serving.","headline":"Solid systems paper that makes communication volume a first-class objective for large-expert MoE inference and backs it with clear hardware-regime results.","tokens_in":17762,"tokens_out":483,"would_cite":true,"duration_ms":4918,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"CAP places and prunes MoE experts by communication cost, cutting distributed inference latency 1.23–1.86× while keeping accuracy higher at the same speedup.","keywords":["Mixture-of-Experts","expert parallelism","expert placement","expert pruning","communication-aware scheduling","distributed LLM inference","load balance"],"falsifier":"Measure device-per-token and max/avg load on a held-out traffic mix (different domain or much longer session) after applying a grouping built on only 8 000 tokens; if either metric drifts enough that the profiled spectrum no longer contains the true latency-optimal placement, the stability claim fails.","tokens_in":17855,"feed_emoji":"⚡","tokens_out":663,"duration_ms":6478,"temperature":0.7,"pith_summary":"Mixture-of-experts models now pack dozens or hundreds of experts per layer, so where those experts sit and which ones a token actually visits decide how much data must cross GPUs and nodes. CAP treats that communication volume as a first-class design variable alongside compute balance and accuracy. It first groups experts that tend to fire together onto the same devices and nodes, then walks a spectrum of placements that trade a little more communication for better load balance, and finally prunes whole devices rather than individual experts when the accuracy budget allows. A short offline profile picks the right point on that spectrum for each machine. The result is higher throughput on both bandwidth-poor and bandwidth-rich hardware, and better accuracy than ordinary pruning when the same latency target must be met.","feed_headline":"MoE experts placed by co-activation cut inference traffic 1.23–1.86×","feed_subtitle":"CAP groups, balances and prunes routing destinations so communication no longer dominates multi-GPU MoE latency.","key_machinery":"The three-stage CAP pipeline: (1) co-activation graph partitioning that seeds high-degree experts and greedily grows balanced device and node groups, (2) iterative expert swaps that generate a Pareto spectrum of placements trading C(π) against load imbalance B(π), and (3) device-level pruning that retains destinations ordered by accuracy-contribution over communication-cost until a user threshold 1−p is met.","core_discovery":"In modern large-expert-count MoE models the volume of all-to-all traffic is governed by the number of distinct devices a token must visit, not merely by how many experts it activates. CAP shows that co-activation-aware placement, a tunable communication–load spectrum, and device-level communication-aware pruning together let a single offline pipeline select an operating point that simultaneously reduces communication, balances work, and preserves accuracy better than load-balance-only or sequential baselines.","pith_inferences":[],"forward_implications":[],"fun_headline_variants":["Co-activation placement cuts MoE all-to-all traffic 1.23-1.86×","CAP groups co-activated experts to slash multi-GPU MoE hops","Communication-aware pruning trims MoE routing for higher throughput","Expert co-activation and pruning yield 1.23-1.86× MoE inference gain","CAP balances MoE load and communication across GPUs and nodes"],"cache_read_input_tokens":128,"weakest_assumption_plain":"A single offline co-activation snapshot taken from a few thousand tokens stays stable enough, both inside one workload and across different domains, that the same expert grouping can be reused without frequent re-profiling.","fun_headline_variants_meta":{"raw":{"variants":["Co-activation placement cuts MoE all-to-all traffic 1.23-1.86×","CAP groups co-activated experts to slash multi-GPU MoE hops","Communication-aware pruning trims MoE routing for higher throughput","Expert co-activation and pruning yield 1.23-1.86× MoE inference gain","CAP balances MoE load and communication across GPUs and nodes"]},"model":"grok-4.5","effort":"low","cost_usd":0.006382,"raw_usage":{"total_tokens":1609,"prompt_tokens":770,"num_sources_used":0,"completion_tokens":108,"cost_in_usd_ticks":63820000,"prompt_tokens_details":{"text_tokens":770,"audio_tokens":0,"image_tokens":0,"cached_tokens":128},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":731,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":770,"tokens_out":108,"duration_ms":5917,"temperature":1.0,"reasoning_tokens":731,"cache_read_input_tokens":128,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-11T08:34:47.630411+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Measure device-per-token and max/avg load on a held-out traffic mix (different domain or much longer session) after applying a grouping built on only 8 000 tokens; if either metric drifts enough that the profiled spectrum no longer contains the true latency-optimal placement, the stability claim fails.","supporting_citations":[],"review_version":1}