{"id":"d0c6f187-c55c-46e8-b638-29b64e59895e","arxiv_id":"2411.10003","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Pro-Prophet balances MoE training load by moving each expert to only the devices that need it and by pre-scheduling those moves to hide communication behind computation.","lead":"This paper describes a load balancer for training mixture-of-experts (MoE) models on many GPUs, combining a planner that repositions experts and a scheduler that overlaps communication with computation. It reports up to 2.66x faster training than DeepSpeed-MoE and up to 1.48x faster than FasterMoE on small clusters and models.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim depends on inter-iteration locality, but the only evidence is qualitative and from late training (Fig. 4), while the speedups are measured in the first 100 iterations; this mismatch is the load-bearing weakness.","rationale":"After reading in good faith, I find the paper's mechanism coherent when the locality assumption holds: the planner's greedy search and the scheduler's block-wise overlap both reduce exposed communication, and the ablation study gives internal support. I do not see an internal inconsistency in the performance model or the scheduling order that would invalidate the construction. The most vulnerable point is exactly the bridge between the profiling observation and the scheduler: all precomputation is justified by inter-iteration locality, yet the quantitative evidence for that locality is a qualitative figure from late training, and the evaluation is limited to the first 100 iterations. These two facts create a mismatch: the speedup numbers are not backed by a measurement of the assumption in the relevant window. The reader's weakest_assumption identifies the same issue, and the proposed change, run longer, inject a distribution shift, and quantify locality, would settle whether the central claim generalizes. I therefore keep the conditional verdict rather than accepting the headline numbers at face value.","tokens_in":20248,"tokens_out":9119,"duration_ms":98481,"concrete_test":"Reproduce the 8-HPNV-node MoE-GPT-M k=1 configuration and run at least 1000 iterations. At iteration 500, apply an abrupt input-distribution shift (e.g., reorder the training corpus or switch the routing input modality) and record per-iteration time for Pro-Prophet, Deepspeed-MoE, and FasterMoE for 100 iterations before and after the shift. If Pro-Prophet's speedup over FasterMoE collapses or its per-iteration time spikes after the shift while FasterMoE's does not, the locality assumption is not load-bearing in the claimed regime. As a complementary check, compute the Jensen-Shannon divergence between consecutive input distributions during the first 100 iterations of the original setup; if the divergence is not small relative to late-training values, Fig. 4's evidence does not transfer to the evaluation window.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The argument's load-bearing premise is the locality property introduced in Section II-B and exploited in Section V-A: the input distribution of iteration j+1 is approximated by that of iteration j, so the Plan for block i of iteration j+1 is precomputed during the A2A of block i in iteration j, and Trans of expert parameters is launched early. Every claimed gain over Deepspeed-MoE and FasterMoE flows through this prediction; without it, the precomputed lightweight placement is stale, the early Trans moves the wrong parameters, and there is no described fallback mechanism in the paper to recover within the iteration. The support for the premise is Fig. 4, a qualitative plot drawn from iterations 2500-2600, while the reported end-to-end speedups are measured in the first 100 iterations (Section VI, default settings), a regime the paper itself describes as the distribution stabilizing. Thus the locality that justifies the scheduler is not actually demonstrated in the measurement window where the headline speedups are claimed, and no experiment injects a distribution shift or trains long enough to stress the assumption.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Pro-Prophet, a system-level load-balancing method for training large-scale Mixture-of-Experts (MoE) models. It consists of a planner, which searches over 'lightweight expert placements' using a cost model and a greedy algorithm, and a scheduler, which exploits the observed locality of input distributions across adjacent iterations to precompute placement plans and to overlap parameter/gradient communication with expert and non-expert computation. The authors evaluate Pro-Prophet on four clusters of up to 32 GPUs with five MoE-GPT model variants and report end-to-end speedups of up to 2.66x over Deepspeed-MoE and up to 1.50x over FasterMoE, together with a load-balancing improvement of up to 11.01x over FasterMoE.","tokens_in":20494,"tokens_out":6902,"duration_ms":67604,"significance":"If the reported speedups are robust, Pro-Prophet would be a useful systems contribution: it addresses a real bottleneck in MoE training and combines two mechanisms (placement search and communication-computation overlap) that are usually treated separately. The paper gives a concrete performance model, a greedy planner, a block-wise scheduling strategy, and an ablation study separating the contribution of the planner and the scheduler. The evaluation covers multiple clusters, models, and top-k settings, which is a reasonable breadth for a systems paper. However, the central claim depends on an inter-iteration locality assumption whose empirical support is currently narrow, and the measurement methodology lacks variance information and a released artifact. These issues are load-bearing because the scheduler's precomputation and early parameter transfer are only beneficial if the next iteration's routing distribution is predictable.","major_comments":[{"comment":"The load-bearing assumption of the scheduler is the locality property: the input distribution of iteration j+1 is estimated from iteration j, so Plan_{i}^{j+1} is launched during A2A_i^j and Trans is started early (Algorithm 2, line 5). However, the only evidence for this property is Fig. 4, which shows a qualitative plot for iterations 2500-2600 of the second MoE layer, while the default evaluation is explicitly restricted to the first 100 iterations. The paper does not quantify how similar adjacent-iteration distributions are in the first 100 iterations, nor does it test a scenario with an abrupt distribution shift (e.g., changed data order, checkpoint restart, or altered batch composition). Because there is no described fallback if the predicted Plan or early Trans is stale, the reported end-to-end speedups may be confined to a regime where the locality assumption happens to hold. Please add locality measurements within the evaluation window itself and a stress test with a distribution shift, or describe a recovery mechanism.","section":"Section V-A and Section VI (Default settings)"},{"comment":"All speedup results are reported as single numbers without error bars, standard deviations, or the number of repeated runs. The headline ranges (e.g., 1.47-2.66x vs. Deepspeed-MoE, 1.01-1.48x vs. FasterMoE) are therefore not yet distinguished from run-to-run noise. In addition, Fig. 12 shows per-iteration times only for iterations 62-70, even though the evaluation is said to cover the first 100 iterations; a complete time curve or an average over the full window would be more convincing. I ask for at least three repeated runs per configuration and reporting of the mean and spread, together with the average throughput over the full 100-iteration window.","section":"Section VI (Figs. 10-12, Tables IV-V)"},{"comment":"The performance model uses constants B (average communication bandwidth) and t (computation throughput) in Eqs. (1)-(5), and the scheduler's split of Trans/Agg in Algorithm 2 relies on pre-training estimates of FNEC and BNEC duration. The paper does not state how these constants are obtained, whether they are re-measured for each cluster, or how sensitive the planner's chosen placement is to their values. Since the planner's greedy search uses this model to select the placement, and the scheduler's split depends on static estimates, the portability of the method to a new cluster is not yet demonstrated. Please report the calibration procedure and, if possible, a sensitivity analysis over B and t.","section":"Section IV-B and Section V-C"}],"minor_comments":[{"comment":"The abstract and introduction state that Pro-Prophet achieves 'up to 2.66x speedup compared to Deepspeed-MoE and FasterMoE,' but the measured speedups against FasterMoE are at most about 1.50x. Please rephrase to avoid implying that 2.66x was obtained against both baselines.","section":"Abstract and Introduction"},{"comment":"The sentence 'We evaluate Pro-Prophet within the first 100 iterations as the input distribution tends to stabilize with the training process' is ambiguous: it could mean the first 100 iterations are chosen because the distribution is stabilizing, or that the distribution only stabilizes later. Please clarify the intended meaning, since it bears directly on the locality assumption.","section":"Section VI (Default settings)"},{"comment":"The text introducing Fig. 12 says 'We also evaluate the single-layer performance of Pro-Prophet' but the experiment measures per-iteration time; this should read 'single-iteration performance.'","section":"Section VI-B (Single-iteration speedups)"},{"comment":"The denominator in T_Trans(s,n) and T_Agg(s,n) is D*B, but only D-n devices participate in the communication. If the intended model is aggregate bandwidth across D devices, please justify this; otherwise the denominator should presumably involve D-n.","section":"Eqs. (4)-(5)"},{"comment":"The pseudocode uses the variable i both for the index of the heaviest device (line 6) and for 'expert-i' in the comment on line 11; since a lightweight placement maps experts to subsets of devices, please disambiguate the notation so the mapping between devices and experts is clear.","section":"Algorithm 1 and Section IV-C"},{"comment":"No code or artifact release is mentioned. For a systems paper whose claims rest on careful implementation details (e.g., the split of Trans into sub-primitives), releasing the implementation or at least a detailed reproducibility appendix would substantially strengthen the paper.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The paper is a plausible systems contribution, but the reviewer's main concern is that the locality assumption is not validated in the exact evaluation window where the speedups are claimed. I would ask the editor to require the locality stress test and a small number of repeated runs as part of the revision. The absence of a code release is also a concern for a systems paper of this type, though it is not by itself disqualifying."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Pro-Prophet is a real engineering contribution, not a repackaging. The lightweight expert placement—each expert mapped to a subset of devices, with Trans/Agg only among those devices—is a concrete step beyond FasterMoE's coarse-grained shadowing. The block-wise scheduler that precomputes next iteration's Plan during the current A2A and splits Trans/Agg across neighboring blocks is also new in this combination. The planner's greedy search over placements, guided by a simple performance model, is sensible, and the reported <5% mean estimation error on that model is credible evidence that the model captures the right variables. The experiments span four clusters and five models, and the speedups over Deepspeed-MoE and FasterMoE are consistent, which suggests the effect is real rather than a single-configuration accident.\n\nThe soft spots are real but mostly fixable. First, the abstract overstates: the 2.66x figure is against Deepspeed-MoE only; against FasterMoE the speedups are 1.01-1.48x, which is modest. Second, the evaluation is confined to the first 100 iterations with no variance statistics, and no code is released. Third, and most important, the scheduler's load-bearing premise is inter-iteration locality of the input distribution. The only support for that premise is Fig. 4, which shows iterations 2500-2600, while the speedups are measured in the first 100 iterations. The stress-test note is right to flag this mismatch. It is not fatal, because Pro-Prophet still wins in the early window where locality should be weakest, but the paper never directly demonstrates that the prediction stays accurate there, and there is no fallback if the distribution shifts sharply. A distribution-shift experiment or a longer training run would close this gap. Finally, Lina, PipeMoE, and ScheMoE are mentioned in related work but not benchmarked; they are closer baselines than Deepspeed-MoE for the scheduling component.\n\nWho should read this: systems people working on MoE training infrastructure. It deserves a serious referee, but the review should request code, longer runs, a distribution-shift test, and honest abstract wording. The central idea is sound and worth building on.","headline":"A solid MoE load-balancing system with a genuinely new placement-plus-overlap combination, whose abstract overstates and whose locality assumption needs stress-testing.","tokens_in":21032,"tokens_out":2394,"would_cite":true,"duration_ms":28621,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper argues that by predicting one iteration ahead, MoE load balancing can be reduced to lightweight expert replication and hidden communication, achieving up to 2.66x end-to-end speedup.","keywords":["Mixture-of-Experts","expert parallelism","dynamic load imbalance","lightweight expert placement","communication-computation overlap","block-wise scheduling","distributed training","MoE model training"],"falsifier":"Feed a MoE model inputs whose routing is deliberately perturbed every few iterations so that adjacent-iteration expert-load vectors have high divergence, and measure Pro-Prophet's per-iteration time; if it degrades toward the no-overlap baseline and planning or prefetch corrections dominate, the locality assumption is falsified. Quantitatively, compute the Jensen-Shannon divergence between consecutive iterations' expert-load distributions; Pro-Prophet's speedup should decrease monotonically as this divergence increases.","tokens_in":20084,"feed_emoji":"⚡","tokens_out":9595,"duration_ms":93649,"temperature":0.7,"pith_summary":"This paper tries to establish that dynamic load imbalance in large-scale Mixture-of-Experts training can be fixed at the system level without paying the usual costs of moving whole model states or disrupting model convergence. It relies on an observed locality: the distribution of tokens among experts in one iteration closely resembles the next iteration's distribution. On that basis, a planner finds a lightweight expert placement, replicating only the busy experts' parameters to the devices that need them, and a scheduler prefetches those parameters and hides balancing communication behind ordinary forward and backward computation. If correct, this would let MoE models scale to more experts without throughput collapsing from skewed routing; the reported experiments claim end-to-end speedups of up to 2.66x against established MoE training systems and up to 11.01x better load balance.","feed_headline":"Mixture-of-Experts training speedup up to 2.66x by predicting loads","feed_subtitle":"Pro-Prophet pre-places experts and overlaps communication with compute, hiding load imbalance in MoE training.","key_machinery":"The central object is the lightweight expert placement: each expert is mapped independently to one or more devices, and only its parameters (forward Trans) and gradients (backward Agg) move among those devices, rather than whole model states. The planner's performance model estimates a MoE layer's time as $T' = 4T_{A2A}(R)+3T_{FEC}(H)+T_{Trans}(s,n)+T_{Agg}(s,n)$, with the last two terms replaced by their optimally overlapped versions after scheduling, and a locality-based greedy algorithm iteratively moves the most-loaded experts' parameters to the devices holding the most of their inputs until the load spread satisfies $\\max(H)-\\min(H)<\\alpha I/E$. The scheduler's block-wise strategy partitions Trans and Agg into sub-operators and launches them in parallel with the forward expert and non-expert computations and the backward non-expert and expert computations of the neighboring block, while Plan is precomputed during the previous iteration's all-to-all communication.","core_discovery":"Pro-Prophet's central claim is that the dynamic device load imbalance that plagues expert-parallel training of Mixture-of-Experts models can be corrected cheaply instead of expensively. The paper observes that the per-expert input distribution of a MoE layer changes little between adjacent training iterations, and builds two components on this locality. The planner searches over lightweight expert placements, placements where each expert is replicated to exactly the subset of devices that currently receives its tokens and only that expert's parameters and gradients are communicated, using a performance model and a greedy search to produce a communication-efficient placement. The scheduler then moves the three data-dependent load-balancing primitives, planning, parameter transfer, and gradient aggregation, off the critical path by computing the plan one iteration ahead, prefetching expert parameters, and splitting transfers into sub-operators overlapped with neighboring blocks' forward and backward computations. The paper reports end-to-end speedups of 1.18-2.66x over the main baseline and 1.01-1.50x over the dynamic-shadowing baseline across four clusters and five MoE-GPT variants, with up to 11.01x improvement in its load-balance metric.","pith_inferences":["Beyond the paper: the same one-iteration-ahead prediction could be applied to expert-parallel inference serving, where request routing is also temporally correlated, so prefetching expert weights to the devices that will receive the next batch could hide much of the serving latency.","Beyond the paper: the scheduler uses a static offline split of Trans and Agg based on fixed non-MoE computation durations; making the split adaptive to measured per-iteration load divergence is a natural testable extension that would also stress the locality assumption.","Beyond the paper: the performance model prices communication with a fixed average bandwidth, so on clusters where achievable bandwidth depends on which device pairs communicate, the placement search would likely benefit from topology-aware communication costs.","Beyond the paper: the planner's search space grows with the number of experts per layer, so the method's advantage should be largest in the regime MoE models are headed toward, many more experts than devices."],"forward_implications":["If Pro-Prophet's central claim holds, system-level load balancing for MoE training no longer requires moving whole model states: replicating only the parameters and gradients of selected experts to a subset of devices is enough to even out device load.","The locality of expert input distributions implies the load-balancing plan can be computed one iteration ahead and expert parameters prefetched, so balancing overhead is largely removed from the critical path.","Splitting communication primitives into sub-operators and overlapping them with forward and backward computation of neighboring MoE blocks can hide most remaining balancing communication, improving device utilization without changing model convergence behavior.","Across four clusters of 8-32 GPUs and five MoE-GPT model sizes, the method reports end-to-end speedups of 1.18-2.66x over the primary baseline, up to 1.50x over the dynamic-shadowing baseline, and up to 11.01x improvement in its load-balance metric."],"supporting_citations":[{"why":"introduces expert parallelism and the all-to-all MoE layer workflow whose per-device load imbalance Pro-Prophet targets","marker":"[1]"},{"why":"the primary end-to-end baseline; its expert-parallel training workflow defines the standard that Pro-Prophet compares against","marker":"[4]"},{"why":"supplies the efficient all-to-all communication implementation and adaptive parallelism ideas reused in Pro-Prophet's performance model","marker":"[5]"},{"why":"the representative systematic load-balancing baseline that transfers parameters to shadow devices; Pro-Prophet compares its speedup and balance metric against it","marker":"[8]"},{"why":"demonstrates pipelining all-to-all communication with expert computation, the scheduling idea Pro-Prophet extends to block-level overlapping","marker":"[34]"},{"why":"a communication scheduler for MoE training that partitions computation and communication operators; Pro-Prophet claims compatibility with such schedulers","marker":"[59]"},{"why":"another pipeline-based MoE scheduler that partitions operations; cited as compatible prior art","marker":"[60]"}],"fun_headline_variants":["Predicting MoE load imbalance yields 2.66x training speedup","Pro-Prophet hides expert imbalance for 2.66x MoE speedup","Load-aware expert placement cuts MoE training time by 2.66x","MoE training 2.66x faster with predicted expert placement"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The scheduling rests on locality, the premise that each expert's share of inputs changes slowly between adjacent training iterations, and the paper tests this only in the first 100 iterations, where the distribution is still stabilizing; a sharp shift in routing would make the precomputed plan and prefetched parameters stale.","fun_headline_variants_meta":{"raw":{"variants":["Predicting MoE load imbalance yields 2.66x training speedup","Pro-Prophet hides expert imbalance for 2.66x MoE speedup","Load-aware expert placement cuts MoE training time by 2.66x","MoE training 2.66x faster with predicted expert placement"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000229,"raw_usage":{"total_tokens":1559,"prompt_tokens":1104,"completion_tokens":455,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":720,"completion_tokens_details":{"reasoning_tokens":372}},"tokens_in":720,"tokens_out":455,"duration_ms":4747,"temperature":1.0,"reasoning_tokens":372,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T20:04:25.955119+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Feed a MoE model inputs whose routing is deliberately perturbed every few iterations so that adjacent-iteration expert-load vectors have high divergence, and measure Pro-Prophet's per-iteration time; if it degrades toward the no-overlap baseline and planning or prefetch corrections dominate, the locality assumption is falsified. Quantitatively, compute the Jensen-Shannon divergence between consecutive iterations' expert-load distributions; Pro-Prophet's speedup should decrease monotonically as this divergence increases.","supporting_citations":[{"cited_title":"Faster- moe: modeling and optimizing training of large-scale dynamic pre- trained models,","cited_arxiv_id":null,"evidence_quote":"the representative systematic load-balancing baseline that transfers parameters to shadow devices; Pro-Prophet compares its speedup and balance metric against it"},{"cited_title":"Accelerating distributed moe training and inference with lina,","cited_arxiv_id":null,"evidence_quote":"demonstrates pipelining all-to-all communication with expert computation, the scheduling idea Pro-Prophet extends to block-level overlapping"}],"review_version":1}