Pith. sign in

REVIEW 3 major objections 4 minor 22 references

TEMPO: Makespan-Aware Expert-Parallel Load Balancing Across Memory- and Compute-Bound Regimes

T0 review · 3 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that per-expert fp8 grouped-GEMM time is flat up to about 156–168 tokens and linear above it, and that a makespan-aware dispatcher over the resulting two-regime cost model stays within about 1% of the best fixed policy…

desk verdict A useful two-regime cost model and a well-built dispatcher, but the paper's headline mixed-regime statistic is arithmetically impossible under its own inflection point. read the letter →

arxiv 2608.13057 v1 pith:TKHUEXQ2 submitted 2026-08-13 cs.DC cs.AIcs.CLcs.GT

Jie Li , Chenxin Jia , Jinliang Shen , Cunzhuang Liu , Ruiyi Ding , Jianwen Xian , Kang He , Chengru Song This is my paper · ORCID
classification cs.DCcs.AIcs.CLcs.GT
keywords mixture-of-expertsexpertparallelismloadbalancingmakespangroupedGEMMmemory-boundphasediagramfp8inference
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

What the paper tries to establish is that the standard proxies used to balance expert-parallel MoE serving — token counts and activated-expert counts — both misprice the hardware cost, and that a makespan-aware dispatcher over a measured two-regime cost model is the right objective. Measurements on two GPU generations show per-expert fp8 grouped-GEMM time is flat up to about 156–168 tokens and linear above it; the flat regime is HBM weight streaming, the linear regime is token compute rounded to 128-token tiles. On recorded decode batches, hot experts sit in the linear regime and cold experts in the flat regime at the same time, so no fixed proxy is safe. The paper formalizes per-batch dispatch as a fixed-charge makespan problem and presents TEMPO, which solves it in milliseconds and stays within about 1% of the best fixed policy at every phase-grid point while winning up to 15.5% where regimes mix. A phase diagram, not a universal win, is the claim.

What carries the argument

The load-bearing object is the two-piece max-affine cost profile $t_g=\max(a+bG_g,\,c+\beta N_g)$; $b$ is the per-replica activation floor (measured at 1.13–1.24$\times$ the HBM roofline for the expert's weights), $\beta$ is the linear per-token compute term, and the tile-aware extension adds $b_2(T-G)$ with $T=\sum_e \lceil n_e/128\rceil$. On top of this, dispatch is cast as a fixed-charge makespan problem (minimize $\max_g \max(a+b\sum_e z_{e,g},\,c+\beta\sum_e x_{e,g})$), which is NP-hard on two fully replicated GPUs yet polynomial in each degenerate limit ($b\to0$ reduces to the token LP; $\beta\to0$ to semi-matching). The practical solver — cost-aware seeding, augmenting-chain activation rebalancing, bottleneck local search with partial splits, and an ensemble with a 1% switching tolerance — solves it in about 2 ms off the critical path and inherits an additive approximation guarantee via a round-robin whole-expert placement.

What would settle it

Measure per-layer wall-clock MoE-block makespan on a 32- or 64-GPU deployment across a batch-size ladder from 32 to 2048 tokens per GPU and compare it with the model's predicted $t_g=\max(a+bG_g,\,c+\beta N_g)$; if the flat-to-linear inflection near 156–168 tokens per expert, or the claimed 92–100% fraction of decode batches containing both regimes, fails to appear on another kernel generation or another routing distribution, the central claim collapses.

Watch

Extended reading notes

Core claim

The central discovery is that per-GPU MoE-block time obeys a max-affine law, $t_g=\max(a+bG_g,\,c+\beta N_g)$, where $G_g$ is the number of activated expert replicas and $N_g$ is the total token count. Below the inflection $n^*\approx156$–$168$ tokens per expert, time is essentially flat in tokens because streaming the expert's weights from HBM dominates; above it, grouped GEMM rounds each expert's tokens up to 128-token M-tiles, so fragmenting an expert manufactures padded compute. Real decode batches mix both regimes simultaneously (92–100% of recorded batches), which is why a dispatcher optimizing the max of the two terms, i.e., the makespan, can beat both token-only and activation-only policies. The same measured shape transfers to a newer GPU generation and is captured with one extra parameter for the full tile staircase, though the largest claimed gains (up to 15.5%) are model-scored extrapolations to EP32–64.

Load-bearing premise

The biggest wins rest on the assumption that the two-regime cost model calibrated on one 8-GPU testbed transfers to full serving pipelines and to the larger expert-parallel sizes (EP32–64) where the headline gains are extrapolated, with the fitted traffic term as the least independently validated part.

Editorial extensions

If this is right

  • If the two-regime cost shape is correct, all token-only and activation-only dispatchers are systematically wrong outside their home regime, with modeled block-time spreads of 1.4–1.6$\times$ (p95 up to 1.7$\times$) between proxy dispatches of the same batch.
  • A deployment can locate itself on a phase diagram rather than guess: the policy flip boundary $B^*$ is analytically predictable from expert popularity, replication, and batch size, and it lands in the observed flip band in 12/12 grid columns.
  • Inside the win region, makespan-aware dispatch delivers end-to-end gains (4–6% throughput and roughly a 15.6% p99 latency cut on one flagship model) while staying within 1% of the best fixed policy elsewhere.
  • The dispatch layer repairs the latency tail and makes tight replica budgets safe to exploit, but it cannot repair the mean: under stale placement, throughput loss is a provisioning error that only the placement layer can fix.
  • On multi-node expert parallelism, source-to-replica pairing is part of the cost model: a topology-aware split flips a 3.5% regression into a 4–7% gain at the all-to-all-bound point.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same two-regime mechanism should appear on other GPU generations and fp8 kernels, but with different $b$, $\beta$, and tile heights, so the phase diagram is a template to be recalibrated per hardware, not a universal constant.
  • Monitoring per-GPU activation counts and tile counts in a live server could let an operator detect which regime the current batch is in and switch dispatcher mode without running a solver.
  • The cost model prices weight movement itself (the activation floor is the cost of touching an expert's weights), which suggests that token-moving dispatch is the only zero-marginal-cost lever at decode scale.
  • The NP-hardness at dispatch granularity hints that placement and dispatch should be co-designed: a placement that minimizes the variance of activation floors would shrink the region where dispatch must act at all.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper studies per-batch load balancing for expert-parallel (EP) Mixture-of-Experts serving, where the per-layer time is the makespan across GPUs. Measurements on two GPU generations show that per-expert fp8 grouped-GEMM time is flat below roughly 156-168 tokens (HBM weight streaming) and linear above, leading to a max-affine per-GPU cost model t_g = max(a+bG_g, c+beta N_g). The authors argue that real decode batches contain both regimes simultaneously, that no fixed token-count or expert-count proxy is therefore safe, and that the dispatch problem is a fixed-charge makespan problem that is NP-hard with two GPUs yet polynomial in each degenerate limit. They present TEMPO, a solver that runs in milliseconds off the critical path, integrates with SGLang via a fused in-graph kernel, and is validated by an 8-GPU microbenchmark, calibrated simulation, and end-to-end serving on Testbed B with Qwen3-235B (inside the predicted win region) and DeepSeek-V3 (outside). The paper's central claim is a phase diagram predicting when time-aware dispatch pays, rather than a universal win.

Significance. If the two-regime cost model is correct, the paper makes a substantive contribution: it explains why token-based and activation-based balancing fail in complementary regimes, provides a principled makespan objective, and delivers a deployable dispatcher with off-critical-path solving and a fused kernel. The paper is unusually careful about evidence: it includes an 8-GPU wall-clock microbenchmark with 93% pairwise ranking agreement, explicit negative controls, a like-for-like architectural decomposition of the LPLB gap, and honest limitations (L1-L6) distinguishing model-scored results from measured serving results. The NP-hardness and additive-approximation proofs are self-contained and appear sound. The main weakness is that the 'pivotal empirical fact' of 92-100% mixed-regime batches at decode sizes is not consistent with the stated kernel inflection, and the deployment-time inflection used for that statistic is not identifiable from the full-pipeline fit. This weakens the motivational narrative and the precision of the phase-diagram boundary, though the algorithmic and engineering contributions stand.

major comments (3)
  1. [Section 2.4, Fig. 5] The claim that 92-100% of batches at B=128 tokens per GPU contain both regimes is mathematically incompatible with the stated inflection n* approximately 156-168. In decode, each active sequence contributes one token and an expert can receive at most one token per sequence, so with B=128 tokens per GPU every expert has n_e <= 128 < n*, meaning 0% of tokens can be in the linear regime. Yet Fig. 5(a) reports 92-100% mixed batches at B=128. The paper must specify which inflection was used as the classifier. If it is the full-pipeline value b/beta approximately 46 from Table 7, then Section 5.2 shows this quantity is not identifiable (corr(a_hat,b_hat) = -0.997, b in [5.1,18.8] microseconds at 95% CI), so the statistic is not robust. This is load-bearing because the paper's motivation states that the pivotal empirical fact is that regimes coexist in 92-100% of decode batches, and the phase-diagram win region is where regimes mix. Please correct the statistic, the stated n*, or the narrative, and report the sensitivity of the mixed-batch fraction to the choice of n*.
  2. [Section 5.2, Table 7, Section 3.2] The paper conflates the measured kernel inflection (n* approximately 156) with the deployment-time inflection (b/beta approximately 46 for DSv3 from Table 7). The full-pipeline fit observes only G in [9,20], where a and b are nearly collinear, so b/beta could range roughly 14-52 within the reported 95% CI. This unidentifiability matters because the phase diagram (Fig. 6) and the analytic boundary B* of Section 3.2 depend on n*, and the scale extrapolation (Fig. 12) uses one point on the (a,b) ridge. The authors should demonstrate that the predicted win region and the B* boundary are stable when (a,b) are moved along the ridge within the CI, or state explicitly which calibration is used for each figure and why the phase-diagram conclusions are insensitive to the ridge.
  3. [Abstract and Section 5.1] The abstract states that TEMPO stays within 1% of the best fixed baseline everywhere, but the wall-clock microbenchmark in Section 5.1 reports that TEMPO is within 5% (approximately run noise) of the per-B best fixed policy. The 1% figure is a model-space guarantee from Proposition 1 and the phase-grid evaluation, not a measured wall-clock result. Please qualify the abstract claim so that model-scored and wall-clock claims are not conflated.
minor comments (4)
  1. [Fig. 5 caption] The caption should state explicitly which calibrated n* is used as the classifier and the source of the parameters, given that different sections use different n* values.
  2. [Section 3.2, Eq. for B*] The analytic boundary formulas B*_avg and B*_hot should define all symbols (e.g., K, n_gpus) at first use, and the fixed-point solution for B* should be described in one sentence for reproducibility.
  3. [Section 5.10, Fig. 8] The left panels of Fig. 8 use the y=0 line as static baseline; the caption says normalized to same-window static but the DSv3 EP16 panel appears to show different scales. Please clarify the normalization and the number of windows for each panel.
  4. [General] The paper uses the phrase model-scored extensively but not always in the abstract and introduction; adding a sentence early on that the headline phase-diagram numbers are simulated under the calibrated model would improve clarity.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the measured cost model, the makespan solver, and the wall-clock/end-to-end anchors form a self-contained derivation chain.

full rationale

The paper's derivation chain is self-contained rather than circular. The two-regime cost model (Eq. 1; tile-aware Eq. 2) is obtained by direct microbenchmark measurement, with the activation floor cross-checked against an HBM roofline; it is not defined in terms of the dispatcher's output. The dispatch objective (Eq. 3) is a well-defined optimization problem over that measured model, and the solver's guarantees are proven rather than imported from a self-citation. The phase diagram and scale extrapolation are explicitly labeled calibrated simulation and model-scored, and the paper repeatedly discloses that Table 4 is self-evaluation; it does not rest the central claim on those alone. Independent support comes from the 8-GPU Testbed A wall-clock microbenchmark (Section 5.1), the transfer test with pairwise ranking agreement and recalibration (Section 5.2), the like-for-like token-LP port (Section 5.12), the predicted-then-measured B*_a2a boundary (Section 5.13), and the Testbed B end-to-end serving that brackets the predicted win region with Qwen3-235B and DSv3 (Section 5.10). The analytic flip boundary of Section 3.2 is a consistency check of a mean-field formula against the same calibrated model's simulation rather than an independent empirical validation, but the paper does not use that agreement to establish the model's validity. The identified (a,b) ridge and the self-admitted limitations (L1-L5) are honesty about identifiability and scope, not circular reasoning. No load-bearing prediction reduces by construction to its own input.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central claim depends on the empirically fitted cost model (Eqs. 1, 2, 4) and on the transfer of that model from microbenchmark to serving. The parameters b, beta, a, c, b2, c2, gamma are fitted to data; the paper validates them with roofline arguments and wall-clock tests but the EP32-64 gains are extrapolations. The main axioms are the functional form, the representativeness of the traces, the linear traffic term, and the assumption that grouped-GEMM time is a deterministic function of (G,N). No new physical entities are introduced.

free parameters (6)
  • activation floor b = 1.74/2.99/14.78 us (Qwen3-30B/DSv2-Lite/DSv3, Testbed A offline); 8.32/3.95 us (DSv3/Qwen3-235B, Testbed B tile)
    Fitted to grouped-GEMM microbenchmark grid; prices the flat memory-bound regime; roofline check places it at 80-88% of HBM bandwidth bound.
  • per-token slope beta = 0.0108/0.0179/0.0945 us/tok (Testbed A offline); 0.358 us/tok full-pipe refit; 0.0336/0.0154 us/tok (Testbed B tile)
    Fitted slope of the linear compute-bound regime; part of the max-affine fit.
  • offsets a and c = a in 116-488 us, c in 176-294 us across full-pipe refits; a=16.4, c=15.6 us in proxy serving
    Max-affine intercepts fitted with slopes; in full-pipeline fits a and b are nearly collinear (corr -0.997) and only their combination is well identified.
  • tile staircase parameter b2 = 2.65 us (DSv3), 1.23 us (Qwen3-235B), about b/3
    Extra per-tile cost in the tile-aware model (Eq. 2), fitted on Testbed B fine sweep around 128-token boundaries.
  • traffic pair (c2, gamma) = c2=25.0 us, gamma=0.10 us/token
    Fitted once from recorded static-vs-uniform crossover; the paper labels it the least independently validated part of the model (Section 7).
  • comm slope kr = 0.0037 us/token
    Used only for EP32-64 scale extrapolation; fitted from microbenchmark all-to-all logs.
assumptions (5)
  • domain assumption The grouped-GEMM per-expert time is a deterministic function of (G, tokens-per-expert) and additive across experts; inter-expert interference beyond the modeled terms is negligible.
    The model (1)/(2) sums activation floors and token time per GPU; Section 5.2 shows fit error 4-8% but L5 admits a third cost dimension (uniform per-slot loads) that decides near-ties.
  • domain assumption The recorded routing traces and synthetic Zipf/Dirichlet workloads are representative of production decode traffic.
    Mixed-regime statistics (Fig. 5) and phase-diagram sweeps rely on these; share of batches with both regimes is 92-100% across the traces.
  • domain assumption All-to-all communication cost is captured by a linear term gamma*N per rank; source-destination pairing matters only through the topology-aware extension.
    Eq. (4) treats the flat table as rank-agnostic; Section 5.15 shows the flat table loses 3.5% at the a2a-bound point, repaired by the two-stage split; (c2,gamma) is least validated.
  • ad hoc to paper The max-affine two-regime form is the correct functional shape for expert cost.
    Chosen after measurement; validated by fit error and the 128-tile staircase, but it is a fitted model, not derived from first principles.
  • standard math Standard NP-completeness and semi-matching results (Garey-Johnson [8], Harvey et al. [10]) are correct.
    Used in Theorem 1 and Lemmas 1-2; no novel mathematics needed beyond these cited results.

how reviews work

0 comments
Cite this review

Pith. "Pith review of TEMPO: Makespan-Aware Expert-Parallel Load Balancing Across Memory- and Compute-Bound Regimes." pith.science (2026). https://pith.science/paper/TKHUEXQ2

@misc{pith2026260813057,
  author       = {Pith},
  title        = {Pith review of: TEMPO: Makespan-Aware Expert-Parallel Load Balancing Across Memory- and Compute-Bound Regimes},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TKHUEXQ2}},
  note         = {Machine review of arXiv:2608.13057}
}
abstract

In expert-parallel (EP) MoE serving, every layer synchronizes at the slowest GPU. Dispatchers balance token counts (EPLB, LPLB, UltraEP) or activated-expert counts (METRO), assuming expert time is linear in one. Measurements on two datacenter GPU generations show it is neither: below $\nstar\!\approx\!156$--$168$ tokens, HBM weight streaming dominates---cost attaches to \emph{activated replicas}, not tokens; above it, grouped GEMM rounds tokens to 128-tile $M$-tiles, so \emph{splitting} an expert adds padded compute. A max-affine profile $t=\max(a+bG,\,c+\beta N)$ captures both regimes. Realistic decode batches hold hot experts in the linear regime and cold in the flat \emph{simultaneously}; recorded batches show proxy dispatches differ by $1.4$--$1.6\times$ in modeled block time (p95 up to $1.7\times$), and \emph{which} proxy wins flips with the regime. We formalize per-batch dispatch as a fixed-charge makespan problem---NP-hard on two fully replicated GPUs, polynomial in degenerate limits---and present \sys{}, a makespan-aware dispatcher solving it in milliseconds off the critical path; its SGLang integration runs out-of-process and fuses dispatch with count collection into one in-graph kernel. Anchored by an 8-GPU Testbed~A microbenchmark, \sys{} stays within 1\% of the best fixed baseline everywhere and wins by up to $15.5\%$ where regimes mix. End-to-end on Testbed~B, Qwen3-235B (inside the win region) gains $4$--$6\%$ throughput and cuts p99 latency by ${\sim}15.6\%$; DeepSeek-V3 (outside, communication-dominated) shows only mechanism cost. A phase diagram, not a universal win, is the claim: it predicts both outcomes before deployment.

Figures

Figures reproduced from arXiv: 2608.13057 by the authors.

Figure 1
Figure 1. The idea in one picture. (a) Measured per-expert time (DSv3 shape, fp8): flat below [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. (a) DSv3 fp8 grouped GEMM per-expert time: flat [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Testbed B fp8 masked DeepGEMM, DSv3 shape. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (7 more)
Figure 5
Figure 5. Figure 5: Fraction of batches in the mixed regime ( [PITH_FULL_IMAGE:figures/full_fig_p004_5.png]
Figure 6
Figure 6. Figure 6: Phase diagram, DSv3 shape, EP8. Letters mark the winning [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: 8-GPU Testbed A EP8 wall clock, DSv3 shape: [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Testbed B serving, all policies normalized to same-window static (the [PITH_FULL_IMAGE:figures/full_fig_p011_8.png]
Figure 9
Figure 9. Figure 9: Proxy-equivalence sets versus true time on the raw [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]
Figure 10
Figure 10. Figure 10: Stale-placement serving on the DSv3-shape proxy [PITH_FULL_IMAGE:figures/full_fig_p017_10.png]
Figure 13
Figure 13. Figure 13: Topology-aware dispatch on 2-node EP16 (Testbed B, stale placement). (a) The solve fixes per-GPU loads; a same-node-first transportation split then assigns sources to replicas, giving one table per source node at zero kernel cost. (b) End-to-end: the flat table loses …

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

22 extracted references · 17 canonical work pages

  1. [1]

    MoonEP: A perfectly balanced expert parallelism library via dynamic redundant experts.https://github.com/ MoonshotAI/MoonEP, 2026

    Yutian Chen, Cong Li, Yucheng Wang, and Ming Wei. MoonEP: A perfectly balanced expert parallelism library via dynamic redundant experts.https://github.com/ MoonshotAI/MoonEP, 2026

    work page 2026

  2. [2]

    DeepSeek-V3 technical report.arXiv preprint arXiv:2412.19437, 2024

    DeepSeek-AI. DeepSeek-V3 technical report.arXiv preprint arXiv:2412.19437, 2024

    arXiv 2024

  3. [3]

    DeepEP: an efficient expert-parallel communication library

    DeepSeek-AI. DeepEP: an efficient expert-parallel communication library. https://github.com/ deepseek-ai/DeepEP, 2025

    work page 2025

  4. [4]

    DeepGEMM: clean and efficient FP8 GEMM kernels

    DeepSeek-AI. DeepGEMM: clean and efficient FP8 GEMM kernels. https://github.com/ deepseek-ai/DeepGEMM, 2025

    work page 2025

  5. [5]

    EPLB: Expert parallelism load balancer

    DeepSeek-AI. EPLB: Expert parallelism load balancer. https://github.com/deepseek-ai/EPLB, 2025

    work page 2025

  6. [6]

    LPLB: An LP-based load bal- ancer for expert parallelism

    DeepSeek-AI. LPLB: An LP-based load bal- ancer for expert parallelism. https://github.com/ deepseek-ai/LPLB, 2025. README notes nonlinear expert cost as an open problem

    work page 2025

  7. [7]

    Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity.JMLR, 23(120), 2022

    William Fedus, Barret Zoph, and Noam Shazeer. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity.JMLR, 23(120), 2022

    work page 2022

  8. [8]

    Garey and David S

    Michael R. Garey and David S. Johnson.Comput- ers and Intractability: A Guide to the Theory of NP- Completeness. W. H. Freeman, 1979

    work page 1979

Show all 22 references
  1. [9]

    ViBE: Co-optimizing workload skew and hardware variability for MoE serving.arXiv preprint arXiv:2606.00735, 2026

    Seokjin Go, Marko Scrbak, Ephrem Wu, Srilatha Manne, and Divya Mahajan. ViBE: Co-optimizing workload skew and hardware variability for MoE serving.arXiv preprint arXiv:2606.00735, 2026

    2026 arXiv

  2. [10]

    Nicholas J. A. Harvey, Richard E. Ladner, L´aszl´o Lov´asz, and Tami Tamir. Semi-matchings for bipartite graphs and load balancing.Journal of Algorithms, 59(1):53–78, 2006

    2006

  3. [11]

    Mixture-of- experts serving.arXiv preprint arXiv:2607.17880, 2026

    Zhiyi Huang, Tao Xiao, and Qinpei Lou. Mixture-of- experts serving.arXiv preprint arXiv:2607.17880, 2026

    2026 arXiv

  4. [12]

    GShard: Scaling gi- ant models with conditional computation and automatic sharding

    Dmitry Lepikhin, HyoukJoong Lee, Yuanzhong Xu, De- hao Chen, Orhan Firat, Yanping Huang, Maxim Krikun, Noam Shazeer, and Zhifeng Chen. GShard: Scaling gi- ant models with conditional computation and automatic sharding. InICLR, 2021

    2021

  5. [13]

    Effi- cient MoE serving in the memory-bound regime: Bal- ance activated experts, not tokens.arXiv preprint arXiv:2512.09277, 2025

    Haiyue Ma, Krish Agarwal, Nicolai Oswald, Qijing Huang, Hugo Linsenmaier, Chunhui Mei, Ritchie Zhao, Ritika Borkar, Bita Darvish Rouhani, David Nellans, Ronny Krashinsky, and Anurag Khandelwal. Effi- cient MoE serving in the memory-bound regime: Bal- ance activated experts, no...

    2025

  6. [14]

    Least-loaded expert parallelism: Load balancing an imbalanced mixture-of-experts.arXiv preprint arXiv:2601.17111, 2026

    Xuan-Phi Nguyen, Shrey Pandit, Austin Xu, Caiming Xiong, and Shafiq Joty. Least-loaded expert parallelism: Load balancing an imbalanced mixture-of-experts.arXiv preprint arXiv:2601.17111, 2026

    2026

  7. [15]

    Flex- MoE: Scaling large-scale sparse pre-trained model train- ing via dynamic device placement

    Xiaonan Nie, Xupeng Miao, Zilong Wang, Zichao Yang, Jilong Xue, Lingxiao Ma, Gang Cao, and Bin Cui. Flex- MoE: Scaling large-scale sparse pre-trained model train- ing via dynamic device placement. InSIGMOD, 2023

    2023

  8. [16]

    Outrageously large neural networks: The sparsely-gated mixture-of-experts layer

    Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, and Jeff Dean. Outrageously large neural networks: The sparsely-gated mixture-of-experts layer. InICLR, 2017

    2017

  9. [17]

    UltraEP: Unleash MoE training and inference on rack-scale nodes with near-optimal load balancing.arXiv preprint arXiv:2606.04101, 2026

    Xinming Wei, Chao Jin, Tuo Dai, Yinmin Zhong, Shan Yu, Chengxu Yang, Bingyang Wu, Zili Zhang, Jing Mai, Qianchao Zhu, Zhouyang Li, Yuliang Liu, and Guojie Luo. UltraEP: Unleash MoE training and inference on rack-scale nodes with near-optimal load balancing.arXiv preprint arXiv...

    2026 arXiv

  10. [18]

    Parallel machine scheduling with splitting jobs.Discrete Applied Mathe- matics, 103(1-3):259–269, 2000

    Wenxun Xing and Jiawei Zhang. Parallel machine scheduling with splitting jobs.Discrete Applied Mathe- matics, 103(1-3):259–269, 2000

    2000

  11. [19]

    Taming latency-memory trade-off in MoE- based LLM serving via fine-grained expert offloading

    Hanfei Yu, Xingqi Cui, Hong Zhang, Hao Wang, and Hao Wang. Taming latency-memory trade-off in MoE- based LLM serving via fine-grained expert offloading. InEuroSys, 2026. arXiv:2502.05370

    2026

  12. [20]

    Be- yond uniform experts: Cost-aware expert execution for efficient multi-device MoE inference.arXiv preprint arXiv:2606.29982, 2026

    Hui Zang, Pengfei Xia, Hong Liu, Jiajia Chu, Tuo Hao, Minghao Chen, Rui Zhang, and Ziyang Zhang. Be- yond uniform experts: Cost-aware expert execution for efficient multi-device MoE inference.arXiv preprint arXiv:2606.29982, 2026

    2026 arXiv

  13. [21]

    SmartMoE: Efficiently training sparsely-activated models through combining offline and online parallelization

    Mingshu Zhai, Jiaao He, Zixuan Ma, Zan Zong, Runqing Zhang, and Jidong Zhai. SmartMoE: Efficiently training sparsely-activated models through combining offline and online parallelization. InUSENIX ATC, 2023

    2023

  14. [22]

    fewest-experts replica, token tie- break

    Lingyun Zhang, Henghua Zhang, Shilei Gu, Kai Mo, Shuai Han, Shiyong Li, Yanpeng Wang, and Dou Shen. TAOT: Topology-aware optimal transport for dynamic expert replica placement in MoE training.arXiv preprint arXiv:2608.03676, 2026. 14 A Proofs A.1 Theorem 1 Membership.Given (x,...

    2026 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.