Pith. sign in

REVIEW 3 major objections 5 minor 2 cited by

Expert-choice routing, not token-choice, is the right default for diffusion language models with mixture-of-experts layers: it removes load imbalance by construction and turns per-step compute into a schedulable quantity.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 05:31 UTC pith:AWRVZGGY

load-bearing objection Honest, well-controlled empirical case for EC routing in DLM MoEs, with a practical scheduler finding; the matched-FLOPs claim needs the masking sampler stated and the mechanism is thinner than advertised. the 3 major comments →

arxiv 2604.01622 v2 pith:AWRVZGGY submitted 2026-04-02 cs.LG cs.CL

Expert-Choice Routing Enables Adaptive Computation in Diffusion Language Models

Shuibai Zhang , Caspian Zhuang , Chihan Cui , Zhihan Yang , Fred Zhangzhi Peng , Yanxin Zhang , Haoyue Bai , Zack Jia
show 3 more authors
Yang Zhou Guanhua Chen Ming Liu
This is my paper
classification cs.LG cs.CL
keywords expert-choice routingtoken-choice routingdiffusion language modelsmixture-of-expertstimestep-dependent capacityadaptive computationload balancingmasked diffusion
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that diffusion language models should abandon token-choice routing for expert-choice routing: because DLMs denoise all tokens at once with bidirectional attention, a global view is available, so each expert can pick its own tokens, guaranteeing perfectly balanced load with no auxiliary loss. This yields 1.5–2.1x higher training throughput and roughly 2x faster wall-clock convergence to the same training loss. The paper then exploits the fact that expert capacity is an externally controllable knob and schedules it across denoising steps, finding that giving more experts to low-mask-ratio steps (where most tokens are already visible) consistently improves perplexity and downstream accuracy under matched compute. A convergence-rate analysis shows those low-mask-ratio steps learn an order of magnitude faster, explaining the allocation. Finally, the paper shows a pretrained token-choice DLM can be converted to expert-choice by replacing only the router, achieving faster finetuning convergence, faster decoding, and—with the dynamic schedule—slightly higher accuracy.

Core claim

The central claim is that expert-choice routing is not merely a load-balancing fix but the right routing paradigm for diffusion language models, because it turns per-timestep compute into a controllable policy. Using expert-choice with a linear-reverse capacity schedule—more experts at low mask ratios, fewer at high—an 8B-parameter MoE DLM reaches a given validation perplexity with less compute than any token-choice variant or static-capacity schedule, and the same schedule improves MMLU and ARC-Challenge scores at matched FLOPs. The mechanism is that tokens in low-mask-ratio contexts have up to an order-of-magnitude higher convergence rate, so extra expert capacity there has the largest mar

What carries the argument

Expert-choice routing (each expert selects its top-c tokens, rather than each token selecting its top-k experts), which guarantees deterministic per-expert load balance by construction and exposes expert capacity as an external hyperparameter. On top of it, the paper defines a timestep-dependent capacity function k(r) = clamp(kmin + (kmax - kmin)·s(r), kmin, kmax) and considers schedulers s(r); the winning scheduler is linear-reverse, s(r) = 1 - r, which allocates most experts to low-mask-ratio denoising steps. The mechanistic quantity is the per-bin convergence rate ηr = -d ln Lr / dt, measuring the fractional rate of loss decrease per training step.

Load-bearing premise

The matched-FLOPs comparison assumes the masking ratios seen during training are uniformly distributed; if the actual training sampler concentrates on certain ratios, the dynamic and static schedules may not consume equal compute.

What would settle it

Record the histogram of masking ratios actually sampled during one training run and compute E[k(r)] under the true distribution; if it differs from the static baseline k=20 by more than a few percent, the reported perplexity comparison is confounded and must be redone with exact FLOPs matching.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • EC routing gives 1.5–2.1x higher training throughput than all tested TC variants and reaches training loss 3.75 in 10.6 hours versus roughly 20 hours for TC.
  • A linear-reverse timestep-dependent capacity schedule outperforms static EC at matched expected FLOPs, improving perplexity as well as MMLU and ARC-Challenge accuracy at 8B-A1B scale.
  • Low-mask-ratio denoising steps exhibit up to an order-of-magnitude higher convergence rate, so concentrating expert capacity on those steps produces the largest marginal return.
  • Pretrained token-choice DLMs can be retrofitted to expert-choice by replacing only the router, yielding faster convergence, 1.3–1.5x faster decoding, and comparable or better peak accuracy.
  • The auxiliary load-balancing loss is not the dominant bottleneck in TC; the structural load imbalance of token-choice routing is the main cause of its slower wall-clock convergence.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the training-time masking-ratio sampler is not uniform, the FLOPs-equivalence proof (which assumes r∼Uniform(0,1)) would need to be re-derived under the true distribution; the scheduler ranking could shift if compute is not exactly matched.
  • The convergence-rate proxy ηr explains why reverse schedulers beat forward schedulers, but the paper itself notes it cannot distinguish linear-reverse from cosine-reverse, so a different measure (e.g., gradient noise or per-token loss variance) may be needed to further optimize capacity schedules.
  • The same capacity-scheduling idea could transfer to continuous diffusion models or to inference-time compute allocation, where a learned capacity predictor could replace hand-designed schedulers.
  • Because expert-choice requires a global view of all tokens, DLM-specific MoE architectures may naturally diverge from autoregressive LLM routing designs, suggesting new infrastructure choices for diffusion-based generation.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper argues that expert-choice (EC) routing is preferable to token-choice (TC) routing for mixture-of-experts diffusion language models (DLMs). It claims EC provides deterministic load balance, higher throughput, and faster convergence than TC; that EC's externally controllable expert capacity enables timestep-dependent capacity schedules; that a linear-reverse schedule allocating more capacity to low-mask-ratio steps achieves the best perplexity under matched FLOPs; that this is explained by an order-of-magnitude higher convergence rate at low mask ratios; and that pretrained TC DLMs can be retrofitted to EC by replacing only the router. The empirical support is a 30B-token OpenWebText scheduler comparison, an 8B-A1B pretraining comparison, and SFT retrofitting experiments on LLaDA-MoE.

Significance. If the central claims hold, the paper would establish a new default routing paradigm for DLM MoE models and a practical principle for adaptive computation. The work has real strengths: the routing comparison in §3 is cleanly controlled (same architecture/data/hyperparameters, only routing differs); Appendix E provides a thorough ablation of TC load-balancing and capacity variants, ruling out the auxiliary-loss confound; the 8B-scale check in §4.4 is valuable; and the retrofitting result in §5 has immediate practical utility. The code release is a concrete reproducibility asset. However, two load-bearing supports are currently weaker than the headline claims: the "matched FLOPs" comparison depends on an unstated mask-ratio sampler assumption, and the mechanistic explanation is explicitly acknowledged in Appendix F.1 to be incomplete. The single-run nature of all key numbers additionally makes the "consistently" language premature.

major comments (3)
  1. [§4.3 / Table 2 / Appendix G.2] The headline claim that linear-reverse capacity allocation outperforms static EC under matched FLOPs depends on the equality E[k(r)] = 20. Appendix G.2 proves this equality only under r ~ Uniform(0,1). The paper never states the actual training-time masking-ratio sampler. If the DLM training sampler is non-uniform (or the time-to-mask-ratio map is nonlinear), the expectations differ across schedulers and from the static k=20 baseline, so the lower PPL of dynamic runs could be a capacity effect rather than an allocation effect. Please state the true sampler, compute E[k(r)] under it, and/or report actual FLOPs consumed per run.
  2. [§4.5 / Appendix F.1] The mechanistic claim that the convergence rate ηr (Eq. 2) measures the marginal return of additional expert capacity is asserted in §4.5 without derivation. Moreover, Appendix F.1 reports that linear-reverse and cosine-reverse have nearly identical per-bin ηr (ratio range 0.97–1.04) despite a 0.7 PPL gap, so ηr cannot explain the ordering among reverse schedulers. The mechanism is therefore at best partial. Please add a direct test, e.g., measuring loss reduction per additional expert in each mask-ratio bin, or soften the causal language throughout the abstract and §4.5.
  3. [Tables 2–3, Figures 4–5] All key empirical numbers are single-run with no seeds or error bars. In Table 2, the margin between linear-reverse (36.5) and the next best scheduler (37.1) is small, and at 8B scale the final PPL differences in Figure 5 are on the order of 0.01–0.02. Without variance estimates, "consistently achieves the best performance" and "strictly superior" are not supported. Please provide multiple seeds with error bars for the OpenWebText scheduler comparison, and at minimum report the seed/variance behavior of the 8B runs if additional seeds are infeasible.
minor comments (5)
  1. [§4.5] The magnitude of the learning-efficiency gap is stated inconsistently: Finding 1 says low-mask-ratio bins converge "up to 7× faster," while the Summary says "ηr up to ∼20×." The values in Figure 6 imply roughly a 20–25× ratio at the first stage; please reconcile the numbers.
  2. [Figure 2 caption] The caption refers to "LLaDA-2.0-mini (16B)" but Section 5 experiments use LLaDA-MoE. Clarify the relationship between these models or correct the caption.
  3. [§5 / Table 3] The text claims EC and dynamic EC "match or exceed TC" in accuracy, but on GSM8K TC achieves 74.8% while both EC and dynamic EC achieve 73.8%. Qualify the claim to avoid overstatement.
  4. [Related Work] The in-text citation "Ni & team (2025)" is informal; please use the proper citation for OpenMoE2 from the reference list.
  5. [Appendix C] The token-drop analysis reports a mean drop ratio of 8.0% for dynamic EC, with layer-0 drop ratios of 20–32%. Since dynamic EC reduces capacity at high-mask-ratio steps, please report whether drop ratios are concentrated in those steps and how this interacts with the convergence-rate story.

Circularity Check

0 steps flagged

No significant circularity: core comparisons are measured experiments, the matched-FLOPs identity is conditional but not circular, and the mechanistic account is post hoc rather than load-bearing.

full rationale

The paper's main claims rest on controlled pretraining experiments (EC vs. TC, Table 1/Figure 1; scheduler comparison, Table 2/Figure 4; 8B-A1B scaling, Figure 5; retrofitting, Figure 7), where the routing mechanism or capacity schedule is varied and perplexity/throughput/accuracy are measured. No parameter is fitted to the target result and then renamed a prediction. The 'matched FLOPs' calibration is a mathematical identity computed in Appendix G.2 under an explicitly stated condition: 'under uniform masking ratio r∼Uniform(0,1), the expected top-k is identical across all schedulers and matches the static baseline.' The paper does not state that the actual training-time masking-ratio sampler is uniform, so the matched-FLOPs comparison depends on an unverified premise; this is a correctness/robustness concern, not a circular reduction, because the scheduler ordering itself is not derived from that identity. The mechanistic analysis in Section 4.5 measures convergence rates ηr from the same training runs and interprets them as marginal returns of additional capacity. That is a post hoc explanatory story, and Appendix F.1 explicitly concedes that 'the mechanistic analysis based on ηr cannot explain the PPL gap between them (36.5 vs. 37.2 in Table 2),' which undermines the mechanism but does not constitute a circular derivation. Self-citations appear only in related-work or inference-acceleration contexts (e.g., Zhang et al. 2026, Peng et al. 2026b, Kang et al. 2025) and are not load-bearing for any central claim. Overall, the central empirical results are self-contained and not forced by definition or by a self-citation chain.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 0 invented entities

The central empirical comparisons are experiments, not derivations. The main hand-set knobs are capacity bounds, scheduler shapes, and the Gaussian width; the FLOPs-equivalence proof depends on a uniform-masking assumption, and the mechanistic 'learning efficiency' measure depends on chosen loss bins and training-stage windows. No new physical or architectural entities are introduced beyond the capacity schedule itself.

free parameters (4)
  • k_min, k_max for OpenWebText scheduler comparison = kmin=8, kmax=32
    Hand-chosen bounds for the dynamic capacity schedule; static baseline k=20 is their midpoint. The claim that linear-reverse is best is conditional on these bounds.
  • k_min, k_max for 8B-A1B pretraining = kmin=2, kmax=14
    Hand-chosen for the scaled validation; E[k]=8 matches the static baseline k=8.
  • Gaussian scheduler width sigma = 0.22
    Appendix G.1 sets sigma=0.22 by hand; the Gaussian and Gaussian-reverse scheduler results depend on this choice.
  • Mask-ratio bins and training-stage windows for convergence rate = bins [0,0.25),[0.25,0.5),[0.5,0.75),[0.75,1.0); stages 16K-32K, 32K-64K, 64K-128K, 128K-256K
    The mechanistic conclusion (low-mask bins learn fastest) is measured with these hand-selected partitions; different binning could change the quantitative claims.
axioms (5)
  • domain assumption The training-time masking ratio r is distributed Uniform(0,1).
    Appendix G.2 explicitly assumes r~Uniform(0,1) to prove FLOPs equivalence across schedulers. The paper never states the actual training sampler, so this is load-bearing for the 'matched FLOPs' claim.
  • ad hoc to paper A higher fractional loss-decrease rate eta_r implies a higher marginal return from additional expert capacity.
    Section 4.5 equates learning efficiency with eta_r and asserts concentrating compute on high-eta_r steps yields largest marginal return. This link is asserted, not derived, and Appendix F.1 shows eta_r cannot explain the linear-reverse vs. cosine-reverse gap.
  • domain assumption Dropped tokens in EC lose no information because shared experts process all tokens and per-layer drop events are independent.
    Appendix C argues from shared-expert coverage and an independence product over layers (p ~ 10^-32) that no token is ever completely unprocessed. Independence is an approximation and layer-0 drop ratios are 20-32%.
  • domain assumption The theoretical FLOPs formula Ffwd in Eq. 3 captures the relevant per-step compute for throughput comparisons.
    Table 1 and Table 4 report throughput using this formula. For capacity-bounded TC, the paper notes actual computation is lower than Ffwd, so reported TC throughput is an upper bound; EC throughput may be overestimated relative to actual useful FLOPs.
  • domain assumption A pretrained TC router's score signal remains usable after replacing the router with an EC gate and brief finetuning.
    Section 5 keeps expert weights frozen at conversion and only swaps the router. The success of the retrofit is shown empirically, but there is no analysis of how TC-vs-EC score distributions differ.

pith-pipeline@v1.3.0-alltime-deepseek · 23192 in / 12138 out tokens · 129143 ms · 2026-08-04T05:31:53.846115+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of Expert-Choice Routing Enables Adaptive Computation in Diffusion Language Models." pith.science (2026). https://pith.science/paper/AWRVZGGY

@misc{pith2026260401622,
  author       = {Pith},
  title        = {Pith review of: Expert-Choice Routing Enables Adaptive Computation in Diffusion Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AWRVZGGY}},
  note         = {Machine review of arXiv:2604.01622}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Diffusion language models (DLMs) enable parallel, non-autoregressive text generation, yet existing DLM mixture-of-experts (MoE) models inherit token-choice (TC) routing from autoregressive systems, leading to load imbalance and rigid computation allocation. We show that expert-choice (EC) routing is a better fit for DLMs: it provides deterministic load balancing by design, yielding higher throughput and faster convergence than TC. Building on the property that EC capacity is externally controllable, we introduce timestep-dependent expert capacity, which varies expert allocation according to the denoising step. We find that allocating more capacity to low-mask-ratio steps consistently achieves the best performance under matched FLOPs, and provide a mechanistic explanation: tokens in low-mask-ratio contexts exhibit an order-of-magnitude higher learning efficiency, so concentrating compute on these steps yields the largest marginal return. Finally, we show that existing pretrained TC DLMs can be retrofitted to EC by replacing only the router, achieving faster convergence and improved accuracy across diverse downstream tasks. Together, these results establish EC routing as a superior paradigm for DLM MoE models and demonstrate that computation in DLMs can be treated as an adaptive policy rather than a fixed architectural constant. Code is available at https://github.com/zhangshuibai/EC-DLM.

Figures

Figures reproduced from arXiv: 2604.01622 by Caspian Zhuang, Chihan Cui, Fred Zhangzhi Peng, Guanhua Chen, Haoyue Bai, Ming Liu, Shuibai Zhang, Yang Zhou, Yanxin Zhang, Zack Jia, Zhihan Yang.

Figure 1
Figure 1. Figure 1: Training loss vs. wall-clock time. EC reaches loss 3.75 in 10.6h, 2.0× faster than TC (20.7h). Diffusion language models (DLMs) have emerged as a promising alternative to au￾toregressive generation, enabling parallel decoding through iterative denoising (Nie et al., 2025; Ye et al., 2025; Arriola et al., 2025; Liu et al., 2025b;a). Scaling lan￾guage models to hundreds of billions of pa￾rameters increasingl… view at source ↗
Figure 2
Figure 2. Figure 2: Left: TC (top-1) vs. EC (capacity c = 2) routing on a 6×3 gating score matrix. Both methods assign the same total of 6 token–expert pairs, but TC produces imbalanced per-expert loads (1/4/1) while EC guarantees uniform loads (2/2/2) by construction. Right: GPU memory snapshot during inference of LLaDA-2.0-mini (16B) with expert parallelism across 8 H100 GPUs. TC exhibits high variance (std 3.6 GB) with one… view at source ↗
Figure 3
Figure 3. Figure 3: Linear-reverse scheduling: as mask ratio [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Scheduler comparison on OpenWebText (30B tokens, matched average FLOPs). [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: 8B-A1B pretraining comparison: dynamic EC (linear-reverse, [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Mechanistic analysis of learning efficiency (8B-A1B, Nemotron-CC). [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Retrofitting a pretrained TC DLM (LLaDA-MoE) to EC and dynamic EC across [PITH_FULL_IMAGE:figures/full_fig_p008_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Per-layer token drop ratio at step 300K for static and dynamic EC (8B-A1B, [PITH_FULL_IMAGE:figures/full_fig_p019_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: TC routing ablations. (a) Removing or replacing the auxiliary load-balancing loss does not close the gap with EC (1.7× faster). (b) Increasing the capacity factor slows TC further due to padding overhead (2.0× faster for EC vs. TC cap=1.5). E.1 Auxiliary Loss The TC baselines in §3 use an auxiliary load-balancing loss to encourage uniform expert utilization. A natural question is whether this auxiliary los… view at source ↗
Figure 10
Figure 10. Figure 10: Per-step training loss. The auxiliary load-balancing loss improves per-step [PITH_FULL_IMAGE:figures/full_fig_p021_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Convergence rate comparison between linear-reverse and cosine-reverse (both [PITH_FULL_IMAGE:figures/full_fig_p023_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: SFT retrofitting results vs. number of training samples for TC, EC, and dynamic [PITH_FULL_IMAGE:figures/full_fig_p024_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: TC routing ablation: comparing four TC variants (no [PITH_FULL_IMAGE:figures/full_fig_p025_13.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Fixed-Point Masked Generative Modeling

    cs.LG 2026-05 unverdicted novelty 6.0

    FP-MGMs with consistency loss and three-state reuse (CoFRe) reduce parameters by up to 38.8% and improve low-budget perplexity and FID versus standard masked generative models on text and images.

  2. dMoE: dLLMs with Learnable Block Experts

    cs.CL 2026-05 unverdicted novelty 6.0

    dMoE aggregates token expert distributions to block level in dLLMs, cutting unique experts from 69.5 to 14.6, memory by 76-80%, and latency by 1.14-1.66x while retaining 99.11% performance.

Reference graph

Works this paper leans on

7 extracted references · 3 linked inside Pith · cited by 2 Pith papers

  1. [2]

    ✓ Cosine / Cosine-reverse

    Thus E[k(r)] = 8 + 24 × 0.5 = 20.00. ✓ Cosine / Cosine-reverse. E h 1 2 (1−cosπr) i = 1 2 − 1 2 R 1 0 cos(πr)dr= 1 2 − 1 2 · sinπr π 1 0 = 1 2. ThusE[k(r)] =8+24×0.5=20.00. The reverse case follows identically.✓ Gaussian / Gaussian-reverse.For the normalized Gaussian ˜g(r), a closed-form expecta- tion is not available, but the symmetry ˜g(r) = ˜g(1 −r) co...

    arXiv 2000

  2. [5]

    Learning rate 2×10−4

    (50/50 blend of High-Quality and Diverse-QA splits). Learning rate 2×10−4. TC and EC models are identical except for the routing mechanism; all other hyperparameters are shared. Scheduler comparison on OpenWebText (§4).16 Transformer layers, hidden size 512, 16 attention heads, 512 fine-grained experts (ffn hidden size 384), 2 shared experts (shared ffn h...

    2048

  3. [70]

    URLhttps://aclanthology.org/2024.acl-long.70/. DeepSeek-AI, Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Daya Guo, Dejian Yang, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang, Han Bao, Hanwei Xu, ...

    Pith/arXiv arXiv 2024

  4. [2021]

    Zigeng Chen, Gongfan Fang, Xinyin Ma, Ruonan Yu, and Xinchao Wang

    URLhttps://arxiv.org/abs/2107.03374. Zigeng Chen, Gongfan Fang, Xinyin Ma, Ruonan Yu, and Xinchao Wang. dparallel: Learn- able parallel decoding for dLLMs. InThe Fourteenth International Conference on Learning Representations, 2026. URLhttps://openreview.net/forum?id=hVOcstAURb. 11 Preprint. Under review. Kun Cheng, Xiao He, Lei Yu, Zhijun Tu, Mingrui Zhu...

    Pith/arXiv arXiv 2026

  5. [2024]

    All of these prior methods target continuous diffusion for vision

    and DyDiT (Zhao et al., 2025) dynamically adjust model width and depth per timestep, while MoE-based approaches target vision diffusion transformers: EC-DIT (Sun et al., 2025) shows that EC routing implicitly adapts to timesteps, DiffMoE (Shi et al., 2025) learns a capacity predictor for per-timestep allocation, and Diff-MoE (Cheng et al., 2025) injects t...

    2025

  6. [2025]

    William Fedus, Barret Zoph, and Noam Shazeer

    URLhttps://openreview.net/forum?id=2edigk8yoU. William Fedus, Barret Zoph, and Noam Shazeer. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity, 2022. URL https://arxiv.org/abs/ 2101.03961. Jonas Geiping, Sean Michael McLeish, Neel Jain, John Kirchenbauer, Siddharth Singh, Brian R. Bartoldson, Bhavya Kailkhura, Ab...

    Pith/arXiv arXiv 2022

  7. [2026]

    Haotian Sun, Tao Lei, Bowen Zhang, Yanghao Li, Haoshuo Huang, Ruoming Pang, Bo Dai, and Nan Du

    URLhttps://arxiv.org/abs/2603.11535. Haotian Sun, Tao Lei, Bowen Zhang, Yanghao Li, Haoshuo Huang, Ruoming Pang, Bo Dai, and Nan Du. EC-DIT: Scaling diffusion transformers with adaptive expert-choice routing. InThe Thirteenth International Conference on Learning Representations, 2025. URL https: //openreview.net/forum?id=PxlfzEePC0. Shengkun Tang, Yaqing ...

    arXiv 2025