Pith. sign in

REVIEW 4 major objections 4 minor 1 cited by

In decentralized diffusion models, routing inputs to experts trained on similar data determines generation quality, not numerical stability: the most stable ensemble strategy produces the worst samples (FID 47.9) while sparse Top-2 routing

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review

2026-08-03 05:18 UTC pith:26UGG72W

load-bearing objection Empirical dissociation likely real, but the alignment claim rests on a circular validation and a missing control. the 4 major comments →

arxiv 2602.02685 v3 pith:26UGG72W submitted 2026-02-02 cs.LG

Expert-Data Alignment Governs Generation Quality in Decentralized Diffusion Models

Marcos Villagra , Bidhan Roy , Raihan Seraj , Zhiying Jiang This is my paper
classification cs.LG
keywords decentralized diffusion modelsexpert-data alignmentroutingtrajectory sensitivitystability-quality dissociationmixture of expertsdiffusion sampling
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper asks what controls sample quality in decentralized diffusion models (DDMs), where several diffusion models are trained independently on disjoint data clusters and combined at inference by a router. The intuitive answer—that minimizing trajectory sensitivity should yield better samples—is shown to be wrong: the full ensemble of all experts is the most numerically stable and has the lowest step-refinement disagreement, yet produces the worst FID (47.9), while sparse Top-2 routing is less stable but yields the best FID (22.6). The authors argue instead that expert-data alignment is the governing principle: quality depends on sending each denoising state to experts whose training distribution covers that state. They support this with three analyses: cluster-distance ranking, per-expert velocity alignment, and a disagreement–quality correlation, replicated on a second MNIST-based system.

Core claim

The central claim is a dissociation: numerical stability does not govern generation quality in decentralized diffusion. Full ensemble routing—which averages all expert velocity predictions—has the lowest effective Lipschitz constant along trajectories and the smallest refinement disagreement, yet the worst generation quality; sparse Top-2 routing, with higher sensitivity, produces the best samples. The governing factor is expert-data alignment: routing inputs to experts trained on similar data. Evidence includes cluster-distance analysis (selected experts' training clusters rank close to the current input, mean rank 1.96 vs 4.5 random), per-expert analysis (selected experts are 29% closer in

What carries the argument

The central object is the routed velocity field v_t(x_t) = Σ_k w_k(t)(x_t) v_k(t)(x_t), where w_k are router weights over independently trained diffusion experts. The paper's key conceptual identity is a decomposition of the flow Jacobian into an expert term Σ_k w_k ∇_x v_k and a router term Σ_k v_k ∇_x w_k; since the router term is two to four orders of magnitude larger but nearly identical across routing strategies, the authors use the expert term as the sensitivity measure. The governing notion is expert-data alignment, operationalized by three measurable quantities: the rank of a selected expert's training-cluster centroid in a learned visual-embedding space, the angular deviation betwee

Load-bearing premise

The dissociation rests on measuring trajectory sensitivity with the expert-only Jacobian norm; if the full Jacobian (whose router-gradient term is 2–4 orders larger and slightly favors Top-2 over full ensemble) is the proper sensitivity measure, the full ensemble may no longer be the most stable and the stability–quality dissociation collapses.

What would settle it

Compute the effective Lipschitz constant along sampled trajectories using the full Jacobian including the router term (rather than the expert-only term) for Top-1, Top-2, and full ensemble. If full ensemble is not the lowest, the dissociation evaporates. Alternatively, run FID on 'misaligned Top-2' (random expert selection preserving sparsity): if FID stays near 22.6, alignment is not the governing factor.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • Practitioners should route toward experts whose training distribution covers the current denoising state; optimizing numerical stability metrics alone can actively hurt sample quality.
  • Sparse routing (Top-2) delivers the best quality with 4× fewer active experts, so alignment-based sparsity yields both better FID and lower compute.
  • Full ensemble underperforms not because it is unstable but because it forces most experts to process out-of-distribution inputs; improving expert OOD robustness may rescue ensemble strategies.
  • Within a fixed routing strategy, trajectory-local sensitivity can still flag numerically sensitive samples, though it is not a cross-strategy quality predictor.
  • Expert disagreement can serve as a cheap quality monitor during decentralized inference: when selected experts disagree, expected quality drops.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A direct testable extension would train the router with an explicit alignment objective (e.g., maximizing predicted per-expert velocity alignment) and check whether FID improves independently of sparsity; the paper's analysis suggests it should.
  • The stability–quality dissociation may generalize to other ensemble generative models (e.g., mixture-of-experts diffusion transformers), where the same mechanism—forcing all experts to process every token—could explain quality losses despite smooth activations.
  • A practical diagnostic for decentralized serving: compute trajectory-integrated expert disagreement at sampling time and use it to trigger a fallback (e.g., more steps or a different router) when disagreement spikes.
  • The paper's counterfactual 'misaligned Top-2' is reported only for refinement error; testing FID under random expert selection would separate the effect of sparsity from alignment, a clean next experiment.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper studies inference-time routing in Decentralized Diffusion Models (DDMs), where independently trained experts are combined by a router. It claims a stability–quality dissociation: full ensemble routing has the lowest trajectory sensitivity and step-refinement disagreement but the worst FID, while sparse Top-2 routing has the best FID despite higher sensitivity. The paper proposes that expert-data alignment — routing inputs to experts trained on similar data — is the governing principle of generation quality, and it attempts to validate this via cluster-distance analysis, per-expert prediction alignment, expert-disagreement analysis, and a separate MNIST-based DDM. It also formalizes a trajectory-local sensitivity measure and gives a conditional convergence argument. The central claim is plausible and counterintuitive, but the current evidence is weakened by several circular or incomplete validation steps.

Significance. If the dissociation and alignment principle were fully established, the result would be practically valuable: it would tell DDM practitioners to prioritize alignment-aware sparse routing over stability optimization, and it would challenge a common numerical-stability intuition for generative ODEs. The paper has real strengths: it works on two DDM systems, includes a formal conditional convergence proposition (Prop. B.1), openly acknowledges the retrospective nature of its sensitivity diagnostic (Remark 6.3), and provides a useful Jacobian decomposition (Table 6). However, the main empirical validations are partly circular or lack the natural control, so the manuscript currently supports a weaker statement than its title and conclusions claim.

major comments (4)
  1. [§5.2, Table 1] The cluster-distance validation is circular. The manuscript states that the Paris router was 'trained post-hoc on the full dataset, effectively learning to route inputs to the expert trained on the most similar data.' Therefore the finding that Top-1/Top-2 select clusters with mean ranks 1.54/1.96 is largely a restatement of the router's training objective, not an independent confirmation of expert-data alignment. This measurement cannot serve as direct evidence for the paper's central claim unless compared against a router that was not trained to match data proximity.
  2. [§F.2, Table 8; Table 4] The paper's central causal claim is untested. The headline quality comparison (Top-2 FID 22.60 vs. Full ensemble 47.89, Table 4) varies both sparsity and alignment simultaneously. The natural control is Misaligned Top-2 (random expert selection), which preserves sparsity while breaking proximity-based alignment, but Table 8 reports only Δrefine for this condition (0.040±0.035), not FID or any perceptual quality metric. If Misaligned Top-2's FID is close to aligned Top-2's 22.60, the FID gap is due to sparsity/averaging rather than alignment; if close to 47.89, the alignment claim is supported. Without this number, the 'governs' formulation is unsupported. Additionally, Table 4 and Table 1 show that Top-1 achieves better mean cluster rank (1.54) than Top-2 (1.96) but worse FID (30.60 vs. 22.60), so the alignment-quality relationship is not monotone and the claimed mechanism needs a sharpe
  3. [§D.3, Table 6; Table 4; Definition 6.1] The reported bLeff in Table 4 does not match Definition 6.1. Definition 6.1 defines the effective Lipschitz constant for the full routed flow v_t = Σ_k w_k v_k, i.e., using ||J_x v_t||. Appendix D.3 reports only the expert term ||Σ_k w_k ∇_x v_k|| and states that the router term ||Σ_k v_k ∇_x w_k|| is 2–4 orders of magnitude larger (Top-2: 923±1.4K; Full: 1161±2.1K). With the full Jacobian, the Full ensemble does not have the lowest mean sensitivity; the router-term mean is higher for Full than for Top-2. The claimed stability ordering in Table 4 therefore depends entirely on omitting the dominant term. The authors need to justify why the expert-only Jacobian is the relevant sensitivity measure for cross-strategy quality comparisons, or report both quantities and show that the dissociation is robust to this modeling choice.
  4. [§5.3, Table 2] The per-expert prediction-quality metric is partially circular. The 'blended velocity' v_t is defined as the weighted combination of the selected experts, so a selected expert is guaranteed to have nonzero influence on the reference vector. The finding that selected experts have smaller angular deviation from the blended velocity (3.6° vs. 5.1°) may reflect the weighting scheme rather than intrinsic prediction quality. An independent ground-truth reference, such as the clean-data velocity or per-expert contribution to final sample quality, is needed to support the claim that routing systematically identifies more accurate experts.
minor comments (4)
  1. [§6.3 footnote] The preliminary analysis of the router Jacobian term reports ρ = -0.07 with p = 0.62 and n = 50. This is severely underpowered and should not be used to conclude that router-term sensitivity is uninformative; either report a properly powered analysis or remove the claim.
  2. [Table 4 and Tables 7–8] It is unclear whether the bLeff values in Table 4 are full Jacobian norms or expert-only norms; Appendix D.3 says expert-only, but Section 6.2.2 and Definition 6.1 suggest full Jacobian. Also, baseline bLeff for Full ensemble in Table 7 (15.97±5.51) differs from Table 4 (17.07±6.33) despite both being 'baseline'; with n=50 vs n=1000 this may be sampling noise, but the inconsistency should be explained.
  3. [§5.6] The sentence 'this correlation provides causal evidence' overstates the design: the disagreement–quality analysis is correlational and uses LPIPS to the Top-2 output as a reference. Please soften the causal language or add an explicit causal manipulation (e.g., Misaligned Top-2 with FID).
  4. [Figure 1] The y-axis label 'LPIPS to Top-2' makes clear that the reference is the Top-2 output. This is a reasonable relative-quality reference, but it should be stated as a relative degradation measure, not as an absolute quality metric, throughout the text.

Circularity Check

3 steps flagged

Two of three alignment validations are built into the router's training objective or the definition of the blended velocity; the disagreement-quality metric takes Top-2 as its quality reference.

specific steps
  1. fitted input called prediction [Section 5.2 (Cluster Distance Analysis), Table 1]
    "The router was trained post-hoc on the full dataset, effectively learning to route inputs to the expert trained on the most similar data. ... We test whether sparse routing selects experts whose training clusters match the input. ... Top-1 and Top-2 achieve mean cluster ranks of 1.54 and 1.96, far below the 4.5 random baseline... Full (8) 4.50±0.00 25.0%"

    The cluster-distance result is the router's own training objective. Paris's router was trained post-hoc to send each input to the expert trained on the most similar data, so observing that Top-1/Top-2 select the closest clusters is a check that the router learned its objective, not independent evidence for a governing quality principle. The Full(8) baseline is also analytic for a uniformly random expert (mean rank of a uniform rank is 4.5; chance of being in the two closest is 2/8=25%), so the contrast is definitional.

  2. self definitional [Section 5.3 (Per-Expert Prediction Quality), Eq. (1), Table 2]
    "For each expert k at each timestep, we compute the velocity alignment score ... a(k)(x_t)= (v_t(k)^T · v_t)/(||v_t(k)|| ||v_t||) ... measures how well expert k's prediction aligns with the routed velocity used for successful generation."

    The reference 'routed velocity' v_t is defined in Eq. (1) as the weighted sum over the selected set S: v_t = Σ_{k∈S} w_k v_k. A selected expert's prediction is one of the summands of the reference vector; a non-selected expert's prediction is absent from it. The reported 3.6° vs 5.1° angular-deviation reduction is therefore mechanically favored for selected experts by construction and does not independently establish that routing identifies more coherent experts.

  3. self definitional [Section 5.4 (Expert Disagreement and Sample Quality), Figure 1]
    "perceptual quality as LPIPS distance [29] to the corresponding Top-2 output (matched initial noise). ... The monotonic increase in LPIPS across disagreement quartiles confirms that expert disagreement drives quality degradation in full ensemble routing."

    Quality degradation is defined as LPIPS distance to the Top-2 output, the very sparse-routing output whose superiority the alignment principle is meant to explain. Full ensemble's 'degradation' is thus measured against the reference under investigation. The monotonic trend across disagreement quartiles is empirical, but the metric already encodes that closeness to Top-2 is the definition of quality, making part of the explanatory conclusion a matter of definition.

full rationale

The paper's genuinely empirical core is the stability-quality dissociation (Table 4): Full ensemble has the lowest bLeff and Δrefine yet worst FID (47.89 vs 22.60). That FID contrast is not reduced to the inputs by construction, and the paper explicitly uses an independent error signal (Δrefine, Eq. 2) and even flags the retrospective nature of bLeff in Remark 6.3. However, the three supporting validations of the alignment principle are not independent. Table 1 validates that sparse routing selects the closest clusters, which is exactly what the post-hoc-trained router was trained to do; Table 2 compares angular deviations to a blended velocity that contains the selected experts' own vectors; and Figure 1 defines full-ensemble quality degradation as distance to Top-2 output, the target being explained. The paper also includes the natural causal control (Misaligned Top-2 random selection, Appendix F.2, Table 8) but reports only Δrefine, not FID or perceptual quality, so the causal role of alignment relative to sparsity remains untested; that is a correctness limitation rather than a circularity. Overall, the central claim is partially supported by the external FID dissociation, but the alignment mechanism is validated substantially by construction, warranting a 6 rather than a higher score.

Axiom & Free-Parameter Ledger

3 free parameters · 5 axioms · 0 invented entities

The central claim depends on the Paris router's training objective, on DINOv2 cluster distance as a proxy for alignment, on ignoring the dominant router Jacobian term in sensitivity comparisons, and on using LPIPS-to-Top-2 as a quality reference. These are domain assumptions or ad hoc choices rather than fitted parameters; only threshold choices count as free parameters. No new physical entities or mediators are introduced.

free parameters (3)
  • tau_refine (high-Δrefine threshold) = 99th percentile of Top-2 Δrefine
    Used to define 'high Δrefine' events for failure prediction and AUC; chosen post hoc on Top-2 runs and then fixed across methods (Appendix D.1).
  • Power method convergence tolerance = 0.5% relative change between iterations 9 and 10
    Stopping criterion for Jacobian spectral norm estimation; directly affects the reported bLeff values (Section 6.2.2).
  • Top-p truncation p* = 0.9
    Best variant in full-ensemble rescue attempts; not central to the main claim but part of the inference-time modifications tested (Appendix F.4).
axioms (5)
  • domain assumption The Paris router was trained post hoc on the full dataset so that it routes inputs to the expert trained on the most similar data.
    Section 5.2 states this. If true, the cluster-distance validation in Table 1 is guaranteed by the router's training objective rather than being independent evidence.
  • domain assumption DINOv2-ViT-L/14 embedding Euclidean distance to training-cluster centroids measures expert-data alignment.
    Used throughout Section 5.2 to rank cluster proximity. The paper claims this is the canonical embedding because clusters were built in DINOv2 space, but it remains an assumption that this geometric proxy captures the alignment relevant to generation quality.
  • ad hoc to paper The expert-only Jacobian term ||sum_k w_k grad v_k|| is the relevant trajectory sensitivity for comparing routing strategies; the router term ||sum_k v_k grad w_k|| can be ignored.
    Appendix D.3/Table 6: the router term is 2-4 orders of magnitude larger, yet Table 4 reports bLeff computed only on the expert term. The stability ordering depends on this modeling choice.
  • ad hoc to paper LPIPS distance to the Top-2 output is a valid measure of full-ensemble quality degradation.
    Section 5.4 uses LPIPS to the Top-2 reference as 'perceptual quality'; this treats the method being advocated as the quality reference.
  • standard math Standard Grönwall-based global error bounds for one-step ODE solvers hold under trajectory-local Lipschitz bounds.
    Used in Proposition B.1. The textbook bound typically requires Lipschitz conditions around the numerical trajectory or a neighborhood of the exact solution, not only along the exact trajectory; the paper does not discuss this gap.

reviewed 2026-08-03 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Expert-Data Alignment Governs Generation Quality in Decentralized Diffusion Models." pith.science (2026). https://pith.science/paper/26UGG72W

@misc{pith2026260202685,
  author       = {Pith},
  title        = {Pith review of: Expert-Data Alignment Governs Generation Quality in Decentralized Diffusion Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/26UGG72W}},
  note         = {Machine review of arXiv:2602.02685}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Decentralized Diffusion Models (DDMs) route denoising through experts trained independently on disjoint data clusters, which can strongly disagree in their predictions. What governs the quality of generations in such systems? We present the first ever systematic investigation of this question. A priori, the expectation is that minimizing denoising trajectory sensitivity -- minimizing how perturbations amplify during sampling -- should govern generation quality. We demonstrate this hypothesis is incorrect: a stability-quality dissociation. Full ensemble routing, which combines all expert predictions at each step, achieves the most stable sampling dynamics and best numerical convergence while producing the worst generation quality (FID 47.9 vs. 22.6 for sparse Top-2 routing). Instead, we identify expert-data alignment as the governing principle: generation quality depends on routing inputs to experts whose training distribution covers the current denoising state. Across two distinct DDM systems, we validate expert-data alignment using (i) data-cluster distance analysis, confirming sparse routing selects experts with data clusters closest to the current denoising state, and (ii) per-expert analysis, showing selected experts produce more accurate predictions than non-selected ones, and (iii) expert disagreement analysis, showing quality degrades when experts disagree. For DDM deployment, our findings establish that routing should prioritize expert-data alignment over numerical stability metrics.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Paris 2.0: A Decentralized Diffusion Model for Video Generation

    cs.CV 2026-05 unverdicted novelty 7.0

    Paris 2.0 is the first decentralized diffusion model for text-to-video generation and reports roughly 2x lower FVD than a monolithic baseline under matched total compute.

Reference graph

Works this paper leans on

31 extracted references · 11 linked inside Pith · cited by 1 Pith paper

  1. [1]

    Diff-MoE: Diffusion transformerwithtime-awareandspace-adaptiveexperts

    Cheng, K., He, X., Yu, L., Tu, Z., Zhu, M., Wang, N., Gao, X., and Hu, J. Diff-MoE: Diffusion transformerwithtime-awareandspace-adaptiveexperts. InForty-secondInternationalConference on Machine Learning, 2025

    2025

  2. [2]

    A., Levinson, N., and Teichmann, T

    Coddington, E. A., Levinson, N., and Teichmann, T. Theory of ordinary differential equations, 1956

    1956

  3. [3]

    StableMoE: Stable routing strategy for mixture of experts.arXiv preprint arXiv:2204.08396, 2022

    Dai, D., Dong, L., Ma, S., Zheng, B., Sui, Z., Chang, B., and Wei, F. StableMoE: Stable routing strategy for mixture of experts.arXiv preprint arXiv:2204.08396, 2022

    Pith/arXiv arXiv 2022

  4. [4]

    Efficient and accurate estimation of lipschitz constants for deep neural networks.Advances in neural information processing systems, 32, 2019

    Fazlyab, M., Robey, A., Hassani, H., Morari, M., and Pappas, G. Efficient and accurate estimation of lipschitz constants for deep neural networks.Advances in neural information processing systems, 32, 2019

    2019

  5. [5]

    Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity.Journal of Machine Learning Research, 23(120):1–39, 2022

    Fedus, W., Zoph, B., and Shazeer, N. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity.Journal of Machine Learning Research, 23(120):1–39, 2022

    2022

  6. [6]

    Scaling diffusion transformers to 16 billion parameters.arXiv preprint arXiv:2407.11633, 2024

    Fei, Z., Fan, M., Yu, C., Li, D., and Huang, J. Scaling diffusion transformers to 16 billion parameters.arXiv preprint arXiv:2407.11633, 2024

    Pith/arXiv arXiv 2024

  7. [7]

    P.Solving ordinary differential equations I: Nonstiff problems

    Hairer, E., Wanner, G., and Nørsett, S. P.Solving ordinary differential equations I: Nonstiff problems. Springer, 1993

    1993

  8. [8]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium.Advances in neural information processing systems, 30, 2017

    Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., and Hochreiter, S. Gans trained by a two time-scale update rule converge to a local nash equilibrium.Advances in neural information processing systems, 30, 2017

    2017

  9. [9]

    and Salimans, T

    Ho, J. and Salimans, T. Classifier-free diffusion guidance.arXiv preprint arXiv:2207.12598, 2022

    Pith/arXiv arXiv 2022

  10. [10]

    Denoising diffusion probabilistic models.Advances in neural information processing systems, 33:6840–6851, 2020

    Ho, J., Jain, A., and Abbeel, P. Denoising diffusion probabilistic models.Advances in neural information processing systems, 33:6840–6851, 2020

    2020

  11. [11]

    Paris: A decentralized trained open-weight diffusion model.arXiv preprint arXiv:2510.03434, 2025

    Jiang, Z., Seraj, R., Villagra, M., and Roy, B. Paris: A decentralized trained open-weight diffusion model.arXiv preprint arXiv:2510.03434, 2025

    Pith/arXiv arXiv 2025

  12. [12]

    and Dimakis, A

    Jordan, M. and Dimakis, A. G. Exactly computing the local lipschitz constant of relu networks. Advances in Neural Information Processing Systems, 33:7344–7353, 2020

    2020

  13. [13]

    Elucidating the design space of diffusion-based generative models.Advances in neural information processing systems, 35:26565–26577, 2022

    Karras, T., Aittala, M., Aila, T., and Laine, S. Elucidating the design space of diffusion-based generative models.Advances in neural information processing systems, 35:26565–26577, 2022

    2022

  14. [14]

    K., Zaheer, M., Sanjabi, M., Talwalkar, A., and Smith, V

    Li, T., Sahu, A. K., Zaheer, M., Sanjabi, M., Talwalkar, A., and Smith, V. Federated optimization in heterogeneous networks.Proceedings of Machine learning and systems, 2:429–450, 2020

    2020

  15. [15]

    T., Ben-Hamu, H., Nickel, M., and Le, M

    Lipman, Y., Chen, R. T., Ben-Hamu, H., Nickel, M., and Le, M. Flow matching for generative modeling.arXiv preprint arXiv:2210.02747, 2022. 13 Expert-Data Alignment Governs Generation Quality in Decentralized Diffusion Models

    Pith/arXiv arXiv 2022

  16. [16]

    Efficient training of diffusion mixture-of-experts models: A practical recipe.arXiv preprint arXiv:2512.01252, 2025

    Liu, Y., Yue, Y., Zhang, J., Sun, C., Zhou, Y., Zeng, W., Tang, R., and Zhou, G. Efficient training of diffusion mixture-of-experts models: A practical recipe.arXiv preprint arXiv:2512.01252, 2025

    arXiv 2025

  17. [17]

    Decentralized diffusion models.Proceed- ings of the Computer Vision and Pattern Recognition Conference, pp

    McAllister, D., Tancik, M., Song, J., and Kanazawa, A. Decentralized diffusion models.Proceed- ings of the Computer Vision and Pattern Recognition Conference, pp. 23323–23333, 2025

    2025

  18. [18]

    Dinov2: Learning robust visual features without supervision.arXiv preprint arXiv:2304.07193, 2023

    Oquab, M., Darcet, T., Moutakanni, T., Vo, H., Szafraniec, M., Khalidov, V., Fernandez, P., Haziza, D., Massa, F., El-Nouby, A., et al. Dinov2: Learning robust visual features without supervision.arXiv preprint arXiv:2304.07193, 2023

    Pith/arXiv arXiv 2023

  19. [19]

    and Xie, S

    Peebles, W. and Xie, S. Scalable diffusion models with transformers. InProceedings of the IEEE/CVF international conference on computer vision, pp. 4195–4205, 2023

    2023

  20. [20]

    Laion-5b: An open large-scale dataset for training next generation image-text models.Advances in neural information processing systems, 35: 25278–25294, 2022

    Schuhmann, C., Beaumont, R., Vencu, R., Gordon, C., Wightman, R., Cherti, M., Coombes, T., Katta, A., Mullis, C., Wortsman, M., et al. Laion-5b: An open large-scale dataset for training next generation image-text models.Advances in neural information processing systems, 35: 25278–25294, 2022

    2022

  21. [21]

    Outra- geously large neural networks: The sparsely-gated mixture-of-experts layer.arXiv preprint arXiv:1701.06538, 2017

    Shazeer, N., Mirhoseini, A., Maziarz, K., Davis, A., Le, Q., Hinton, G., and Dean, J. Outra- geously large neural networks: The sparsely-gated mixture-of-experts layer.arXiv preprint arXiv:1701.06538, 2017

    Pith/arXiv arXiv 2017

  22. [22]

    DiffMoE: Dynamic token selection for scalable diffusion transformers.arXiv preprint arXiv:2503.14487, 2025

    Shi, M., Yuan, Z., Yang, H., Wang, X., Zheng, M., Tao, X., Zhao, W., Zheng, W., Zhou, J., Lu, J., et al. DiffMoE: Dynamic token selection for scalable diffusion transformers.arXiv preprint arXiv:2503.14487, 2025

    Pith/arXiv arXiv 2025

  23. [23]

    P., Kumar, A., Ermon, S., and Poole, B

    Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Ermon, S., and Poole, B. Score-based generative modeling through stochastic differential equations.arXiv preprint arXiv:2011.13456, 2020

    Pith/arXiv arXiv 2011

  24. [24]

    Ec-dit: Scaling diffusion transformers with adaptive expert-choice routing.arXiv preprint arXiv:2410.02098, 2024

    Sun, H., Lei, T., Zhang, B., Li, Y., Huang, H., Pang, R., Dai, B., and Du, N. Ec-dit: Scaling diffusion transformers with adaptive expert-choice routing.arXiv preprint arXiv:2410.02098, 2024

    Pith/arXiv arXiv 2024

  25. [25]

    L., and Ryu, E

    Tan, Z., Wang, W., Bertozzi, A. L., and Ryu, E. K. Stork: Faster diffusion and flow matching sampling by resolving both stiffness and structure-dependence.arXiv:2505.24210, 2025

    arXiv 2025

  26. [26]

    and Scaman, K

    Virmaux, A. and Scaman, K. Lipschitz regularity of deep neural networks: analysis and efficient estimation.Advances in Neural Information Processing Systems, 31, 2018

    2018

  27. [27]

    Lipschitz singularities in diffusion models

    Yang, Z., Feng, R., Zhang, H., Shen, Y., Zhu, K., Huang, L., Zhang, Y., Liu, Y., Zhao, D., Zhou, J., et al. Lipschitz singularities in diffusion models. InThe Twelfth International Conference on Learning Representations, 2023

    2023

  28. [28]

    Expert race: A flexible routing strategy for scaling diffusion transformer with mixture of experts.arXiv preprint arXiv:2503.16057, 2025

    Yuan, Y., Wang, Z., Huang, Z., Zhu, D., Zhou, X., Yu, J., and Min, Q. Expert race: A flexible routing strategy for scaling diffusion transformer with mixture of experts.arXiv preprint arXiv:2503.16057, 2025

    Pith/arXiv arXiv 2025

  29. [29]

    A., Shechtman, E., and Wang, O

    Zhang, R., Isola, P., Efros, A. A., Shechtman, E., and Wang, O. The unreasonable effectiveness of deep features as a perceptual metric. InProceedings of the IEEE conference on computer vision and pattern recognition, pp. 586–595, 2018

    2018

  30. [30]

    Dense2MoE: Restructuring diffusion transformer to moe for efficient text-to-image generation

    Zheng, Y., Ren, Y., Xia, X., Xiao, X., and Xie, X. Dense2MoE: Restructuring diffusion transformer to moe for efficient text-to-image generation. InProceedings of the IEEE/CVF International Conference on Computer Vision, pp. 18661–18670, 2025

    2025

  31. [31]

    M., Le, Q

    Zhou, Y., Lei, T., Liu, H., Du, N., Huang, Y., Zhao, V., Dai, A. M., Le, Q. V., Laudon, J., et al. Mixture-of-experts with expert choice routing.Advances in Neural Information Processing Systems, 35:7103–7114, 2022. Appendix 14 Expert-Data Alignment Governs Generation Quality in Decentralized Diffusion Models A. When b𝐿(ℎ) eff approximates𝐿 eff Assume the...

    2022

This paper was first reviewed by deepseek-v4-flash on August 3, 2026.