Pith. sign in

REVIEW 5 major objections 4 minor 44 references

Diffusion experts trained in isolation with different objectives can be unified at inference by a deterministic velocity conversion, and the mixed ensemble beats homogeneous baselines in FID and diversity.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

A decentralized diffusion framework that mixes DDPM and Flow Matching experts, converts all predictions to velocity space at inference, and claims 16x less compute and 14x less data than prior DDM work.

T0 review reviewed 2026-08-04 challenge →

load-bearing objection The conversion recipe is real, but the headline heterogeneity gain rests on a single run and a cherry-picked cluster assignment; worth peer review with a request for seeds and ablations. the 5 major comments →

arxiv 2603.06741 v3 pith:FYPQ6U6Q submitted 2026-03-06 cs.LG cs.AIcs.CV

Heterogeneous Decentralized Diffusion Models

Zhiying Jiang , Raihan Seraj , Marcos Villagra , Bidhan Roy This is my paper
classification cs.LG cs.AIcs.CV
keywords decentralized diffusion modelsheterogeneous objectivesflow matchingDDPMinference-time conversionexpert routingtext-to-image generationefficient training
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether decentralized diffusion training can work when different experts use different training objectives, not just the same one. It argues yes: a DDPM expert's epsilon prediction can be converted into a flow-matching velocity prediction by differentiating the forward schedule, and the converted velocities can be fused by a learned router. On an aesthetics-filtered image benchmark, an 8-expert ensemble with two DDPM and six flow-matching experts achieves better FID (11.88 vs. 12.45) and higher intra-prompt diversity than a homogeneous flow-matching ensemble, at a 16x reduction in compute and 14x reduction in data versus prior decentralized diffusion training. If correct, this makes heterogeneous, low-budget, single-GPU participation in foundation-model training practical.

Core claim

The central claim is that heterogeneous diffusion objectives are not a liability but an advantage: experts trained in complete isolation—some with DDPM epsilon-prediction, some with Flow Matching velocity-prediction—can be combined at inference without retraining, provided each prediction is mapped into a shared velocity space by the deterministic conversion v = (dα_t/dt) x̂_0 + (dσ_t/dt) ε_θ, built from the forward process x_t = α_t x_0 + σ_t ε. Under aligned inference settings, the mixed 2DDPM:6FM configuration improves FID from 12.45 to 11.88 and mean pairwise LPIPS from 0.617 to 0.631 relative to 8FM, and the paper attributes the complementarity to an implicit timestep weighting: velocit

What carries the argument

The load-bearing identity is Eq. 7: v(x_t,t) = (dα_t/dt) x̂_0 + (dσ_t/dt) ε_θ, where x̂_0 = (x_t − σ_t ε_θ)/α_t. This deterministically converts any DDPM epsilon prediction into a flow-matching velocity for an arbitrary schedule, letting a router fuse DDPM and FM experts as u_t = Σ_k p_φ(k|x_t,t) v_k. The paper also proves a weighting ratio (Prop. 1) showing velocity-prediction losses upweight high-noise timesteps by 1/α_t^2 relative to epsilon-prediction, which it uses to explain complementary specialization.

Load-bearing premise

The DDPM-to-velocity conversion used at inference remains numerically stable and accurate enough across the noise levels where DDPM experts are active; the paper itself shows the conversion breaks down at high noise, so the reported gains rely on hand-tuned clamps and scaling that restrict converted DDPM experts to low-noise stages.

What would settle it

Reproduce the 2DDPM:6FM heterogeneous ensemble while assigning the two DDPM experts to the high-noise phase (t > 0.85) rather than low-noise, and drop the hand-tuned scaling s(t). The paper's own Fig. 14 predicts blocky artifacts and severe FID degradation; if FID still beats the homogeneous 8FM baseline, the stability assumption is not load-bearing, whereas degradation would confirm the central improvement depends on keeping converted experts in the numerically stable regime.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Decentralized training no longer requires homogeneous objectives or coordinated noise schedules; each contributor can pick the objective they prefer.
  • The inference-time unification allows existing pretrained DDPM checkpoints to be reused inside flow-matching pipelines, lowering the cost of switching objectives.
  • Heterogeneous ensembles give practitioners a knob for diversity: adjusting the DDPM:FM ratio or the router threshold trades FID against per-prompt variation without retraining.
  • The 16x compute and 14x data reduction relative to previously reported decentralized diffusion training suggests that frontier-scale image models might be obtainable from single-GPU contributors with substantially less centralized compute.
  • The mathematical conversion is schedule-agnostic in principle, so the framework extends beyond the evaluated cosine/linear schedules to other diffusion paths.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • We infer that the same velocity-conversion identity could unify other prediction targets (x0-prediction, v-prediction, consistency) as long as each target provides a clean-sample estimator; the paper leaves this open, but the derivative-of-forward-process derivation suggests it generalizes.
  • A plausible implication the paper does not spell out: if objective heterogeneity is beneficial in diffusion, similar objective diversity may help other ensemble generative models, where members trained with different losses are usually treated as a nuisance.
  • The quality–diversity trade-off controlled by routing threshold suggests a user-facing dial: downstream applications could select a threshold to emphasize FID or diversity at inference time; we would expect this to generalize to other router-based generative ensembles.
  • A concrete next experiment: measure whether the heterogeneous gain persists as the number of DDPM experts increases beyond a 2:6 ratio; the paper only tests that single allocation and states the optimal mix is open.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The paper proposes a decentralized training framework for diffusion models in which independent experts can be trained with different objectives: DDPM (epsilon-prediction) and Flow Matching (velocity-prediction). The key methodological claim is that these heterogeneous experts can be combined at inference time in a common velocity space via the deterministic conversion in Eq. (7), without retraining or synchronization. The paper also introduces a checkpoint-conversion procedure for initializing Flow Matching experts from a pretrained ImageNet-DDPM DiT checkpoint, and adopts PixArt-alpha's AdaLN-Single architecture. Empirically, the authors report a 16x compute and 14x data reduction relative to the prior DDM work, and, under aligned inference settings, a 0.57 FID improvement (11.88 vs. 12.45) and higher intra-prompt diversity (LPIPS 0.631 vs. 0.617) for a heterogeneous 2DDPM:6FM configuration over a homogeneous 8FM baseline. The paper is clearly written and the conversion algebra is standard, but the headline empirical claims rest on a small number of unablated runs.

Significance. If the main claim holds, the paper makes a useful practical contribution: objective heterogeneity among independently trained experts could be a cheap way to improve generation diversity and quality in decentralized diffusion systems, without requiring synchronized training or objective-specific pretraining. The derivation in Sec. 3.2 is correct as far as it goes, and the authors are transparent about several limitations, including hand-tuned numerical safeguards and the narrow set of DDPM:FM ratios tested. However, the central empirical assertion — that heterogeneity itself drives the reported FID/LPIPS gains — is supported by only a single configuration with no seed variance and no ablation on the cluster assignment. The paper also contains an ambiguity in the timestep handling for DDPM experts (Eq. 20) and a misleading comparison to DDM in Table 2. These issues do not invalidate the conceptual contribution, but they currently prevent the evidence from being load-bearing.

major comments (5)
  1. [Sec. 4.4 / Table 4 / Sec. 6.2] The headline heterogeneity result (FID 11.88 vs. 12.45, LPIPS 0.631 vs. 0.617) is based on a single training run and a deliberately chosen assignment of DDPM experts to clusters 0 and 3 ('high-fidelity subjects like cars and flowers'). Because clusters differ in content and difficulty, and because all experts are initialized from an ImageNet-DDPM checkpoint, the gain could reflect cluster-specific initialization or data difficulty rather than objective heterogeneity per se. Please provide: (a) multiple seeds for both homogeneous and heterogeneous configurations; (b) an ablation that assigns DDPM experts to different clusters (e.g., random or uniform); and (c) the exact inference expert-selection strategy used for the rows of Table 4 (Top-1, Top-K, or Full), which is not stated. Without these, the 0.57 FID difference is not statistically or causally established.
  2. [Sec. 3.3 / Proposition 1 and Remark] The complementary-specialization explanation is essentially a restatement of the known SNR-dependent weighting of different diffusion objectives. The ratio w_v/w_epsilon = 1/alpha_t^2 in Eq. (11) follows algebraically from writing both losses in terms of clean-sample estimation error, and the claim that velocity-prediction experts receive stronger gradients at high noise does not by itself predict that a heterogeneous ensemble will improve FID. The empirical heterogeneity effect must carry the argument; the derivation alone is not evidence of complementarity. Please either soften the causal language or provide per-noise-level diagnostics (e.g., FID or reconstruction error as a function of timestep) showing that the two objectives differ in the way the text claims. Also note that Proposition 1 uses the Salimans-Ho v-parameterization, not the linear-path FM velocity used in Eq. (4); the Re
  3. [Sec. 7.3 / Sec. 8.3] The conversion in Eq. (7) is presented as a general 'schedule-aware deterministic conversion' without retraining, but its practical viability relies on several hand-tuned safeguards: alpha_safe = max(alpha_t, 0.01) in Eq. (28), the clamp radius in Eq. (27), and the piecewise scaling s(t) in Eq. (30). Section 7.3 itself concludes that the conversion 'may be best restricted to low-noise regimes' and reports blocky artifacts when converted DDPM experts handle high-noise stages. The paper should quantify how much the main FID result (Table 4) depends on these safeguards, for example by ablating s(t) or alpha_safe, and should state more clearly which parts of the conversion claim are general and which are empirical fixes tuned for the cosine schedule.
  4. [Eq. (20) / Sec. 3.5] The timestep scaling in Eq. (20) is ambiguous and potentially inconsistent. The unified sampling trajectory is described as operating in t in [0,1], and the router is trained with t_DiT = 999*t. For DDPM experts, however, the equation says t_DiT = t when t in [0,999]. If the inference trajectory supplies t in [0,1], then DDPM experts would receive raw values near 0-1 rather than the 0-999 range used in their sinusoidal timestep embedding and MLP. Please clarify what timestep is actually fed to DDPM experts during training and inference, and whether the FDPM experts' timestep embedding is scaled by 999. If the current procedure is correct, the equation should be rewritten to avoid the apparent mismatch.
  5. [Table 2 / Sec. 4.2.1] The phrase 'competitive generation quality' is not supported by the numbers shown: DDM's estimated FID range is 5.5-10.5, while the proposed method achieves 11.88. The 16x/14x claims are appropriately framed as resource reductions relative to DDM's reported training scale, but the accompanying FID comparison makes the qualitative statement misleading. Please either remove 'competitive' or provide a matched-compute comparison (same data budget, same model scale, same inference protocol) against DDM or a monolithic baseline, so that the efficiency claim can be evaluated on a level playing field.
minor comments (4)
  1. [Sec. 4.3 / Table 3] The combined-expert results in Table 3 use a deterministic router with a threshold at t=0.5 and a different inference setting (CFG=6, 75 steps) from the main aligned comparison (CFG=7.5, 50 steps). The FID degradation (32.67 vs. 20.23) is therefore not directly comparable to the gains in Table 4. Please state this explicitly in the text and consider reporting the combined-expert result under the aligned settings as well.
  2. [Sec. 3.4 / Eq. (14)-(15)] The AdaLN-Single formulation is described as reducing parameters by 'approximately 30% for text-conditioned DiT-XL/2'. Please give the measured parameter counts for the homogeneous and heterogeneous models so the claim can be verified.
  3. [Sec. 4.4] The LPIPS values are reported with standard deviations (0.631 +/- 0.078 vs. 0.617 +/- 0.074), but FID is reported without any uncertainty or multiple runs. Since LPIPS and FID are computed on different bases (100 prompts x 10 images vs. 50K images), the differing error bars may mislead readers. Please report confidence intervals for FID as well, or explain why they are not available.
  4. [Sec. 6.2 / Objective Assignment] The paper says 'we specifically assign DDPM experts to cluster 0 and cluster 3 as they contain high-fidelity subjects like cars and flowers.' This is exactly the assignment whose effect needs to be ablated. In addition to the ablation requested above, please report what the homogeneous 8FM baseline would have achieved if clusters 0 and 3 had been trained with the same initialization and data but with additional training steps, to disentangle cluster-specific effects from objective effects.

Circularity Check

0 steps flagged

No significant circularity: the conversion is an algebraic identity and the heterogeneity improvements are empirical comparisons, not fitted predictions.

full rationale

Walking the derivation chain, the central conversion (Eq. 7) is not circular: starting from the forward process x_t = alpha_t x0 + sigma_t eps and the model estimate x0_hat, the velocity v = (d alpha_t/dt) x0_hat + (d sigma_t/dt) eps_theta is literally the time-derivative of the path tilde x_t = alpha_t x0_hat + sigma_t eps_theta. This is a deterministic algebraic identity, not a quantity fitted to the data and then renamed as a prediction. Proposition 1 similarly restates the epsilon- and velocity-prediction MSE losses in terms of clean-sample estimation error; the ratio w_v/w_eps = 1/alpha_t^2 follows from algebra and is used only as a motivating interpretation, not as the evidence for the headline improvement. The headline claim (heterogeneous 2DDPM:6FM improving FID and LPIPS over homogeneous 8FM, Table 4) is an empirical comparison against a real homogeneous baseline under matched inference settings; it is not derived from the conversion identity. The checkpoint-transfer and architecture contributions are validated by external loss curves and adopted architecture components, and no load-bearing argument reduces to a self-citation: all cited works (DDM, PixArt-alpha, Kingma et al.) have no author overlap with the present paper. The manuscript's own limitations—hand-tuned numerical safeguards, restricted low-noise conversion reliability, and the narrow 2:6 objective ratio—are honest scope restrictions, not signs that an output was built into its inputs. Robustness concerns such as the single unseeded run and the specific choice of DDPM clusters are methodological validity questions, not circularity. No circular step meets the quoted-evidence bar, so the appropriate score is 0.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 1 invented entities

The paper introduces no new physical entities. Its central algorithmic contribution relies on five fitted/cherry-picked choices (clamp value, alpha floor, scaling schedule, cluster assignment, DDPM-to-FM ratio) plus several domain assumptions inherited from prior DDM work. The conversion formula itself is a change of variables, so the free parameters are the main things that make the empirical result work.

free parameters (5)
  • alpha_safe floor (0.01) = 0.01
    Sec. 8.3.2: safe division threshold to stabilize DDPM-to-velocity conversion; 'chosen to balance stability against accuracy'.
  • Schedule-aware velocity scaling s(t) = 0.88 / 0.93 / 0.96
    Sec. 8.3.4, Eq. 30: piecewise damping of converted velocity at high noise levels; manually tuned and used in all heterogeneous FID results.
  • Clamp radius for x0_hat = 20.0 for VAE latents
    Sec. 8.3.1, Eq. 27: 'empirically determined based on the typical distribution of clean samples'.
  • DDPM-to-FM ratio (2 DDPM : 6 FM) = 2:6
    Chosen in Sec. 4.1; the conclusion explicitly says the optimal allocation is not studied and 'likely depends on data'.
  • Assignment of DDPM experts to clusters 0 and 3 = clusters 0 and 3
    Sec. 6.2: assigned because they 'contain high-fidelity subjects like cars and flowers' — a post hoc selection that can favor the heterogeneous-over-homogeneous comparison.
axioms (5)
  • domain assumption The router posterior p_t(k|x_t) can be learned and used to weight independently trained expert velocity fields (Eq. 1).
    Sec. 3.1: taken from DDM [22] and assumed to remain valid when experts use different objectives.
  • domain assumption Treating xhat_0 and eps_theta as fixed at their current-timestep values while differentiating defines a valid velocity (Eq. 7).
    Sec. 3.2: standard Heun-like approximation, but it discards the time-dependence of the model prediction; the paper relies on this being a stable trajectory for sampling.
  • ad hoc to paper The DiT ImageNet-DDPM checkpoint transfers to Flow Matching via the given layer reinitialization (Eq. 19).
    Sec. 3.5: assumed transferability of low-level visual features; validated only by loss curves and samples, with no ablations on which transferred components matter.
  • standard math Euler ODE integration of the fused velocity field produces valid samples.
    Sec. 3.2: standard numerical integration for probability-flow ODEs.
  • domain assumption FID-50K and LPIPS on the chosen held-out test set are sufficient measures of generation quality.
    Sec. 4: standard for generative models, but no confidence intervals; 50K samples vs 50K reference images.
invented entities (1)
  • None. no independent evidence
    purpose: The paper introduces no new particle, force, dimension, or physical entity.
    All contributions are algorithmic and architectural.

reviewed 2026-08-04 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Heterogeneous Decentralized Diffusion Models." pith.science (2026). https://pith.science/paper/FYPQ6U6Q

@misc{pith2026260306741,
  author       = {Pith},
  title        = {Pith review of: Heterogeneous Decentralized Diffusion Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FYPQ6U6Q}},
  note         = {Machine review of arXiv:2603.06741}
}
Share X Bluesky LinkedIn Reddit HN
abstract

Training frontier-scale diffusion models often requires substantial computational resources concentrated in tightly-coupled clusters, limiting participation to well-resourced institutions. While Decentralized Diffusion Models (DDM) enable training multiple experts in isolation, existing approaches require 1176 GPU-days and homogeneous training objectives across all experts. We present an efficient framework that dramatically reduces resource requirements while supporting heterogeneous training objectives. Our approach combines three key contributions: (1) a heterogeneous decentralized training paradigm that allows experts to use different objectives (DDPM and Flow Matching), unified at inference time without any retraining; (2) pretrained checkpoint conversion from ImageNet-DDPM to Flow Matching objectives, accelerating convergence and enabling initialization without objective-specific pretraining; and (3) PixArt-$\alpha$'s efficient AdaLN-Single architecture, reducing parameters while maintaining quality. Experiments on LAION-Aesthetics show that, relative to the training scale reported for prior DDM work, our approach reduces the compute by 16$\times$ and data by 14$\times$. Under aligned inference settings, our heterogeneous configuration achieves better FID and higher intra-prompt diversity than the homogeneous baseline. By eliminating synchronization requirements and enabling mixed DDPM/FM objectives, our framework makes decentralized generative model training accessible to contributors with single GPUs requiring only 24--48GB VRAM.

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

44 extracted references · 7 linked inside Pith

  1. [1]

    Pixart-𝛼: Fast training of diffusion transformer for photorealistic text-to-image synthesis.arXiv preprint arXiv:2310.00426, 2023

    Junsong Chen, Jincheng Yu, Chongjian Ge, Lewei Yao, Enze Xie, Yue Wu, Zhongdao Wang, James Kwok, Ping Luo, Huchuan Lu, and Zhenguo Li. Pixart-𝛼: Fast training of diffusion transformer for photorealistic text-to-image synthesis.arXiv preprint arXiv:2310.00426, 2023

    Pith/arXiv arXiv 2023

  2. [2]

    Diffusion models beat gans on image synthesis.Advances in neural information processing systems, 34:8780–8794, 2021

    Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis.Advances in neural information processing systems, 34:8780–8794, 2021

    2021

  3. [3]

    Decentralized learning made easy with decentralizepy

    Akash Dhasade, Anne-Marie Kermarrec, Rafael Pires, Rishi Sharma, and Milos Vujasinovic. Decentralized learning made easy with decentralizepy. InProceedings of the 3rd Workshop on Machine Learning and Systems, pages 34–41, 2023

    2023

  4. [4]

    Efficient diffusion training via min-snr weighting strategy

    Tiankai Hang, Shuyang Gu, Chen Li, Jianmin Bao, Dong Chen, Han Hu, Xin Geng, and Baining Guo. Efficient diffusion training via min-snr weighting strategy. InProceedings of the IEEE/CVF international conference on computer vision, pages 7441–7451, 2023

    2023

  5. [5]

    Improved noise schedule for diffusion training

    Tiankai Hang, Shuyang Gu, Jianmin Bao, Fangyun Wei, Dong Chen, Xin Geng, and Baining Guo. Improved noise schedule for diffusion training. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 4796–4806, 2025

    2025

  6. [6]

    Faster training of diffusion models and improved density estimation via parallel score matching

    Etrit Haxholli and Marco Lorenzi. Faster training of diffusion models and improved density estimation via parallel score matching. InAdvances in Neural Information Processing Systems, 2023. 12 Heterogeneous Decentralized Diffusion Models

    2023

  7. [7]

    Multistep consistency models.arXiv preprint arXiv:2403.06807, 2024

    Jonathan Heek, Emiel Hoogeboom, and Tim Salimans. Multistep consistency models.arXiv preprint arXiv:2403.06807, 2024

    Pith/arXiv arXiv 2024

  8. [8]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. InAdvances in Neural Information Processing Systems, 2017

    2017

  9. [9]

    Classifier-free diffusion guidance.arXiv preprint arXiv:2207.12598, 2022

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance.arXiv preprint arXiv:2207.12598, 2022

    Pith/arXiv arXiv 2022

  10. [10]

    Denoising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. InAdvances in Neural Information Processing Systems, pages 6840–6851, 2020

    2020

  11. [11]

    Fast-ddpm: Fast denoising diffusion probabilistic models for medical image-to-image generation.IEEE Journal of Biomedical and Health Informatics, 2025

    Hongxu Jiang, Muhammad Imran, Teng Zhang, Yuyin Zhou, Muxuan Liang, Kuang Gong, and Wei Shao. Fast-ddpm: Fast denoising diffusion probabilistic models for medical image-to-image generation.IEEE Journal of Biomedical and Health Informatics, 2025

    2025

  12. [12]

    Kingma and Ruiqi Gao

    Diederik P. Kingma and Ruiqi Gao. Understanding diffusion objectives as the ELBO with simple data augmentation. InAdvances in Neural Information Processing Systems, 2023

    2023

  13. [13]

    Kingma, Tim Salimans, Ben Poole, and Jonathan Ho

    Diederik P. Kingma, Tim Salimans, Ben Poole, and Jonathan Ho. Variational diffusion models. InAdvances in Neural Information Processing Systems, pages 21696–21707, 2021

    2021

  14. [14]

    Decentralized stochastic optimization and gossip algorithms with compressed communication

    Anastasia Koloskova, Sebastian Stich, and Martin Jaggi. Decentralized stochastic optimization and gossip algorithms with compressed communication. InInternational conference on machine learning, pages 3478–3487. PMLR, 2019

    2019

  15. [15]

    Laion-aesthetics v2: Aesthetic-filtered subset of laion-5b

    LAION-AI. Laion-aesthetics v2: Aesthetic-filtered subset of laion-5b. https://laion.ai/blog/ laion-aesthetics/, 2022. Subset of LAION-5B filtered for high predicted aesthetic scores

    2022

  16. [16]

    Improving the training of rectified flows.Advances in neural information processing systems, 37:63082–63109, 2024

    Sangyun Lee, Zinan Lin, and Giulia Fanti. Improving the training of rectified flows.Advances in neural information processing systems, 37:63082–63109, 2024

    2024

  17. [17]

    Gshard: Scaling giant models with conditional computation and automatic sharding.arXiv preprint arXiv:2006.16668, 2021

    Dmitry Lepikhin, HyoukJoong Lee, Yuanzhong Xu, Dehao Chen, Orhan Firat, Yanping Huang, Maxim Krikun, Noam Shazeer, and Zhifeng Chen. Gshard: Scaling giant models with conditional computation and automatic sharding.arXiv preprint arXiv:2006.16668, 2021

    Pith/arXiv arXiv 2006

  18. [18]

    Distrifusion: Distributed parallel inference for high-resolution diffusion models

    Muyang Li, Tianle Cai, Jiaxin Cao, Qinsheng Zhang, Han Cai, Junjie Bai, Yangqing Jia, Ming-Yu Liu, Kai Li, and Song Han. Distrifusion: Distributed parallel inference for high-resolution diffusion models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 7183–7193, 2024

    2024

  19. [19]

    Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Le. Flow matching for generative modeling. InThe Eleventh International Conference on Learning Representations (ICLR), 2023

    2023

  20. [20]

    Flow straight and fast: Learning to generate and transfer data with rectified flow.arXiv preprint arXiv:2209.03003, 2023

    Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow.arXiv preprint arXiv:2209.03003, 2023

    Pith/arXiv arXiv 2023

  21. [21]

    Optimal complexity in decentralized training

    Yucheng Lu and Christopher De Sa. Optimal complexity in decentralized training. InInternational conference on machine learning, pages 7111–7123. PMLR, 2021

    2021

  22. [22]

    Decentralized diffusion models

    David McAllister, Matthew Tancik, Jiaming Song, and Angjoo Kanazawa. Decentralized diffusion models. arXiv preprint arXiv:2501.05450, 2025

    Pith/arXiv arXiv 2025

  23. [23]

    On distillation of guided diffusion models

    Chenlin Meng, Robin Rombach, Ruiqi Gao, Diederik Kingma, Stefano Ermon, Jonathan Ho, and Tim Salimans. On distillation of guided diffusion models. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 14297–14306, 2023

    2023

  24. [24]

    Improved denoising diffusion probabilistic models

    Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In Proceedings of the 38th International Conference on Machine Learning, pages 8162–8171. PMLR, 2021

    2021

  25. [25]

    Dinov2: Learning robust visual features without supervision.Transactions on Machine Learning Research, 2024

    Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, Mahmoud Assran, Nicolas Ballas, Wojciech Galuba, Russell Howes, Po-Yao Huang, Shang-Wen Li, Ishan Misra, Michael Rabbat, Vasu Sharma, Gabriel Synnaeve, Hu Xu, Hervé Jegou, Julien Mairal, Patrick Lab...

    2024

  26. [26]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 4195–4205, 2023

    2023

  27. [27]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. InInternational conference on machine learning, pages 8748–8763. PMLR, 2021. 13 Heterogeneous Decent...

    2021

  28. [28]

    High-resolution image synthesis with latent diffusion models.Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models.Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022

    2022

  29. [29]

    Sara Mahdavi, Rapha Gontijo Lopes, Tim Salimans, Jonathan Ho, David J Fleet, and Mohammad Norouzi

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kamyar Seyed Ghasemipour, Burcu Karagol Ayan, S. Sara Mahdavi, Rapha Gontijo Lopes, Tim Salimans, Jonathan Ho, David J Fleet, and Mohammad Norouzi. Photorealistic text-to-image diffusion models with deep language understanding.Advances in Neural Information Processing S...

    2022

  30. [30]

    Progressive distillation for fast sampling of diffusion models.arXiv preprint arXiv:2202.00512, 2022

    Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models.arXiv preprint arXiv:2202.00512, 2022

    Pith/arXiv arXiv 2022

  31. [31]

    LAION-5b: An open large-scale dataset for training next generation image-text models.Advances in Neural Information Processing Systems, 35:25278–25294, 2022

    Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, Patrick Schramowski, Srivatsa Kun- durthy, Katherine Crowson, Ludwig Schmidt, Robert Kaczmarczyk, and Jenia Jitsev. LAION-5b: An open large-scale dataset for training next generation image-text model...

    2022

  32. [32]

    Diff2flow: Training flow matching models via diffusion model alignment

    Johannes Schusterbauer, Ming Gui, Frank Fundel, and Björn Ommer. Diff2flow: Training flow matching models via diffusion model alignment. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 28347–28357, 2025

    2025

  33. [33]

    Outrageously large neural networks: The sparsely-gated mixture-of-experts layer

    Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, and Jeff Dean. Outrageously large neural networks: The sparsely-gated mixture-of-experts layer. InInternational Conference on Learning Representations, 2017

    2017

  34. [34]

    Consistency models

    Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. InInternational Conference on Machine Learning, pages 32211–32252. PMLR, 2023

    2023

  35. [35]

    Swarm learning for decentralized and confidential clinical machine learning.Nature, 594(7862):265–270, 2021

    Stefanie Warnat-Herresthal, Hartmut Schultze, Krishnaprasad Lingadahalli Shastry, Sathyanarayanan Manamohan, SaikatMukherjee, VisheshGarg, RaviSarveswara, KristianHändler, PeterPickkers, NAhmad Aziz, Sofia Ktena, Florian Tran, Michael Bitzer, Stephan Ossowski, Nicolas Casadei, Christian Herr, Daniel Petersheim, Uta Behrends, Fabian Kern, Tobias Fehlmann, ...

    2021

  36. [36]

    Communication-efficient federated learning via knowledge distillation.Nature communications, 13(1):2032, 2022

    Chuhan Wu, Fangzhao Wu, Lingjuan Lyu, Yongfeng Huang, and Xing Xie. Communication-efficient federated learning via knowledge distillation.Nature communications, 13(1):2032, 2022

    2032

  37. [37]

    Decentralized federated learning towards communication efficiency, robustness, and personalization.ACM Transactions on Sensor Networks, 21(3): 1–20, 2025

    Anqi Zhang, Ping Zhao, Wenke Lu, and Guanglin Zhang. Decentralized federated learning towards communication efficiency, robustness, and personalization.ACM Transactions on Sensor Networks, 21(3): 1–20, 2025

    2025

  38. [38]

    Redi: efficient learning-free diffusion inference via trajectory retrieval

    Kexun Zhang, Xianjun Yang, William Yang Wang, and Lei Li. Redi: efficient learning-free diffusion inference via trajectory retrieval. InInternational Conference on Machine Learning, pages 41770–41785. PMLR, 2023

    2023

  39. [39]

    Efros, Eli Shechtman, and Oliver Wang

    Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018

    2018

  40. [40]

    Communication-efficient decentralized federated learning via one-bit compressive sensing

    Shenglong Zhou, Kaidi Xu, and Geoffrey Ye Li. Communication-efficient decentralized federated learning via one-bit compressive sensing. In2024 IEEE 99th Vehicular Technology Conference (VTC2024-Spring), pages 1–5. IEEE, 2024. 1 Heterogeneous Decentralized Diffusion Models LAION Dataset D DINOv2 Feature Extraction Hierarchical Clustering Cluster𝑆 1 Cluster...

    2024

  41. [41]

    Data Preprocessing and Clustering Dataset.We train on the LAION-Aesthetics subset

    Training Details 6.1. Data Preprocessing and Clustering Dataset.We train on the LAION-Aesthetics subset. For DiT-B/2, we utilize LAION-Art whose aesthetic score is≥ 8, containing around 3.9M image-text pairs. For DiT-XL/2, we filter LAION-Aesthetic for aesthetic score≥ 4.5and resolution≥ 256× 256. Images are center-cropped to square aspect ratio and resiz...

  42. [42]

    baked into

    Additional Qualitative Analysis In this section, we provide extensive qualitative results to demonstrate the capabilities of our het- erogeneous decentralized diffusion framework. All images are generated at256× 256resolution with 75 Euler sampling steps and CFG scale 6. Importantly, all text prompts used for generation are either from a held-out test set...

  43. [43]

    DDPM to Flow Matching Conversion: Implementation Details The conversion of DDPM expert outputs to Flow Matching velocity predictions is critical for het- erogeneous ensemble inference. While theoretically straightforward, the practical implementation requires careful handling of numerical instabilities, schedule-dependent derivatives, and multi-expert coo...

  44. [44]

    These large derivatives can amplify velocity magnitudes and cause integration instability during sampling. 8.2.3. Accumulation of Conversion Errors In multi-expert ensembles where some experts use DDPM objectives, conversion errors accumulate across the sampling trajectory. Small biases in individual conversions compound through the iterative sampling pro...

This paper was first reviewed by deepseek-v4-flash on August 4, 2026.