REVIEW 3 major objections 5 minor 13 references
Morse: Dual-Sampling for Lossless Acceleration of Diffusion Models
T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read Morse claims that any pretrained diffusion model can run 1.78x to 3.31x faster with no loss in output quality by pairing it with a fast residual-correction model that fills in the skipped denoising steps.
desk verdict Solid new dual-sampling acceleration method with broad experiments, but the 'lossless' claim is overstated because the speedup depends on best-guidance-scale selection and interpolated curves without error bars. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the dual-sampling noise-estimation rule of Eq. (5): $z_{t_i} = \theta(x_{t_i}, t_i)$ if $t_i$ is a Dash step in $S$, and $z_{t_i} = z_{t_s} + \eta(x_{t_s}, x_{t_i}, z_{t_s}, t_s, t_i)$ otherwise. It turns the denoising trajectory into a time-interleaved alternation of a slow accurate estimator and a fast corrective one. The Dot model $\eta$ is constructed by weight sharing: lightweight down-sampling and up-sampling blocks are stacked onto the top and bottom of the frozen pretrained model, shrinking the resolution seen by every pretrained block by $4^m$ and dropping self-attention from the new blocks, which is what makes Dot several times faster, with LoRA adapters as the trainable part. Dot is trained with an L2 loss on the residual against a single matched step of Dash (Algorithm 1), then applied at arbitrary jump intervals at inference (Algorithm 2). The trajectory inputs — the anchor sample $x_{t_s}$, the time step $t_s$, and Dash's estimate $z_{t_s}$ — are what let Dot behave like Dash without holding Dash's full compute; the ablation in Table 3 shows each input contributes.
What would settle it
Run the DDIM-with-Morse procedure of Algorithm 2 on CIFAR-10 at an exchanged-steps ratio above 70%, where the paper's own Fig. 13 shows speedups dropping below 1.0x, and measure the matched-latency FID curve precisely: if the baseline's FID is not reproduced there, the 'lossless' claim fails exactly in the maximal-acceleration regime. Alternatively, measure the per-step residual error $\|\theta(x_{t_o}, t_o) - (z_{t_s} + \eta(x_{t_s}, x_{t_o}, z_{t_s}, t_s, t_o))\|$ over a chain of several consecutive Dot steps; if the error grows with chain length rather than staying flat, Dot drifts and the losslessness is approximate, not structural.
Extended reading notes
Core claim
Morse's central claim is that the information lost when a diffusion model jumps over timesteps can be recovered by a much cheaper model, so the total compute per generated image drops without changing what the images look like in aggregate. The paper formalizes the generation process as an interleaving of two estimators: at jump-sampling anchor steps $t_s$ the pretrained model $\theta$ ('Dash') produces its usual estimate, and at every skipped step $t_i$ the fast model $\eta$ ('Dot') produces $z_{t_i} = z_{t_s} + \eta(x_{t_s}, x_{t_i}, z_{t_s}, t_s, t_i)$, trained so this corrected estimate equals $\theta(x_{t_i}, t_i)$, the estimate Dash would have made without jumping. Because Dot runs $N$ times faster than Dash ($N$ between roughly 5 and 10 in the reported setups), the paired system performs more denoising steps inside the same latency budget and matches the baseline's FID, or improves the CLIP score, across DDPM, DDIM, SDE and DPM-Solver samplers on six benchmarks. The same construction applied to LCM-SDXL, a model already accelerated by consistency distillation, produces a further 1.43x average speedup on CLIP score. Losslessness here is defined at the level of the aggregate evaluation metric: a speedup of $n/l$ means the Morse process at latency $l$ and the baseline at latency $n$ achieve the same metric.
Load-bearing premise
The whole scheme rests on the assumption that the small Dot model, trained to match Dash on single skipped steps, keeps matching Dash's noise estimates accurately when many of its corrections are chained together at jump lengths and guidance scales beyond its training, and the paper gives no bound on how that error grows.
Editorial extensions
If this is right
- Any pretrained diffusion model can be converted into a Morse pair by freezing its weights and training only the small Dot module; reported matched-FID speedups average 1.78x to 3.31x across 9 baselines and 6 benchmarks.
- The scheme is sampler- and architecture-agnostic: it accelerates DDPM, DDIM, SDE and DPM-Solver, works for unconditional, class-conditional, classifier-guided and classifier-free generation, and stacks on top of distillation-based acceleration such as LCM-SDXL (about 1.43x further speedup on CLIP score).
- Dot training is cheap relative to the base model: for Stable Diffusion, roughly 2M text-image pairs, 100,000 iterations, and about 0.1% of the base model's training cost, adding 97.84M parameters to an 859.52M model.
- When Dot exactly matched Dash, the speedup would reach its upper bound of $(n-k+Nk)/n$, and the appendix recommends keeping this factor between 2.0 and 3.0, which it reports as yielding the best results.
Reading between the lines
- The Dash/Dot split is structurally a speculative-decoding scheme for diffusion: a cheap model proposes what the slow accurate model would have computed, and the residual correction is the verification. Read this way, the same 'fast proposer plus residual corrector' pattern could transfer to video, audio, or 3D diffusion, where adjacent frames or tokens have similar redundancy.
- The paper's lossless claim is empirical, resting on aggregate FID/CLIP at matched latency. A natural strengthening would be a stability bound: if the noise estimator $\theta$ is Lipschitz in its inputs and Dot's per-step residual error is bounded, accumulated error over a chain of corrections would grow roughly linearly with chain length, turning the measured speedups into a certified guarantee.
- The appendix's observation that speedups collapse below 1.0x once more than about 70% of steps are handed to Dot suggests the method's limit is set by jump distance. Making the Dot conditioning explicitly depend on the jump length $t_s - t_i$, rather than on the endpoints alone, is a testable way to push that limit.
- Because the measured speed-up factor $N$ varies with hardware (from 4.7 to 11.3 across GPUs in the appendix), the reported speedups are device-dependent constants; the mechanism itself should survive, but a deployment would need to re-measure $N$ and re-tune the schedule on its target hardware.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes Morse, a dual-sampling acceleration framework for pretrained diffusion models. The pretrained model (Dash) runs at a reduced set of jump-sampling steps, and a faster auxiliary model (Dot), built from the Dash backbone with lightweight down/up blocks and LoRA, predicts residual corrections to the noise estimate at intermediate steps. The authors report lossless speedups of 1.78x to 3.31x on average across 9 diffusion baselines and 6 image generation tasks, using FID and CLIP as quality metrics, and also apply Morse to LCM-SDXL. The framework is designed to be training- and inference-efficient through weight sharing and is presented as complementary to distillation-based accelerators.
Significance. If the reported speedups hold under standard evaluation protocols, Morse is a practically valuable and broadly applicable acceleration method: the paper covers multiple samplers, architectures, resolutions, and tasks, and it ships code and models, which strengthens reproducibility. The residual-feedback formulation and the weight-sharing construction are clear and the training cost of the Dot model is very low relative to the base model. However, the headline 'lossless' claim currently rests on an evaluation protocol that selects the best guidance scale per latency and on interpolated latency-FID curves without confidence intervals, so the quantitative claim is not yet supported as stated.
major comments (3)
- [§3.3, Table 1 and Fig. 7] The claimed Stable Diffusion speedup is computed under a best-scale oracle rather than the officially default scale. In Table 1, the baseline at 10 LSD with scale 7.5 has FID 11.92, while Morse at 15 LSD with scale 7.5 has FID 10.07, giving roughly 1.5x, and Morse at 20 LSD has FID 10.93, giving roughly 1.8x; the advertised 2.29x average comes from selecting the best FID across guidance scales 2-10 at each latency. Because users do not typically re-tune the guidance scale for each latency budget, the lossless claim should be reported at a fixed scale (e.g., 7.5), with the best-scale results clearly labeled as an oracle upper bound.
- [§3.1] Speedups are computed by linear interpolation between FID-latency curves, and no confidence intervals, multiple seeds, or per-run variances are reported for any of the figures. The headline range 1.78x-3.31x is therefore an estimate whose statistical uncertainty is unquantified; a lossless claim requires demonstrating that Morse and the baseline achieve the same metric within error at matched latencies. Please report the raw FID/CLIP values at each sampled latency, the interpolation points, and error bars over multiple evaluation runs.
- [§2.3, Algorithm 2] The speedup upper bound (n-k+Nk)/n assumes that Dot and Dash produce identical noise estimates, but Dot is trained only to match a single transition (Algorithm 1) and is then applied at arbitrary jump intervals and chained over multiple steps (Algorithm 2). The paper provides no error-accumulation analysis or bound, so 'lossless' is at present an empirical statement about aggregate FID/CLIP, not a proven property. This should be stated explicitly, and an ablation should report how Dot's residual error grows with the number of chained Dot steps and with the distance between consecutive Dash steps.
minor comments (5)
- [§2.1, Eq. (1)] Equation (1) appears to contain a typo: the integrand includes p(x0) and the expression is not a valid conditional distribution for p(xt|x0); it should be the integral of the transition densities over x1:t-1 without the marginal term.
- [§3.1] The notation 'N/A' for DPM-Solver at 100 LSDs is confusing because N is already defined as the speed ratio between Dash and Dot; please use a different symbol or phrase to indicate that no speedup is reported.
- [Table 3] The formatting of Table 3 is hard to read: the checkmarks for trajectory information and the FID values are not clearly aligned with the rows, making it difficult to see which input combination corresponds to which FID.
- [Table 5] In Table 5, Morse is evaluated with 10,000 generated samples following PFDiff settings, while other Stable Diffusion tables use 30,000 samples; FID values depend on the number of samples, so please state the sample count used for PFDiff and ensure both methods are compared on the same sample size.
- [Abstract] The units '1.78X' and '3.31X' should be typeset as '1.78x' and '3.31x' for consistency with the rest of the paper.
Circularity Check
No significant circularity; acceleration is judged on external FID/CLIP benchmarks, not on Dot's training target.
full rationale
The derivation chain is not circular. The only learned component, the Dot model, is explicitly trained against Dash in Algorithm 1 by minimizing ||z_to - (z_ts + eta(x_ts, x_to, z_ts, ts, to))||^2; this is the proposed distillation/distillation-like mechanism, not a hidden premise. The paper's evidence for 'lossless' acceleration is external: FID and CLIP scores on MS-COCO, CIFAR-10, ImageNet, CelebA, CelebA-HQ and LSUN-Church, computed with official pre-trained baseline models and standard samplers (Figs. 4-8, Tables 1-6), so the central speedup claim does not reduce to the Dot training objective. The speedup definition in Sec. 3.1 (n/l when the two processes 'get the same evaluation metric') is an evaluation convention, not a prediction derived from the method; the upper-bound formula (n-k+Nk)/n is explicitly introduced 'under ideal conditions where Dot and Dash perform exactly the same for noise estimation' and is not used as evidence of empirical correctness. No load-bearing self-citation, no uniqueness theorem imported from the authors, and no renamed-known-result pattern was found. Concerns that remain--best-guidance-scale selection for Stable Diffusion, linear interpolation without confidence intervals, and the absence of a drift bound for chained Dot corrections--are correctness/reporting risks under the evaluation protocol, not circularity of the derivation.
Assumptions & free parameters
free parameters (5)
- Number of extra down and up blocks m =
m=2 for image generation; m=1 for LCM-SDXL
- LoRA rank =
64 for Stable Diffusion
- Extra block channel counts =
96 and 160 for Stable Diffusion Dot
- Exchanged steps ratio k/n =
Suggested (n-k+Nk)/n between 2.0 and 3.0
- Dot training data budget =
2M text-image pairs for Stable Diffusion and LCM-SDXL; 100k iterations
assumptions (5)
- domain assumption Pretrained diffusion models admit jump sampling with a schedule function phi (Eq. 4), and the visited subsequence can be chosen uniformly.
- ad hoc to paper The Dot model, built from fixed Dash blocks at reduced resolution plus trainable down and up blocks and LoRA, can approximate Dash's next-step noise estimate across unvisited timesteps.
- domain assumption The measured latency ratio N remains representative for speedup on other GPUs and batch sizes.
- domain assumption Linear interpolation between FID or CLIP scores at discrete latencies accurately estimates performance at intermediate latencies used in speedup averaging.
- domain assumption Equal or better aggregate FID and CLIP scores constitute lossless generation quality.
Cite this review
Pith. "Pith review of Morse: Dual-Sampling for Lossless Acceleration of Diffusion Models." pith.science (2026). https://pith.science/paper/D2FOLTLN
@misc{pith2026250618251,
author = {Pith},
title = {Pith review of: Morse: Dual-Sampling for Lossless Acceleration of Diffusion Models},
year = {2026},
howpublished = {\url{https://pith.science/paper/D2FOLTLN}},
note = {Machine review of arXiv:2506.18251}
}
read the original abstract
In this paper, we present Morse, a simple dual-sampling framework for accelerating diffusion models losslessly. The key insight of Morse is to reformulate the iterative generation (from noise to data) process via taking advantage of fast jump sampling and adaptive residual feedback strategies. Specifically, Morse involves two models called Dash and Dot that interact with each other. The Dash model is just the pre-trained diffusion model of any type, but operates in a jump sampling regime, creating sufficient space for sampling efficiency improvement. The Dot model is significantly faster than the Dash model, which is learnt to generate residual feedback conditioned on the observations at the current jump sampling point on the trajectory of the Dash model, lifting the noise estimate to easily match the next-step estimate of the Dash model without jump sampling. By chaining the outputs of the Dash and Dot models run in a time-interleaved fashion, Morse exhibits the merit of flexibly attaining desired image generation performance while improving overall runtime efficiency. With our proposed weight sharing strategy between the Dash and Dot models, Morse is efficient for training and inference. Our method shows a lossless speedup of 1.78X to 3.31X on average over a wide range of sampling step budgets relative to 9 baseline diffusion models on 6 image generation tasks. Furthermore, we show that our method can be also generalized to improve the Latent Consistency Model (LCM-SDXL, which is already accelerated with consistency distillation technique) tailored for few-step text-to-image synthesis. The code and models are available at https://github.com/deep-optimization/Morse.
Figures
Figures from the paper (14 more)
Reference graph
Works this paper leans on
-
[4]
10 Morse: Dual-Sampling for Lossless Acceleration of Diffusion Models Mohamed, S. and Lakshminarayanan, B. Learn- ing in implicit generative models.arXiv preprint arXiv:1610.03483,
-
[6]
Learning to efficiently sample from diffusion probabilistic models
Watson, D., Ho, J., Norouzi, M., and Chan, W. Learning to efficiently sample from diffusion probabilistic models. arXiv preprint arXiv:2106.03802,
-
[7]
Yu, F., Seff, A., Zhang, Y ., Song, S., Funkhouser, T., and Xiao, J. Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop.arXiv preprint arXiv:1506.03365,
-
[8]
11 Morse: Dual-Sampling for Lossless Acceleration of Diffusion Models A. Appendix A.1. Benchmarks and Evaluation Details Image Generation.In the experiments described in Sec. 3.2, we consider 5 mainstream image generation benchmarks with various resolutions for evaluating the generalization ability of our Morse, including CIFAR-10 (32×32, 50 thousand imag...
work page 2009
-
[9]
dataset. Following the popular evaluation protocol, we evaluate the text-to-image diffusion models under zero-shot text-to-image generation on the MS-COCO 2014 validation set (Lin et al.,
work page 2014
-
[12]
Latency (second) per sampling step of the Dash models and the Dot models on different GPUs. N denotes that the Dot model is Ntimes faster than the Dash model.h×wdenotes the resolution of input feature maps. Model Source Benchmark RTX 3090 RTX 4090 Tesla V100 Dash Dot N Dash Dot N Dash Dot N DDPM CIFAR-10 (32×32) 0.072 0.012 6.0 0.035 0.006 5.8 0.082 0.015...
-
[13]
For a DM, we collect its official pre-trained model as the Dash model
and ADM (Dhariwal & Nichol, 2021). For a DM, we collect its official pre-trained model as the Dash model. To construct the corresponding Dot model, we add two lightweight down-sampling blocks and up-sampling blocks on the top and under the bottom of each pre-trained Dash model respectively. With the weight sharing strategy, all the Dot models are trained ...
work page 2021
-
[20]
With the official settings, Stable Diffusion v1.4 is pre-trained with around 2 billion text-image pairs at resolution 256 ×256 and fine-tuned with around 600M text-image pairs at resolution 512×512 from LAION-5B dataset (Schuhmann et al., 2022). We add two trainable down-sampling blocks and up-sampling blocks, with the numbers of channels 96 and 160, on t...
work page 2022
Show all 13 references
-
[2014]
All the generated images are down-sampled from 512×512 to 256×256 for evaluation
(256×256). All the generated images are down-sampled from 512×512 to 256×256 for evaluation. For each DM, we generate 30000 samples with the prompts from the validation set. The CLIP scores are calculated using ViT-g/14. A.2. Implementation Details for Stable Diffusion Impleme...
2022
-
[2021]
Hierarchical text-conditional image generation with clip latents.arXiv preprint arXiv:2204.06125,
Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., and Chen, M. Hierarchical text-conditional image generation with clip latents.arXiv preprint arXiv:2204.06125,
-
[2022]
La- tent consistency models: Synthesizing high-resolution images with few-step inference.arXiv preprint arXiv:2310.04378,
Luo, S., Tan, Y ., Huang, L., Li, J., and Zhao, H. La- tent consistency models: Synthesizing high-resolution images with few-step inference.arXiv preprint arXiv:2310.04378,
-
[2023]
Accelerating large language model decoding with speculative sampling.arXiv preprint arXiv:2302.01318, 2023a
Chen, C., Borgeaud, S., Irving, G., Lespiau, J.-B., Sifre, L., and Jumper, J. Accelerating large language model decoding with speculative sampling.arXiv preprint arXiv:2302.01318, 2023a. Chen, T., Zhang, R., and Hinton, G. Analog bits: Gen- erating discrete data using diffusio...
-
[2024]
S., Li, L., Yang, S., Wang, Y ., Cheng, M.-M., and Yang, J
Li, S., Hu, T., Khan, F. S., Li, L., Yang, S., Wang, Y ., Cheng, M.-M., and Yang, J. Faster diffusion: Rethinking the role of unet encoder in diffusion models.arXiv preprint arXiv:2312.09608, 2023a. Li, X., Wang, Y ., Zhou, Z., and Qiao, Y . Smallbignet: Inte- grating core and...
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.