Recognition: unknown
MixFlow: Mixed Source Distributions Improve Rectified Flows
Pith reviewed 2026-05-10 17:05 UTC · model grok-4.3
The pith
Linear mixtures of unconditional and conditioned source distributions reduce curvature in rectified flows.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
Rectified flow models trained with MixFlow optimize their velocity fields on linear combinations of an unconditional Gaussian source and a kappa-FC conditioned source. The mixture produces lower path curvature, tighter source-to-data alignment, quicker training convergence, and improved sample quality measured by FID.
What carries the argument
MixFlow training on linear mixtures of unconditional Gaussian and kappa-FC source distributions, which straightens the learned generative trajectories by improving initial alignment.
Load-bearing premise
The linear mixture reliably lowers path curvature without harming sample diversity or creating new training instabilities.
What would settle it
No measurable drop in FID or no reduction in required sampling steps when the same model is trained with MixFlow versus standard rectified flow on image benchmarks.
Figures
read the original abstract
Diffusion models and their variations, such as rectified flows, generate diverse and high-quality images, but they are still hindered by slow iterative sampling caused by the highly curved generative paths they learn. An important cause of high curvature, as shown by previous work, is independence between the source distribution (standard Gaussian) and the data distribution. In this work, we tackle this limitation by two complementary contributions. First, we attempt to break away from the standard Gaussian assumption by introducing $\kappa\texttt{-FC}$, a general formulation that conditions the source distribution on an arbitrary signal $\kappa$ that aligns it better with the data distribution. Then, we present MixFlow, a simple but effective training strategy that reduces the generative path curvatures and considerably improves sampling efficiency. MixFlow trains a flow model on linear mixtures of a fixed unconditional distribution and a $\kappa\texttt{-FC}$-based distribution. This simple mixture improves the alignment between the source and data, provides better generation quality with less required sampling steps, and accelerates the training convergence considerably. On average, our training procedure improves the generation quality by 12\% in FID compared to standard rectified flow and 7\% compared to previous baselines under a fixed sampling budget. Code available at: $\href{https://github.com/NazirNayal8/MixFlow}{https://github.com/NazirNayal8/MixFlow}$
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes κ-FC, a general formulation for conditioning the source distribution in rectified flows on an arbitrary signal κ to better align it with the data, and MixFlow, a training strategy that optimizes flow models on linear mixtures of a fixed unconditional source and a κ-FC source. It claims this mixture reduces generative path curvature, accelerates convergence, and yields average FID improvements of 12% over standard rectified flow and 7% over prior baselines under fixed sampling budgets, with code released.
Significance. If the reported FID gains are reproducible and the mechanism is confirmed, the work could offer a practical, low-overhead way to improve sampling efficiency in flow-based generative models without architectural changes. The public code release supports reproducibility and is a clear strength.
major comments (2)
- [Experimental results] Experimental results section: the central claim attributes the 12% FID gain and faster convergence to reduced path curvature from the linear mixture, yet no direct curvature diagnostics (e.g., average ∫||v_t|| dt, integrated squared acceleration, or OT cost between effective source and data) are reported; FID at fixed NFE alone cannot isolate this mechanism from optimization or regularization effects.
- [Ablation studies] Ablation studies: the manuscript lacks a controlled comparison of the full MixFlow mixture against training on the κ-FC component alone, so it is impossible to determine whether the reported gains require the mixture or are already achieved by the conditioning term.
minor comments (1)
- [Abstract] Abstract: experimental details (dataset, model size, exact baselines, number of runs, statistical significance) are omitted, making it difficult to assess the strength of the 12% and 7% claims without the full text.
Simulated Author's Rebuttal
We thank the referee for the constructive review and for recognizing the potential practical value of our approach along with the code release. We address each major comment below and will update the manuscript accordingly to strengthen the experimental support for our claims.
read point-by-point responses
-
Referee: Experimental results section: the central claim attributes the 12% FID gain and faster convergence to reduced path curvature from the linear mixture, yet no direct curvature diagnostics (e.g., average ∫||v_t|| dt, integrated squared acceleration, or OT cost between effective source and data) are reported; FID at fixed NFE alone cannot isolate this mechanism from optimization or regularization effects.
Authors: We agree that direct curvature diagnostics would provide stronger mechanistic evidence and help rule out confounding factors. The original manuscript relies on indirect indicators (FID at fixed NFEs, convergence speed, and qualitative path visualizations). In the revised version we will add quantitative metrics including average integrated velocity norm along trajectories and OT cost between the effective source and data distributions to better isolate the contribution of reduced curvature. revision: yes
-
Referee: Ablation studies: the manuscript lacks a controlled comparison of the full MixFlow mixture against training on the κ-FC component alone, so it is impossible to determine whether the reported gains require the mixture or are already achieved by the conditioning term.
Authors: We acknowledge that the current ablations do not include a direct head-to-head comparison of training on the κ-FC source in isolation versus the proposed linear mixture. While the paper already shows gains relative to standard rectified flow, adding this controlled ablation will clarify whether the mixture itself is necessary. We will include these results in the revised manuscript. revision: yes
Circularity Check
Empirical FID gains from mixture training; no derivation reduces to self-definition or fitted prediction
full rationale
The paper introduces κ-FC and MixFlow as a training procedure that mixes source distributions, then reports average 12% FID improvement over standard rectified flow under fixed sampling budgets. All central claims are supported by external experimental metrics on image datasets rather than any equation that defines the output in terms of itself or renames a fitted parameter as a prediction. The curvature-reduction mechanism is presented as a hypothesis supported by prior literature and empirical outcomes, with no self-citation chain or uniqueness theorem invoked to force the result. This is a standard non-circular empirical contribution.
Axiom & Free-Parameter Ledger
Reference graph
Works this paper leans on
-
[1]
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...
-
[2]
David Berthelot, Arnaud Autef, Jierui Lin, Dian Ang Yap, Shuangfei Zhai, Siyuan Hu, Daniel Zheng, Walter Talbot, and Eric Gu. Tract: Denoising diffusion models with transitive closure time-distillation. ArXiv, abs/2303.04248, 2023. URL https://api.semanticscholar.org/CorpusID:257404979
-
[3]
Ricky T. Q. Chen. torchdiffeq, 2018. URL https://github.com/rtqichen/torchdiffeq
work page 2018
-
[4]
Stargan v2: Diverse image synthesis for multiple domains
Yunjey Choi, Youngjung Uh, Jaejun Yoo, and Jung-Woo Ha. Stargan v2: Diverse image synthesis for multiple domains. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020
work page 2020
-
[5]
GENIE : Higher-order denoising diffusion solvers
Tim Dockhorn, Arash Vahdat, and Karsten Kreis. GENIE : Higher-order denoising diffusion solvers. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Processing Systems, 2022. URL https://openreview.net/forum?id=LKEYuYNOqx
work page 2022
-
[6]
Zhengyang Geng, Ashwini Pokle, Weijian Luo, Justin Lin, and J Zico Kolter. Consistency models made easy. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=xQVxo9dSID
work page 2025
-
[7]
Variational rectified flow matching
Pengsheng Guo and Alex Schwing. Variational rectified flow matching. In ICLR 2025 Workshop on Deep Generative Model in Machine Learning: Theory, Principle and Efficacy, 2025. URL https://openreview.net/forum?id=ZLL6SYNptz
work page 2025
-
[8]
Coupled variational autoencoder
Xiaoran Hao and Patrick Shafto. Coupled variational autoencoder. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp.\ 12546--12555. PMLR, 23--29 Jul 2023. URL https:...
work page 2023
-
[9]
Denoising diffusion probabilistic models
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp.\ 6840--6851. Curran Associates, Inc., 2020. URL https://proceedings.neurips.cc/paper_files/paper/2020/file/4c5bcfec8584af0d967f1ab10...
work page 2020
-
[10]
A style-based generator architecture for generative adversarial networks
Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2019
work page 2019
-
[11]
Elucidating the design space of diffusion-based generative models
Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems, volume 35, pp.\ 26565--26577. Curran Associates, Inc., 2022 a
work page 2022
-
[12]
Elucidating the design space of diffusion-based generative models
Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems, volume 35, pp.\ 26565--26577. Curran Associates, Inc., 2022 b . URL https://proceedings.neurips.cc/paper_...
work page 2022
-
[13]
Simple reflow: Improved techniques for fast flow models
Beomsu Kim, Yu-Guan Hsieh, Michal Klein, marco cuturi, Jong Chul Ye, Bahjat Kawar, and James Thornton. Simple reflow: Improved techniques for fast flow models. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=fpvgSDKXGY
work page 2025
-
[14]
Adam: A Method for Stochastic Optimization
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. CoRR, abs/1412.6980, 2014. URL https://api.semanticscholar.org/CorpusID:6628106
work page internal anchor Pith review Pith/arXiv arXiv 2014
-
[15]
Learning multiple layers of features from tiny images
Alex Krizhevsky. Learning multiple layers of features from tiny images. pp.\ 32--33, 2009. URL https://www.cs.toronto.edu/ kriz/learning-features-2009-TR.pdf
work page 2009
-
[16]
Minimizing trajectory curvature of ODE -based generative models
Sangyun Lee, Beomsu Kim, and Jong Chul Ye. Minimizing trajectory curvature of ODE -based generative models. In Proceedings of the 40th International Conference on Machine Learning, Proceedings of Machine Learning Research. PMLR, 23--29 Jul 2023
work page 2023
-
[17]
Learning quantized adaptive conditions for diffusion models
Yuchen Liang, Yuchan Tian, Lei Yu, Huaao Tang, Jie Hu, Xiangzhong Fang, and Hanting Chen. Learning quantized adaptive conditions for diffusion models. In Computer Vision – ECCV 2024: 18th European Conference, Milan, Italy, September 29–October 4, 2024, Proceedings, Part LXXXI, 2024
work page 2024
-
[18]
Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Le. Flow matching for generative modeling. In The Eleventh International Conference on Learning Representations, 2023
work page 2023
-
[19]
Flow straight and fast: Learning to generate and transfer data with rectified flow
Xingchao Liu, Chengyue Gong, and qiang liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. In The Eleventh International Conference on Learning Representations, 2023
work page 2023
-
[20]
DPM -solver: A fast ODE solver for diffusion probabilistic model sampling in around 10 steps
Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. DPM -solver: A fast ODE solver for diffusion probabilistic model sampling in around 10 steps. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Processing Systems, 2022. URL https://openreview.net/forum?id=2uAaGwlP_V
work page 2022
-
[21]
Knowledge Distillation in Iterative Generative Models for Improved Sampling Speed
Eric Luhman and Troy Luhman. Knowledge distillation in iterative generative models for improved sampling speed. ArXiv, abs/2101.02388, 2021. URL https://api.semanticscholar.org/CorpusID:230799531
work page internal anchor Pith review arXiv 2021
-
[22]
Robert J. McCann. A convexity principle for interacting gases. Advances in Mathematics, 128 0 (1): 0 153--179, 1997. ISSN 0001-8708. doi:https://doi.org/10.1006/aima.1997.1634. URL https://www.sciencedirect.com/science/article/pii/S0001870897916340
-
[23]
Aram-Alexandre Pooladian, Heli Ben-Hamu, Carles Domingo-Enrich, Brandon Amos, Yaron Lipman, and Ricky T. Q. Chen. Multisample flow matching: Straightening flows with minibatch couplings. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Proceedings of the 40th International Conference on Mach...
work page 2023
-
[24]
Progressive distillation for fast sampling of diffusion models
Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. In International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=TIdIXIpzhoI
work page 2022
-
[25]
VCT : Training consistency models with variational noise coupling
Gianluigi Silvestri, Luca Ambrogioni, Chieh-Hsin Lai, Yuhta Takida, and Yuki Mitsufuji. VCT : Training consistency models with variational noise coupling. In Forty-second International Conference on Machine Learning, 2025. URL https://openreview.net/forum?id=CMoX0BEsDs
work page 2025
-
[26]
Denoising diffusion implicit models
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In International Conference on Learning Representations, 2021 a . URL https://openreview.net/forum?id=St1giarCHLP
work page 2021
-
[27]
Improved techniques for training consistency models
Yang Song and Prafulla Dhariwal. Improved techniques for training consistency models. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=WNzy9bRDvG
work page 2024
-
[28]
Score-based generative modeling through stochastic differential equations
Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, 2021 b . URL https://openreview.net/forum?id=PxTIG12RRHS
work page 2021
-
[29]
Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. In International Conference on Machine Learning, 2023. URL https://api.semanticscholar.org/CorpusID:257280191
work page 2023
-
[30]
Improving and generalizing flow-based generative models with minibatch optimal transport
Alexander Tong, Kilian FATRAS, Nikolay Malkin, Guillaume Huguet, Yanlei Zhang, Jarrid Rector-Brooks, Guy Wolf, and Yoshua Bengio. Improving and generalizing flow-based generative models with minibatch optimal transport. Transactions on Machine Learning Research, 2024. ISSN 2835-8856. URL https://openreview.net/forum?id=CD9Snc73AW. Expert Certification
work page 2024
-
[31]
Methods17, 261–272, DOI: 10.1038/s41592-019-0686-2 (2020)
Pauli Virtanen, Ralf Gommers, Travis E. Oliphant, Matt Haberland, Tyler Reddy, David Cournapeau, Evgeni Burovski, Pearu Peterson, Warren Weckesser, Jonathan Bright, St \'e fan J. van der Walt , Matthew Brett, Joshua Wilson, K. Jarrod Millman, Nikolay Mayorov, Andrew R. J. Nelson, Eric Jones, Robert Kern, Eric Larson, C J Carey, \.I lhan Polat, Yu Feng, Er...
-
[32]
Rectified diffusion: Straightness is not your need in rectified flow
Fu-Yun Wang, Ling Yang, Zhaoyang Huang, Mengdi Wang, and Hongsheng Li. Rectified diffusion: Straightness is not your need in rectified flow. In The Thirteenth International Conference on Learning Representations, 2025 a . URL https://openreview.net/forum?id=nEDToD1R8M
work page 2025
-
[33]
Block flow: Learning straight flow on data blocks, 2025 b
Zibin Wang, Zhiyuan Ouyang, and Xiangyun Zhang. Block flow: Learning straight flow on data blocks, 2025 b
work page 2025
-
[34]
Tackling the generative learning trilemma with denoising diffusion GAN s
Zhisheng Xiao, Karsten Kreis, and Arash Vahdat. Tackling the generative learning trilemma with denoising diffusion GAN s. In International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=JprM0p-q0Co
work page 2022
-
[35]
Kingma, Tingbo Hou, Ying Nian Wu, Kevin Patrick Murphy, Tim Salimans, Ben Poole, and Ruiqi Gao
Sirui Xie, Zhisheng Xiao, Diederik P. Kingma, Tingbo Hou, Ying Nian Wu, Kevin Patrick Murphy, Tim Salimans, Ben Poole, and Ruiqi Gao. Em distillation for one-step diffusion models. ArXiv, abs/2405.16852, 2024. URL https://api.semanticscholar.org/CorpusID:270062581
-
[36]
Consistency flow matching: Defining straight flows with velocity consistency,
Ling Yang, Zixiang Zhang, Zhilong Zhang, Xingchao Liu, Minkai Xu, Wentao Zhang, Chenlin Meng, Stefano Ermon, and Bin Cui. Consistency flow matching: Defining straight flows with velocity consistency. CoRR, abs/2407.02398, 2024. URL https://doi.org/10.48550/arXiv.2407.02398
-
[37]
Simple and fast distillation of diffusion models
Zhenyu Zhou, Defang Chen, Can Wang, Chun Chen, and Siwei Lyu. Simple and fast distillation of diffusion models. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang (eds.), Advances in Neural Information Processing Systems, volume 37, pp.\ 40831--40860. Curran Associates, Inc., 2024. URL https://proceedings.neurips.cc/paper...
work page 2024
-
[38]
\@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should not add it explicitly Type <Return> for now, but then later remove the command n...
-
[39]
\@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@first@sw \@firstoftwo \@ifundefined NAT@b*@#2 \@firstoftwo @num @NAT@ctr \@secondoft...
-
[40]
@open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibsetup #1 @NAT@ctr @ @openbib .11em \@plus.33em \@minus.07em 4000 4000 `\.\@m @bibit...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.