REVIEW 3 major objections 5 minor 71 references
Joint Flow Matching makes a single reversible flow both a generator and a classifier, with a provable consistency relation between the two conditionals.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-07-31 23:27 UTC pith:Z4VPQXSI
load-bearing objection Neat idea with a sound consistency theorem, but the experiments violate the theorem's key assumption and the empirical comparison is too thin to carry the claims. the 3 major comments →
Joint Flow Matching for Generator-Consistent Classification
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The central claim is that placing image and label variables at opposite temporal endpoints turns one continuous normalising flow into both a generator and a classifier. With endpoint distributions p0(x,y)=p_noise(x)p_data(y) and p1(x,y)=p_data(x)p_noise(y), the flow implicitly defines a joint distribution pJFM as the pair of the forward-flow image output and the label input. Because the flow is reversible, integrating forward from a label samples pJFM(x|y), and integrating backward from an image samples pJFM(y|x), with the consistency identity pJFM(x|y)p_data(y)=pJFM(y|x)p_data(x) holding exactly. This is internal consistency: pJFM is the coupling induced by the trained flow, not necessarily
What carries the argument
The load-bearing object is the map Psi(x0,y0) = (pi_x(Phi_1(x0,y0)), y0), which pairs the image at the end of the forward flow with the label at its start, and its pushforward pJFM = Psi_* p0. This map isolates the learned coupling between image and label in one joint distribution; the consistency relation then follows from Bayes' theorem applied to that joint. The construction works because one variable travels noise-to-data while the other travels data-to-noise across the same time interval, so a single ODE provides both conditional directions.
Load-bearing premise
The load-bearing premise is that each label slice acts like a one-to-one smooth map — no two different noise inputs with the same label land on the same image — combined with treating one-hot labels as continuous variables; if either fails, the joint density and the consistency identity are not well-defined.
What would settle it
On a trained JFM model, reverse-flow a fixed set of images with many label-noise samples to estimate pJFM(y|x), then forward-flow many noise samples from those labels to estimate pJFM(x|y), and check whether pJFM(x|y)p_data(y) equals pJFM(y|x)p_data(x) within Monte Carlo error; a systematic mismatch would falsify the consistency claim. Separately, look for two distinct noise seeds with the same label that flow to nearly identical images — frequent collisions would violate the slice-diffeomorphism assumption.
If this is right
- A single trained JFM model performs class-conditional generation and classification without a separate classifier or approximate inference, with a guaranteed consistency between the two.
- The consistency relation holds for any velocity model satisfying the endpoint conditions, so trajectory straightening or optimal-transport couplings can be added without breaking the guarantee.
- Reverse flow recovers a class-independent noise vector from an image; re-flowing that noise with a different label swaps class while preserving context, yielding counterfactual visualisations from the same model.
- Confidence scores are naturally calibrated without post-hoc calibration in the reported experiments: ECE 0.003 on MNIST and 0.057 on CIFAR-10.
- The induced joint pJFM gives a model-internal view of what a classifier relies on: grouping generated images by reverse-flow confidence reveals features that drive class decisions.
Where Pith is reading between the lines
- This consistency suggests a benchmark for post-hoc explainability: a saliency method should agree with the features that actually change when JFM re-flows an image's recovered noise under a different label.
- Because pJFM is model-internal, 'consistent' means internally coupled, not externally correct; this distinction, noted in the paper, is easy to miss and should be highlighted in any application.
- Extending JFM to continuous Y for regression or structured outputs is natural; the paper leaves that reverse conditional uncharacterised, so the first check would be whether Eq. 7 holds empirically for continuous targets.
- The slice-diffeomorphism assumption can be tested directly by searching for collisions in forward integration from different noise seeds with the same label; non-negligible collisions would break the density derivation.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper introduces Joint Flow Matching (JFM), a continuous normalising flow framework for paired variables (X,Y). The key design is to assign the data distribution of X and the noise distribution of Y at t=0, and the noise distribution of X and the data distribution of Y at t=1. The paper defines an induced joint distribution pJFM over (x1,y0) and claims that forward integration (generation) and backward integration (classification) through the same flow yield the conditionals pJFM(x1|y0) and pJFM(y0|x1), giving a provable consistency between generation and classification (Eq. 7). The authors prove this under a slice-diffeomorphism assumption and validate on a spiral toy problem, MNIST, and CIFAR-10, reporting competitive accuracy and low expected calibration error.
Significance. The idea of exploiting opposite endpoint roles to obtain bidirectional consistency from a single flow is elegant and, to my knowledge, novel. The core change-of-variables derivation for the continuous case is sound, and the paper correctly identifies that the consistency relation is internal to the model rather than a statement about matching the true data distribution. The empirical results show the framework can be trained on standard image benchmarks and that calibration emerges without post-hoc processing; the interpretability applications (Figures 3 and 4) are suggestive. However, the central guarantee is formulated for continuous densities, while the main classification experiments use discrete one-hot labels, so the provable consistency claim does not apply to the paper's primary setting. This gap is not merely cosmetic and must be addressed before the claims can be accepted.
major comments (3)
- [§2.2, Eq. (6)-(7); §4.2] The theory assumes p0 and p1 have densities w.r.t. Lebesgue measure. In the classification experiments, pdata(Y) is discrete on one-hot vectors, so p0 is supported on a finite union of D-dimensional affine subspaces of R^{D+K}. Since Φ1 is a diffeomorphism, (Φ1)_*p0 is also singular, and cannot equal p1 = pdata(X)N(0, σ_yI), which is absolutely continuous. Thus the endpoint condition cannot be exactly satisfied, Eq. (6) is not valid, and Eq. (7) is not defined. The manuscript acknowledges the discreteness in §4.2 but does not reconcile it with the theory; the statement that the formulation was 'sufficient' does not rescue the provable-consistency claim for MNIST/CIFAR-10. Please provide a rigorous treatment (e.g., formalize the model with label noise/smoothing so Y has a density) or restrict the theorem to continuous Y and reframe the experimental claims as an approximation.
- [§2.3] The proof of the backward direction is only a sketch. The sentence 'by bijectivity of Φ1' is not enough to show that sampling y1~pnoise(Y) and inverting the flow yields pJFM(y0|x1). Please write out the full argument: since (x1,y1) ~ p1 implies (x0,y0)=Φ^{-1}(x1,y1) ~ p0, and pJFM = Ψ_*p0 with Ψ(x0,y0)=(πxΦ1(x0,y0),y0), the induced distribution of (x1,y0) under this sampling scheme is exactly pJFM; the conditional then follows. This derivation also clarifies where the slice-diffeomorphism assumption is used.
- [§2.2] The slice-diffeomorphism assumption, that πx∘Φ1(·,y0) is a diffeomorphism for almost every y0, is essential for Eq. (6) yet is stated to be not derivable from ODE regularity. Since the paper's central formal object pJFM is defined through this map, the assumption should be either proved under suitable conditions or empirically verified (e.g., by checking invertibility of the slice Jacobian on test labels). As written, the abstract's unconditional 'We prove' overstates the result.
minor comments (5)
- [§4.2] The sentence 'We trained U-Net to model the objective described in Equation 7 using the loss defined in Equation 8' is confusing; the objective is defined in Equation 1 or 8, not 7.
- [Related Work] Typo: 'mutliple' should be 'multiple'; 'form a set' should be 'from a set'. Also, the citation 'CycleGAN Zhao et al. [2020]' appears to mis-attribute the CycleGAN method; the cited paper is not the original CycleGAN paper (Zhu et al., 2017).
- [§2.2] The phrase 'or equivalently the Y-output of the reverse flow paired with the Y-input' should read 'paired with the X-input'.
- [Conclusion] Typo: 'JMF' should be 'JFM'.
- [Eq. (7)] Clarify that the equality holds exactly only under the ideal endpoint condition (Φ1)_*p0 = p1; in practice it is an approximate identity, since pJFM's x1-marginal may differ from pdata(x1) under approximate training.
Circularity Check
No significant circularity: JFM's consistency guarantee is a formal theorem from the definition of pJFM and flow bijectivity, not a fitted prediction or self-citation chain.
full rationale
The central consistency claim (Eq. 7) is derived, not fitted. pJFM is explicitly defined as Ψ_*p0 (Eqs. 5-6), so forward sampling x1|y0 is pJFM(x1|y0) by definition; the paper states this directly ('Both claims follow directly from the definition of pJFM as Ψ∗p0'). The backward direction is the substantive step: it uses bijectivity of Φ1 and the endpoint condition (Φ1)_*p0 = p1 to recover the same conditional pJFM(y0|x1) and to identify the marginals with pdata, yielding Bayes' identity. No trainable parameters are fit to enforce consistency; the result holds for any diffeomorphic flow satisfying the endpoint conditions. There are no load-bearing self-citations (the paper has no self-citations), no imported uniqueness theorem, and no ansatz smuggled in via citation. The acknowledged limitations—the slice-diffeomorphism assumption (§2.2) and the use of discrete one-hot labels (§4.2, §5.2)—are correctness/assumption gaps that can invalidate the density derivation in practice, but they are not circular: they do not make Eq. 7 an input to itself. Hence score 0.
Axiom & Free-Parameter Ledger
free parameters (3)
- label noise scale σy =
0.1
- image noise scale σx =
1.0
- number of Monte Carlo samples for confidence =
10
axioms (4)
- standard math Standard flow matching theory: minimizing the conditional flow matching loss (Eq. 8) is equivalent to learning the marginal vector field that pushes p0 to p1.
- standard math Regularity conditions on v ensure Φt is a diffeomorphism for each t.
- domain assumption Slice-diffeomorphism: for almost every y0, πx∘Φ1(·,y0) is a diffeomorphism.
- domain assumption One-hot label vectors with Gaussian noise approximate a continuous endpoint distribution.
invented entities (1)
-
pJFM(X,Y) joint distribution
independent evidence
read the original abstract
We introduce Joint Flow Matching (JFM), a training framework for continuous normalising flows over multiple variables. Standard flow matching transports variables from noise to data simultaneously, offering no natural mechanism for forward and reverse conditional inference from a shared joint model. JFM resolves this by assigning opposite roles to each variable at the temporal endpoints. We prove that JFM produces a consistent joint distribution where that forward or reverse integration are conditionals of the same joint. We explore this consistency in the context of joint classification and generation as the basis for interpretability in discriminative-generative models. We validate JFM on conditional datasets producing competitive accuracy with inherently well-calibrated confidence scores without post-hoc calibration, and classifier-consistent image generation.
Figures
Reference graph
Works this paper leans on
-
[1]
Albergo, Michael S. and. Building. doi:10.48550/arXiv.2209.15571 , archiveprefix =. 2209.15571 , primaryclass =
-
[2]
and Welling, Max , year = 2019, number =
van den Berg, Rianne and Hasenclever, Leonard and Tomczak, Jakub M. and Welling, Max , year = 2019, number =. Sylvester. doi:10.48550/arXiv.1803.05649 , archiveprefix =. 1803.05649 , primaryclass =
-
[3]
Bertrand, Quentin and Gagneux, Anne and Massias, Mathurin and Emonet, R. On the
-
[4]
Chen, Ricky T. Q. and Rubanova, Yulia and Bettencourt, Jesse and Duvenaud, David K , year = 2018, volume =. Neural. Advances in
2018
-
[5]
Chen, Ricky T. Q. and Lipman, Yaron , year = 2024, booktitle =. Flow
2024
-
[6]
doi:10.48550/arXiv.1410.8516 , archiveprefix =
Dinh, Laurent and Krueger, David and Bengio, Yoshua , year = 2015, number =. doi:10.48550/arXiv.1410.8516 , archiveprefix =. 1410.8516 , primaryclass =
-
[7]
Density Estimation Using
Dinh, Laurent and. Density Estimation Using
-
[8]
TO-FLOW: Efficient Continuous Normalizing Flows with Temporal Optimization adjoint with Moving Speed
Du, Shian and Luo, Yihong and Chen, Wei and Xu, Jian and Zeng, Delu , year = 2022, number =. doi:10.48550/arXiv.2203.10335 , archiveprefix =. 2203.10335 , primaryclass =
work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2203.10335 2022
-
[9]
Gagneux, Anne and Martin, S. The. arXiv preprint arXiv:s2510.24830 , eprint =
-
[10]
Grathwohl, Will and Chen, Ricky T. Q. and Bettencourt, Jesse and Sutskever, Ilya and Duvenaud, David , year = 2019, booktitle=
2019
-
[11]
Gu, Jiatao and Shen, Ying and Chen, Tianrong and Dinh, Laurent and Wang, Yuyang and Bautista, Miguel Angel and Berthelot, David and Susskind, Josh and Zhai, Shuangfei , year = 2025, journal =. 2511.20462 , primaryclass =
arXiv 2025
-
[12]
Gu, Jiatao and Chen, Tianrong and Berthelot, David and Zheng, Huangjie and Wang, Yuyang and Zhang, Ruixiang and Dinh, Laurent and Bautista, Miguel Angel and Susskind, Josh and Zhai, Shuangfei , year =
-
[13]
Generalization in Diffusion Models Arises from Geometry-Adaptive Harmonic Representations , author =
-
[14]
and Dhariwal, Prafulla , year = 2018, number =
Kingma, Diederik P. and Dhariwal, Prafulla , year = 2018, number =. Glow:. doi:10.48550/arXiv.1807.03039 , archiveprefix =. 1807.03039 , primaryclass =
-
[15]
and Salimans, Tim and Jozefowicz, Rafal and Chen, Xi and Sutskever, Ilya and Welling, Max , year = 2016, booktitle =
Kingma, Diederik P. and Salimans, Tim and Jozefowicz, Rafal and Chen, Xi and Sutskever, Ilya and Welling, Max , year = 2016, booktitle =. Improving
2016
-
[16]
Neural Information Processing Systems , author =
Optimal. Neural Information Processing Systems , author =
-
[17]
Kumar, Manoj and Babaeizadeh, Mohammad and Erhan, Dumitru and Finn, Chelsea and Levine, Sergey and Dinh, Laurent and Kingma, Durk , year = 2020, booktitle=
2020
-
[18]
L. A Survey of the. doi:10.48550/arXiv.1308.0215 , archiveprefix =. 1308.0215 , primaryclass =
-
[19]
International Conference on Learning Representations , year=
Flow Matching for Generative Modeling , author=. International Conference on Learning Representations , year=
-
[20]
Lipman, Yaron and Chen, Ricky T. Q. and. Flow
-
[21]
International Conferenceon Learning Representations , author =
Flow. International Conferenceon Learning Representations , author =
-
[22]
Towards a
Niedoba, Matthew and Zwartsenberg, Berend and Murphy, Kevin and Wood, Frank , year = 2025, booktitle=. Towards a
2025
-
[23]
Pokle, Ashwini and Muckley, Matthew J. and Chen, Ricky T. Q. and Karrer, Brian , year = 2024, number =. Training-Free. doi:10.48550/arXiv.2310.04432 , archiveprefix =. 2310.04432 , primaryclass =
-
[24]
International Conference on Machine Learning , author =
Multisample. International Conference on Machine Learning , author =
-
[25]
Variational
Rezende, Danilo Jimenez and Mohamed, Shakir , year = 2015, booktitle =. Variational
2015
-
[26]
Tabak, E. G. and Turner, Cristina V. , year = 2013, journal =. A
2013
-
[27]
and Welling, Max , year = 2017, number =
Tomczak, Jakub M. and Welling, Max , year = 2017, number =. Improving. doi:10.48550/arXiv.1611.09630 , archiveprefix =. 1611.09630 , primaryclass =
-
[28]
Improving and Generalizing Flow-Based Generative Models with Minibatch Optimal Transport , author =
-
[29]
Generalization through Variance: How Noise Shapes Inductive Biases in Diffusion Models , author =
-
[30]
Zhang, Yasi and Yu, Peiyu and Zhu, Yaxuan and Chang, Yingshan and Gao, Feng and Wu, Ying Nian and Leong, Oscar , booktitle =. Flow
-
[31]
Goodfellow, Ian J. and. Generative. Advances in
-
[32]
and Sutton, Charles , year = 2017, volume =
Srivastava, Akash and Valkov, Lazar and Russell, Chris and Gutmann, Michael U. and Sutton, Charles , year = 2017, volume =. Advances in
2017
-
[33]
Catastrophic Forgetting and Mode Collapse in
-
[34]
and Rezende, Danilo J
Kingma, Diederik P. and Rezende, Danilo J. and Mohamed, Shakir and Welling, Max , year = 2014, volume =. Semi-Supervised. Advances in
2014
-
[35]
Denoising
Ho, Jonathan and Jain, Ajay and Abbeel, Pieter , year = 2020, volume =. Denoising. Advances in
2020
-
[36]
International Conference on Machine Learning , author =
Deep. International Conference on Machine Learning , author =
-
[37]
Generative
Song, Yang and Ermon, Stefano , year = 2019, journal =. Generative
2019
-
[38]
Song, Yang and. Score-
-
[39]
Diffusion
Yang, Ling and Zhang, Zhilong and Song, Yang and Hong, Shenda and Xu, Runsheng and Zhao, Yue and Zhang, Wentao and Cui, Bin and Yang, Ming-Hsuan , year = 2023, journal =. Diffusion
2023
-
[40]
Implicit
Du, Yilun and Mordatch, Igor , year = 2019, volume =. Implicit. Advances in
2019
-
[41]
Song, Yang and Kingma, Diederik P. , year = 2021, journal =. How to. 2101.03288 , primaryclass =
Pith/arXiv arXiv 2021
-
[42]
International Conference Machine Learning , author =
Learning the. International Conference Machine Learning , author =
-
[43]
Grathwohl, Will and Kelly, Jacob and Hashemi, Milad and Norouzi, Mohammad and Swersky, Kevin and Duvenaud, David , year = 2021, booktitle=. No
2021
-
[44]
Lu, You and Huang, Bert , year = 2020, journal =. Structured. doi:10.1609/aaai.v34i04.5940 , copyright =
-
[45]
Sorkhei, Moein and Henter, Gustav Eje and Kjellstr. Full-. Pattern. doi:10.1007/978-3-030-92659-5_45 , isbn =
-
[46]
Classifier-
Ho, Jonathan and Salimans, Tim , booktitle=. Classifier-
-
[47]
Li, Xiao-Hui and Shi, Yuhan and Li, Haoyang and Bai, Wei and Cao, Caleb Chen and Chen, Lei , year = 2021, pages =. An. Proceedings of the
2021
-
[48]
Simonyan, Karen and Vedaldi, Andrea and Zisserman, Andrew , year = 2014, journal =. Deep. 1312.6034 , primaryclass =
Pith/arXiv arXiv 2014
-
[49]
and Fergus, Rob , editor =
Zeiler, Matthew D. and Fergus, Rob , editor =. Visualizing and. Computer
-
[50]
Proceedings of the
Goetschalckx, Lore and Andonian, Alex and Oliva, Aude and Isola, Phillip , year = 2019, pages =. Proceedings of the
2019
-
[51]
Lundberg, Scott M and Lee, Su-In , year = 2017, volume =. A. Advances in
2017
-
[52]
Axiomatic
Sundararajan, Mukund and Taly, Ankur and Yan, Qiqi , year = 2017, pages =. Axiomatic. Proceedings of the
2017
-
[53]
Ronneberger, Olaf and Fischer, Philipp and Brox, Thomas , year = 2015, number =. U-. doi:10.48550/arXiv.1505.04597 , archiveprefix =. 1505.04597 , primaryclass =
-
[54]
Attention Is
Vaswani, Ashish and Shazeer, Noam and Parmar, Niki and Uszkoreit, Jakob and Jones, Llion and Gomez, Aidan N and Kaiser,. Attention Is. Advances in
-
[55]
Ioffe, Sergey and Szegedy, Christian , year = 2015, booktitle =. Batch
2015
-
[56]
He, Kaiming and Zhang, Xiangyu and Ren, Shaoqing and Sun, Jian , booktitle =. Deep
-
[57]
Gal, Yarin and Ghahramani, Zoubin , year = 2015, number =. Dropout as a. doi:10.48550/arXiv.1506.02142 , archiveprefix =. 1506.02142 , primaryclass =
-
[58]
Peebles, William and Xie, Saining , year = 2023, number =. Scalable. doi:10.48550/arXiv.2212.09748 , archiveprefix =. 2212.09748 , primaryclass =
-
[59]
Learning
Krizhevsky, Alex and Hinton, Geoffrey , year = 2009, journal =. Learning
2009
-
[60]
LeCun, Yann and Cortes, Corrina and Burges, Christopher , year = 1998, keywords =
1998
-
[61]
and Ba, Jimmy , year = 2017, number =
Kingma, Diederik P. and Ba, Jimmy , year = 2017, number =. Adam:. doi:10.48550/arXiv.1412.6980 , archiveprefix =. 1412.6980 , publisher =
-
[62]
, year = 2016, volume =
Krotov, Dmitry and Hopfield, John J. , year = 2016, volume =. Dense. Advances in
2016
-
[63]
Krotov, Dmitry and Hopfield, John , year = 2018, journal =. Dense. doi:10.1162/neco_a_01143 , keywords =
-
[64]
Learning with
Seeger, Matthias , year = 2000, langid =. Learning with
2000
-
[65]
Proceedings of the 40th International Conference on Machine Learning , pages =
One Transformer Fits All Distributions in Multi-Modal Diffusion at Scale , author =. Proceedings of the 40th International Conference on Machine Learning , pages =. 2023 , editor =
2023
-
[66]
Text-to-Image Diffusion Models are Zero Shot Classifiers , volume =
Clark, Kevin and Jaini, Priyank , booktitle =. Text-to-Image Diffusion Models are Zero Shot Classifiers , volume =
-
[67]
and Prabhudesai, Mihir and Duggal, Shivam and Brown, Ellis and Pathak, Deepak , title =
Li, Alexander C. and Prabhudesai, Mihir and Duggal, Shivam and Brown, Ellis and Pathak, Deepak , title =. Proceedings of the IEEE/CVF International Conference on Computer Vision , year =
-
[68]
Arnold and S
Barry C. Arnold and S. James Press , title =. Journal of the American Statistical Association , volume =. 1989 , publisher =
1989
-
[69]
Proceedings of the European Conference on Computer Vision , pages =
Zhao, Yihao and Wu, Ruihai and Dong, Hao , title =. Proceedings of the European Conference on Computer Vision , pages =. 2020 , publisher =
2020
-
[70]
Diffusion Models Beat GANs on Image Synthesis , volume =
Dhariwal, Prafulla and Nichol, Alexander , booktitle =. Diffusion Models Beat GANs on Image Synthesis , volume =
-
[71]
and van der Sommen, Fons , title =
Caetano, Francisco and Viviers, Christiaan and De With, Peter H.N. and van der Sommen, Fons , title =. Proceedings of the AAAI Conference on Artificial Intelligence , articleno =. 2026 , publisher =
2026
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.