REVIEW 2 major objections 1 minor 70 references
Rectified flows exhibit a bell-shaped gap in reconstruction accuracy between training and test data along their interpolation paths.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · grok-4.3
2026-06-27 22:43 UTC pith:DFSWM5U4
load-bearing objection Rectified flows leak a bell-shaped train-test gap along the interpolation path, with a closed-form peak location only under Gaussian assumptions. the 2 major comments →
Where Rectified Flows Leak: Characterising Membership Signals Along the Interpolation Path
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The reconstruction error gap between train and test points in a rectified flow follows a bell-shaped curve over the interpolation coefficient λ. This gap increases as training progresses while standard validation losses remain stable. When the data and noise are Gaussian, the λ value that maximizes the gap can be calculated exactly from the variances.
What carries the argument
The interpolation path defined by X_λ = (1-λ)X_0 + λ X_1, and the resulting λ-resolved reconstruction gap that serves as a membership signal.
Load-bearing premise
Deriving the exact location of the signal's maximum requires assuming Gaussian distributions for both the data points and the noise.
What would settle it
Collect reconstruction errors for train and test points at many values of λ; if the difference is not bell-shaped or its maximum is far from the predicted λ on Gaussian-like data, the central claim would be falsified.
If this is right
- The signal can be used to distinguish training members from non-members via a membership inference attack.
- The leakage persists even when overall model performance on validation sets looks normal.
- The bell-shaped structure appears consistently across audio and image domains.
- The peak location matches the closed-form prediction when the Gaussian assumption holds.
Where Pith is reading between the lines
- Privacy risks may extend to other generative models that rely on similar linear interpolations during training.
- Regularization targeted at the peak λ could reduce this specific leakage without harming generation quality.
- Combining this signal with existing membership inference methods could improve attack success rates.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper claims that Rectified Flow models exhibit a membership signal consisting of a bell-shaped gap in reconstruction error between train and test data along the interpolation path X_λ = (1-λ)X_0 + λX_1. This gap accumulates during training while validation metrics remain stable. The location of the maximum is derived in closed form under Gaussian assumptions for data and noise. The bell-shaped structure is reported as universal across audio and image domains, with the peak prediction holding when assumptions are met, and the structure is used as the basis for a membership inference attack.
Significance. If the results hold, the work supplies a theoretically grounded characterization of subtle membership leakage in rectified flows, a model family seeing increasing deployment. The explicit closed-form derivation under stated assumptions together with cross-modal empirical checks constitutes a concrete advance over purely observational studies of data retention in generative models, with direct relevance to privacy and copyright questions.
major comments (2)
- [Abstract] Abstract: the manuscript asserts universality of the bell-shaped structure and efficacy for membership inference, yet provides no details on experimental setup (datasets, model sizes, training hyperparameters), statistical significance testing, or controls for confounding factors such as reconstruction metric choice. These omissions are load-bearing for the empirical claims.
- [Theoretical derivation] Theoretical derivation (closed-form peak location): while the Gaussian assumption is stated explicitly, the manuscript does not include sensitivity analysis or synthetic experiments showing how mild departures from normality affect either the existence of the bell shape or the accuracy of the predicted peak location, limiting the practical utility of the closed-form result.
minor comments (1)
- [Abstract] Abstract: 'wich' is a typo and should read 'which'.
Simulated Author's Rebuttal
We thank the referee for their thoughtful review and constructive suggestions. We address each of the major comments point by point below, agreeing where revisions are warranted to strengthen the manuscript.
read point-by-point responses
-
Referee: [Abstract] Abstract: the manuscript asserts universality of the bell-shaped structure and efficacy for membership inference, yet provides no details on experimental setup (datasets, model sizes, training hyperparameters), statistical significance testing, or controls for confounding factors such as reconstruction metric choice. These omissions are load-bearing for the empirical claims.
Authors: The abstract is intentionally concise, as is standard, with full experimental details provided in the main text (Sections 3-5). However, we recognize that including key details in the abstract would improve clarity. We will revise the abstract to briefly mention the datasets used (audio and image domains), model scales, and note that results are statistically significant across multiple runs with controls for the reconstruction metric. This addresses the concern without exceeding typical abstract length. revision: yes
-
Referee: [Theoretical derivation] Theoretical derivation (closed-form peak location): while the Gaussian assumption is stated explicitly, the manuscript does not include sensitivity analysis or synthetic experiments showing how mild departures from normality affect either the existence of the bell shape or the accuracy of the predicted peak location, limiting the practical utility of the closed-form result.
Authors: We agree that a sensitivity analysis would be valuable to assess the robustness of the closed-form result. The manuscript explicitly states the Gaussian assumption and validates the peak prediction when assumptions hold, but does not explore departures. In the revision, we will include synthetic experiments with non-Gaussian data (e.g., using t-distributions or Gaussian mixtures) to evaluate the persistence of the bell shape and accuracy of the predicted peak location under mild violations. revision: yes
Circularity Check
No circularity; closed-form peak derivation is standard math under explicit Gaussian assumptions with separate empirical validation
full rationale
The paper derives the λ-location of the membership signal maximum in closed form under stated Gaussian assumptions for data and noise. This is a direct mathematical derivation from the interpolation path definition and distributional premises, not a reduction to fitted inputs or self-referential definitions. The bell-shaped structure is reported as observed universally on audio/image data, with the peak formula validated only when assumptions hold. No self-citations, uniqueness theorems, or ansatzes are invoked in the provided text. The central claim remains independent of its own outputs and is externally checkable via the Gaussian premise.
Axiom & Free-Parameter Ledger
axioms (1)
- domain assumption The data and noise distributions allow for Gaussian assumptions in deriving the maximum location of the membership signal.
read the original abstract
Understanding what generative models retain from training data remains challenging, with implications for copyright and privacy. Beyond verbatim reproduction, models can encode subtler traces of their training data that never surface in their outputs yet remain exploitable. We study this regime for Rectified Flows, which are increasingly used in deployed generative systems. We analyse the interpolation path $X_\lambda = (1-\lambda)X_0 + \lambda X_1$ that defines the Rectified Flow training. We show that a gap exists between the reconstruction of train and test data that follows a bell-shaped curve over $\lambda$, wich accumulates during training, while the validation metrics remain stable. The signal has a maximum whose location we derive in closed form under Gaussian assumptions. We validate these predictions on both audio and images and show that the bell-shaped structure is universal, while the peak prediction holds when our assumptions are satisfied. As a proof of concept, we exploit this specific $\lambda$-resolved structure to perform a Membership Inference Attack, distinguishing members of the training set from non-members.
Figures
Reference graph
Works this paper leans on
-
[1]
International Conference on Learning Representations (ICLR) , year =
Chiyuan Zhang and Samy Bengio and Moritz Hardt and Benjamin Recht and Oriol Vinyals , title =. International Conference on Learning Representations (ICLR) , year =
-
[2]
Proceedings of the 33rd International Conference on Machine Learning (ICML) , pages =
Moritz Hardt and Ben Recht and Yoram Singer , title =. Proceedings of the 33rd International Conference on Machine Learning (ICML) , pages =. 2016 , volume =
2016
-
[3]
Proceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing (STOC) , pages =
Vitaly Feldman , title =. Proceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing (STOC) , pages =. 2020 , publisher =
2020
-
[4]
Advances in Neural Information Processing Systems (NeurIPS) , pages =
Behnam Neyshabur and Srinadh Bhojanapalli and David McAllester and Nathan Srebro , title =. Advances in Neural Information Processing Systems (NeurIPS) , pages =
-
[5]
Roy , title =
Gintare Karolina Dziugaite and Daniel M. Roy , title =. Proceedings of the 33rd Conference on Uncertainty in Artificial Intelligence (UAI) , year =
-
[6]
The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks , booktitle =
Nicholas Carlini and Chang Liu and. The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks , booktitle =. 2019 , publisher =
2019
-
[7]
Extracting Training Data from Diffusion Models , booktitle =
Nicholas Carlini and Jamie Hayes and Milad Nasr and Matthew Jagielski and Vikash Sehwag and Florian Tram. Extracting Training Data from Diffusion Models , booktitle =. 2023 , publisher =
2023
-
[8]
Quantifying Memorization Across Neural Language Models , booktitle =
Nicholas Carlini and Daphne Ippolito and Matthew Jagielski and Katherine Lee and Florian Tram. Quantifying Memorization Across Neural Language Models , booktitle =
-
[9]
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages =
Gowthami Somepalli and Vasu Singla and Micah Goldblum and Jonas Geiping and Tom Goldstein , title =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages =
-
[10]
Advances in Neural Information Processing Systems (NeurIPS) , year =
Gowthami Somepalli and Vasu Singla and Micah Goldblum and Jonas Geiping and Tom Goldstein , title =. Advances in Neural Information Processing Systems (NeurIPS) , year =
-
[11]
Preventing Verbatim Memorization in Language Models Gives a False Sense of Privacy , booktitle =
Daphne Ippolito and Florian Tram. Preventing Verbatim Memorization in Language Models Gives a False Sense of Privacy , booktitle =
-
[12]
Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (ACL) , year =
Katherine Lee and Daphne Ippolito and Andrew Nystrom and Chiyuan Zhang and Douglas Eck and Chris Callison-Burch and Nicholas Carlini , title =. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (ACL) , year =
-
[13]
IEEE Symposium on Security and Privacy (S&P) , pages =
Reza Shokri and Marco Stronati and Congzheng Song and Vitaly Shmatikov , title =. IEEE Symposium on Security and Privacy (S&P) , pages =
-
[14]
IEEE 31st Computer Security Foundations Symposium (CSF) , pages =
Samuel Yeom and Irene Giacomelli and Matt Fredrikson and Somesh Jha , title =. IEEE 31st Computer Security Foundations Symposium (CSF) , pages =
-
[15]
Gunter and Kai Chen , title =
Yunhui Long and Vincent Bindschaedler and Lei Wang and Diyue Bu and Xiaofeng Wang and Haixu Tang and Carl A. Gunter and Kai Chen , title =. Computers & Security , volume =
-
[16]
Proceedings of the 40th International Conference on Machine Learning (ICML) , series =
Jinhao Duan and Fei Kong and Shiqi Wang and Xiaoshuang Shi and Kaidi Xu , title =. Proceedings of the 40th International Conference on Machine Learning (ICML) , series =
-
[17]
Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , year =
Fei Kong and Jinhao Duan and RuiPeng Ma and Heng Tao Shen and Xiaofeng Zhu and Xiaoshuang Shi and Kaidi Xu , title =. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , year =
-
[18]
arXiv preprint arXiv:2306.12983 , year =
Jan Dubiński and Antoni Kowalczuk and Stanisław Pawlak and Przemysław Rokita and Tomasz Trzciński and Paweł Morawiecki , title =. arXiv preprint arXiv:2306.12983 , year =
-
[19]
International Conference on Learning Representations (ICLR) , year =
Xingchao Liu and Chengyue Gong and Qiang Liu , title =. International Conference on Learning Representations (ICLR) , year =
-
[20]
Yaron Lipman and Ricky T. Q. Chen and Heli Ben-Hamu and Maximilian Nickel and Matt Le , title =. International Conference on Learning Representations (ICLR) , year =
-
[21]
Scaling Rectified Flow Transformers for High-Resolution Image Synthesis , booktitle =
Patrick Esser and Sumith Kulal and Andreas Blattmann and Rahim Entezari and Jonas M. Scaling Rectified Flow Transformers for High-Resolution Image Synthesis , booktitle =
-
[22]
Zach Evans and Julian D. Parker and C. J. Carr and Zack Zukowski and Josiah Taylor and Jordi Pons , title =. arXiv preprint arXiv:2407.14358 , year =
-
[23]
2024 , howpublished =
2024
-
[24]
arXiv preprint arXiv:2305.16860 , year=
Joe Benton and George Deligiannidis and Arnaud Doucet , title =. arXiv preprint arXiv:2305.16860 , year =
-
[25]
Aram-Alexandre Pooladian and Heli Ben-Hamu and Carles Domingo-Enrich and Brandon Amos and Yaron Lipman and Ricky T. Q. Chen , title =. International Conference on Machine Learning (ICML) , year =
-
[26]
Kingma and Max Welling , title =
Diederik P. Kingma and Max Welling , title =. International Conference on Learning Representations (ICLR) , year =
-
[27]
International Conference on Learning Representations (ICLR) , year =
Irina Higgins and Loic Matthey and Arka Pal and Christopher Burgess and Xavier Glorot and Matthew Botvinick and Shakir Mohamed and Alexander Lerchner , title =. International Conference on Learning Representations (ICLR) , year =
-
[28]
International Conference on Machine Learning (ICML) , pages =
Alec Radford and Jong Wook Kim and Chris Hallacy and Aditya Ramesh and Gabriel Goh and Sandhini Agarwal and Girish Sastry and Amanda Askell and Pamela Mishkin and Jack Clark and Gretchen Krueger and Ilya Sutskever , title =. International Conference on Machine Learning (ICML) , pages =
-
[29]
International Conference on Learning Representations (ICLR) Workshop , year =
Guillaume Alain and Yoshua Bengio , title =. International Conference on Learning Representations (ICLR) Workshop , year =
-
[30]
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages =
Yujun Shen and Jinjin Gu and Xiaoou Tang and Bolei Zhou , title =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages =
-
[31]
Advances in Neural Information Processing Systems (NeurIPS) , year =
Erik H. Advances in Neural Information Processing Systems (NeurIPS) , year =
-
[32]
Hamprecht and Yoshua Bengio and Aaron Courville , title =
Nasim Rahaman and Aristide Baratin and Devansh Arpit and Felix Draxler and Min Lin and Fred A. Hamprecht and Yoshua Bengio and Aaron Courville , title =. International Conference on Machine Learning (ICML) , pages =
-
[33]
International Conference on Machine Learning (ICML) , year =
Ronen Basri and Meirav Galun and Amnon Geifman and David Jacobs and Yoni Kasten and Shira Kritchman , title =. International Conference on Machine Learning (ICML) , year =
-
[34]
International Joint Conference on Artificial Intelligence (IJCAI) , year =
Yuan Cao and Zhiying Fang and Yue Wu and Ding-Xuan Zhou and Quanquan Gu , title =. International Joint Conference on Artificial Intelligence (IJCAI) , year =
-
[35]
Machine Learning for Music Discovery Workshop, International Conference on Machine Learning (ICML) , year =
Dmitry Bogdanov and Minz Won and Philip Tovstogan and Alastair Porter and Xavier Serra , title =. Machine Learning for Music Discovery Workshop, International Conference on Machine Learning (ICML) , year =
-
[36]
International Society for Music Information Retrieval Conference (ISMIR) , year =
Micha. International Society for Music Information Retrieval Conference (ISMIR) , year =
-
[37]
arXiv preprint arXiv:2408.06500 , year =
Marco Pasini and Stefan Lattner and George Fazekas , title =. arXiv preprint arXiv:2408.06500 , year =
-
[38]
2023 , url =
William Peebles and Saining Xie , title =. 2023 , url =
2023
-
[39]
2024 , note =
Record Companies Bring Landmark Cases for Responsible. 2024 , note =
2024
-
[40]
Music Business Worldwide , year =
Newton-Rex, Ed , title =. Music Business Worldwide , year =
-
[41]
and Finch, John F
West, Stephen G. and Finch, John F. and Curran, Patrick J. , title =. Structural Equation Modeling , year =
-
[42]
But is it trained on copyrighted recordings? , author=
Suno is a music AI company aiming to generate \ 120 billion per year. But is it trained on copyrighted recordings? , author=. 2024 , url=
2024
-
[43]
2025 , eprint=
Membership and Dataset Inference Attacks on Large Audio Generative Models , author=. 2025 , eprint=
2025
-
[44]
Jaakkola , title =
Bowen Jing and Bonnie Berger and Tommi S. Jaakkola , title =. Forty-first International Conference on Machine Learning,. 2024 , url =
2024
-
[45]
Takuya Akiba and Shotaro Sano and Toshihiko Yanase and Takeru Ohta and Masanori Koyama , editor =. Optuna:. Proceedings of the 25th. 2019 , url =
2019
-
[46]
Kanwal and Tegan Maharaj and Asja Fischer and Aaron C
Devansh Arpit and Stanislaw Jastrzebski and Nicolas Ballas and David Krueger and Emmanuel Bengio and Maxinder S. Kanwal and Tegan Maharaj and Asja Fischer and Aaron C. Courville and Yoshua Bengio and Simon Lacoste. A Closer Look at Memorization in Deep Networks , booktitle =. 2017 , url =
2017
-
[47]
Enabling Factorized Piano Music Modeling and Generation with the
Curtis Hawthorne and Andriy Stasyuk and Adam Roberts and Ian Simon and Cheng. Enabling Factorized Piano Music Modeling and Generation with the. 7th International Conference on Learning Representations,. 2019 , url =
2019
-
[48]
The MTG-Jamendo Dataset for Automatic Music Tagging
Bogdanov, Dmitry and Won, Minz and Tovstogan, Philip and Porter, Alastair and Serra, Xavier. The MTG-Jamendo Dataset for Automatic Music Tagging. Machine Learning for Music Discovery Workshop, International Conference on Machine Learning (ICML 2019). 2019
2019
-
[49]
Proceedings of the 18th International Society for Music Information Retrieval Conference,
Micha. Proceedings of the 18th International Society for Music Information Retrieval Conference,. 2017 , url =
2017
-
[50]
Counterfactual Memorization in Neural Language Models , booktitle =
Chiyuan Zhang and Daphne Ippolito and Katherine Lee and Matthew Jagielski and Florian Tram. Counterfactual Memorization in Neural Language Models , booktitle =
-
[51]
Membership Inference Attacks From First Principles , booktitle =
Nicholas Carlini and Steve Chien and Milad Nasr and Shuang Song and Andreas Terzis and Florian Tram. Membership Inference Attacks From First Principles , booktitle =. 2022 , url =
2022
-
[52]
26th Annual Network and Distributed System Security Symposium,
Ahmed Salem and Yang Zhang and Mathias Humbert and Pascal Berrang and Mario Fritz and Michael Backes , title =. 26th Annual Network and Distributed System Security Symposium,
-
[53]
Yu and Xuyun Zhang , title =
Hongsheng Hu and Zoran Salcic and Lichao Sun and Gillian Dobbie and Philip S. Yu and Xuyun Zhang , title =. 2022 , url =
2022
-
[54]
Tomoya Matsumoto and Takayuki Miura and Naoto Yanai , title =. 2023. 2023 , url =
2023
-
[55]
Advances in Neural Information Processing Systems , volume =
Why Diffusion Models Don't Memorize: The Role of Implicit Dynamical Regularization in Training , author =. Advances in Neural Information Processing Systems , volume =. 2025 , note =
2025
-
[56]
Transactions on Machine Learning Research , year =
On Memorization in Diffusion Models , author =. Transactions on Machine Learning Research , year =
-
[57]
arXiv preprint arXiv:2410.23594 , year=
How Do Flow Matching Models Memorize and Generalize in Sample Data Subspaces? , author =. arXiv preprint arXiv:2410.23594 , year =
-
[58]
Advances in Neural Information Processing Systems , volume =
On the Closed-Form of Flow Matching: Generalization Does Not Arise from Target Stochasticity , author =. Advances in Neural Information Processing Systems , volume =
-
[59]
Advances in Neural Information Processing Systems , volume =
Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models , author =. Advances in Neural Information Processing Systems , volume =
-
[60]
Proceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing , pages =
Does Learning Require Memorization? A Short Tale about a Long Tail , author =. Proceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing , pages =. 2020 , publisher =
2020
-
[61]
Getty Images lawsuit against Stability AI , year =
-
[62]
Voicebox: Text-Guided Multilingual Universal Speech Generation at Scale , booktitle =
Matthew Le and Apoorv Vyas and Bowen Shi and Brian Karrer and Leda Sari and Rashel Moritz and Mary Williamson and Vimal Manohar and Yossi Adi and Jay Mahadeokar and Wei. Voicebox: Text-Guided Multilingual Universal Speech Generation at Scale , booktitle =
-
[63]
Physical Review , volume=
Information Theory and Statistical Mechanics , author=. Physical Review , volume=. 1957 , publisher=
1957
-
[64]
Proceedings of the 32nd International Conference on Machine Learning (ICML) , pages=
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , author=. Proceedings of the 32nd International Conference on Machine Learning (ICML) , pages=. 2015 , volume=
2015
-
[65]
Neural Information Processing (ICONIP) , pages=
Training Behavior of Deep Neural Network in Frequency Domain , author=. Neural Information Processing (ICONIP) , pages=. 2019 , volume=
2019
-
[66]
Proceedings of International Conference on Computer Vision (ICCV) , month =
Deep Learning Face Attributes in the Wild , author =. Proceedings of International Conference on Computer Vision (ICCV) , month =
-
[67]
Retrieval-Augmented Diffusion Models , booktitle =
Andreas Blattmann and Robin Rombach and Kaan Oktay and Jonas M. Retrieval-Augmented Diffusion Models , booktitle =
-
[68]
Jianlin Su and Murtadha H. M. Ahmed and Yu Lu and Shengfeng Pan and Wen Bo and Yunfeng Liu , title =. Neurocomputing , year =
-
[69]
and Ermon, Stefano and Rudra, Atri and R
Dao, Tri and Fu, Daniel Y. and Ermon, Stefano and Rudra, Atri and R. Flash. Advances in Neural Information Processing Systems (NeurIPS) , year=
-
[70]
Parker and CJ Carr and Zack Zukowski and Josiah Taylor and Jordi Pons , title =
Zach Evans and Julian D. Parker and CJ Carr and Zack Zukowski and Josiah Taylor and Jordi Pons , title =. 2025. 2025 , url =
2025
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.