REVIEW 3 major objections 4 minor 30 references
Flow Stochastic Segmentation Networks
T0 review · 3 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read Flow-SSNs replace the low-rank covariance assumption in stochastic segmentation with a lightweight flow over a learned prior, reaching state-of-the-art medical imaging results with fewer parameters.
desk verdict Solid model contribution and empirical work, but the theoretical proof of sublinear effective-rank growth is invalid and should be fixed or removed. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The machinery is a two-part generative model: a conditional base distribution $p_{U|\mathcal X}(u|x;\lambda) = \mathcal N(u; \mu(x), \operatorname{diag}(\sigma^2(x)))$ that carries most of the network parameters, and a lightweight flow $\phi$ that transforms samples $u$ into logits $\eta$, whose softmax produces label maps. The paper proves (Proposition 5.1) that a single linear autoregressive transformation is already enough to turn a diagonal Gaussian into a full covariance, so no rank parameter is ever chosen or stored. For continuous time, $\phi$ is defined by the interpolation path $y_t = (1-t)u + t y$ and trained by regressing the conditional expectation $\mathbb E[y|y_t]$ through an expected categorical likelihood objective, a variant of flow matching. The theoretical spine is Lemma 4.1 (softmax pushforward raises covariance rank) and Theorem 4.4 (the effective rank, the exponential of the Shannon entropy of the normalized singular values, then grows sublinearly in the assumed rank), which the paper uses to justify abandoning low-rank parameterisations.
What would settle it
Numerically compute the effective rank of the softmax pushforward covariance for low-rank Gaussian logits across assumed ranks $r = 1, 2, 4, 8, \dots, 512$ at fixed image dimension, and fit $\operatorname{erank}(\operatorname{Cov}(y))$ versus $r$. If the growth is linear with a slope bounded away from zero, Theorem 4.4's sublinear conclusion fails. A simpler target: exhibit one covariance with non-uniform singular values whose effective rank is proportional to $r$.
Extended reading notes
Core claim
The paper's central discovery is that the standard stochastic segmentation network parameterization, a rank-$r$ Gaussian over logits, is not just practically but theoretically restrictive, and that moving the modelling burden from the covariance to a flow over a learned base distribution dissolves the restriction. Concretely, Flow-SSNs define $p(y|x) = \int p(y|\eta)\, p_{U|\mathcal X}(u|x;\lambda)\, |\det J_\phi(u)|^{-1} d\eta$ with $\eta = \phi(u)$, so the logit distribution is the pushforward of a diagonal Gaussian through an autoregressive or continuous-time flow. The implied pixel-wise covariance can have full rank without assuming or storing the rank. The paper supports this with two formal results: the softmax pushforward of a low-rank logit Gaussian has covariance rank strictly larger than the assumed rank $r$ when $r < d(k-1)$, but its effective rank grows sublinearly with $r$, so adding rank factors is a losing strategy. Empirically, a discrete-time one-step inverse autoregressive flow and a continuous-time flow variant both outperform previous stochastic segmentation and diffusion baselines on LIDC-IDRI and REFUGE-MultiRater while using about 14-15M parameters.
Load-bearing premise
The advertised limit on low-rank stochastic segmentation relies on the step that a non-uniform singular value distribution makes the effective rank grow sublinearly with assumed rank; this does not follow from effective rank being smaller than the true rank, since sublinearity is a claim about growth rate, not just size.
Editorial extensions
If this is right
- Flow-SSNs remove the rank hyperparameter from stochastic segmentation: the model can represent correlations among any subset of pixels or classes, and the user only chooses the flow architecture and the number of Monte Carlo samples.
- Sampling no longer scales with a full diffusion or score model: with a 150K-parameter flow network, competitive results are reached at 10 Euler steps, and discrete-time IAF sampling is one forward pass.
- The two-part design makes it natural to plug in a large foundation model as the prior and keep the flow tiny, so future gains can come from better priors without raising sampling cost.
- The theoretical critique targets any low-rank softmax-Gaussian likelihood in high dimensions, so it would motivate re-examining other low-rank approximation methods in segmentation and classification, not just SSNs.
- On the reported benchmarks, the method sets new results for both discrete and continuous variants, with continuous-time Flow-SSNs best on LIDC and the discrete-time variant best on REFUGE.
Reading between the lines
- If the empirical pattern generalises, a useful design rule is to put almost all parameters in the conditional prior and keep the flow deliberately small; the paper's roughly 100:1 prior-to-flow parameter ratio on LIDC suggests the flow's job is purely to add correlation, not to represent the marginal structure.
- The continuous-time formulation is essentially a learned stochastic interpolant conditioned on the image; one could extend it to interactive editing by conditioning the prior on user scribbles and keeping the same lightweight ODE solve for refinement.
- A direct comparison of Flow-SSN against a full-rank SSN with matched training budget would isolate how much of the gain comes from removing the rank assumption versus from the flow's nonlinear transform; the paper compares against low-rank SSNs and diffusion models but does not report a full-rank SSN at the same scale.
- The 'without storing distributional parameters' claim should be read as implicit parametrisation: the flow network's weights encode the covariance, so the memory trade-off is exactly the flow's parameter count, which the paper makes very small.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces Flow Stochastic Segmentation Networks (Flow-SSNs), a class of generative segmentation models that replace the low-rank Gaussian logit distribution of standard SSNs with a learned diagonal base distribution followed by a lightweight normalizing flow. Two variants are developed: a discrete-time inverse autoregressive flow and a continuous-time flow trained with a categorical expected-likelihood objective. The paper claims to prove that the effective rank of the pushforward covariance in low-rank SSNs grows only sublinearly with the assumed rank, and reports state-of-the-art results on LIDC-IDRI and REFUGE-MultiRater with fewer parameters than diffusion-based baselines.
Significance. If the theoretical claims were sound, this paper would provide both a useful negative result about low-rank SSNs and a practical generative alternative with strong empirical performance. The experimental work is genuinely careful: the MarkovShapes toy problem controls the true covariance rank, the ablations cover rank scalability, Monte Carlo sample counts, ODE solver steps, and fixed versus learned priors, and the code is publicly available. The sampling-efficiency comparison against CCDM is informative. However, the headline theoretical justification for the 'fundamental limitations' of low-rank SSNs, stated in the abstract and Section 4, rests on a proof with a clear logical gap. The significance is therefore conditional on repairing or reframing that theoretical claim.
major comments (3)
- [§4 / Appendix A.2, Theorem 4.4 (Eqs. 25–28)] The proof of Theorem 4.4 contains an invalid inference: from the singular value distribution of Cov(y) being almost surely non-uniform, it concludes that erank(Cov(y)) grows sublinearly with the assumed rank r. Non-uniformity only gives the strict inequality erank(Cov(y)) < rank(Cov(y)) at each fixed r; it does not constrain how erank scales as r grows. For example, if rank(Cov(y)) = r and the singular values are σ1 = 1 and σ2 = ... = σr = 1/2, then the normalized distribution satisfies H(p) = log r + O(1), so erank = Θ(r), i.e. linear growth despite non-uniformity. Moreover, the theorem never specifies a family of covariance matrices over which growth is measured, so 'sublinear w.r.t. r' is not formally defined. Because this theorem is advertised in the abstract as proving fundamental limitations of low-rank SSNs, this gap is load-bearing.
- [§4 / Appendix A.1, Lemma 4.1 (Eqs. 21–24)] Lemma 4.1 states an 'iff' result, but the proof only shows that the first-order Taylor term has rank r and that higher-order terms are non-zero. The rank subadditivity inequality rank(A+B) ≤ rank(A) + rank(B) does not imply rank(Cov(y)) > r; the higher-order contribution could lie in the column space of the first-order term. The proof also invokes a 'Taylor expansion of Cov(y)' without a formal derivation (e.g., a delta-method expansion), so the claimed equivalence is not established. The 'only if' direction is plausible from the softmax rank bound, but the 'if' direction, which Theorem 4.4 relies on, remains unproven.
- [§5.1 / §6 and Table 5] The abstract and Section 5 claim that Flow-SSNs 'can estimate arbitrarily high-rank pixel-wise covariances without assuming the rank or storing the distributional parameters.' The toy experiment demonstrates this only for a small synthetic problem with true rank 12. On the real datasets, the autoregressive flow uses patches of size (1,8) or (8,8), which induces a block-structured covariance, and no effective-rank or covariance-rank statistics are reported for the learned distributions. The paper should either directly estimate and report the effective rank of the learned covariance on real data, or qualify the claim to what is actually demonstrated.
minor comments (4)
- [§5.2, Eq. (18)] The approximation E[y|y_t] ≈ softmax(η(φ_t(u|y);θ)) should clarify the reshaping: y is a one-hot map in R^{k×d}, while softmax(η) is a vector in R^{kd}, so the equality is only meaningful after an explicit reshaping of the right-hand side.
- [§D.3 / Table 5] The text says 'we require a lightweight autoregressive model' and later notes that a single Transformer layer was used. It would help to state explicitly how the patch ordering is chosen and whether the induced block covariance is treated as a limitation or an intentional approximation.
- [Appendix E.2, Figure 10] The caption says 'mild increases in the assumed rank can cause SSNs to collapse,' but the rightmost point uses r=100, which is not mild relative to the r=10 default. Please rephrase the caption to distinguish the rank sweep from the collapse regime.
- [§6.2, Table 1] The table reports results for many baselines, but for the SSN row the D2_GED(16) entry is missing while other entries are present. Adding this value or a placeholder such as 'not reported' would make the table easier to read.
Circularity Check
No significant circularity found: the training objectives and rank-capacity claims are derived from flow equations and Cholesky factorization, not from the evaluation metrics; self-citations are background only.
full rationale
The derivation chain is self-contained. The Flow-SSN objectives (Eqs. 14 and 19) follow directly from the flow change-of-variables formula and the interpolation path y_t = (1-t)u + t y; they are expected categorical likelihood estimators, not parameters fitted to the test metrics and then renamed as predictions. Proposition 5.1 is a standard Cholesky argument showing that any full covariance can be induced by a linear autoregressive transformation, so the 'arbitrarily high-rank' claim is a capacity statement rather than a disguised fit. The experiments compare against external baselines and held-out data; no fitted constant is relabeled as a prediction. Self-citations, including the SSN predecessor [55] that shares an author, supply the baseline and background but are not load-bearing: the paper's central construction does not depend on an unverified result imported from those citations. A caveat belongs in correctness rather than circularity: the proof of Theorem 4.4 (Appendix A.2) asserts that non-uniform singular values imply sublinear growth of the effective rank, which does not follow from erank < rank; that is an invalid inference, not an input-output equivalence, and therefore does not count as circularity under the stated rubric.
Assumptions & free parameters
free parameters (3)
- Patch size for autoregressive flow =
(1,8) for LIDC, (8,8) for REFUGE
- Training Monte Carlo samples =
16 or 32 (discrete), 1 (continuous)
- Inference ODE steps (Euler) =
T=50 and T=250 in main results; 8 for validation
assumptions (5)
- domain assumption The Markovian projection ODE preserves marginals p_t when E[y|y_t] is exact
- ad hoc to paper The Taylor expansion of Cov(y) has non-zero higher-order terms that increase rank
- ad hoc to paper Non-uniform singular values imply sublinear effective-rank growth
- standard math A single linear autoregressive layer can represent any full covariance Gaussian
- domain assumption The categorical-likelihood objective (Eq. 19) is a valid surrogate for training the sampling ODE
Cite this review
Pith. "Pith review of Flow Stochastic Segmentation Networks." pith.science (2026). https://pith.science/paper/2TYVKWUL
@misc{pith2026250718838,
author = {Pith},
title = {Pith review of: Flow Stochastic Segmentation Networks},
year = {2026},
howpublished = {\url{https://pith.science/paper/2TYVKWUL}},
note = {Machine review of arXiv:2507.18838}
}
read the original abstract
We introduce the Flow Stochastic Segmentation Network (Flow-SSN), a generative segmentation model family featuring discrete-time autoregressive and modern continuous-time flow variants. We prove fundamental limitations of the low-rank parameterisation of previous methods and show that Flow-SSNs can estimate arbitrarily high-rank pixel-wise covariances without assuming the rank or storing the distributional parameters. Flow-SSNs are also more efficient to sample from than standard diffusion-based segmentation models, thanks to most of the model capacity being allocated to learning the base distribution of the flow, constituting an expressive prior. We apply Flow-SSNs to challenging medical imaging benchmarks and achieve state-of-the-art results. Code available: https://github.com/biomedia-mira/flow-ssn.
Reference graph
Works this paper leans on
-
[1]
In12th USENIX symposium on operating systems design and implementation (OSDI 16), pages 265–283, 2016
Mart´ın Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghe- mawat, Geoffrey Irving, Michael Isard, et al.{TensorFlow}: a system for {Large-Scale} machine learning. In12th USENIX symposium on operating systems design and implementation (OSDI 16), pages 265–283, 2016. 17
2016
-
[2]
Build- ing normalizing flows with stochastic interpolants
Michael Samuel Albergo and Eric Vanden-Eijnden. Build- ing normalizing flows with stochastic interpolants. InThe Eleventh International Conference on Learning Representa- tions, 2023. 2, 3, 5
2023
-
[3]
Tomer Amit, Tal Shaharbany, Eliya Nachmani, and Lior Wolf. Segdiff: Image segmentation with diffusion probabilistic mod- els.arXiv preprint arXiv:2112.00390, 2021. 2, 5, 20
arXiv 2021
-
[4]
Samuel G Armato III, Geoffrey McLennan, Luc Bidaut, Michael F McNitt-Gray, Charles R Meyer, Anthony P Reeves, Binsheng Zhao, Denise R Aberle, Claudia I Henschke, Eric A Hoffman, et al. The lung image database consortium (lidc) and image database resource initiative (idri): a completed ref- erence database of lung nodules on ct scans.Medical physics, 38(2)...
2011
-
[5]
Average calibration error: A differentiable loss for improved reliability in image seg- mentation
Theodore Barfoot, Luis C Garcia Peraza Herrera, Ben Glocker, and Tom Vercauteren. Average calibration error: A differentiable loss for improved reliability in image seg- mentation. InInternational Conference on Medical Image Computing and Computer-Assisted Intervention, pages 139–
-
[6]
Phiseg: Capturing uncertainty in medical image segmentation
Christian F Baumgartner, Kerem C Tezcan, Krishna Chai- tanya, Andreas M H ¨otker, Urs J Muehlematter, Khoschy Schawkat, Anton S Becker, Olivio Donati, and Ender Konukoglu. Phiseg: Capturing uncertainty in medical image segmentation. InMedical Image Computing and Computer Assisted Intervention–MICCAI 2019: 22nd International Con- ference, Shenzhen, China, ...
2019
-
[7]
Failure detection in medical image classification: A reality check and benchmarking testbed.Transactions on Machine Learning Research, 2022
M´elanie Bernhardt, Fabio De Sousa Ribeiro, and Ben Glocker. Failure detection in medical image classification: A reality check and benchmarking testbed.Transactions on Machine Learning Research, 2022. 1
2022
-
[8]
Flowsdf: Flow matching for medical image segmentation using distance transforms
Lea Bogensperger, Dominik Narnhofer, Alexander Falk, Kon- rad Schindler, and Thomas Pock. Flowsdf: Flow matching for medical image segmentation using distance transforms. arXiv preprint arXiv:2405.18087, 2024. 2
arXiv 2024
Show all 30 references
-
[9]
Qubiq: Uncertainty quantification for biomedical image seg- mentation challenge.arXiv preprint arXiv:2405.18435, 2024
Hongwei Bran, Fernando Navarro, Ivan Ezhov, Amirhossein Bayat, Dhritiman Das, Florian Kofler, Suprosanna Shit, Di- ana Waldmannstetter, Johannes C Paetzold, Xiaobin Hu, et al. Qubiq: Uncertainty quantification for biomedical image seg- mentation challenge.arXiv preprint arXiv:...
2024 arXiv
-
[10]
Yuille, and Yuyin Zhou
Jieneng Chen, Yongyi Lu, Qihang Yu, Xiangde Luo, Ehsan Adeli, Yan Wang, Le Lu, Alan L. Yuille, and Yuyin Zhou. Transunet: Transformers make strong encoders for medical image segmentation.arXiv preprint arXiv:2102.04306, 2021. 8
2021 arXiv
-
[11]
Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L Yuille. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolu- tion, and fully connected crfs.IEEE transactions on pattern analysis and machine intelligence, 40(4):834–8...
2017
-
[12]
Neural ordinary differential equations
Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary differential equations. Advances in neural information processing systems, 31, 2018. 2, 3, 5
2018
-
[13]
Ricky T. Q. Chen. torchdiffeq, 2018. https://github.com/ rtqichen/torchdiffeq. 18
2018
-
[14]
Steffen Czolbe, Kasra Arnavaz, Oswin Krause, and Aasa Feragen. Is segmentation uncertainty useful? InInforma- tion Processing in Medical Imaging: 27th International Con- ference, IPMI 2021, Virtual Event, June 28–June 30, 2021, Proceedings 27, pages 715–726. Springer, 2021. 1
2021
-
[15]
Deep bayesian self-training.Neural Computing and Applications, 32(9):4275–4291, 2020
Fabio De Sousa Ribeiro, Francesco Caliv´a, Mark Swainson, Kjartan Gudmundsson, Georgios Leontidis, and Stefanos Kol- lias. Deep bayesian self-training.Neural Computing and Applications, 32(9):4275–4291, 2020. 2
2020
-
[16]
Introducing routing uncertainty in capsule networks
Fabio De Sousa Ribeiro, Georgios Leontidis, and Stefanos Kollias. Introducing routing uncertainty in capsule networks. Advances in neural information processing systems, 33:6490– 6502, 2020. 2
2020
-
[17]
High fidelity image counterfac- tuals with probabilistic causal models
Fabio De Sousa Ribeiro, Tian Xia, Miguel Monteiro, Nick Pawlowski, and Ben Glocker. High fidelity image counterfac- tuals with probabilistic causal models. InProceedings of the 40th International Conference on Machine Learning, pages 7390–7425, 2023. 17
2023
-
[18]
Aleatory or epis- temic? does it matter?Structural safety, 31(2):105–112,
Armen Der Kiureghian and Ove Ditlevsen. Aleatory or epis- temic? does it matter?Structural safety, 31(2):105–112,
-
[19]
Diffusion models beat gans on image synthesis.Advances in neural information processing systems, 34:8780–8794, 2021
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis.Advances in neural information processing systems, 34:8780–8794, 2021. 7, 18, 19
2021
-
[20]
Nice: Non- linear independent components estimation.arXiv preprint arXiv:1410.8516, 2014
Laurent Dinh, David Krueger, and Yoshua Bengio. Nice: Non- linear independent components estimation.arXiv preprint arXiv:1410.8516, 2014. 3
2014 arXiv
-
[21]
Den- sity estimation using real NVP
Laurent Dinh, Jascha Sohl-Dickstein, and Samy Bengio. Den- sity estimation using real NVP. InInternational Conference on Learning Representations, 2017. 2
2017
-
[22]
Naesseth, Max Welling, and Jan-Willem van de Meent
Floor Eijkelboom, Grigory Bartosh, Christian A. Naesseth, Max Welling, and Jan-Willem van de Meent. Variational flow matching for graph generation. InThe Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. 5
2024
-
[23]
Refuge2 challenge: A treasure trove for multi- dimension analysis and evaluation in glaucoma screening
Huihui Fang, Fei Li, Junde Wu, Huazhu Fu, Xu Sun, Jaemin Son, Shuang Yu, Menglu Zhang, Chenglang Yuan, Cheng Bian, et al. Refuge2 challenge: A treasure trove for multi- dimension analysis and evaluation in glaucoma screening. arXiv preprint arXiv:2202.08994, 2022. 7
2022 arXiv
-
[24]
Modeling multimodal aleatoric uncertainty in segmentation with mixture of stochastic experts.International Conference on Representation Learning (ICLR), 2022
Zhitong Gao, Yucong Chen, Chuyu Zhang, and Xuming He. Modeling multimodal aleatoric uncertainty in segmentation with mixture of stochastic experts.International Conference on Representation Learning (ICLR), 2022. 2
2022
-
[25]
Modeling multimodal aleatoric uncertainty in segmentation with mixture of stochastic experts
Zhitong Gao, Yucong Chen, Chuyu Zhang, and Xuming He. Modeling multimodal aleatoric uncertainty in segmentation with mixture of stochastic experts. InThe Eleventh Inter- national Conference on Learning Representations, 2023. 7, 20
2023
-
[26]
Made: Masked autoencoder for distribution es- timation
Mathieu Germain, Karol Gregor, Iain Murray, and Hugo Larochelle. Made: Masked autoencoder for distribution es- timation. InInternational conference on machine learning, pages 881–889. PMLR, 2015. 6
2015
-
[27]
Partha Ghosh, Mehdi S. M. Sajjadi, Antonio Vergari, Michael Black, and Bernhard Scholkopf. From variational to determin- istic autoencoders. InInternational Conference on Learning Representations, 2020. 2
2020
-
[28]
Starflow: Scaling latent normalizing flows for high-resolution image synthesis.arXiv preprint arXiv:2506.06276, 2025
Jiatao Gu, Tianrong Chen, David Berthelot, Huangjie Zheng, Yuyang Wang, Ruixiang Zhang, Laurent Dinh, Miguel An- gel Bautista, Josh Susskind, and Shuangfei Zhai. Starflow: Scaling latent normalizing flows for high-resolution image synthesis.arXiv preprint arXiv:2506.06276, 2025. 2, 21
2025 arXiv
-
[29]
Mimicking the one-dimensional marginal dis- tributions of processes having an itˆo differential.Probability theory and related fields, 71(4):501–516, 1986
Istv´an Gy¨ongy. Mimicking the one-dimensional marginal dis- tributions of processes having an itˆo differential.Probability theory and related fields, 71(4):501–516, 1986. 5
1986
-
[30]
Keeping the neural networks simple by minimizing the description length of the weights
Geoffrey E Hinton and Drew Van Camp. Keeping the neural networks simple by minimizing the description length of the weights. InProceedings of the sixth annual conference on Computational learning theory, pages 5–13, 1993. 2
1993
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.