REVIEW 4 major objections 8 minor 31 references
Optimal Transport Driven Asymmetric Image-to-Image Translation for Nuclei Segmentation of Histological Images
T0 review · 4 major / 8 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read This paper claims that asymmetric image-to-image translation for nuclei segmentation can be made both simpler and more accurate by adding an embedding space and an invertible generator, which removes the explicit cycle-consistency loss.
desk verdict The performance advantage over supervised baselines is mostly the H-channel SSIM prior, not the optimal transport / invertible generator claim. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the invertible generator $G_\theta$, whose sections $G_\theta^y$ and $G_\theta^z$ are treated as inverse operators of the forward generators $E_\omega$ and $F_\varphi$; Theorem 1 uses this invertibility to cancel the cycle-consistency terms and collapse three GAN objectives into one paired discriminator. The invertible generator is implemented with invertible $1\times 1$ convolutions and general coupling layers, together with a spatially constrained squeeze-unsqueeze operation that keeps neighboring pixels in the same sub-patch, and an SSIM-based regularization term built from the hematoxylin channel via Otsu thresholding and Voronoi labeling.
What would settle it
Measure reconstruction errors on held-out patches of the trained model, comparing an input image $x$ with $G_\theta^z(F_\varphi(x))$ and a segmentation map $y$ with $F_\varphi(G_\theta^z(y))$; if these errors are large, the exact-inverse assumption behind Theorem 1 fails, and the reported accuracy gains cannot be attributed to the cycle-consistency-free optimal-transport reduction.
Extended reading notes
Core claim
The central claim is an optimal-transport reduction that makes asymmetric image-to-image translation cheap enough for nuclei segmentation. The paper formalizes three maps: $F_\varphi: X \to Y$ from images to segmentation maps, $E_\omega: X \to Z$ from images to an embedding space, and $G_\theta: (Y \times Z) \to X$ that reconstructs images from map-plus-embedding pairs. Theorem 1 states that if the sections of $G_\theta$ are exact inverses of $F_\varphi$ and $E_\omega$, the cycle-consistency loss and the image-domain discriminator vanish, and the minimax problem reduces to two generators and one paired discriminator with loss $2\,l_{GAN}(F_\varphi) + l_{GAN}(E_\omega) + \lambda_1 l_{SSIM}(F_\varphi)$. The implementation approximates invertibility by letting the reconstruction network share the parameters of $F_\varphi$, builds the invertible generator from invertible $1\times 1$ convolutions and general coupling layers, and adds a spatially constrained squeeze operation to preserve spatial continuity. On TCGA and CoNIC, Ostrich outperforms the compared state-of-the-art methods in most evaluation cases and is significantly better in 87.5% of cases across the two datasets, using 7.27 million parameters versus 18.04 million for CycleGAN and 27.05 million for the asymmetric CycleGAN variant.
Load-bearing premise
The proof that cycle-consistency can be dropped assumes the sections of the generator are exact inverses of the forward generators, but the trained network only shares parameters between them, so exact invertibility is not guaranteed.
Editorial extensions
If this is right
- On TCGA data, Ostrich reaches a Dice coefficient of 0.788, ahead of every compared state-of-the-art method on Dice, Jaccard, and precision, with recall slightly below BoNuS.
- On CoNIC data, Ostrich outperforms all compared methods on all four metrics, with Dice 0.741 versus the best competitor HoVer-Net at 0.740.
- Because the cycle-consistency loss and one discriminator are removed, the trained model uses 7.27 million parameters, far fewer than CycleGAN's 18.04 million and the asymmetric CycleGAN variant's 27.05 million.
- The spatially constrained squeeze operation separates overlapping nuclei better than checkerboard masking, and the difference is statistically significant on TCGA.
- The model performs both instance and binary semantic segmentation without assuming any particular nucleus shape, so it can capture heterogeneous nuclei across organs.
Reading between the lines
- Beyond the paper, the same invertible-generator reduction should apply to other asymmetric translation tasks, such as stain transfer or low-dose CT denoising, where one domain contains far more information than the other; the paper only demonstrates histology segmentation.
- Because the implementation only approximates exact invertibility by parameter sharing, a version that retains a light reconstruction or cycle-consistency loss might be more stable on domains where the hematoxylin-channel SSIM regularizer is unavailable or weak.
- The SSIM regularization term injects a hand-built weak supervision signal derived from Otsu thresholding and Voronoi labeling; ablating it against a learned or data-driven prior would reveal how much of the reported gain comes from optimal-transport structure versus this domain-specific regularizer.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents Ostrich, an asymmetric image-to-image translation model for nuclei segmentation from H&E histology images. It models translation between the image domain X and the segmentation domain Y through an embedding space Z, with forward generators Fφ and Eω and an invertible generator Gθ. The authors derive an optimal-transport GAN objective (Eq. 6), prove (Theorem 1) that under invertibility of the sections of Gθ the problem reduces to two generators and one discriminator (Eq. 9), and introduce an H-channel SSIM regularization l_ssim(Fφ) to guide early training. Experiments on TCGA and CoNIC compare Ostrich with CycleGAN variants and supervised segmentation networks, reporting the best Dice/Jaccard/precision/recall in most cases and fewer parameters than several I2I baselines.
Significance. The proposed direction is timely and the paper contains useful ingredients: an OT-inspired asymmetric I2I formulation with an embedding space, a parameter-sharing invertible-generator idea, a spatially constrained squeeze operation, and experiments on two public datasets with parameter counts and paired statistical tests. If the attribution issues were resolved, the result would be a comparatively lightweight I2I segmentation model with competitive Dice/Jaccard performance on TCGA and CoNIC. The main weaknesses, namely the uncontrolled auxiliary prior, the unproven theoretical reduction, and the lack of repeated-run uncertainty, currently prevent the paper from supporting its central claims. The contribution is nonetheless potentially valuable to the medical image analysis community and could be made publishable with a controlled experimental design and a corrected theoretical statement.
major comments (4)
- [Section III-B-2, Eq. (3), Table I] The main empirical claim is confounded by an asymmetric evaluation setup. The SSIM regularization l_ssim(Fφ) of Eq. (3) is added to Ostrich and to the three I2I baselines (CycleGAN, OT-CycleGAN, Asym-CycleGAN), but it is not added to the supervised segmentation baselines (U-Net, Mask-R-CNN, U-Net++, HoVer-Net, MoNS, StarDist, Swin-MIL, BoNuS). Table I shows that removing R from Ostrich lowers Dice from 0.788035 to 0.761814, a drop of 0.026221, which is approximately 8 times the remaining margin of 0.003295 over BoNuS (0.784740). Without the H-channel prior, Ostrich would fall below BoNuS on Dice. Therefore the reported superiority cannot be attributed to the OT/invertible-generator architecture unless a controlled comparison is run in which the same auxiliary prior is given to the supervised baselines, or in which Ostrich without R is compared against baselines without the prior.
- [Section II-B, Theorem 1, Eqs. (16)-(24)] The proof of Theorem 1 is not valid as written. The step claiming that Phi_3 is contained in Phi_1 is asserted rather than proved; membership in Phi_1 requires representation as phi o G_theta_z with phi in Lip_1(X), and the fact that an element of Phi_3 is 1-Lipschitz on Y does not imply such a representation, especially since a bound on G_theta_z does not transfer to a bound on its inverse F_phi. In addition, the tightness argument after Eq. (25) constructs phi(x)=psi*(F_phi(x)) and needs F_phi to be 1-Lipschitz, which is not assumed. Finally, the theorem assumes exact invertibility G_theta_z = F_phi^{-1} and G_theta_y = E_omega^{-1}, but the architecture in Section II-C only shares parameters between F_phi and G_theta_z; it does not enforce exact inversion, so the cycle-consistency terms in Eq. (8) need not vanish in the implemented model. The reduction to Eq. (9) should be restated as an approximation or proved under conditions satisfied by the network.
- [Tables I, V, II, VI] Tables I and V report single-run performance without error bars, confidence intervals, or the number of runs. The p-values in Tables II and VI are computed with paired tests over image patches, not over independent model trainings, so they conflate per-patch variation with model-instance variation and do not establish that the observed differences are due to the method rather than to random initialization or training seed. The experimental section should report mean plus/minus standard deviation over multiple random seeds and should apply significance tests at the level of repeated runs.
- [Abstract, Tables II and VI] The abstract's claim that the model 'performs significantly better than the state-of-the-art methods in 87.5% cases' overstates the evidence in Tables II and VI. The 87.5% figure aggregates 63 of 72 model-metric pairs, but on TCGA the differences against BoNuS are not significant for Dice (p=1.42e-01), Jaccard (p=1.05e-01), or Recall (p=9.43e-01), and the differences against Asym-CycleGAN are not significant for Dice (p=6.49e-02), Jaccard (p=5.42e-02), or Recall (p=8.31e-02). The 'significantly better' claim should be restricted to the cases where the paired tests give p<0.05, or a global multiple-comparison procedure should be used.
minor comments (8)
- [Eq. (3), Section III-A] Please specify how V_O is implemented (Otsu threshold plus Voronoi labeling details, including any morphological post-processing); as written, the H-channel prior is not fully reproducible.
- [Section II-C] The statement that Z is assumed to follow a standard normal distribution is not reconciled with the measure eta introduced in Section II-B; state explicitly whether eta is fixed to N(0,I) or learned.
- [Fig. 2 caption] The caption says 'x[:,:,0:2] denotes 3 channel input image', but the slice 0:2 gives only two channels; this is likely a typo for x[:,:,0:3].
- [Table II] BoNuS is labeled '(2022)' in the first block but '(2024)' in the second block; align the year with reference [18].
- [Algorithm 1 and Section III-A-3] The text states that lambda_1 decreases gradually during training, but Algorithm 1 shows a constant lambda_1; please specify the decay schedule.
- [Section III-B-5] The sentence '29 out of 36 cases, better but not significantly in 6 cases' accounts for only 35 cases; check whether a third category is needed or whether the numbers are misreported.
- [Appendix, Eq. (37)] The notation x = G_theta(y,z) = G_theta_y(z) = G_theta_z(y) conflates the two sections of G_theta; these are generally different and should be written separately with the intended equality stated explicitly.
- [Eq. (6) and Eq. (10)] The cycle-consistency term with weight lambda_2 appears in Eq. (6), but Theorem 1 eliminates it and Eq. (10) has no lambda_2 term; clarify whether any cycle-consistency weight is used in the final objective.
Circularity Check
No significant circularity: the OT/invertible-generator derivation is conditional and self-contained, and the H-channel regularization is transparently an auxiliary self-supervised term rather than a hidden fitted input.
full rationale
The paper's central derivation (Theorem 1) explicitly assumes Gθ_z = Fφ^{-1} and Gθ_y = Eω^{-1}; under that assumption the cycle-consistency terms vanish by composition, so the reduction to Eq. (9) is a valid conditional equivalence rather than a circular inference. No self-citations are load-bearing: the cited invertible-generator and normalizing-flow building blocks [22,28,29] are external prior work and are not used to smuggle in the paper's own conclusions. The l_ssim term in Eq. (3) uses a deterministic transform VO(H-channel of x) as a pseudo-label; while this is a strong inductive bias and arguably confounds the comparison with supervised baselines that do not receive the same prior, it is openly defined, explicitly ablated (Table I shows Ostrich Dice 0.788 vs Ostrich\R 0.762), and it is not a parameter fitted to a subset of the evaluation data, so it does not make the reported prediction equivalent to its input by construction. The main legitimate concerns are experimental fairness (baselines without l_ssim) and the gap between the exact invertibility assumed in Theorem 1 and the parameter-sharing implementation, but these are correctness and benchmarking risks, not circularity.
Assumptions & free parameters
free parameters (3)
- lambda_1 =
not specified; annealed during training
- lambda_2 =
not used; cycle loss vanishes by assumption
- embedding dimension of Z =
not specified
assumptions (6)
- standard math Kantorovich duality for optimal transport
- standard math Fubini's theorem for product measures
- standard math 1-Lipschitz continuity of Kantorovich potentials
- ad hoc to paper Gθ_z = Fφ^{-1} and Gθ_y = Eω^{-1} (exact invertibility)
- domain assumption The embedding space Z follows a standard normal distribution
- domain assumption The Voronoi-labeled H-channel (via Otsu) provides a useful pseudo-segmentation for the SSIM regularization
invented entities (1)
-
Embedding space Z
Cite this review
Pith. "Pith review of Optimal Transport Driven Asymmetric Image-to-Image Translation for Nuclei Segmentation of Histological Images." pith.science (2026). https://pith.science/paper/KJK2M6JN
@misc{pith2026250607023,
author = {Pith},
title = {Pith review of: Optimal Transport Driven Asymmetric Image-to-Image Translation for Nuclei Segmentation of Histological Images},
year = {2026},
howpublished = {\url{https://pith.science/paper/KJK2M6JN}},
note = {Machine review of arXiv:2506.07023}
}
read the original abstract
Segmentation of nuclei regions from histological images enables morphometric analysis of nuclei structures, which in turn helps in the detection and diagnosis of diseases under consideration. To develop a nuclei segmentation algorithm, applicable to different types of target domain representations, image-to-image translation networks can be considered as they are invariant to target domain image representations. One of the important issues with image-to-image translation models is that they fail miserably when the information content between two image domains are asymmetric in nature. In this regard, the paper introduces a new deep generative model for segmenting nuclei structures from histological images. The proposed model considers an embedding space for handling information-disparity between information-rich histological image space and information-poor segmentation map domain. Integrating judiciously the concepts of optimal transport and measure theory, the model develops an invertible generator, which provides an efficient optimization framework with lower network complexity. The concept of invertible generator automatically eliminates the need of any explicit cycle-consistency loss. The proposed model also introduces a spatially-constrained squeeze operation within the framework of invertible generator to maintain spatial continuity within the image patches. The model provides a better trade-off between network complexity and model performance compared to other existing models having complex network architectures. The performance of the proposed deep generative model, along with a comparison with state-of-the-art nuclei segmentation methods, is demonstrated on publicly available histological image data sets.
Figures
Figures from the paper (5 more)
Reference graph
Works this paper leans on
-
[1]
Diagnostic Concordance Among Patho logists Interpreting Breast Biopsy Specimens,
J. G. Elmore et al., “Diagnostic Concordance Among Patho logists Interpreting Breast Biopsy Specimens,” The Journal of the American Medical Association , vol. 313, no. 11, pp. 1122–1132, 2015
work page 2015
-
[2]
P . Filipczuk et al., “Computer-Aided Breast Cancer Diag nosis Based on the Analysis of Cytological Images of Fine Needle Biopsie s,” IEEE Transactions on Medical Imaging , vol. 32, no. 12, pp. 2169–2178, 2013
work page 2013
-
[3]
Automatic Nuclei Segmentation in H&E Sta ined Breast Cancer Histopathology Images,
M. V eta et al., “Automatic Nuclei Segmentation in H&E Sta ined Breast Cancer Histopathology Images,” Plos One , vol. 8, no. 7, p. e70221, 2013
work page 2013
-
[4]
S. Ali and A. Madabhushi, “An Integrated Region-, Bounda ry-, Shape- Based Active Contour for Multiple Object Overlap Resolutio n in Histo- logical Imagery,” IEEE Transactions on Medical Imaging , vol. 31, no. 7, pp. 1448–1460, 2012
work page 2012
-
[5]
Nucleus Detection Using Gradient Orie ntation Information and Linear Least Squares Regression,
J. T. Kwak et al., “Nucleus Detection Using Gradient Orie ntation Information and Linear Least Squares Regression,” in Proceedings of Medical Imaging: Digital Pathology , vol. 9420, 2015, pp. 152–159
work page 2015
-
[6]
Cell Segmentation Algorithm Using D ouble Thresh- olding with Morphology-Based Techniques,
C.-S. Chang et al., “Cell Segmentation Algorithm Using D ouble Thresh- olding with Morphology-Based Techniques,” in Proceedings of IEEE International Conference on Consumer Electronics, Taiwan , 2018, pp. 1–5
work page 2018
-
[7]
J. Song, L. Xiao, and Z. Lian, “Contour-Seed Pairs Learni ng-Based Framework for Simultaneously Detecting and Segmenting V arious Over- lapping Cells/Nuclei in Microscopy Images,” IEEE Transactions on Image Processing, vol. 27, no. 12, pp. 5759–5774, 2018
work page 2018
-
[8]
J. Song et al., “Multi-Layer Boosting Sparse Convolutio nal Model for Generalized Nuclear Segmentation from Histopathology Images,” Knowledge-Based Systems , vol. 176, pp. 40–53, 2019
work page 2019
Show all 31 references
-
[9]
U-Net: Convolutional Networks for Biomedi cal Image Seg- mentation,
O. R. et al., “U-Net: Convolutional Networks for Biomedi cal Image Seg- mentation,” in Proceedings of Medical Image Computing and Computer- Assisted Intervention, 2015, pp. 234–241
2015
-
[10]
UNet++: A Nested U-Net Architecture for Medical Image Segmentation,
Z. Zhou et al., “UNet++: A Nested U-Net Architecture for Medical Image Segmentation,” in Proceedings of Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decisi on Support, 2018, pp. 3–11
2018
-
[11]
Micro-Net: A Unified Model for Segme ntation of V arious Objects in Microscopy Images,
S. E. A. Raza et al., “Micro-Net: A Unified Model for Segme ntation of V arious Objects in Microscopy Images,” Medical Image Analysis , vol. 52, pp. 160–173, 2019
2019
-
[12]
Soft Att ention Mech- anism Based Network to Extract Blood V essels From Retinal Im age Modality,
Preity, A. K. Bhandari and S. Shahnawazuddin, “Soft Att ention Mech- anism Based Network to Extract Blood V essels From Retinal Im age Modality,” IEEE Transactions on Artificial Intelligence , vol. 5, no. 7, pp. 3408–3418, 2024
2024
-
[13]
Mask R-CNN,
K. He et al., “Mask R-CNN,” in Proceedings of IEEE International Conference on Computer Vision , 2017, pp. 2961–2969
2017
-
[14]
Hover-Net: Simultaneous Segmentati on and Classi- fication of Nuclei in Multi-Tissue Histology Images,
S. Graham et al., “Hover-Net: Simultaneous Segmentati on and Classi- fication of Nuclei in Multi-Tissue Histology Images,” Medical Image Analysis, vol. 58, p. 101563, 2019
2019
-
[15]
Nuclei Instance Segmentati on and Classi- fication in Histopathology Images with Stardist,
M. Weigert and U. Schmidt, “Nuclei Instance Segmentati on and Classi- fication in Histopathology Images with Stardist,” in Proceedings of IEEE International Symposium on Biomedical Imaging Challenges , 2022, pp. 1–4
2022
-
[16]
Dual Encoder–Decoder Shifted Window- Based Trans- former Network for Polyp Segmentation With Self-Learning A pproach,
Lijin P . et al., “Dual Encoder–Decoder Shifted Window- Based Trans- former Network for Polyp Segmentation With Self-Learning A pproach,” IEEE Transactions on Artificial Intelligence , vol. 5, no. 7, pp. 3456– 3469, 2024
2024
-
[17]
Transformer Based Multiple Instance Le arning for Weakly Supervised Histopathology Image Segmentation,
Z. Qian et al., “Transformer Based Multiple Instance Le arning for Weakly Supervised Histopathology Image Segmentation, ” in Pro- ceedings of Medical Image Computing and Computer Assisted I nter- vention, 2022, pp. 160–170
2022
-
[18]
BoNuS: Boundary Mining for Nuclei Segmen tation with Partial Point Labels,
Y . Lin et al., “BoNuS: Boundary Mining for Nuclei Segmen tation with Partial Point Labels,” IEEE Transactions on Medical Imaging , vol. 43, no. 6, pp. 2137–2147, 2024
2024
-
[19]
Unpaired Image-to-Image Translatio n Using Cycle- Consistent Adversarial Networks,
J. -Y . Zhu et al., “Unpaired Image-to-Image Translatio n Using Cycle- Consistent Adversarial Networks,” in Proceedings of IEEE International Conference on Computer Vision , 2017, pp. 2242–2251
2017
-
[20]
Deep Adversarial Training for Multi -Organ Nuclei Segmentation in Histopathology Images,
F. Mahmood et al., “Deep Adversarial Training for Multi -Organ Nuclei Segmentation in Histopathology Images,” IEEE Transactions on Medical Imaging, vol. 39, no. 11, pp. 3257–3267, 2020
2020
-
[21]
Unpaired Stain Transfer Using Pathology -Consistent Constrained Generative Adversarial Networks,
S. Liu et al., “Unpaired Stain Transfer Using Pathology -Consistent Constrained Generative Adversarial Networks,” IEEE Transactions on Medical Imaging , vol. 40, no. 8, pp. 1977–1989, 2021
1977
-
[22]
Cycle-Free CycleGAN Using Inverti ble Gener- ator for Unsupervised Low-Dose CT Denoising,
T. Kwon and J. C. Y e, “Cycle-Free CycleGAN Using Inverti ble Gener- ator for Unsupervised Low-Dose CT Denoising,” IEEE Transactions on Computational Imaging , vol. 7, pp. 1354–1368, 2021
2021
-
[23]
Villani, Optimal Transport: Old and New , ser
C. Villani, Optimal Transport: Old and New , ser. Grundlehren Der Mathematischen Wissenschaften. Springer Berlin Heidelbe rg, 2008
2008
-
[24]
Image Quality Assessment: From Error Vi sibility to Structural Similarity,
W. Zhou et al., “Image Quality Assessment: From Error Vi sibility to Structural Similarity,” IEEE Transactions on Image Processing , vol. 13, no. 4, pp. 600–612, 2004. 13
2004
-
[25]
A Threshold Selection Method from Gray-Level Histograms,
N. Otsu, “A Threshold Selection Method from Gray-Level Histograms,” IEEE Transactions on Systems, Man, and Cybernetics , vol. 9, no. 1, pp. 62–66, 1979
1979
-
[26]
Peyr´ e and M
G. Peyr´ e and M. Cuturi, Computational Optimal Transport: With Ap- plications to Data Science , ser. Foundations and Trends in Machine Learning. Now Publishers, 2019
2019
-
[27]
D. L. Cohn, Measure Theory , ser. Birkh¨ auser Advanced Texts Basler Lehrb¨ ucher. Springer New Y ork, 2013
2013
-
[28]
Density Esti mation Using Real NVP,
L. Dinh and J. S. -Dickstein and S. Bengio, “Density Esti mation Using Real NVP,” in Proceedings of International Conference on Learning Representations, 2017
2017
-
[29]
General Invertible Transformations fo r Flow-based Generative Modeling,
J. M. Tomczak, “General Invertible Transformations fo r Flow-based Generative Modeling,” in Proceedings of International Conference on Machine Learning W orkshop on Invertible Neural Networks, N ormaliz- ing Flows, and Explicit Likelihood Models , 2021
2021
-
[30]
A Dataset and a Technique for Generaliz ed Nuclear Segmentation for Computational Pathology,
N. Kumar et al., “A Dataset and a Technique for Generaliz ed Nuclear Segmentation for Computational Pathology,” IEEE Transactions on Medical Imaging , vol. 36, no. 7, pp. 1550–1560, 2017
2017
-
[31]
Conic: Colon nuclei identification an d counting challenge 2022,
S. Graham et al., “Conic: Colon nuclei identification an d counting challenge 2022,” arXiv preprint arXiv:2111.14485 , 2021
2022 arXiv
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.