REVIEW 3 major objections 4 minor 26 references
ORGAN, a CycleGAN that translates images into object lists and back, is the only tested approach that reliably detects separate objects in dense, low-contrast real-world scenes such as microscopy images.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-02 19:27 UTC pith:L7STVWJN
load-bearing objection Clever CycleGAN-based object-centric model with a real architectural contribution, but the paper's unsupervised claim rests on an unspecified target in Eq. (3) that may leak object locations. the 3 major comments →
ORGAN: Object-Centric Representation Learning using Cycle Consistent Generative Adversarial Networks
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
ORGAN's central claim is that an unpaired image-to-list translation objective, enforced by two GANs and a cycle-consistency loss, yields object-centric representations that are competitive on synthetic data and superior on challenging real-world low-contrast multi-object scenes. The model detects objects by partitioning the image into overlapping patches, scoring them, suppressing non-maxima, and selecting the top k patches with a differentiable operator; each selected patch is encoded into a list entry with position, presence probability, and feature vector. The reverse generator places Gaussian blobs from the list into a multi-channel image and refines it with a convolutional network. The
What carries the argument
The key mechanism is the cycle-consistent generative adversarial network with two domains, image and list. The image-to-list generator uses a differentiable top-k patch selection after non-maximum suppression, so the model can output a variable number of object proposals; the list-to-image generator turns each list entry into an isotropic Gaussian blob whose channel values are projections of the feature vector, then a U-Net-style convolutional network renders the final image. Training is driven by adversarial losses for each domain plus two cycle-consistency terms—mean absolute error for images and a matched element-wise loss for lists, where matching is solved by a linear-sum assignment (Hu
Load-bearing premise
The training signal includes a term that requires the object locations to compute a target score map; if these locations come from ground-truth labels, the method is not fully unsupervised, and if they are derived from the model's own output, the cycle-consistency signal becomes circular—the paper does not specify which.
What would settle it
Run ORGAN on a dataset with no available object locations and a fixed, known object count; if the model cannot be trained without the t_p term, or if detection fails when t_p is removed, the claimed unsupervised capability is not established. Alternatively, compute t_p from the model's own patch scores and show that the cycle loss is minimized even when object detection is wrong.
If this is right
- If ORGAN's scaling claim holds, a single model trained on small patches can be deployed on arbitrarily large images, removing the need for tiling and its boundary artifacts.
- The list domain provides a natural interface for object manipulation: editing positions, swapping features, or adding or removing entries before cycling back to an image, which could support data augmentation and interactive editing.
- The success on low-contrast microscopy cells suggests GAN-based cycle-consistency is a viable alternative to autoencoder slot models for biological imaging, where objects are numerous and visually similar.
- The method's requirement for a list domain could be extended to other structured outputs (e.g., keypoints, text descriptions) as long as a differentiable cycle can be defined.
Where Pith is reading between the lines
- The auxiliary scorer target t_p in Eq. (3) is a potential leakage of ground-truth object locations; if it is computed from labels, the method is not fully unsupervised, and if it is recomputed from the model's own predictions, the cycle loss could be trivially satisfied. The paper does not resolve this ambiguity, and it is the main open question for reproducing the unsupervised claim.
- Because the list cycle loss relies on a discrete assignment, the method may be sensitive to the top-k hyperparameter; inference suggests that a mismatch between k and the true object count could degrade detection, and a learned or adaptive k would be a natural extension.
- The method's generality beyond homogeneous-background scenes is untested; given the paper's own limitation about complex backgrounds, a testable extension is to combine ORGAN with a background model or to evaluate on datasets with textured backgrounds.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces ORGAN, a CycleGAN-based architecture for object-centric representation learning that maps between images and 'lists' (object locations, presence probabilities, and feature vectors). It reports competitive F1 scores on synthetic datasets (Tetrominoes, Sprites, Multi-MNIST) and claims to be the only method tested that successfully detects objects on a real-world low-contrast Cells dataset, while also scaling to larger images. Comparisons are made against SPACE, SLATE, LSD, and SPOT, with additional experiments on latent space structure.
Significance. If the claims hold, ORGAN would be a novel GAN-based alternative to autoencoder/slot-attention models, particularly valuable for microscopy data with many small objects. The paper includes a reasonable architectural contribution, a broad comparison, and a scaling analysis. However, the central 'unsupervised' claim and the scaling results contain ambiguities and apparent inconsistencies that must be resolved before the contribution can be assessed.
major comments (3)
- [Eq. (3), §3.2, §4.2] The list cycle loss in Eq. (3) includes λ_loc ||s_ϕ(p) − t_p||²/|t_p|, where t_p is 'the target for the scorer calculated from the locations of the objects.' For synthetic datasets these locations come from the simulator. For Cells, the paper does not state how t_p or the real list domain is constructed; §4.2 mentions only 50 expert-annotated images for evaluation. If t_p is derived from these annotations during training, ORGAN receives object-location supervision that the baselines do not, invalidating the 'unsupervised' and 'only approach' claims. If t_p is derived from the model's own output, the scorer gradient is circular. The authors must specify the data flow for t_p and the list domain for all datasets.
- [§3.3, §4.4] The list generator extracts exactly k patches, with k 'tuned according to the maximum expected number of objects' (§3.1.1). For Sprites, k=10 (§3.3). In the scaling experiment (§4.4), ORGAN is trained on 128×128 Sprites (k=10) and evaluated on 256×256 images with up to 39 objects, yet Fig. 5A reports recall of 0.934 at 39 objects. With k=10, recall cannot exceed 10/39 ≈ 0.256, so either k was increased at inference without disclosure, or the reported recall is inconsistent with the architecture. This also bears on the claim in §1 that 'CycleGANs correctly identify object counts' — the count is upper-bounded by k. Please clarify how k is set in the scaling experiment and discuss the implications.
- [§4.2, Table 1] Table 1 reports F1 scores with large standard deviations (e.g., LSD Tetrominoes 20.0±39.8, SPOT 13.8±44.6) but does not state the number of seeds/runs. For Cells, where the main claim rests, ORGAN's F1=77.9±8.6, but the baselines' near-zero scores are plausible; still, the synthetic-dataset 'performs similarly' claim is weakened by variance. Additionally, §4.2 says SPOT's object locations come from its segmentation masks, but Fig. 3 and the caption state SPOT does not allow image reconstruction; explain how masks are obtained.
minor comments (4)
- [§4.4] Typo: 'ORGAN outperformed ORGAN' should be 'ORGAN outperformed SPACE'.
- [Table 1] Check units/typos: SLATE Tetrominoes training time is 6.75 h while other SLATE entries are 0.75 h; SPACE Sprites training time '4.08' may be '40.8'.
- [Availability Statement] The GitHub link lacks a commit hash; the code and dataset pipeline (especially list construction for Cells) are not reproducible from the information given.
- [§3.3, §5] The paper's framing as 'unsupervised' should be tempered given that ORGAN explicitly requires a list domain. For real-world datasets, deriving lists typically requires object-location information, which is a form of supervision unless the paper explains otherwise.
Circularity Check
ORGAN's detection result is driven by Eq. (3)'s t_p term computed from object locations and by the preset list size k, so the 'unsupervised / only approach' claims reduce to the method's own inputs.
specific steps
-
fitted input called prediction
[Eq. (3), §3.2; §4.2; §5]
"LCyc L =E L(1/k Σ_l [CObj(l,lCyc) + ||ηl−ηCyc l||^2/|ηl|] + λ_loc ||sϕ(p)−tp||^2/|tp|), with tp being the target for the scorer calculated from the locations of the objects. ... ORGAN requires a dataset that contains lists, where the size is controlled by a hyperparameter."
The scorer sϕ is trained with a direct regression term λ_loc||sϕ(p)−tp||^2/|tp| whose target is 'calculated from the locations of the objects.' For synthetic data those locations are simulator ground truth; for Cells the only ground-truth locations mentioned are the 50 expert-annotated images in §4.2, and §5 concedes that ORGAN requires a dataset that contains lists. Thus the predicted patch scores and therefore the reported object locations are fitted to the supplied list locations, and the F1/recall numbers measure that fit rather than an unsupervised discovery. The paper's 'unsupervised' and 'only approach tested here capable' claims thereby reduce to the availability of a location-annotated list domain. If instead t_p were derived from the model's own output, the gradient would be circ
-
self definitional
[Intro, §3.1.1, §5]
"we show that CycleGANs correctly identify object counts even when test-time properties, such as object spacing and object count, differ from the training data. ... The number of extracted patches k is a hyperparameter that can be tuned according to the maximum expected number of objects within an image. ... ORGAN requires a dataset that contains lists, where the size is controlled by a hyperparameter."
The output list has exactly k entries, and k is chosen as the maximum expected number of objects. The datasets are defined with lists of fixed size (Tetrominoes: three entries; Sprites, Multi-MNIST, Cells: ten entries). When ORGAN is evaluated on 256×256 images 'with up to 39 objects,' k must be set to that maximum, so the object count is an input rather than a predicted quantity. The advertised ability to 'correctly identify object counts' is therefore the prescribed list size by construction, not an emergent count-estimation capability.
full rationale
The core CycleGAN image/list cycle, the GAN losses, the Hungarian assignment, and the feature-space experiments are defined independently and are not circular by themselves. There is no load-bearing self-citation chain: Ihle et al. (2019) is used only as background for CycleGAN segmentation. However, the central detection claim is undermined by Eq. (3), where the scorer is trained against t_p 'calculated from the locations of the objects.' For a real-world dataset, the paper never explains how the list domain is obtained, and §5 admits that ORGAN requires a dataset that contains lists; the only cell ground truth mentioned is expert annotation. If those locations provide t_p, ORGAN's detection is supervised rather than unsupervised; if t_p is model-derived, the cycle-loss gradient is circular. Additionally, the fixed hyperparameter k prescribes the output list size, so the 'object count' generalization claim reduces to the chosen k. Together these two issues make the headline claims—unsupervised, only approach capable on low-contrast real data, and scaling with object count—depend on the method's own inputs. I therefore set the circularity score at 6: partial circularity in the central claim, while other components retain independent content.
Axiom & Free-Parameter Ledger
free parameters (5)
- k (number of list entries / top-k) =
3 (Tetrominoes), 10 (Sprites/Multi-MNIST/Cells)
- Patch size ph=pw and stride =
per-dataset; stride = pw/8
- Gaussian blob sigma =
pw/10
- Loss weights lambda_DisL, lambda_DisI, lambda_CycI, lambda_CycL, lambda_pres, lambda_loc =
not reported
- Feature dimensionality |eta| =
dataset-dependent
axioms (4)
- domain assumption Cycle-consistent translation between image and list domains is a well-posed training signal for object discovery.
- ad hoc to paper Object locations are available for the scorer target t_p in Eq. (3).
- standard math Differentiable top-k with non-maximum suppression provides usable gradients for selection.
- domain assumption Objects can be represented as fixed isotropic Gaussian blobs plus a U-Net style transfer.
read the original abstract
Although data generation is often straightforward, extracting information from data is more difficult. Object-centric representation learning can extract information from images in an unsupervised manner. It does so by segmenting an image into its subcomponents: the objects. Each object is then represented in a low-dimensional latent space that can be used for downstream processing. Object-centric representation learning is dominated by autoencoder architectures (AEs). Here, we present ORGAN, a novel approach for object-centric representation learning, which is based on cycle-consistent Generative Adversarial Networks instead. We show that it performs similarly to other state-of-the-art approaches on synthetic datasets, while at the same time being the only approach tested here capable of handling more challenging real-world datasets with many objects and low visual contrast. Complementing these results, ORGAN creates expressive latent space representations that allow for object manipulation. Finally, we show that ORGAN scales well both with respect to the number of objects and the size of the images, giving it a unique edge over current state-of-the-art approaches.
Figures
Reference graph
Works this paper leans on
-
[1]
Adeel Pervez, Phillip Lippe, and Efstratios Gavves. Differentiable mathematical programming for object-centric representation learning.arXiv preprint arXiv:2210.02159,
-
[8]
Adversarial synthesis learning enables segmentation without target modality ground truth
Yuankai Huo, Zhoubing Xu, Shunxing Bao, Albert Assad, Richard G Abramson, and Bennett A Landman. Adversarial synthesis learning enables segmentation without target modality ground truth. In2018 IEEE 15th international symposium on biomedical imaging (ISBI 2018), pages 1217–1220. IEEE,
2018
-
[12]
Data synthesis based on generative adversarial networks.arXiv preprint arXiv:1806.03384,
Noseong Park, Mahmoud Mohammadi, Kshitij Gorde, Sushil Jajodia, Hongkyu Park, and Youngmin Kim. Data synthesis based on generative adversarial networks.arXiv preprint arXiv:1806.03384,
-
[13]
Cyclegan-vc2: Improved cyclegan-based non-parallel voice conversion
Takuhiro Kaneko, Hirokazu Kameoka, Kou Tanaka, and Nobukatsu Hojo. Cyclegan-vc2: Improved cyclegan-based non-parallel voice conversion. InICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 6820–6824. IEEE,
2019
-
[14]
(un) paired signal-to-signal translation with 1d conditional gans.arXiv preprint arXiv:2403.04800,
Eric Easthope. (un) paired signal-to-signal translation with 1d conditional gans.arXiv preprint arXiv:2403.04800,
-
[15]
Text-to-image-to-text translation using cycle consistent adversarial networks
Satya Krishna Gorti and Jeremy Ma. Text-to-image-to-text translation using cycle consistent adversarial networks. arXiv preprint arXiv:1808.04538,
-
[16]
Object discovery with a copy-pasting gan.arXiv preprint arXiv:1905.11369,
Relja Arandjelovi´c and Andrew Zisserman. Object discovery with a copy-pasting gan.arXiv preprint arXiv:1905.11369,
Pith/arXiv arXiv 1905
-
[18]
Ssd: Single shot multibox detector
Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, and Alexander C Berg. Ssd: Single shot multibox detector. InComputer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part I 14, pages 21–37. Springer,
2016
-
[19]
Yolov3: An incremental improvement.arXiv preprint arXiv:1804.02767,
Joseph Redmon. Yolov3: An incremental improvement.arXiv preprint arXiv:1804.02767,
-
[20]
Illiterate dall-e learns to compose.arXiv preprint arXiv:2110.11405,
Gautam Singh, Fei Deng, and Sungjin Ahn. Illiterate dall-e learns to compose.arXiv preprint arXiv:2110.11405,
-
[23]
Sindy Löwe, Phillip Lippe, Maja Rudolph, and Max Welling
URL https://arxiv.org/abs/2410.00728. Sindy Löwe, Phillip Lippe, Maja Rudolph, and Max Welling. Complex-valued autoencoders for object discovery.arXiv preprint arXiv:2204.02075,
-
[24]
U-net: Convolutional networks for biomedical image segmenta- tion
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmenta- tion. InMedical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18, pages 234–241. Springer,
2015
-
[25]
Spectral normalization for generative adversarial networks.arXiv preprint arXiv:1802.05957,
Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida. Spectral normalization for generative adversarial networks.arXiv preprint arXiv:1802.05957,
-
[1995]
Auto-encoding variational bayes.arXiv preprint arXiv:1312.6114,
Diederik P Kingma. Auto-encoding variational bayes.arXiv preprint arXiv:1312.6114,
-
[2010]
Movi: A large multipurpose motion and video dataset
Saeed Ghorbani, K Mahdaviani, Anne Thaler, K Kording, DJ Cook, G Blohm, and NF Troje. Movi: A large multipurpose motion and video dataset. arxiv 2020.arXiv preprint arXiv:2003.01888, 9,
Pith/arXiv arXiv 2020
-
[2012]
Blood dataset
Irfan Bin Azmi. Blood dataset. 2024-07-08. url:.https://www.kaggle.com/datasets/irfanbinazmi/blood-dataset,
2024
-
[2014]
Contrastive learning of structured world models.arXiv preprint arXiv:1911.12247,
Thomas Kipf, Elise Van der Pol, and Max Welling. Contrastive learning of structured world models.arXiv preprint arXiv:1911.12247,
Pith/arXiv arXiv 1911
-
[2016]
Gmair: Unsupervised object detection based on spatial attention and gaussian mixture model.Computational Intelligence and Neuroscience, 2022(1): 7254462,
Weijin Zhu, Yao Shen, Mingqian Liu, and Lizeth Patricia Aguirre Sanchez. Gmair: Unsupervised object detection based on spatial attention and gaussian mixture model.Computational Intelligence and Neuroscience, 2022(1): 7254462,
2022
-
[2018]
Semantic-aware generative adversarial nets for unsupervised domain adaptation in chest x-ray segmentation
Cheng Chen, Qi Dou, Hao Chen, and Pheng-Ann Heng. Semantic-aware generative adversarial nets for unsupervised domain adaptation in chest x-ray segmentation. InMachine Learning in Medical Imaging: 9th International Workshop, MLMI 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 16, 2018, Proceedings 9, pages 143–151. Springer,
2018
-
[2019]
Benchmarking and analysis of unsupervised object segmentation from real-world single images.International Journal of Computer Vision, 132(6):2077–2113,
Yafei Yang and Bo Yang. Benchmarking and analysis of unsupervised object segmentation from real-world single images.International Journal of Computer Vision, 132(6):2077–2113,
2077
-
[2020]
Synthesizing tabular data using generative adversarial networks.arXiv preprint arXiv:1811.11264,
Lei Xu and Kalyan Veeramachaneni. Synthesizing tabular data using generative adversarial networks.arXiv preprint arXiv:1811.11264,
-
[2021]
Bridging the gap to real-world object-centric learning
Maximilian Seitzer, Max Horn, Andrii Zadaianchuk, Dominik Zietlow, Tianjun Xiao, Carl-Johann Simon-Gabriel, Tong He, Zheng Zhang, Bernhard Schölkopf, Thomas Brox, et al. Bridging the gap to real-world object-centric learning. arXiv preprint arXiv:2209.14860,
-
[2022]
On the binding problem in artificial neural networks
Klaus Greff, Sjoerd Van Steenkiste, and Jürgen Schmidhuber. On the binding problem in artificial neural networks. arXiv preprint arXiv:2012.05208,
Pith/arXiv arXiv 2012
-
[2023]
Tianyang Chen, Yufei Huang, Zhiqiang Shen, Jian Zhang, Yao Li, and Lei Zhang
URL https://arxiv.org/ abs/2303.10834. Tianyang Chen, Yufei Huang, Zhiqiang Shen, Jian Zhang, Yao Li, and Lei Zhang. Learning global object-centric representations via disentangled slot attention.Machine Learning, 114(40):1–22,
-
[2024]
Zhixuan Lin, Yi-Fu Wu, Skand Vishwanath Peri, Weihao Sun, Gautam Singh, Fei Deng, Jindong Jiang, and Sungjin Ahn. Space: Unsupervised object-oriented scene representation via spatial attention and decomposition.arXiv preprint arXiv:2001.02407,
Pith/arXiv arXiv 2001
-
[2025]
Karl Stelzner, Robert Peharz, and Kristian Kersting
doi:10.1007/s10994-024- 06687-9. Karl Stelzner, Robert Peharz, and Kristian Kersting. Faster attend-infer-repeat with tractable probabilistic models. In International Conference on Machine Learning, pages 5966–5975. PMLR,
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.