REVIEW 4 major objections 3 minor 14 references
unMORE: Unsupervised Multi-Object Segmentation via Center-Boundary Reasoning
T0 review · 4 major / 3 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read Explicit center and boundary fields, followed by network-free reasoning, let a single image yield many discovered objects without human labels, outperforming all unsupervised baselines on six benchmarks.
desk verdict A genuinely new mechanism for unsupervised multi-object segmentation, but the reported SOTA margins are not yet credible given the evaluation protocol. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the objectness network plus its network-free reasoning loop. The network is trained to output three representations: a scalar existence score; a center field in which every pixel inside an object carries a unit vector pointing toward that object's bounding-box center; and a boundary distance field in which each pixel carries a normalized signed distance to the nearest boundary, positive inside the object and negative outside. The boundary field has a property that makes it load-bearing: the gradient norm at any interior pixel recovers the maximum distance from the boundary to the innermost pixel, so the reasoning module can convert boundary distances into physical pixel movements. The reasoning module seeds proposals from anchor boxes, removes patches with low existence scores, splits patches whose center fields produce a high anti-center response under a fixed outward-pointing kernel, and iteratively moves each proposal's four borders inward or outward according to the boundary field until convergence. Final masks are read off by thresholding the center and boundary fields, and duplicate boxes are removed by non-maximum suppression.
What would settle it
Take a test set of crowded images where the VoteCut pseudo masks demonstrably merge adjacent instances, run unMORE on them, and count whether the center-field splitting step separates those pairs; if unMORE's separations coincide with boundaries already present in the pseudo masks, the claimed gain over feature-grouping baselines reduces to pseudo-mask quality, while if it separates pairs the pseudo masks merged, the reasoning itself is doing genuine object recovery. A second decisive check is the paper's own reported failure on overlapping objects with similar textures: modifying the boundary field to incorporate texture and observing whether that failure disappears would test whether boundary geometry alone carries the claim.
Extended reading notes
Core claim
The central claim is that three levels of explicitly defined object-centric representations, learned from single-object images, are sufficient to discover many objects in complex scenes without any human annotation or additional trainable modules. The paper argues that the right representations are a binary object existence score, a unit-vector center field pointing to the object center, and a normalized signed boundary distance field, and that these three fields together let a frozen network answer where objects are, how many are present, and how far each object extends. The subsequent reasoning module is network-free: it generates anchor proposals, filters them by existence score, splits proposals whose center fields show multiple centers, and refines each proposal's borders from boundary-distance values until the bounding box is tight. The paper reports that this pipeline outperforms all existing unsupervised methods on six real-world benchmarks, and that the improvement is most pronounced on crowded images, where feature-grouping baselines under-segment by grouping adjacent objects into one.
Load-bearing premise
The load-bearing premise is that VoteCut's pseudo masks on ImageNet mark objects reliably enough to teach the objectness network real objectness; if those masks systematically merge adjacent objects, miss small ones, or drag in background, the learned center and boundary fields inherit those errors and the network-free reasoning cannot recover objects the pseudo masks never represented.
Editorial extensions
If this is right
- Because the reasoning module is network-free, future improvements to the objectness network can be dropped in without retraining the discovery procedure.
- A class-agnostic detector trained on unMORE's discovered objects inherits the multi-object recall and, the paper reports, beats detectors trained on earlier pseudo-label pipelines on COCO* and six zero-shot datasets.
- Crowded images, where feature-grouping baselines merge adjacent objects into one, become tractable because the center-field splitting step is designed specifically to separate multiple centers inside one proposal.
- The boundary distance field carries most of the performance gain in the ablations, suggesting that explicit boundary supervision, rather than a binary mask or reconstruction loss, is what lets the reasoning module tighten boxes accurately.
Reading between the lines
- Because the reasoning module reads only geometric fields from a frozen network, the same center and boundary representations could be reused for interactive segmentation or point-and-click object editing, where the center field supplies the object anchor and the boundary field supplies the extent.
- The COCO* re-annotation effort, which adds 197 categories and many previously unlabelled objects, may shift how unsupervised discovery methods are evaluated, since standard COCO annotations undercount objects and can turn correct discoveries into apparent false positives.
- A testable extension is to train the objectness network from multiple different pseudo-mask generators and compare discovery quality; if unMORE's advantage shrinks when the pseudo masks are already excellent, most of the gain comes from supervision quality, whereas if it persists, the center-boundary reasoning is doing the work.
- The paper's reported failure on overlapping objects with similar textures points to adding a texture-disambiguation term to the boundary field as the next step, rather than simply scaling the network.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes unMORE, a two-stage unsupervised multi-object segmentation pipeline. In the first stage, an objectness network is trained on ImageNet using VoteCut pseudo masks to predict three object-centric representations: a binary object existence score, an object center field, and an object boundary distance field. In the second stage, a network-free multi-object reasoning module initializes box proposals, filters them by existence scores, splits them by multi-center detection, and refines them by boundary-distance reasoning; optionally, the discovered objects are used as pseudo labels to train a class-agnostic detector. The paper claims state-of-the-art results on COCO* and several other real-world datasets, with particular strength in crowded images.
Significance. If the empirical claims are validated, unMORE would be a meaningful advance in unsupervised multi-object segmentation: it replaces pure feature-grouping with explicit learned object representations and a transparent, network-free reasoning procedure, and the mathematical property underlying the boundary update (Eq. 4) is correct. The paper also contributes a large COCO* re-annotation and promises code/data release. However, the central SOTA claim is currently conditioned on evaluation-protocol issues: hyperparameters are selected on the same COCO* validation set as the final results, the COCO20K 'zero-shot' experiment is not zero-shot for COCO-trained detectors, and no variance estimates are reported. These issues are fixable and do not appear to indicate a fundamental flaw in the method, but they must be addressed before the headline claims can be considered established.
major comments (4)
- [§4.1 / App. A.12 / Table 1] The headline result on COCO* val is not as clean as presented because hyperparameters are tuned on that same set. Appendix A.12 reports selection of tau_e_conf, tau_c_conf, tau_b_conf on the COCO* validation set (Table 10) and selection of the binary-baseline step size on a 100-image subset of COCO* val (Table 9), while Table 1 reports the final numbers on the full COCO* val set; the baselines are taken at their original-paper settings without equivalent tuning. This makes the comparison asymmetric and weakens the statement in Sec. 4 that 'all final evaluation is conducted on COCO* val set which is completely held out.' I request an independent tuning split with the final table computed on the untouched COCO* val, or a sensitivity analysis demonstrating that the reported margins are stable over a range of thresholds, applied to the baselines as well.
- [§4.2 / App. A.9 / Table 2] The 'zero-shot' result on COCO20K is not zero-shot for unMORE (or for CuVLER in its best setting). According to App. A.9, the best unMORE detector (Setting #2) is trained on pseudo labels from the COCO 2017 train set, and CuVLER's best Setting #4 also uses COCO train pseudo labels, while CutLER's best Setting #3 is ImageNet-only. COCO20K is, per App. A.7, a subset of COCO 2014 trainval, from which the COCO 2017 train images are drawn; the COCO20K test images are therefore not held out for the COCO-trained detectors. The comparison against CutLER is thus unfair, and the phrase 'zero-shot detection' is misleading. Please report an ImageNet-only unMORE detector on COCO20K, or explicitly reframe this part of Table 2 as in-domain generalization and compare only with detectors having the same training-data overlap.
- [§4.1 / Table 1 / §4.2 / Table 2] No error bars or multiple-seed variance are reported for any of the headline numbers. Proposal generation and detector training involve stochastic sampling (App. A.3, A.6), so margins such as +4.6 APbox50 over CuVLER on COCO* val in Table 1 and +1.8 APbox50 over CuVLER on COCO20K in Table 2 could lie within run-to-run variability. I ask the authors to report mean plus/minus standard deviation over at least three random seeds for the main tables, or to provide a formal significance test, for at least unMORE and the strongest baselines.
- [§4 / App. A.16] The primary evaluation benchmark, COCO*, is an author-created re-annotation of COCO val2017 with 10,336 additional SAM-assisted labels across 197 new categories. There is no independent validation of these annotations, such as inter-annotator agreement statistics or a detailed public annotation protocol. Because the method is explicitly designed to discover precisely the kinds of objects that the authors chose to label, this benchmark may favor unMORE in a way that is not yet demonstrated. The manuscript should report the full annotation protocol, provide agreement statistics on a subsample, and give the original COCO val results (currently in App. A.10) equal prominence in the main text.
minor comments (3)
- [App. A.3, Eq. (9)] Equation (9) in the appendix appears to have a sign typo: 'P_u1 <- -P_u1 - ...' should likely read 'P_u1 <- P_u1 - ...' to match the corresponding update in Eq. (6) of the main text.
- [Abstract / Sec. 4] The abstract states '6 real-world benchmark datasets,' but Section 4 lists seven named datasets (COCO*, COCO20K, LVIS, VOC, KITTI, Object365, OpenImages) in addition to the original COCO val in the appendix; please clarify the intended count.
- [App. A.16, Table 15] Table 15 contains several spelling and truncation errors ('pinapple', 'spagatti', 'guita', 'envolop', 'vegatable', 'applicance', 'tomato 53' ordering), which should be corrected in the released annotation metadata as well as in the paper.
Circularity Check
No circular derivation: outputs are produced by an externally bootstrapped objectness network and scored against external annotations; the COCO* tuning and COCO20K overlap are evaluation-protocol risks, not derivation circularity.
full rationale
The paper's derivation chain is: VoteCut (an external, published unsupervised method from CuVLER) generates single-object pseudo masks on ImageNet; these masks define the three supervision targets in Eqs. 1-3 (existence score, center field, and signed-distance boundary field); a ResNet/DPT objectness network is trained on these targets with Eq. 5; the network-free reasoning module in Sec. 3.3 queries the frozen network to split and refine proposals; optionally, the discovered objects become pseudo labels for a Cascade Mask R-CNN. No step defines its output in terms of the benchmark being predicted, and no reported number is obtained by re-using the ground-truth annotations as supervision. The VoteCut pseudo-label bootstrap and CuVLER detector recipe are external baselines, not a self-citation chain that forces the result; the paper's contributions are the three-level representation and the hand-designed reasoning procedure, which are evaluated against held-out human annotations. The self-citations to Yang & Yang (2022, 2024) appear only in related-work statements about slot-based methods and are not load-bearing for any experimental claim. The main validity concerns are evaluation-protocol issues rather than circularity: Appendix A.12 tunes pseudo-label thresholds on the same COCO* validation set later reported in Table 1, and Appendix A.7 shows COCO20K is a subset of COCO trainval2014 while the detector is trained on COCO 2017 train, so the 'zero-shot' COCO20K result overlaps the training distribution. These are leakage or fairness problems, not cases where a prediction is equivalent by construction to a fitted input. Accordingly, the paper shows no significant circularity.
Assumptions & free parameters
free parameters (8)
- tau_e (existence score threshold) =
not stated in paper
- tau_c (center split threshold) =
0.25
- tau_adjust (boundary update adjustment ratio) =
0.5
- Maximum proposal update iterations =
50
- Initial anchor scales and aspect ratios =
scales [32,64,128,256,512], ratios [0.5,1,2]
- Pseudo-label selection thresholds tau_e_conf, tau_c_conf, tau_b_conf =
0.5, 0.8, 0.75
- Proposal patch resolution =
128x128
- Stopping margin for boundary shrinkage =
16 pixels
assumptions (6)
- domain assumption VoteCut pseudo masks on ImageNet provide reliable single-object masks for training the objectness network.
- domain assumption DINO/v2 features provide enough object localization information for VoteCut to isolate objects.
- standard math The signed distance field gradient property in Eq. 4 holds at training-time masks and at inference-time predictions.
- domain assumption Iterative border updates converge to the tight bounding box for objects represented by the learned fields.
- domain assumption The connected-component splitting from CuVLER correctly separates objects that are more than 5 pixels apart.
- domain assumption COCO* annotations are an accurate superset of objects in COCO val2017.
Cite this review
Pith. "Pith review of unMORE: Unsupervised Multi-Object Segmentation via Center-Boundary Reasoning." pith.science (2026). https://pith.science/paper/QF3BJKKN
@misc{pith2026250601778,
author = {Pith},
title = {Pith review of: unMORE: Unsupervised Multi-Object Segmentation via Center-Boundary Reasoning},
year = {2026},
howpublished = {\url{https://pith.science/paper/QF3BJKKN}},
note = {Machine review of arXiv:2506.01778}
}
read the original abstract
We study the challenging problem of unsupervised multi-object segmentation on single images. Existing methods, which rely on image reconstruction objectives to learn objectness or leverage pretrained image features to group similar pixels, often succeed only in segmenting simple synthetic objects or discovering a limited number of real-world objects. In this paper, we introduce unMORE, a novel two-stage pipeline designed to identify many complex objects in real-world images. The key to our approach involves explicitly learning three levels of carefully defined object-centric representations in the first stage. Subsequently, our multi-object reasoning module utilizes these learned object priors to discover multiple objects in the second stage. Notably, this reasoning module is entirely network-free and does not require human labels. Extensive experiments demonstrate that unMORE significantly outperforms all existing unsupervised methods across 6 real-world benchmark datasets, including the challenging COCO dataset, achieving state-of-the-art object segmentation results. Remarkably, our method excels in crowded images where all baselines collapse.
Figures
Figures from the paper (14 more)
Reference graph
Works this paper leans on
-
[1]
Both models are from the original paper and are included for reference
For UnSAM, it has two detectors trained under two set- tings below. Both models are from the original paper and are included for reference. • Setting #1: It trains a detector on pseudo objects discov- ered by MaskCut on ImageNet train set, and then the detector is used to infer scene images jointly with Mask- Cut. • Setting #2: The detector trained in its...
-
[2]
For CutLER, it has three detectors trained under three settings below. The Settings #1/#2 are fairly comparable with our Settings #1/#2, whereas its Setting #3 is from the original paper. • Setting #1: It is trained on pseudo objects discovered by its own MaskCut on COCO train set. • Setting #2: It is trained on two groups of pseudo labels: one group from...
-
[3]
For CuVLER, it has four detectors trained under four settings below. The Settings #1/#2 are fairly comparable with our Settings #1/#2, whereas its Settings #3/#4 are from the original paper. • Setting #1: It is trained only on pseudo objects discovered by its own V oteCut on COCO train set. • Setting #2: It is trained on two groups of pseudo labels: one g...
-
[4]
For our method, named unMORE, we train two separate detectors under two settings: • Setting #1: It is trained only on pseudo objects discovered by our method on COCO train set. • Setting #2: It is trained on two groups of pseudo labels: one group from our discovered objects on COCO train set, another from object pseudo labels generated by V oteCut on Imag...
-
[5]
Only using a binary mask as the object-centric repre- sentation: In the task of object segmentation, a binary mask is probably the most commonly-used object representation. In particular, we remove all of our three object-centric rep- resentations, but just train the same objectness network to predict a binary mask. Then, when discovering multi- 16 unMORE...
-
[6]
In the absence of object boundary field, the binary mask representation can update bounding boxes
Only using a binary mask and an object existence score: This is to evaluate whether the object existence score can be useful for better object segmentation. In the absence of object boundary field, the binary mask representation can update bounding boxes
-
[7]
In the absence of object boundary field, the binary mask representation can update bounding boxes
Only using a binary mask and an object center field: This is to evaluate whether the object center field can be useful for better object segmentation. In the absence of object boundary field, the binary mask representation can update bounding boxes
-
[8]
In the absence of object boundary field, the binary mask representation can update bounding boxes
Using a binary mask, an object existence score and center field: This is to evaluate whether both object exis- tence score and center field can be useful for better object segmentation. In the absence of object boundary field, the binary mask representation can update bounding boxes
Show all 14 references
-
[9]
Only using an object boundary field: This is to verify the importance of object boundary field
-
[10]
Only using an object boundary field and existence score: This is to evaluate whether adding the existence score can help object segmentation on top of the object boundary field
-
[11]
Only using an object boundary field and center field: This is to evaluate whether adding the center field can help object segmentation on top of the object boundary field
-
[12]
Our full three-level object-centric representations: This is our full framework for reference. A.12. More Ablations Selection of Fixed Step Size for Binary Baseline.Since the information provided by binary mask representation is very limited, the final discovered objects can b...
-
[13]
It could be possible to leverage reinforcement learning tech- niques to learn an efficient policy net to discover objects
Direct Object Discovery of unMORE disc takes time. It could be possible to leverage reinforcement learning tech- niques to learn an efficient policy net to discover objects
-
[14]
#$Input unMORE!
Our method struggles to separate overlapping objects with similar textures, as shown in the attached Figure 11. Additional language priors may help alleviate this issue. A.15. More Visualizations 18 unMORE: Unsupervised Multi-Object Segmentation via Center-Boundary Reasoning T...
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.