{"id":"b8546292-69f6-4a09-8f0f-197b1b5b72b7","arxiv_id":"2507.11985","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"MPAE discovers object parts by filling randomly masked image patches with learned part descriptors and then restoring the image, reporting state-of-the-art unsupervised part discovery on four benchmarks.","lead":"This paper introduces MPAE, a self-supervised method that finds object parts by randomly masking parts of an image and restoring them with learned part descriptors. It matters because it produces part-level masks without any annotations and works across multiple object categories, potentially reducing the cost of part-level labels in fine-grained visual tasks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper's own Table 4 shows that removing Ls, which the text says makes the model degrade into a foreground segmentation model, raises NMI/ARI at K=8/25 (55.69/76.54 vs 33.51/65.05).","rationale":"I read the paper in good faith: the masked-restoration mechanism in Section 3.1.1 is plausible, and the VGG-19 structural-penalty ablation in Appendix B.1 is honest evidence that the restoration target matters for part boundaries. The load-bearing weakness is not the mechanism itself but the validity of the measurements used to support the headline claim. Table 4 and Appendix B.8 show that a model which, by the authors' own description, degrades into foreground segmentation scores higher on NMI/ARI at K=8/25 than the full model; at K=50 the two are essentially tied. Hence the primary metric cannot tell meaningful part discovery apart from background separation, and the cross-method comparisons in Tables 1-3 inherit this confound. This is more decisive than the implicit-clustering concern: even if the clustering mechanism operates exactly as described, the reported quantitative evidence would not demonstrate it, because the metric rewards the degenerate alternative. The proposed foreground-only NMI/ARI check would settle whether the margins survive once background separation cannot inflate the scores. I keep the reader's CONDITIONAL verdict, so no verdict change is needed; the condition should explicitly require a part-quality metric that is insensitive to foreground/background separation.","tokens_in":17799,"tokens_out":10107,"duration_ms":110957,"concrete_test":"On PartImageNet Segmentation, recompute NMI and ARI after removing the background class from both the ground-truth labels and the predicted similarity maps, so that only the K predicted part masks over foreground pixels are scored. If the full model's margin over the Without-Ls ablation in Table 4 collapses at K=8 and K=25, the headline gains are an artifact of foreground/background separation; if the margin persists, the confound is not the source of the reported improvement.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that MPAE robustly discovers meaningful parts and achieves more competitive performance than state-of-the-art methods (Abstract, Section 1). The quantitative evidence is NMI and ARI, but the paper's own ablations show these metrics cannot distinguish part discovery from a foreground/background split. In Table 4, removing the semantic constraint Ls increases NMI/ARI from 33.51/65.05 to 55.69/76.54 at K=8, from 37.22/68.04 to 57.70/76.97 at K=25, and leaves K=50 essentially unchanged (55.10/73.52 vs 56.42/74.03). Appendix B.8 reports that this model discovers only 3.94 foreground parts per image on average, and the text states it degenerates into a foreground segmentation model even though the numerical metrics improve. Therefore every comparison in Tables 1-3 is contaminated by a metric that rewards coarse foreground/background separation. The comparison with Xia et al. is additionally mismatched in cluster count (K=4Nc, i.e., 436 for OOD and 632 for Segmentation, versus K=8/25/50), which further weakens the claimed margins. This is a measurement-validity problem, not a claim that the method is wrong; the mechanism in Section 3.1.1 may still work, but the reported numbers do not establish it.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes MPAE, a masked autoencoder variant for unsupervised part discovery. The model learns K+1 part descriptors from a trainable ViT, extracts dense features from a frozen pretrained ViT, and uses the similarity between descriptors and local features to fill masked patches in an image restoration objective. Several auxiliary losses are introduced: a presence loss (Lf and Lb), a semantic consistency loss (Ls), and a distribution loss (Ld). The paper reports NMI/ARI comparisons on PartImageNet, CUB, and CelebA, plus extensive ablations over masking ratios, hyperparameters, backbones, and loss components. The code is provided.","tokens_in":18131,"tokens_out":4975,"duration_ms":48221,"significance":"The idea of using masked restoration with part descriptors as fill tokens is interesting, and the qualitative results suggest plausible part masks across categories and scenarios. The paper ships code and includes extensive ablations, which is a strength. If the quantitative claims held, this would be a useful step toward unsupervised part discovery. However, the evaluation issues detailed below mean that the numerical superiority over prior work is not currently established, and the paper's own ablations indicate that the primary metric may reward a degenerate foreground/background solution.","major_comments":[{"comment":"The paper's primary evidence is NMI/ARI, yet its own ablation shows these metrics reward a degenerate solution. Removing Ls increases NMI/ARI at K=8 (55.69/76.54 vs 33.51/65.05) and K=25 (57.70/76.97 vs 37.22/68.04), while the text states the model without Ls 'degrades into a foreground segmentation model' and Appendix B.8 reports only 3.94 foreground parts per image. Thus the higher NMI/ARI of the full model at K=50 (55.10/73.52) could in principle be driven by coarse foreground/background separation rather than meaningful part structure. The comparisons in Tables 1-3 are therefore contaminated: they do not establish that MPAE discovers parts better than a simple foreground segmentation model. Please add baselines that explicitly evaluate part-level quality (e.g., part-aware IoU or semantic part consistency) and report the number of active parts per image for all methods.","section":"Section 4.4, Table 4"},{"comment":"The comparison with Xia et al. is at a different cluster count (K=4Nc, i.e., 436 for OOD and 632 for Segmentation) versus MPAE at K=8/25/50. NMI and ARI are known to depend on the number of clusters, so these numbers are not directly comparable. Notably, Table 6 shows that changing MPAE's K from 50 to 4Nc reduces NMI from 55.10 to 47.59 and ARI from 73.52 to 70.71, which is a large drop. Please either evaluate MPAE at the same K as Xia et al., or evaluate Xia et al. at the same K as MPAE, and discuss the cluster-count dependence.","section":"Table 1 and Table 6"},{"comment":"The paper asserts that the restoration objective 'implicitly clusters' the filled descriptors and unmasked patch features, and that this is the mechanism that aligns descriptors with part shapes. The only direct evidence is the large drop when the VGG-19 structural penalty is removed (NMI 55.10 to 19.65, ARI 73.52 to 49.72 in Table 7). This shows the component matters but does not show that the clustering actually occurs as described. Please provide direct evidence, such as an analysis of the latent space distribution of filled versus unmasked features, or a controlled experiment that varies restoration difficulty, or temper the mechanistic claim to what is demonstrated.","section":"Section 3.1.1 and Appendix B.1"}],"minor_comments":[{"comment":"Typos: 'inpus' should be 'inputs'; 'datasests' should be 'datasets'.","section":"Section 1 and Section 4.1"},{"comment":"The phrase 'as followings' should be 'as follows'.","section":"Contributions list"},{"comment":"The typesetting of the ArcFace-style loss is garbled (e.g., 'M kes' appears to be M_k * exp(...)). Please rewrite with clear mathematical notation and define all symbols in one place.","section":"Equation (7)"},{"comment":"The term 'PartFormer' is used without a formal definition; earlier in Section 3.1.1 it is mentioned in passing. Please introduce the term when the architecture is first described.","section":"Section 3.3"},{"comment":"The sentence 'a higher r results in more features in the filled feature map R comes from part descriptors D' has a subject-verb agreement problem and should be rephrased.","section":"Section 4.4, masking ratio paragraph"}],"recommendation":"major_revision","confidential_remarks":"The comparison to Xia et al. is against the first author's own prior TPAMI work, which is legitimate but should be scrutinized for fairness; the cluster-count mismatch amplifies this concern. The abstract's claim of 'more competitive performance on all datasets' is stronger than what the evidence supports given the metric validity issue. Both issues are fixable in revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nBottom line: this paper is a genuine step forward for unsupervised part discovery, with a new mechanism and unusually thorough ablations, but its own experiments undermine the headline quantitative claims. The descriptor-based masked restoration idea—filling masked patches with soft descriptor mixtures and restoring the image—is genuinely different from the reconstruction and clustering baselines, and the qualitative results suggest it does discover coherent parts across categories. The loose presence constraints and the deep-dive ablations (masking ratio, group size, loss weights, backbone choice, encoder depth) are more careful than most work in this area. Code is promised. I would read it for the mechanism alone.\n\nThe soft spot is the metric. Table 4 shows that removing Ls, the semantic consistency loss, raises NMI and ARI at K=8 and K=25 (33.51 to 55.69 NMI at K=8), and Appendix B.8 confirms the model degrades to ~4 foreground parts per image—essentially foreground segmentation. The authors are upfront that this \"degrades into a foreground segmentation model,\" but the metric that rewards it is the same one used to claim state-of-the-art in Tables 1–3. At K=50 the gap nearly disappears, so the comparison with DINO and Xia et al. at K=8 and K=25 is not measuring part discovery quality; it may be measuring coarseness. That is a measurement-validity problem, not a proof that the method is wrong, but the paper's quantitative evidence does not establish its central claim.\n\nThe other issue is the comparison with Xia et al. uses K=4Nc, i.e., 436 or 632 parts, while MPAE is evaluated at K=8/25/50. Table 6 shows MPAE itself drops from 55.10 to 47.59 NMI when K is changed to 4Nc, so the setting difference is not innocuous. Without a like-for-like K, those margins are not credible. No error bars either, which for a stochastic training pipeline is a real omission.\n\nThe fragility to the VGG-19 structural penalty (19.65 vs 55.10 NMI without it) is worth flagging but the authors discuss it and include it in the main loss, so I would call that a documented sensitivity rather than a hidden flaw.\n\nWho is this for? People working on unsupervised part discovery, fine-grained recognition, or self-supervised feature grouping. The mechanism deserves a serious referee and the paper is well above the desk-reject line. But I would send it back asking for a faithful evaluation protocol: fixed K comparisons, error bars, and at minimum a discussion of the foreground-segmentation degeneracy in the context of the reported metrics. The authors are honest enough to fix this.\n\nRecommendation: engage, conditionally. The mechanism may well be the right idea; the current numbers just don't prove it yet.","headline":"Genuinely new mechanism for unsupervised part discovery, but the paper's own ablation shows the reported NMI/ARI metrics reward a degenerate foreground-segmentation solution, so the quantitative claims need a credibility fix.","tokens_in":18693,"tokens_out":3317,"would_cite":true,"duration_ms":35746,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"MPAE discovers meaningful object parts with no labels: filling randomly masked patches with learned part descriptors and restoring the image aligns descriptor semantics with real part shapes, and the method outperforms baselines on…","keywords":["unsupervised part discovery","masked autoencoder","part descriptors","similarity maps","part segmentation","self-supervised Vision Transformer","perceptual loss","part presence constraints"],"falsifier":"Train MPAE with a decoder strong enough to reconstruct masked patches almost perfectly from visible context alone, for example by using a much deeper decoder or a very low masking ratio, and then measure whether the similarity maps still match annotated part boundaries. If reconstruction stays accurate while the similarity maps no longer track part shapes, the claimed implicit alignment through restoration is not the driving mechanism.","tokens_in":17584,"feed_emoji":"🧩","tokens_out":7337,"duration_ms":81981,"temperature":0.7,"pith_summary":"MPAE (Masked Part Autoencoder) is a training paradigm for discovering object parts without any part or category labels. Instead of reconstructing the whole image directly from part descriptors, it randomly masks patches, fills the masked regions with learned part descriptors according to similarity maps, and then restores the image from that filled feature map. The restoration step is what aligns the high-level semantics of the descriptors with actual part shapes, because the visible patches supply low-level appearance cues such as boundaries and textures. With relaxed presence constraints that only require each part to appear somewhere in a small group of images, the model can handle multi-category data and occlusion. The paper reports that this yields masks that closely match object parts on PartImageNet, CUB, and CelebA, outperforming both unsupervised and weakly supervised baselines.","feed_headline":"Masked-image restoration discovers object parts with no labels","feed_subtitle":"Part descriptors fill hidden patches and align with real object shapes, outperforming label-based methods on four benchmarks.","key_machinery":"The central object is the Masked Part Autoencoder and its matching block. A frozen self-supervised Vision Transformer plus a trainable $1 \\times 1$ convolution produces a dense feature map $F$; the model learns $K+1$ part descriptors $D$ (one per foreground part plus background); softmax similarity $P_{i,j,k}$ scores each pixel against each descriptor; masked positions of the filled feature map $R$ are set to $\\sum_{k=1}^{K+1} P_{i,j,k} D_k$, while visible positions keep the unmasked patch features $F^U$. Restoration with an L1 term plus a perceptual term then pulls descriptors and visible features into shared latent clusters. Three constraints stabilize this: a presence constraint (each part appears somewhere in a mini-group, background near image borders), a semantic consistency constraint with additive angular margin on descriptor-to-region cosine similarity, and a distribution constraint combining total variation and entropy to sharpen part boundaries.","core_discovery":"On the paper's own terms, the central discovery is that masked image restoration can function as a part-alignment mechanism: by using learned part descriptors to fill masked patches and then restoring the image, the model implicitly clusters the filled descriptor features together with unmasked patch features inside each part region, while low-level appearance from visible patches guides the descriptors to the correct part boundaries. The resulting similarity maps can be read directly as pixel-level part masks. The paper further argues that looser constraints, which allow parts to be absent from individual images, let the same descriptors transfer across categories and handle occlusion, and it supports this with experiments showing improved normalized mutual information and adjusted Rand index over existing methods on four benchmarks.","pith_inferences":["A testable extension follows from the masking-ratio curve: if descriptor alignment is driven by restoration pressure, then an annealing schedule that raises the masking ratio during training could improve convergence and final part quality, though the paper does not test this.","The role of the perceptual loss suggests the method's success is tied to structural supervision in pixel space; one could test whether a latent-space perceptual loss, or a stronger structural criterion, changes how tightly similarity maps follow boundaries.","The mini-group presence constraint is an implicit batch-level prior; the paper's ablation shows very large groups fragment rare regions into spurious parts, so a principled choice of group size could be connected to part frequency statistics across categories."],"forward_implications":["With no labels, MPAE can produce part-level masks that follow real object boundaries, which is a prerequisite for part-level features in downstream tasks such as fine-grained recognition and person search.","Because parts may be absent from any single image, the same training procedure transfers across categories, allowing the model to share similar parts such as wheels, heads, or hulls across different object classes.","The method is designed to handle occlusion, since it learns which parts are present rather than assuming all parts appear in every image.","The optimal masking ratio around 90 percent indicates that the alignment mechanism depends on forcing most of the reconstruction through part descriptors rather than through visible context alone."],"supporting_citations":[{"why":"Supplies the random-masking and reconstruction baseline that MPAE turns into a part-alignment mechanism.","marker":"[19]"},{"why":"Establishes the descriptor-extraction design and the area loss that MPAE replaces with looser presence constraints.","marker":"[43]"},{"why":"Provides the frozen-backbone weakly supervised baseline and the presence loss that MPAE relaxes for the unsupervised setting.","marker":"[2]"},{"why":"Gives the concentration loss that MPAE replaces with a total-variation-based distribution constraint.","marker":"[22]"},{"why":"Shows direct clustering of self-supervised features for part discovery, the baseline MPAE improves upon.","marker":"[1]"},{"why":"Provides the frozen self-supervised Vision Transformer backbone whose dense features are matched against part descriptors.","marker":"[28]"},{"why":"Supplies the perceptual or structural penalty in the restoration loss; removing it degrades part alignment sharply.","marker":"[36]"},{"why":"Defines the multi-category PartImageNet benchmarks used for the main quantitative comparisons.","marker":"[16]"}],"fun_headline_variants":["Masked restoration reveals object parts without labels","No labels, just masks: part discovery by restoration","Fill missing patches to align part shapes unsupervised","Restore masked regions to discover object parts automatically"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that filling masked patches with part descriptors and restoring the image will, by itself, pull descriptors and unmasked patch features into shared latent clusters whose similarity maps track part shapes; the paper's ablation shows this alignment is fragile, since removing the perceptual term drops normalized mutual information (NMI) from 55.10 to 19.65 on the multi-category segmentation benchmark.","fun_headline_variants_meta":{"raw":{"variants":["Masked restoration reveals object parts without labels","No labels, just masks: part discovery by restoration","Fill missing patches to align part shapes unsupervised","Restore masked regions to discover object parts automatically"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001487,"raw_usage":{"total_tokens":5959,"prompt_tokens":921,"completion_tokens":5038,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":537,"completion_tokens_details":{"reasoning_tokens":4980}},"tokens_in":537,"tokens_out":5038,"duration_ms":44993,"temperature":1.0,"reasoning_tokens":4980,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T16:57:29.204402+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train MPAE with a decoder strong enough to reconstruct masked patches almost perfectly from visible context alone, for example by using a much deeper decoder or a very low masking ratio, and then measure whether the similarity maps still match annotated part boundaries. If reconstruction stays accurate while the similarity maps no longer track part shapes, the claimed implicit alignment through restoration is not the driving mechanism.","supporting_citations":[{"cited_title":"Masked autoencoders are scalable vision learners","cited_arxiv_id":null,"evidence_quote":"Supplies the random-masking and reconstruction baseline that MPAE turns into a part-alignment mechanism."},{"cited_title":"Unsupervised part dis- covery via dual representation alignment","cited_arxiv_id":null,"evidence_quote":"Establishes the descriptor-extraction design and the area loss that MPAE replaces with looser presence constraints."},{"cited_title":"Pdiscoformer: Relaxing part discovery constraints with vision transformers","cited_arxiv_id":null,"evidence_quote":"Provides the frozen-backbone weakly supervised baseline and the presence loss that MPAE relaxes for the unsupervised setting."},{"cited_title":"Scops: Self-supervised co-part segmentation","cited_arxiv_id":null,"evidence_quote":"Gives the concentration loss that MPAE replaces with a total-variation-based distribution constraint."},{"cited_title":"Deep vit features as dense visual descriptors","cited_arxiv_id":null,"evidence_quote":"Shows direct clustering of self-supervised features for part discovery, the baseline MPAE improves upon."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the frozen self-supervised Vision Transformer backbone whose dense features are matched against part descriptors."},{"cited_title":"Partimagenet: A large, high- quality dataset of parts","cited_arxiv_id":null,"evidence_quote":"Defines the multi-category PartImageNet benchmarks used for the main quantitative comparisons."}],"review_version":1}