Pith. sign in

REVIEW 4 major objections 5 minor 84 references

Dense self-supervised representations are over-dispersed because strict spatial alignment and scene-centric backgrounds weaken semantic concentration; the paper proposes explicit concentration via a noise-tolerant ranking loss and an object

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review

2026-08-04 19:06 UTC pith:Q2CTFJSQ

load-bearing objection A solid dense-SSL paper with a genuinely new ranking loss and object-aware filter; the empirical story is convincing but the paper needs multi-seed reporting and a clarification of the pseudo-label source before I'd trust the ablation. the 4 major comments →

arxiv 2509.09429 v1 pith:Q2CTFJSQ submitted 2025-09-11 cs.CV cs.LG

Semantic Concentration for Self-Supervised Dense Representations Learning

Peisong Wen , Qianqian Xu , Siran Dai , Runmin Cong , Qingming Huang This is my paper
classification cs.CV cs.LG
keywords self-supervised learningdense representationsemantic segmentationranking lossaverage precisionobject-aware filterpatch correspondenceover-dispersion
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that existing self-supervised learning for dense (patch-level) representations suffers from over-dispersion: patches from the same object or category end up scattered in the embedding space, which hurts downstream segmentation and detection. It identifies why image-level self-supervised methods avoid this problem—non-strict spatial alignment and shared patterns across intra-class instances—and explains why dense SSL cannot easily adopt those mechanisms. To fix this, the authors propose two techniques: a Continuous-Target Average Precision (CoTAP) loss that distills noisy and imbalanced patch correspondences from the target branch, and an Object-Aware Filter (OAF) that re-encodes patches in a space spanned by learnable object prototypes. On top of existing SSL models (DINO, iBOT, Leopart, Mugs), their fine-tuning framework yields consistent gains across semantic segmentation, object detection, instance segmentation, video object segmentation, and image classification.

Core claim

The authors claim that dense self-supervised representations can be made semantically concentrated by explicitly distilling cross-image patch correspondences and by filtering features through object prototypes. The paper's central mechanism is the CoTAP loss, which generalizes Average Precision ranking loss to continuous soft targets, thereby tolerating the noisy and skewed patch-pair similarities that the target branch provides. The Object-Aware Filter learns a set of object prototypes from object-centric images and uses cross-attention to represent each patch as a combination of these prototypes, reducing background interference. Experiments show that fine-tuning strong SSL models with the

What carries the argument

The Continuous-Target Average Precision (CoTAP) loss extends the AP loss—a ranking loss that up-weights pairs with high target scores—to accept continuous pseudo-labels, making it robust to noisy correspondence estimates; it is implemented via an upper bound with a Huber surrogate to remain differentiable and efficient. The Object-Aware Filter (OAF) learns M object prototypes from object-centric images and uses cross-attention with patch features as queries, producing object-centered representations that highlight shared patterns across images. A theoretical analysis based on a Lipschitz encoder and k-NN classifier formalizes the roles of non-strict spatial alignment and shared patterns, lea

Load-bearing premise

The ranking signal in the target branch's patch-pair similarities must be reliable enough that focusing on top-ranked pairs via the CoTAP adaptive weights distills genuine semantic correspondences rather than amplifying noise.

What would settle it

Run the method on a strong base model (e.g., DINO) and measure the positive rate of top-ranked patch pairs during fine-tuning: if the positive rate does not rise above the early training level shown in the paper's Fig. 4(a), then the CoTAP loss is likely learning noise. More directly, if ablations show that replacing CoTAP with binary cross-entropy on the same pseudo-labels gives equal performance, the claimed noise tolerance is not essential.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • If correct, the framework provides a recipe for turning any image-level SSL checkpoint into a strong dense representation encoder by fine-tuning with explicit semantic concentration.
  • The CoTAP loss offers a general noise-tolerant mechanism for matching patch across images without labels, potentially useful beyond segmentation, such as video tracking or correspondence learning.
  • The OAF shows that cross-attention over learned object prototypes can compensate for scene-centric background dominance, which could transfer to other scene-understanding tasks.
  • The analysis linking non-strict alignment and shared patterns to k-NN classification error sharpens the understanding of why certain augmentations matter for dense SSL.
  • The consistent gains on instance-level tasks like VOS indicate that semantic concentration does not destroy instance discrimination, contrary to a possible trade-off.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper's diagnosis of over-dispersion could generalize to other dense prediction objectives (e.g., depth estimation, keypoints), where strict spatial alignment is common and scene clutter is high; one could test whether similar concentration losses improve those tasks.
  • The dependency on object-centric images to learn clean prototypes is an acknowledged limitation; a promising extension is to learn prototypes adaptively from uncurated data, as the authors note, which would make the method scalable to larger mixtures of scenes.
  • The CoTAP loss is a specific instance of a broader idea: use ranking-based objectives for noisy continuous supervision. This could be applied to self-supervised learning in other modalities, such as audio or text, where correspondences are soft and imbalanced.
  • The theoretical error bound suggests that increasing q_T (shared pattern probability) and decreasing p_T (poor alignment probability) directly lower the classification error; a testable prediction is that augmentations designed to increase shared patterns (e.g., style transfer) would yield further gains.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper addresses over-dispersion in self-supervised dense representations by introducing explicit 'semantic concentration'. It argues that image-level SSL implicitly concentrates semantics through non-strict spatial alignment and shared patterns, while dense SSL with strict spatial alignment and scene-centric data loses this property. To restore it, the authors propose (i) CoTAP, a continuous-target Average Precision ranking loss that distills patch correspondence rankings from the EMA target branch, and (ii) an Object-Aware Filter (OAF) that maps patch features into a space spanned by learnable object prototypes. The framework is evaluated by fine-tuning DINO, iBOT, Leopart, and Mugs on ImageNet-1k plus COCO, with downstream linear/FCN segmentation, detection, instance segmentation, VOS, and image classification. The main reported gains include +3.4 mIoU from CoTAP and +0.9 mIoU from OAF in the DINO ablation (Table 7), and a ViT-B/8 COCOStuff-27 mIoU of 54.7 (Table 2). A theoretical analysis of the implicit concentration mechanism and appendices with proofs are also provided.

Significance. If the empirical claims hold, the paper makes a useful practical contribution: it provides a recipe for improving dense representations of existing SSL backbones with relatively short fine-tuning, and the consistency of gains across segmentation, detection, VOS, and classification is impressive. The paper also ships several strengths: a code link, formal propositions with proofs in the appendix, and a control experiment (Fig. 7) that addresses the extra-epoch/extra-data confound for at least the ViT-S/16 setting. The proposed CoTAP idea—extending AP loss to continuous soft targets with adaptive weighting—is interesting and reasonably motivated. The main weaknesses are evidential: key definitions and loss composition are ambiguous, pseudo-label reliability is not directly measured, multi-seed variability is absent, and the fairness control is not extended to the largest architectures.

major comments (4)
  1. [§4.2, Eqs. (4)–(5), Eq. (14)] The pseudo-label source and loss composition are unclear. Eq. (4) defines q(x)_i = SK(\hat f_o(x)_i), i.e., using the online branch, not the target branch. But Eq. (5) then treats [q(v2)]_sg as the target for p(v1), and Sec. 4.3 describes St from the target branch as supervision. If q is really online, the dense alignment is not target distillation and the noise-tolerance argument changes substantially. Please define p and q consistently (likely q should be SK(\hat f_t(v2))) and update Fig. 3/Eq. (5). Relatedly, Eq. (14) omits λ3·Lproto despite Sec. 5.1 setting λ3=1 and Fig. 3 showing Lproto; specify exactly how the prototype entropy loss enters the total objective.
  2. [§5.2, Table 2; §5.5, Fig. 7] The headline ViT-B/8 result (DINO+Ours 54.7 vs DINO 45.9) is confounded by additional COCO data and 100k extra iterations. Fig. 7 controls for these factors, but the reported control appears to use the default ViT-S/16 setting; no extended-training/COCO-only baselines are shown for ViT-B/8 or ViT-B/16 in Table 2. Without such controls for the architectures used in the main table, the fair comparison claim is not established for the strongest reported numbers. Add the Fig. 7-style controls for ViT-B/16 and ViT-B/8, or qualify those rows as additional-data comparisons.
  3. [§5.4, Table 7; §5.3, Tables 3–6] All main experiments appear to be single runs with no seeds or error bars. This is especially concerning for the smaller deltas that carry the component analysis: OAF adds +0.9 mIoU (row 6→7 in Table 7), and detection/instance gains are +0.4 to +0.6 AP in Table 5. These margins are well within typical run-to-run variation for fine-tuning ViT backbones. Please report mean±std over at least three seeds for the ablations and the principal comparisons, or otherwise justify that the differences are statistically meaningful.
  4. [§4.3, Fig. 4; Table 7] The load-bearing assumption of CoTAP is that high target-branch similarity is a reliable ranking signal for semantic correspondence, so that the adaptive weights γ̃ focus on informative pairs. Fig. 4(a) shows that in the initial DINO model the positive rate at high cosine similarity is low on both COCO and VOC, which means the teacher is a noisy ranker at the beginning of fine-tuning. Fig. 4(b) is measured after training and cannot establish that the +3.4 mIoU gain comes from semantic concentration rather than from a generic ranking regularizer. Please add a direct precision@top-k measurement of target correspondences during early fine-tuning (e.g., at 0, 10k, 20k, 50k iterations), and include an ablation that replaces CoTAP with a standard margin ranking loss on the same target similarities to verify that the AP-style adaptive weighting is the causal component.
minor comments (5)
  1. [Eq. (2)] The notation 'St(u,v), S t(u,v)' is duplicated in the text; the second should clearly be So(u,v). Please also clarify whether the maps are HW×HW matrices for every pair (u,v), as used in Eq. (10).
  2. [Eq. (12)] There is a stray closing bracket after 'Σ_{q_j<q_i} ℓ(p_i−p_j)]', and 'differential' should be 'differentiable'. The Huber surrogate definition also has an unusual conditional structure; a short explanation of τ2 as a margin in Eq. (11) would help.
  3. [§3.3, Assumption 2 and Remark 1] Assumption 2 effectively postulates the shared-pattern structure that the paper then 'reveals'. Since Sec. 3.2 states the theory is not essential for the method, this is acceptable, but the wording 'analytically reveal' in the introduction overstates the contribution; consider softening.
  4. [References] Reference [24] is a duplicate of [23] (BYOL), and [80] and [81] are the same paper (Zhuo et al.). Please deduplicate.
  5. [Appendix B.1, Table 8] The symbol '■' is used in the table but only explained in the caption as 'means applying the same random cropping'. Please add a legend or use 'SameCrop' as a column label for clarity.

Circularity Check

0 steps flagged

No significant circularity: CoTAP is a self-distillation regularizer evaluated on external benchmarks; the theoretical assumptions are input-space conditions, not restatements of the learned result.

full rationale

The core claim is that the CoTAP ranking loss and the object-aware filter improve dense SSL representations. The CoTAP loss (Eq. 12) trains the online branch to rank patch-pair similarities consistently with the EMA target branch's similarities; this is a standard self-distillation loop (DINO/Leopart style), not a claim that the target provides ground truth. Downstream evaluation is performed on external labeled benchmarks (COCOStuff, PASCAL VOC, ADE20k, Cityscapes, DAVIS), so no fitted parameter is renamed as a prediction. The AP-loss formulation is cited from the authors' prior work [63,64], but the equation is explicitly stated and its adaptive-weighting/noise-tolerance property is visible from the formula itself; no uniqueness theorem or unverified premise is imported. The theoretical Assumption 2 is an input-space condition about shared patterns, and Theorem 1 derives a conditional error bound from it; the conclusion is not identical to the assumption, so this is a standard conditional analysis, not circularity. The post-hoc diagnostics (Fig. 4) and ablations (Table 7) are evaluations, not training inputs. Two textual/consistency issues exist: Eq. (4) defines q(x)_i with SK(ˆfo(x)_i) rather than the target features, and Lproto is listed among hyperparameters but omitted from the total loss Eq. (14). These are correctness/consistency concerns, not circularity, because the central derivation does not reduce to its own inputs by construction.

Axiom & Free-Parameter Ledger

6 free parameters · 6 axioms · 1 invented entities

The central claim rests on a handful of hand-set hyperparameters and a few domain assumptions about pseudo-label reliability, shared patterns, and object-centric prototype transfer. The theory adds Assumptions 1 and 2, which are stated but not independent of the paper's own hypothesis. The invented entity (object prototypes) is internal and not externally validated.

free parameters (6)
  • Loss weights λ1, λ¯1, λ2, λ¯2 (and λ3) = 1 (all set to 1; λ3 absent from Eq. 14)
    Hand-set via validation; they control the balance between alignment, image-level concentration, and dense concentration.
  • CoTAP threshold τ1 = -0.2
    Hand-tuned; appears in γ˜i = [q_i − τ1]_+ and affects which target scores receive weight.
  • CoTAP margin τ2 = 0.5
    Hand-tuned; margin in the one-side Huber surrogate for the ranking indicator.
  • Prototype temperature τ3 = 0.1
    Hand-tuned; controls softmax sharpness when matching prototypes to object-centric features.
  • Number of object prototypes M = 64
    Architecture choice; the prototype tensor is 64×384×3×3 for ViT-S and 64×768×3×3 for ViT-B.
  • k-NN size for image-level concentration
    The paper says 'top-k similar images' but never reports the value of k used in the experiments.
axioms (6)
  • domain assumption Lipschitz continuous encoder (Assumption 1, Sec. 3.3)
    Used to connect input-space similarity to representation similarity in the theoretical bound. Stated explicitly, but not verified on the learned encoders.
  • ad hoc to paper Existence of shared-pattern constants d_T, q_T (Assumption 2, Sec. 3.3)
    This assumes that intra-class input-space distances are small for most samples, which is exactly the 'shared pattern' hypothesis. If false, the theory's bound degenerates (Remark 7).
  • domain assumption Target-branch cosine similarity ranks patch pairs by semantic relatedness (Sec. 4.3)
    The CoTAP loss treats St as supervision. Fig. 4(a) suggests the positive rate at high similarity is low for the pretrained DINO, so this is a real and load-bearing assumption.
  • domain assumption Object-centric prototypes encode foreground objects and transfer to scene-centric data (Sec. 4.4)
    Prototypes are learned from ImageNet object-centric images and then applied to COCO scenes. The authors flag the object-centric data requirement as a limitation in the conclusion.
  • domain assumption k-NN image neighbors are semantic positives (Sec. 4.3, L¯sc)
    For object-centric ImageNet, nearest neighbors are assumed to share the same category; this is approximate but standard in SSL literature.
  • standard math Standard optimization results: KKT conditions, spectral decomposition, Young's inequality (Appendix A)
    Used in the proof of Lemma 2 and Theorem 1. The authors do not formally verify these proofs in a proof assistant.
invented entities (1)
  • Learnable object prototypes U (M×K_s×K_s×D) no independent evidence
    purpose: Represent foreground objects and serve as keys in cross-attention so that patches are encoded in an object-based space, highlighting shared patterns.
    The prototypes are internal parameters trained with an entropy objective. Their quality is only measured through downstream task performance; there is no external falsifiable handle.

reviewed 2026-08-04 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Semantic Concentration for Self-Supervised Dense Representations Learning." pith.science (2026). https://pith.science/paper/Q2CTFJSQ

@misc{pith2026250909429,
  author       = {Pith},
  title        = {Pith review of: Semantic Concentration for Self-Supervised Dense Representations Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Q2CTFJSQ}},
  note         = {Machine review of arXiv:2509.09429}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Recent advances in image-level self-supervised learning (SSL) have made significant progress, yet learning dense representations for patches remains challenging. Mainstream methods encounter an over-dispersion phenomenon that patches from the same instance/category scatter, harming downstream performance on dense tasks. This work reveals that image-level SSL avoids over-dispersion by involving implicit semantic concentration. Specifically, the non-strict spatial alignment ensures intra-instance consistency, while shared patterns, i.e., similar parts of within-class instances in the input space, ensure inter-image consistency. Unfortunately, these approaches are infeasible for dense SSL due to their spatial sensitivity and complicated scene-centric data. These observations motivate us to explore explicit semantic concentration for dense SSL. First, to break the strict spatial alignment, we propose to distill the patch correspondences. Facing noisy and imbalanced pseudo labels, we propose a noise-tolerant ranking loss. The core idea is extending the Average Precision (AP) loss to continuous targets, such that its decision-agnostic and adaptive focusing properties prevent the student model from being misled. Second, to discriminate the shared patterns from complicated scenes, we propose the object-aware filter to map the output space to an object-based space. Specifically, patches are represented by learnable prototypes of objects via cross-attention. Last but not least, empirical studies across various tasks soundly support the effectiveness of our method. Code is available in https://github.com/KID-7391/CoTAP.

Figures

Figures reproduced from arXiv: 2509.09429 by Peisong Wen, Qianqian Xu, Qingming Huang, Runmin Cong, Siran Dai.

Figure 1
Figure 1. Figure 1: Strict spatial alignment applied in dense SSL leads to [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Implicit semantic concentration of image-level SSL. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: An overview of the proposed framework. The model consists of the target branch and the online branch. The [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 5
Figure 5. Figure 5: An illustration of the CoTAP loss. The AP losses under [PITH_FULL_IMAGE:figures/full_fig_p009_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: An illustration of the object-aware filter module. [PITH_FULL_IMAGE:figures/full_fig_p010_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Relative improvement of additional training epochs [PITH_FULL_IMAGE:figures/full_fig_p013_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Self-attention maps (row 2-4) and cross-attention maps (row 5-7) of the OAF module. Best viewed in colors. a more general concept like "foreground objects", object￾prototype attention is effective in capturing the main body of objects, especially for small objects. See Appendix C for more visualizations. Segmentation results. To intuitively demonstrate the improvements, we provide some visualization result… view at source ↗
Figure 9
Figure 9. Figure 9: The models are implemented with ViT-S/16 and an [PITH_FULL_IMAGE:figures/full_fig_p014_9.png] view at source ↗
Figure 9
Figure 9. Figure 9: Visualizations of the segmentation results for FCN transfer learning. Best viewed in colors. [PITH_FULL_IMAGE:figures/full_fig_p015_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Relative improvement of additional training epochs and the training dataset over different baselines. [PITH_FULL_IMAGE:figures/full_fig_p028_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Cross-attention maps of the OAF module. Each column corresponds to one of the heads of an object prototype. The [PITH_FULL_IMAGE:figures/full_fig_p029_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Clustering of patch representations in the PASCAL VOC dataset. The dimension of representations is reduced to 2 [PITH_FULL_IMAGE:figures/full_fig_p030_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Visualization of sparse correspondences between images from the same category. [PITH_FULL_IMAGE:figures/full_fig_p031_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Visualization of sparse correspondences between different views. [PITH_FULL_IMAGE:figures/full_fig_p031_14.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

84 extracted references · 7 linked inside Pith

  1. [1]

    S. Amir, Y. Gandelsman, S. Bagon, and T. Dekel. Deep vit features as dense visual descriptors.European Conference on Computer Vision Workshop, 2021

    2021

  2. [2]

    Asano, C

    Y. Asano, C. Rupprecht, and A. Vedaldi. Self-labelling via simul- taneous clustering and representation learning. InInternational Conference on Learning Representations, 2019

    2019

  3. [3]

    Assran, R

    M. Assran, R. Balestriero, Q. Duval, F. Bordes, I. Misra, P . Bo- janowski, P . Vincent, M. Rabbat, and N. Ballas. The hidden uniform cluster prior in self-supervised learning.arXiv preprint arXiv:2210.07277, 2022

    Pith/arXiv arXiv 2022

  4. [4]

    Balestriero and Y

    R. Balestriero and Y. LeCun. Contrastive and non-contrastive self- supervised learning recover global and local spectral embedding methods. InAdvances in Neural Information Processing Systems, 2022

    2022

  5. [5]

    Caesar, J

    H. Caesar, J. Uijlings, and V . Ferrari. Coco-stuff: Thing and stuff classes in context. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1209–1218, 2018. JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 15 Fig. 9: Visualizations of the segmentation results for FCN transfer learning. Best viewed in colors

    2018

  6. [6]

    Cai and N

    Z. Cai and N. Vasconcelos. Cascade r-cnn: High quality object detection and instance segmentation.IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(5):1483–1498, 2019

    2019

  7. [7]

    Caron, P

    M. Caron, P . Bojanowski, J. Mairal, and A. Joulin. Unsupervised pre-training of image features on non-curated data. InInternational Conference on Computer Vision, pages 2959–2968, 2019

    2019

  8. [8]

    Caron, I

    M. Caron, I. Misra, J. Mairal, P . Goyal, P . Bojanowski, and A. Joulin. Unsupervised learning of visual features by contrasting cluster assignments.Advances in Neural Information Processing Systems, 33:9912–9924, 2020

    2020

  9. [9]

    Caron, H

    M. Caron, H. Touvron, I. Misra, H. Jégou, J. Mairal, P . Bojanowski, and A. Joulin. Emerging properties in self-supervised vision transformers. InInternational Conference on Computer Vision, pages 9650–9660, 2021

    2021

  10. [10]

    C.-F. R. Chen, Q. Fan, and R. Panda. Crossvit: Cross-attention multi-scale vision transformer for image classification. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 357–366, 2021

    2021

  11. [11]

    T. Chen, S. Kornblith, M. Norouzi, and G. Hinton. A simple framework for contrastive learning of visual representations. In International Conference on Machine Learning, pages 1597–1607, 2020

    2020

  12. [12]

    X. Chen, H. Fan, R. Girshick, and K. He. Improved baselines with momentum contrastive learning.arXiv preprint arXiv:2003.04297, 2020

    Pith/arXiv arXiv 2003

  13. [13]

    Chen and K

    X. Chen and K. He. Exploring simple siamese representation learning. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15750–15758, 2021

    2021

  14. [14]

    X. Chen, S. Xie, and K. He. An empirical study of training self-supervised vision transformers. InInternational Conference on Computer Vision, pages 9640–9649, 2021

    2021

  15. [15]

    J. H. Cho, U. Mall, K. Bala, and B. Hariharan. Picie: Unsupervised semantic segmentation using invariance and equivariance in clustering. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16794–16804, 2021

    2021

  16. [16]

    Cordts, M

    M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Be- nenson, U. Franke, S. Roth, and B. Schiele. The cityscapes dataset for semantic urban scene understanding. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3213–3223, 2016

    2016

  17. [17]

    M. Cuturi. Sinkhorn distances: Lightspeed computation of optimal transport.Advances in Neural Information Processing Systems, 26, 2013

    2013

  18. [18]

    S. Dai, Q. Xu, P . Wen, Y. Liu, and Q. Huang. Exploring non- contrastive self-supervised representation learning for image-based profiling.arXiv preprint arXiv:2506.14265, 2025

    arXiv 2025

  19. [19]

    Dosovitskiy, L

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. InInternational Conference on Learning Representations, 2021

    2021

  20. [20]

    Everingham, L

    M. Everingham, L. Van Gool, C. K. Williams, J. Winn, and A. Zisserman. The pascal visual object classes (voc) challenge. International Journal of Computer Vision, 88:303–338, 2010

    2010

  21. [21]

    Garrido, Y

    Q. Garrido, Y. Chen, A. Bardes, L. Najman, and Y. LeCun. On the duality between contrastive and non-contrastive self-supervised learning. InInternational Conference on Learning Representations, 2023

    2023

  22. [22]

    Gidaris, P

    S. Gidaris, P . Singh, and N. Komodakis. Unsupervised repre- sentation learning by predicting image rotations. InInternational Conference on Learning Representations, 2018

    2018

  23. [24]

    Grill, F

    J.-B. Grill, F. Strub, F. Altché, C. Tallec, P . Richemond, E. Buchatskaya, C. Doersch, B. Avila Pires, Z. Guo, M. Ghesh- laghi Azar, et al. Bootstrap your own latent-a new approach to self-supervised learning.Advances in Neural Information Processing Systems, 33:21271–21284, 2020

    2020

  24. [25]

    Hamilton, Z

    M. Hamilton, Z. Zhang, B. Hariharan, N. Snavely, and W. T. Freeman. Unsupervised semantic segmentation by distilling feature correspondences. InInternational Conference on Learning Representations, 2022

    2022

  25. [26]

    K. He, X. Chen, S. Xie, Y. Li, P . Dollár, and R. Girshick. Masked autoencoders are scalable vision learners. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16000–16009, 2022

    2022

  26. [27]

    K. He, H. Fan, Y. Wu, S. Xie, and R. Girshick. Momentum contrast for unsupervised visual representation learning. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9729– 9738, 2020

    2020

  27. [28]

    K. He, R. Girshick, and P . Dollár. Rethinking imagenet pre-training. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4918–4927, 2019

    2019

  28. [29]

    K. He, G. Gkioxari, P . Dollár, and R. Girshick. Mask r-cnn. In International Conference on Computer Vision, pages 2961–2969, 2017

    2017

  29. [30]

    Huang, Q

    J. Huang, Q. Dong, S. Gong, and X. Zhu. Unsupervised deep learning by neighbourhood discovery. InInternational Conference on Machine Learning, pages 2849–2858. PMLR, 2019

    2019

  30. [31]

    Huang, M

    W. Huang, M. Yi, X. Zhao, and Z. Jiang. Towards the generalization of contrastive self-supervised learning. InInternational Conference on Learning Representations, 2022

    2022

  31. [32]

    S. Khan, M. Naseer, M. Hayat, S. W. Zamir, F. S. Khan, and M. Shah. Transformers in vision: A survey.ACM Computing Surveys, 54(10s):1–41, 2022

    2022

  32. [33]

    K. Li, Z. Wang, Z. Cheng, R. Yu, Y. Zhao, G. Song, L. Yuan, and J. Chen. Acseg: Adaptive conceptualization for unsupervised semantic segmentation.arXiv preprint arXiv:2210.05944, 2022. JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 16

    Pith/arXiv arXiv 2022

  33. [34]

    Y. Li, H. Mao, R. Girshick, and K. He. Exploring plain vision transformer backbones for object detection. InEuropean Conference on Computer Vision, pages 280–296. Springer, 2022

    2022

  34. [35]

    Z. Li, J. Tang, and T. Mei. Deep collaborative embedding for social image understanding.IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(9):2070–2083, 2018

    2070

  35. [36]

    Z. Li, Y. Zhu, F. Yang, W. Li, C. Zhao, Y. Chen, Z. Chen, J. Xie, L. Wu, R. Zhao, et al. Univip: A unified framework for self-supervised visual pre-training. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14627–14636, 2022

    2022

  36. [37]

    Z.-Y. Li, S. Gao, and M.-M. Cheng. Sere: Exploring feature self- relation for self-supervised transformer.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023

    2023

  37. [38]

    T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P . Perona, D. Ramanan, P . Dollár, and C. L. Zitnick. Microsoft coco: Common objects in context. InEuropean Conference on Computer Vision, pages 740–755, 2014

    2014

  38. [39]

    Y. Liu, Q. Xu, P . Wen, S. Dai, and Q. Huang. Not all pairs are equal: Hierarchical learning for average-precision-oriented video retrieval. InACM International Conference on Multimedia, pages 3828–3837, 2024

    2024

  39. [40]

    Y. Liu, Q. Xu, P . Wen, S. Dai, and Q. Huang. When the future becomes the past: Taming temporal correspondence for self- supervised video representation learning. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 24033–24044, 2025

    2025

  40. [41]

    Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, and B. Guo. Swin transformer: Hierarchical vision transformer using shifted windows. InInternational Conference on Computer Vision, pages 10012–10022, 2021

    2021

  41. [42]

    Loshchilov and F

    I. Loshchilov and F. Hutter. Decoupled weight decay regularization. InInternational Conference on Learning Representations, 2019

    2019

  42. [43]

    Melas-Kyriazi, C

    L. Melas-Kyriazi, C. Rupprecht, I. Laina, and A. Vedaldi. Deep spectral methods: A surprisingly strong baseline for unsupervised semantic segmentation and localization. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8364–8375, 2022

    2022

  43. [44]

    Noroozi and P

    M. Noroozi and P . Favaro. Unsupervised learning of visual representations by solving jigsaw puzzles. InEuropean Conference on Computer Vision, pages 69–84, 2016

    2016

  44. [45]

    P . O. O Pinheiro, A. Almahairi, R. Benmalek, F. Golemo, and A. C. Courville. Unsupervised learning of dense visual representations. Advances in Neural Information Processing Systems, 33:4489–4500, 2020

    2020

  45. [46]

    Pathak, P

    D. Pathak, P . Krahenbuhl, J. Donahue, T. Darrell, and A. A. Efros. Context encoders: Feature learning by inpainting. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2536– 2544, 2016

    2016

  46. [47]

    Pont-Tuset, F

    J. Pont-Tuset, F. Perazzi, S. Caelles, P . Arbeláez, A. Sorkine- Hornung, and L. Van Gool. The 2017 davis challenge on video object segmentation.arXiv preprint arXiv:1704.00675, 2017

    Pith/arXiv arXiv 2017

  47. [48]

    Raghavan, P

    V . Raghavan, P . Bollmann, and G. S. Jung. A critical investigation of recall and precision as measures of retrieval system performance. ACM Transactions on Information Systems, 7(3):205–229, 1989

    1989

  48. [49]

    P . H. Richemond, A. Tam, Y. Tang, F. Strub, B. Piot, and F. Hill. The edge of orthogonality: A simple view of what makes byol tick. arXiv preprint arXiv:2302.04817, 2023

    Pith/arXiv arXiv 2023

  49. [50]

    Roy and M

    O. Roy and M. Vetterli. The effective rank: A measure of effective dimensionality. InEuropean Signal Processing Conference, pages 606–610, 2007

    2007

  50. [51]

    Russakovsky, J

    O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, et al. Imagenet large scale visual recognition challenge.International Journal of Computer Vision, 115(3):211–252, 2015

    2015

  51. [52]

    Stegmüller, T

    T. Stegmüller, T. Lebailly, B. Bozorgtabar, T. Tuytelaars, and J.- P . Thiran. Croc: Cross-view online clustering for dense visual representation learning. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7000–7009, 2023

    2023

  52. [53]

    Q. Su, A. Netchaev, H. Li, and S. Ji. Flsl: Feature-level self- supervised learning.Advances in Neural Information Processing Systems, 36:6568–6581, 2023

    2023

  53. [54]

    J. Tang, X. Shu, G.-J. Qi, Z. Li, M. Wang, S. Yan, and R. Jain. Tri- clustered tensor completion for social-aware image tag refinement. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(8):1662–1674, 2016

    2016

  54. [55]

    Tarvainen and H

    A. Tarvainen and H. Valpola. Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results.Advances in Neural Information Processing Systems, 30, 2017

    2017

  55. [56]

    Y. Tian, X. Chen, and S. Ganguli. Understanding self-supervised learning dynamics without contrastive pairs. InInternational Conference on Machine Learning, pages 10268–10278, 2021

    2021

  56. [57]

    Tomasev, I

    N. Tomasev, I. Bica, B. McWilliams, L. Buesing, R. Pascanu, C. Blundell, and J. Mitrovic. Pushing the limits of self-supervised resnets: Can we outperform supervised learning without labels on imagenet? InInternational Conference on Machine Learning Pre- training Workshop, 2022

    2022

  57. [58]

    Vaswani, N

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin. Attention is all you need. Advances in Neural Information Processing Systems, 30, 2017

    2017

  58. [59]

    Vincent, H

    P . Vincent, H. Larochelle, Y. Bengio, and P .-A. Manzagol. Extracting and composing robust features with denoising autoencoders. In International Conference on Machine Learning, pages 1096–1103, 2008

    2008

  59. [60]

    A. N. Wang, C. Hoang, Y. Xiong, Y. LeCun, and M. Ren. Poodle: Pooled and dense self-supervised learning from naturalistic videos. InInternational Conference on Learning Representations, 2025

    2025

  60. [61]

    Z. Wang, Q. Xu, Z. Yang, Y. He, X. Cao, and Q. Huang. Optimizing partial area under the top-k curve: Theory and practice.IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(4):5053– 5069, 2022

    2022

  61. [62]

    X. Wei, T. Zhang, Y. Li, Y. Zhang, and F. Wu. Multi-modality cross attention network for image and sentence matching. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10941– 10950, 2020

    2020

  62. [63]

    P . Wen, Q. Xu, Z. Yang, Y. He, and Q. Huang. Exploring the algorithm-dependent generalization of auprc optimization with list stability. InAdvances in Neural Information Processing Systems, volume 35, pages 28335–28349, 2022

    2022

  63. [64]

    P . Wen, Q. Xu, Z. Yang, Y. He, and Q. Huang. Algorithm-dependent generalization of auprc optimization: Theory and algorithm.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024

    2024

  64. [65]

    X. Wen, B. Zhao, A. Zheng, X. Zhang, and X. QI. Self-supervised visual representation learning with semantic grouping. InAdvances in Neural Information Processing Systems, 2022

    2022

  65. [66]

    X. Weng, L. Huang, L. Zhao, R. Anwer, S. H. Khan, and F. Shah- baz Khan. An investigation into whitening loss for self-supervised learning.Advances in Neural Information Processing Systems, 35:29748– 29760, 2022

    2022

  66. [67]

    Z. Wu, Y. Xiong, S. X. Yu, and D. Lin. Unsupervised feature learning via non-parametric instance discrimination. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3733–3742, 2018

    2018

  67. [68]

    Z. Xie, Y. Lin, Z. Yao, Z. Zhang, Q. Dai, Y. Cao, and H. Hu. Self-supervised learning with swin transformers.arXiv preprint arXiv:2105.04553, 2021

    Pith/arXiv arXiv 2021

  68. [69]

    Z. Xie, Y. Lin, Z. Zhang, Y. Cao, S. Lin, and H. Hu. Propagate yourself: Exploring pixel-level consistency for unsupervised visual representation learning. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16684–16693, 2021

    2021

  69. [70]

    Yang and Y

    T. Yang and Y. Ying. Auc maximization in the era of big data and ai: A survey.ACM Computing Surveys (CSUR), 2022

    2022

  70. [71]

    Ye and J

    S. Ye and J. Lu. Sequence unlearning for sequential recommender systems. InAustralasian Joint Conference on Artificial Intelligence, pages 403–415. Springer, 2023

    2023

  71. [72]

    Ye and J

    S. Ye and J. Lu. Robust recommender systems with rating flip noise. ACM Transactions on Intelligent Systems and Technology, 16(1):1–19, 2024

    2024

  72. [73]

    S. Ye, J. Lu, and G. Zhang. Towards safe machine unlearning: A paradigm that mitigates performance degradation. InACM on Web Conference 2025, pages 4635–4652, 2025

    2025

  73. [74]

    Zbontar, L

    J. Zbontar, L. Jing, I. Misra, Y. LeCun, and S. Deny. Barlow twins: Self-supervised learning via redundancy reduction. InInternational Conference on Machine Learning, pages 12310–12320. PMLR, 2021

    2021

  74. [75]

    Zhang, H

    D. Zhang, H. Zhang, J. Tang, X.-S. Hua, and Q. Sun. Causal inter- vention for weakly-supervised semantic segmentation.Advances in Neural Information Processing Systems, 33:655–666, 2020

    2020

  75. [76]

    B. Zhou, H. Zhao, X. Puig, S. Fidler, A. Barriuso, and A. Torralba. Scene parsing through ade20k dataset. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 633–641, 2017

    2017

  76. [77]

    J. Zhou, C. Wei, H. Wang, W. Shen, C. Xie, A. Yuille, and T. Kong. ibot: Image bert pre-training with online tokenizer.International Conference on Learning Representations, 2022

    2022

  77. [78]

    P . Zhou, Y. Zhou, C. Si, W. Yu, T. K. Ng, and S. Yan. Mugs: A multi-granular self-supervised learning framework.arXiv preprint arXiv:2203.14415, 2022. JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 17

    Pith/arXiv arXiv 2022

  78. [79]

    Zhuang, A

    C. Zhuang, A. L. Zhai, and D. Yamins. Local aggregation for unsupervised learning of visual embeddings. InInternational Conference on Computer Vision, pages 6002–6012, 2019

    2019

  79. [81]

    Z. Zhuo, Y. Wang, J. Ma, and Y. Wang. Towards a unified theoretical understanding of non-contrastive learning via rank differential mechanism. InInternational Conference on Learning Representations, 2023

    2023

  80. [82]

    Ziegler and Y

    A. Ziegler and Y. M. Asano. Self-supervised learning of object parts for semantic segmentation. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14502–14511, 2022. Peisong Wenreceived the B.S. degree in intel- ligent science and technology from Nankai Uni- versity in 2020 and the Ph.D. degree in computer science from University of C...

    2022

Showing first 80 references.

This paper was first reviewed by deepseek-v4-flash on August 4, 2026.