Pith. sign in

REVIEW 2 major objections 6 minor 1 cited by

Visual Pre-Training on Unlabeled Images using Reinforcement Learning

T0 review · 2 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper argues that pre-training visual representations on unlabeled images can be strengthened by treating image augmentations as actions in a reinforcement learning problem and learning a value function over them.

desk verdict A solid, substantial RL reformulation of self-supervised image pretraining with real empirical gains, but the central claim is only partially isolated from reward transfer and the method is not as standalone as the abstract suggests. read the letter →

arxiv 2506.11967 v1 pith:MQPUQ5FB submitted 2025-06-13 cs.LG cs.CV

Dibya Ghosh , Sergey Levine This is my paper
classification cs.LGcs.CV
keywords self-supervisedlearningreinforcementtemporaldifferencevaluefunctionvisualpre-trainingcontrastiveimageaugmentationannotationbootstrapping
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that pre-training visual representations on unlabeled images can be strengthened by treating image augmentations as actions in a reinforcement learning problem and learning a value function over them. The central claim is that bootstrapping value estimates across crops—predicting which semantic annotations are likely to appear after a transformation—transfers better to downstream tasks than directly optimizing crop-invariance or the base contrastive loss on its own. This matters for web-crawl, scene, and video data, where the usual invariance-to-crops inductive bias fits poorly and methods like DINO and SimCLR degrade. The authors report consistent gains on top of CLIP, SimCLR, and DINO, with the largest gains when a curated or weakly labeled reward is learned on one dataset and bootstrapped on another.

What carries the argument

The load-bearing mechanism is the Bellman bootstrapping target in Equation 5, which combines an immediate reward estimate from an EMA model at the next crop with the discounted maximum value estimate from that crop. The value function $Q_{\mathrm{AB}}(x,a,\ell)$ takes a contrastive form—a dot product between an image-action embedding (a vision encoder plus a lightweight decoder that consumes bounding-box action tokens) and an annotation embedding—and is trained with binary cross-entropy against the target. A lagging EMA copy of the network supplies the targets, echoing standard deep Q-learning and self-distillation practice. The discount factor $\gamma$ (set to 0.5) interpolates between matching the base reward distribution and propagating value estimates to more distant crops.

What would settle it

Train the decoupled AB setup with the reward head frozen at random initialization while still running the value-bootstrapping objective on unlabeled target images; if downstream probe accuracy still matches or exceeds the base loss, the value objective alone is creating semantics, which would contradict the paper's grounding claim. Alternatively, shuffle annotation-to-image assignment in the reward batch and check whether gains persist.

Watch

Extended reading notes

Core claim

The paper's central discovery is that a temporal-difference value-learning objective over an image-transformation Markov decision process produces better visual features than the base reward loss alone. In annotation bootstrapping, the state is an image view, an action is a relative bounding-box transformation to another crop, and the reward is the likelihood of a semantic annotation given the view, estimated by a contrastive base model such as CLIP, SimCLR, or DINO. The model is trained so that the value at one crop matches a Bellman target computed from the next crop: $(1-\gamma)$ times the reward at the next view plus $\gamma$ times the discounted best value from that next view. The authors claim this yields equivariant, temporally extended predictions rather than invariance, and that it improves representation quality on ImageNet, CC12M, COCO, and EpicKitchens, especially when rewards come from curated data and are bootstrapped on unlabeled target data.

Load-bearing premise

The approach assumes the reward score $p(\ell|x)$ learned by the base contrastive objective is an accurate enough estimate of genuine semantic annotation likelihood that bootstrapping can propagate it across crops; when that reward is weak, the paper's own COCO and EpicKitchens results show bootstrapping cannot create semantics from pixels alone.

Editorial extensions

If this is right

  • Unlabeled data can be steered by the choice of reward function without retuning the masking or augmentation strategy; curated or weakly labeled data can specify semantics while value learning runs on unrelated unlabeled target data.
  • Value-based pre-training extends crop-consistency methods to non-object-centric data, where invariance to random crops is a poor inductive bias and methods like DINO and SimCLR degrade.
  • Combining a weakly supervised loss with bootstrapping aligns gradients with the base loss better than adding invariant self-supervision, so ABCLIP improves downstream classification, detection, and reasoning metrics over CLIP, SLIP, and SiLC.
  • The Bellman structure gives the objective a fixed-point solution, in contrast with pure self-distillation objectives, which may reduce representation collapse and training instability.
  • Because the model learns equivariant value predictions, its outputs are inspectable: value estimates reveal what the model expects to see after zooming out or panning, providing a training-time window into the learned semantics.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The reward lever suggests a family of task-conditioned pre-training objectives: value functions could be trained for arbitrary annotation spaces, such as actions, affordances, or fine-grained captions, and then bootstrapped on unlabeled web or video data—an extension the paper only begins to explore.
  • Since the value head distills predictions from multiple views, it acts as an implicit ensembled reward estimator; a natural test would be whether the AB value head alone outperforms the base CLIP head on zero-shot image-text retrieval.
  • The same Bellman operator could propagate annotations through time rather than only through random crops, by treating temporally adjacent video frames as transitions; the paper evaluates video frames only as independent images.
  • A scaling study beyond the ViT-S backbone would clarify whether the gains come from value bootstrapping itself or from the decoupling of reward and value data; the paper's ablations already show that removing the annotation loss collapses performance.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. The paper proposes “annotation bootstrapping”, a self-supervised visual pre-training method that casts crop-consistency as a reinforcement-learning problem. A base reward loss (CLIP, SimCLR, or DINO) estimates the annotation likelihood p(l|x); the method additionally trains a Q-function over image transformations using the target (1−γ)p_ema(l|x_j) + γ max_a' Q_ema(x_j,a',l) (Eq. 5), with EMA parameters and random crops as transitions. Experiments on ImageNet, CC12M, COCO, and EpicKitchens under a standardized 800M-view budget report gains over the base losses on several probe tasks, and large gains when the reward is trained on ImageNet but bootstrapped on COCO/EpicKitchens. The paper argues that value learning yields equivariant, temporally propagated representations that suit non-object-centric data better than invariance-based crop consistency.

Significance. If the central claim is substantiated, the paper provides a clean interface for injecting curated rewards into self-supervised pre-training and a useful alternative to invariance-based objectives. Strengths include the standardized training budget across methods, multi-dataset evaluation with a unified probing protocol, three instantiations of the base reward, released code, and ablations isolating the propagation loss, target network, and annotation loss (Table 5). The main weakness is that the distinguishing ingredient—the TD bootstrap—is not isolated from reward transfer: on COCO and EpicKitchens with a local reward, the method remains below MAE on several metrics, and the largest gains use an ImageNet-trained reward. The current evidence supports a weaker claim: multi-crop soft-target distillation of a good reward improves representation quality.

major comments (2)
  1. [§3.2, Eq. (5); Table 3; Section 5] The central claim that bootstrapping “trains better representations than methods that directly combine weak supervision and standard self-supervised losses” is not yet isolated from reward transfer. In the fully coupled setting on COCO and EpicKitchens, ABSimCLR and ABDINO improve over their base losses but remain below MAE on several metrics (e.g., COCO object detection 26.4/26.1 vs MAE 29.8; EpicKitchens object detection 32.0/32.6 vs MAE 37.8); the large gains appear only in the decoupled rows where the reward is imported from ImageNet. Section 5 concedes the method “is not standalone.” To substantiate the headline assertion, please include an ablation that keeps the same reward target p_ema but removes the future/max term γ max_a' Q_ema from Eq. (5) (or an equivalent multi-crop self-distillation of the reward), reported on these same weak-reward datasets for both the coupled and decoupled settings. Without this, the results do not distinguish “the value recursion helps” from “a better/curated reward, distilled across crops, helps.”
  2. [Appendix A.1, Eq. (7); §3.3] The paper’s fixed-point and convergence claims (“well-studied convergence properties” in §3.3; “guaranteed solution” in Appendix A.1) apply to the Bellman operator with the true max, but the implemented objective replaces the max with the max over n sampled actions and does not report or ablate n. The sampled max is an upward-biased estimator of the true maximum, and the tabular contractivity argument does not directly carry over to this estimated operator. Please either report an ablation over n (and justify the approximation), or qualify the convergence statements to the estimated operator actually used in training.
minor comments (6)
  1. [Section 4.1] The text refers to “Table ??” for additional probes; this unresolved reference should be fixed (the data appear in Appendix Table 6).
  2. [Section 4.2] The text refers to “Figure 3 (right)”, “Figure 3”, and “Combined with Figure 3” for prediction accuracy and crop-overlap analyses, but the printed Figure 3 is the DINO-mixture comparison; please renumber or split the figure so each reference points to the correct panel.
  3. [Section 2] There is a typo in the first paragraph of Section 2: “ChameleonTeam, 2024). these objectives yield” should capitalize “These”.
  4. [Section 4.2] The phrase “ABCLIPthan” is missing a space.
  5. [Appendix A.1] The Bellman fixed-point line contains a duplicated or corrupted symbol (“Q*(x,a,l) = ... Q*(x,a,l) = ...”); please correct the typesetting.
  6. [Tables 3–4] The table headers render with inserted spaces (e.g., “I m ag e n e t C l s .” and “M e t h o d”); please fix the LaTeX/tabular formatting.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the TD bootstrap is self-referential by design, but the paper's claims are tested on independent downstream probes and the method's reliance on the base reward is explicitly acknowledged.

full rationale

The bootstrapping objective in Eq. (5) uses targets built from the model's own EMA outputs, but this is standard TD learning rather than a circular derivation: the Bellman fixed point in Eq. (1) is a well-defined mathematical characterization of the optimal value function given the reward and dynamics, and optimizing a parametric Q-function toward that fixed point does not presuppose the empirical conclusion that downstream representations improve. The paper's central empirical claims are evaluated with frozen-encoder probes on held-out tasks (ImageNet, Clevr, Food101, COCO detection, EpicKitchens action/object recognition), so the reported gains are not generated by construction from the training objective. The paper also explicitly concedes its limitation in Section 5: 'Annotation bootstrapping also is not standalone: rather, it accentuates a base reward loss,' and ablation Table 5 shows that removing the annotation loss drops performance by 23.9 points, making the input's role transparent rather than hidden. The self-citations to Levine's prior work in Appendix A.1 are background references for goal-conditioned MDP formulations and soft Q-learning, and they are not load-bearing: no uniqueness theorem or restrictive ansatz is imported from those works. The method is closely related to self-distillation and pseudo-labeling, but the paper distinguishes the equivariant value objective from invariance-based objectives and evaluates it against external baselines, so this is not a mere renaming of a known result. Overall, the derivation chain is not circular; the main risks are empirical confounds (e.g., reward transfer in the decoupled setting), which are correctness risks rather than circularity.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The method introduces no new physical or scientific entities; it relies on standard deep learning components (ViT, decoder, EMA) and standard RL machinery. The main free parameters are the discount factor, EMA schedule, crop count, and temperatures, with gamma being the only one explicitly swept.

free parameters (4)
  • Discount factor gamma = 0.5
    Swept in Figure 5 over [0,1]; 0.5 chosen as default, affects probe accuracy by about 1%.
  • EMA schedule tau = 0.004 decaying to 0 (self-supervised) or constant 0.004 (weakly supervised)
    Taken from DINO and SiLC respectively; not ablated in this paper.
  • Number of bootstrap crops n = 4
    Chosen for the bootstrap batch; determines the number of sampled actions used to approximate the max in the Bellman target.
  • Logit temperatures t and tAB = not reported
    Temperatures for the reward and value logits; they scale the contrastive and BCE losses but are not swept or reported.
assumptions (4)
  • domain assumption Random crop transformations define a valid MDP transition distribution P(x'|x,a).
    Section A.1: the transition dynamics are induced by the random cropping mechanism, treating relative bounding box coordinates as actions.
  • domain assumption Q-learning with function approximation and EMA targets converges to a useful representation.
    Section 3.3 invokes tabular Q-learning fixed-point guarantees, but the actual method uses deep networks, a learned reward, and sampled max actions; no convergence proof is given for this setting.
  • domain assumption The contrastive InfoNCE reward p(l|x) is a semantically meaningful annotation distribution.
    Equation (3) and Algorithm 1 use the base loss to define rewards; the paper's own COCO/EpicKitchens results show that weak rewards limit the value objective.
  • ad hoc to paper Sampling n actions approximates the max in the Bellman target sufficiently well.
    Equation (7) replaces the true max with a max over n sampled actions, following EMAQ (Ghasemipour et al., 2020); the bias of this estimator is not analyzed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Visual Pre-Training on Unlabeled Images using Reinforcement Learning." pith.science (2026). https://pith.science/paper/MQPUQ5FB

@misc{pith2026250611967,
  author       = {Pith},
  title        = {Pith review of: Visual Pre-Training on Unlabeled Images using Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MQPUQ5FB}},
  note         = {Machine review of arXiv:2506.11967}
}
read the original abstract

In reinforcement learning (RL), value-based algorithms learn to associate each observation with the states and rewards that are likely to be reached from it. We observe that many self-supervised image pre-training methods bear similarity to this formulation: learning features that associate crops of images with those of nearby views, e.g., by taking a different crop or color augmentation. In this paper, we complete this analogy and explore a method that directly casts pre-training on unlabeled image data like web crawls and video frames as an RL problem. We train a general value function in a dynamical system where an agent transforms an image by changing the view or adding image augmentations. Learning in this way resembles crop-consistency self-supervision, but through the reward function, offers a simple lever to shape feature learning using curated images or weakly labeled captions when they exist. Our experiments demonstrate improved representations when training on unlabeled images in the wild, including video data like EpicKitchens, scene data like COCO, and web-crawl data like CC12M.

Figures

Figures reproduced from arXiv: 2506.11967 by the authors.

Figure 1
Figure 1. In annotation bootstrapping, we train on unlabeled images with RL: predicting and maximizing semantic annotations associated with distant sub-crops of an image (e.g., a kite, or a boy playing). Our key idea is that we can learn this from unlabeled images, even though they provide no supervision of their own. We instead learn by bootstrapping between image crops: using model outputs on one view to pro￾vide a target f… view at source ↗
Figure 2
Figure 2. Visualization of the value function objective in our method with a base CLIP loss. The model processes a crop of an image 𝑥1 and action tokens demarcating how to transform the image to a target crop 𝑥2 (e.g. using the relative bounding box 𝐛𝐛1→2. The targets are created using the outputs of an EMA copy of the model on the view 𝑥2. where the natural distance is the binary cross-entropy loss. As with other RL methods … view at source ↗
Figure 3
Figure 3. We compare decoupled training of ABDINO on ImageNet and COCO or EpicKitchens to running DINO on a mixture for 𝑝 ∈ {0, 0.25, 0.5, 0.75, 1.0}. ABDINO outper￾forms all DINO mixtures, indicating that value bootstrap￾ping learns more useful features than crop-invariance on any combination of the two datasets. These datasets are a poor fit for the inductive bias underlying consistency methods, containing many (small) obje… view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: (left) ABCLIP improves over CLIP across a range of model scales (middle) Controlling the difficulty of the bootstrapping prediction problem, performance degrades as the overlap source and target crop grows, but slower than vanilla DINO. (right). Accuracy of annotation …
Figure 5
Figure 5. Figure 5: Effect of discount factor 𝛾 on ABCLIP; (left) Imagenet probing accuracy, (right) CLIP reward loss. We use 𝛾 = 0.5 for our main experi￾ments as a sensible default. Objective: The loss function used in annotation bootstrapping is described in Algorithm 2. There is one pr…
Figure 6
Figure 6. Figure 6: Visualizations of Annotation Bootstrapping for different base learning algorithms, ABCLIP, ABSimCLR, ABDINO. The model architectures are near-identical on the visual side: a ViT vision encoder, with a head for the base loss, and a Decoder transformer to predict the ann…
Figure 7
Figure 7. Figure 7: Model Predictions: An example of model predictions through training for vanilla CLIP, for ABCLIP, and for the value predictions of ABCLIP. For this particular example, vanilla CLIP recognizes early in training that the dog is on a ship, but loses this knowledge by the …
Figure 8
Figure 8. Figure 8: Learning Dynamics: (left) We measure the cosine similarity of ViT gradients of the self-supervised loss and the CLIP loss on held-out validation batches through training for various algorithms. While the similarity decays to zero for all methods, the value bootstrappin…
Figure 9
Figure 9. Figure 9: Effective Curriculum: The value loss forms an implicit curriculum over images, placing higher weight on scenes that are more visually complex, and down-weighting plain single-color images and signs. In the plot on the right, we plot the CDF of | 𝜕𝐿value 𝜕𝜙 | at various…

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Reinforcement Learning Meets Large Language Models: A Survey of Advancements and Applications Across the LLM Lifecycle

    cs.CL 2025-09 conditional novelty 3.0 of 10

    A survey that maps reinforcement learning methods, datasets, benchmarks, and open-source tools across the full training lifecycle of large language models, focusing on verifiable-reward reasoning.

Reference graph

Works this paper leans on

68 extracted references · 39 canonical work pages · cited by 1 Pith paper

  1. [1]

    Aloimonos, I

    J. Aloimonos, I. Weiss, and A. Bandyopadhyay. Active vision. International journal of computer vision, 1: 0 333--356, 1988

    work page 1988

  2. [2]

    Assran, Q

    M. Assran, Q. Duval, I. Misra, P. Bojanowski, P. Vincent, M. G. Rabbat, Y. LeCun, and N. Ballas. Self-supervised learning from images with a joint-embedding predictive architecture. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 15619--15629, 2023

    work page 2023

  3. [3]

    Y. Bai, X. Geng, K. Mangalam, A. Bar, A. L. Yuille, T. Darrell, J. Malik, and A. A. Efros. Sequential modeling enables scalable learning for large vision models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22861--22872, 2024

    work page 2024

  4. [4]

    R. Bajcsy. Active perception. Proceedings of the IEEE, 76 0 (8): 0 966--1005, 1988

    work page 1988

  5. [5]

    H. Bao, L. Dong, and F. Wei. Beit: Bert pre-training of image transformers. ArXiv, abs/2106.08254, 2021. URL https://api.semanticscholar.org/CorpusID:235436185

    arXiv 2021

  6. [6]

    Beaumont

    R. Beaumont. img2dataset: Easily turn large sets of image urls to an image dataset. https://github.com/rom1504/img2dataset, 2021

    work page 2021

  7. [7]

    Beyer, X

    L. Beyer, X. Zhai, and A. Kolesnikov. Big vision. https://github.com/google-research/big_vision, 2022

    work page 2022

  8. [8]

    Beyer, B

    L. Beyer, B. Wan, G. Madan, F. Pavetic, A. Steiner, A. Kolesnikov, A. S. Pinto, E. Bugliarello, X. Wang, Q. Yu, L.-C. Chen, and X. Zhai. A study of autoregressive decoders for multi-tasking in computer vision. ArXiv, abs/2303.17376, 2023. URL https://api.semanticscholar.org/CorpusID:257833784

    arXiv 2023

Show all 68 references
  1. [9]

    Blier, C

    L. Blier, C. Tallec, and Y. Ollivier. Learning successor states and goal-dependent values: A mathematical viewpoint. arXiv preprint arXiv:2101.07123, 2021

    2021 arXiv

  2. [10]

    Caron, P

    M. Caron, P. Bojanowski, J. Mairal, and A. Joulin. Unsupervised pre-training of image features on non-curated data. 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pages 2959--2968, 2019

    2019

  3. [11]

    Caron, I

    M. Caron, I. Misra, J. Mairal, P. Goyal, P. Bojanowski, and A. Joulin. Unsupervised learning of visual features by contrasting cluster assignments. ArXiv, abs/2006.09882, 2020. URL https://api.semanticscholar.org/CorpusID:219721240

    2006 arXiv

  4. [12]

    Caron, H

    M. Caron, H. Touvron, I. Misra, H. J'egou, J. Mairal, P. Bojanowski, and A. Joulin. Emerging properties in self-supervised vision transformers. 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pages 9630--9640, 2021

    2021

  5. [13]

    Chameleon: Mixed-modal early-fusion foundation models, 2024

    ChameleonTeam. Chameleon: Mixed-modal early-fusion foundation models, 2024. URL https://arxiv.org/abs/2405.09818

    2024 arXiv

  6. [14]

    Changpinyo, P

    S. Changpinyo, P. K. Sharma, N. Ding, and R. Soricut. Conceptual 12m: Pushing web-scale image-text pre-training to recognize long-tail visual concepts. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 3557--3567, 2021. URL https://api.semantics...

    2021

  7. [15]

    Chen and L

    T. Chen and L. Li. Intriguing properties of contrastive losses. CoRR, abs/2011.02803, 2020. URL https://arxiv.org/abs/2011.02803

    2011 arXiv

  8. [16]

    T. Chen, S. Kornblith, M. Norouzi, and G. E. Hinton. A simple framework for contrastive learning of visual representations. ArXiv, abs/2002.05709, 2020 a

    2002 arXiv

  9. [17]

    T. Chen, S. Kornblith, K. Swersky, M. Norouzi, and G. E. Hinton. Big self-supervised models are strong semi-supervised learners. CoRR, abs/2006.10029, 2020 b . URL https://arxiv.org/abs/2006.10029

    2006 arXiv

  10. [18]

    Chen and K

    X. Chen and K. He. Exploring simple siamese representation learning. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 15745--15753, 2020. URL https://api.semanticscholar.org/CorpusID:227118869

    2021

  11. [19]

    X. Chen, S. Xie, and K. He. An empirical study of training self-supervised vision transformers. 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pages 9620--9629, 2021

    2021

  12. [20]

    Damen, H

    D. Damen, H. Doughty, G. M. Farinella, S. Fidler, A. Furnari, E. Kazakos, D. Moltisanti, J. Munro, T. Perrett, W. Price, and M. Wray. The epic-kitchens dataset: Collection, challenges and baselines. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43: 0 4125--41...

    2020

  13. [21]

    Darkhalil, D

    A. Darkhalil, D. Shan, B. Zhu, J. Ma, A. Kar, R. Higgins, S. Fidler, D. Fouhey, and D. Damen. Epic-kitchens visor benchmark: Video segmentations and object relations. In Proceedings of the Neural Information Processing Systems (NeurIPS) Track on Datasets and Benchmarks, 2022

    2022

  14. [22]

    Dosovitskiy, L

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. ArXiv, abs/2010.11929, 2020. URL https://...

    2010 arXiv

  15. [23]

    El-Nouby, M

    A. El-Nouby, M. Klein, S. Zhai, M. A. Bautista, A. Toshev, V. Shankar, J. M. Susskind, and A. Joulin. Scalable pre-training of large autoregressive image models, 2024. URL https://arxiv.org/abs/2401.08541

    2024 arXiv

  16. [24]

    Eysenbach, T

    B. Eysenbach, T. Zhang, R. Salakhutdinov, and S. Levine. Contrastive learning as goal-conditioned reinforcement learning. ArXiv, abs/2206.07568, 2022. URL https://api.semanticscholar.org/CorpusID:249674522

    2022 arXiv

  17. [25]

    E. Fini, P. Astolfi, A. Romero-Soriano, J. Verbeek, and M. Drozdzal. Improved baselines for vision-language pre-training. Transactions on Machine Learning Research, 2023. ISSN 2835-8856. URL https://openreview.net/forum?id=a7nvXxNmdV. Featured Certification

    2023

  18. [26]

    L. Fu, L. Lian, R. Wang, B. Shi, X. Wang, A. Yala, T. Darrell, A. A. Efros, and K. Goldberg. Rethinking patch dependence for masked autoencoders. arXiv preprint arXiv:2401.14391, 2024

    2024 arXiv

  19. [27]

    S. K. S. Ghasemipour, D. Schuurmans, and S. S. Gu. Emaq: Expected-max q-learning operator for simple yet effective offline and online RL . CoRR, abs/2007.11091, 2020. URL https://arxiv.org/abs/2007.11091

    2007 arXiv

  20. [28]

    Grill, F

    J.-B. Grill, F. Strub, F. Altch'e, C. Tallec, P. H. Richemond, E. Buchatskaya, C. Doersch, B. \'A . Pires, Z. D. Guo, M. G. Azar, B. Piot, K. Kavukcuoglu, R. Munos, and M. Valko. Bootstrap your own latent: A new approach to self-supervised learning. ArXiv, abs/2006.07733, 2020

    2006 arXiv

  21. [29]

    J. Z. HaoChen and T. Ma. A theoretical study of inductive biases in contrastive learning, 2023. URL https://arxiv.org/abs/2211.14699

    2023 arXiv

  22. [30]

    K. He, H. Fan, Y. Wu, S. Xie, and R. B. Girshick. Momentum contrast for unsupervised visual representation learning. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9726--9735, 2019

    2020

  23. [31]

    K. He, X. Chen, S. Xie, Y. Li, P. Doll'ar, and R. B. Girshick. Masked autoencoders are scalable vision learners. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 15979--15988, 2021. URL https://api.semanticscholar.org/CorpusID:243985980

    2022

  24. [32]

    Iscen, G

    A. Iscen, G. Tolias, Y. Avrithis, and O. Chum. Label propagation for deep semi-supervised learning. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 5065--5074, 2019. URL https://api.semanticscholar.org/CorpusID:104291869

    2019

  25. [33]

    Jayaraman and K

    D. Jayaraman and K. Grauman. Learning to look around: Intelligently exploring unseen environments for unknown tasks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1238--1247, 2018

    2018

  26. [34]

    A. Jha, M. B. Blaschko, Y. M. Asano, and T. Tuytelaars. The common stability mechanism behind most self-supervised learning approaches, 2024. URL https://arxiv.org/abs/2402.14957

    2024 arXiv

  27. [35]

    C. Jia, Y. Yang, Y. Xia, Y.-T. Chen, Z. Parekh, H. Pham, Q. V. Le, Y.-H. Sung, Z. Li, and T. Duerig. Scaling up visual and vision-language representation learning with noisy text supervision. ArXiv, abs/2102.05918, 2021. URL https://api.semanticscholar.org/CorpusID:231879586

    2021 arXiv

  28. [36]

    L. Jing, P. Vincent, Y. LeCun, and Y. Tian. Understanding dimensional collapse in contrastive self-supervised learning. ArXiv, abs/2110.09348, 2021. URL https://api.semanticscholar.org/CorpusID:239016966

    2021 arXiv

  29. [37]

    D. D. Johnson, A. E. Hanchi, and C. J. Maddison. Contrastive learning can find an optimal basis for approximately view-invariant functions. ArXiv, abs/2210.01883, 2022. URL https://api.semanticscholar.org/CorpusID:252715969

    2022 arXiv

  30. [38]

    Karpathy and F

    A. Karpathy and F. Li. Deep visual-semantic alignments for generating image descriptions. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015, Boston, MA, USA, June 7-12, 2015 , pages 3128--3137. IEEE Computer Society, 2015. doi:10.1109/CVPR.2015.7298932. ...

    2015

  31. [39]

    Kostrikov, A

    I. Kostrikov, A. Nair, and S. Levine. Offline reinforcement learning with implicit q-learning. arXiv preprint arXiv:2110.06169, 2021

    2021 arXiv

  32. [40]

    Lee et al

    D.-H. Lee et al. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In Workshop on challenges in representation learning, ICML, volume 3, page 896. Atlanta, 2013

    2013

  33. [41]

    A. C. Li, A. A. Efros, and D. Pathak. Understanding collapse in non-contrastive siamese representation learning. ECCV, 2022

    2022

  34. [42]

    Y. Li, F. Liang, L. Zhao, Y. Cui, W. Ouyang, J. Shao, F. Yu, and J. Yan. Supervision exists everywhere: A data efficient contrastive language-image pre-training paradigm. ArXiv, abs/2110.05208, 2021. URL https://api.semanticscholar.org/CorpusID:238582773

    2021 arXiv

  35. [43]

    T.-Y. Lin, M. Maire, S. J. Belongie, J. Hays, P. Perona, D. Ramanan, P. Doll \'a r, and C. L. Zitnick. Microsoft coco: Common objects in context. In European Conference on Computer Vision, 2014. URL https://api.semanticscholar.org/CorpusID:14113767

    2014

  36. [44]

    V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. A. Riedmiller. Playing atari with deep reinforcement learning. ArXiv, abs/1312.5602, 2013. URL https://api.semanticscholar.org/CorpusID:15238391

    2013 arXiv

  37. [45]

    N. Mu, A. Kirillov, D. A. Wagner, and S. Xie. Slip: Self-supervision meets language-image pre-training. ArXiv, abs/2112.12750, 2021. URL https://api.semanticscholar.org/CorpusID:245424883

    2021 arXiv

  38. [46]

    M. F. Naeem, Y. Xian, X. Zhai, L. Hoyer, L. Van Gool, and F. Tombari. Silc: Improving vision language pretraining with self-distillation. arXiv preprint arXiv:2310.13355, 2023

    2023 arXiv

  39. [47]

    Papanikolopoulos, P

    N. Papanikolopoulos, P. K. Khosla, and T. Kanade. Vision and control techniques for robotic visual tracking. In ICRA, pages 857--864, 1991

    1991

  40. [48]

    a henb \

    D. Pathak, P. Kr \"a henb \"u hl, J. Donahue, T. Darrell, and A. A. Efros. Context encoders: Feature learning by inpainting. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2536--2544, 2016. URL https://api.semanticscholar.org/CorpusID:2202933

    2016

  41. [49]

    H. Pham, Q. Xie, Z. Dai, and Q. V. Le. Meta pseudo labels. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 11552--11563, 2020

    2021

  42. [50]

    Radford, J

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748--8763. PMLR, 2021

    2021

  43. [51]

    Rivlin and H

    E. Rivlin and H. Rotstein. Control of a camera for active vision: Foveal vision, smooth tracking and saccade. International Journal of Computer Vision, 39: 0 81--96, 2000

    2000

  44. [52]

    T. G. Rudner, V. Pong, R. McAllister, Y. Gal, and S. Levine. Outcome-driven reinforcement learning via variational inference. Advances in Neural Information Processing Systems, 34: 0 13045--13058, 2021

    2021

  45. [53]

    Russakovsky, J

    O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. S. Bernstein, A. C. Berg, and L. Fei-Fei. Imagenet large scale visual recognition challenge. International Journal of Computer Vision, 115: 0 211 -- 252, 2014. URL https://api.s...

    2014

  46. [54]

    R. S. Sutton and A. G. Barto. Reinforcement Learning: An Introduction. The MIT Press, second edition, 2018. URL http://incompleteideas.net/book/the-book-2nd.html

    2018

  47. [55]

    R. S. Sutton, J. Modayil, M. Delp, T. Degris, P. M. Pilarski, A. White, and D. Precup. Horde: a scalable real-time architecture for learning knowledge from unsupervised sensorimotor interaction. In Adaptive Agents and Multi-Agent Systems, 2011. URL https://api.semanticscholar....

    2011

  48. [56]

    Y. Tian, C. Sun, B. Poole, D. Krishnan, C. Schmid, and P. Isola. What makes for good views for contrastive learning? Advances in neural information processing systems, 33: 0 6827--6839, 2020

    2020

  49. [57]

    van den Oord, Y

    A. van den Oord, Y. Li, and O. Vinyals. Representation learning with contrastive predictive coding. ArXiv, abs/1807.03748, 2018. URL https://api.semanticscholar.org/CorpusID:49670925

    2018 arXiv

  50. [58]

    Venkataramanan, M

    S. Venkataramanan, M. N. Rizve, J. Carreira, Y. M. Asano, and Y. Avrithis. Is imagenet worth 1 video? learning strong image encoders from 1 long unlabelled video. In International Conference on Learning Representations, 2024

    2024

  51. [59]

    Watkins and P

    C. Watkins and P. Dayan. Q-learning. Machine Learning, 8 0 (3): 0 279--292, 1992. ISSN 1573-0565. doi:10.1007/BF00992698

    1992 doi

  52. [60]

    Weers, V

    F. Weers, V. Shankar, A. Katharopoulos, Y. Yang, and T. Gunter. Masked autoencoding does not help natural language supervision at scale. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 23432--23444, June 2023

    2023

  53. [61]

    Q. Xie, Z. Dai, E. H. Hovy, M.-T. Luong, and Q. V. Le. Unsupervised data augmentation for consistency training. arXiv: Learning, 2019

    2019

  54. [62]

    Z. Xie, Z. Zhang, Y. Cao, Y. Lin, J. Bao, Z. Yao, Q. Dai, and H. Hu. Simmim: a simple framework for masked image modeling. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9643--9653, 2021. URL https://api.semanticscholar.org/CorpusID:244346275

    2022

  55. [63]

    X. Yang, Z. Song, I. King, and Z. Xu. A survey on deep semi-supervised learning. IEEE Transactions on Knowledge and Data Engineering, 35 0 (9): 0 8934--8954, 2023. doi:10.1109/TKDE.2022.3220219

    2023

  56. [64]

    J. Yu, Z. Wang, V. Vasudevan, L. Yeung, M. Seyedhosseini, and Y. Wu. Coca: Contrastive captioners are image-text foundation models. Trans. Mach. Learn. Res., 2022, 2022. URL https://api.semanticscholar.org/CorpusID:248512473

    2022

  57. [65]

    X. Zhai, A. Oliver, A. Kolesnikov, and L. Beyer. S4l: Self-supervised semi-supervised learning. 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pages 1476--1485, 2019 a

    2019

  58. [66]

    X. Zhai, J. Puigcerver, A. Kolesnikov, P. Ruyssen, C. Riquelme, M. Lucic, J. Djolonga, A. S. Pinto, M. Neumann, A. Dosovitskiy, L. Beyer, O. Bachem, M. Tschannen, M. Michalski, O. Bousquet, S. Gelly, and N. Houlsby. A large-scale study of representation learning with the visua...

    2019

  59. [67]

    X. Zhai, B. Mustafa, A. Kolesnikov, and L. Beyer. Sigmoid loss for language image pre-training. 2023 IEEE/CVF International Conference on Computer Vision (ICCV), pages 11941--11952, 2023. URL https://api.semanticscholar.org/CorpusID:257767223

    2023

  60. [68]

    B. D. Ziebart, A. L. Maas, J. A. Bagnell, A. K. Dey, et al. Maximum entropy inverse reinforcement learning. In Aaai, volume 8, pages 1433--1438. Chicago, IL, USA, 2008

    2008

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.