Pith. sign in

REVIEW 3 cited by

Cluster and Predict Latent Patches for Improved Masked Image Modeling

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2502.08769 v3 pith:WEXOUBOE submitted 2025-02-12 cs.CV cs.AI

Cluster and Predict Latent Patches for Improved Masked Image Modeling

Timoth\'ee Darcet , Federico Baldassarre , Maxime Oquab , Julien Mairal , Piotr Bojanowski This is my paper
classification cs.CV cs.AI
keywords approachcapiimagelatentlossmaskedmodelingmodels
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved
0 comments
read the original abstract

Masked Image Modeling (MIM) offers a promising approach to self-supervised representation learning, however existing MIM models still lag behind the state-of-the-art. In this paper, we systematically analyze target representations, loss functions, and architectures, to introduce CAPI - a novel pure-MIM framework that relies on the prediction of latent clusterings. Our approach leverages a clustering-based loss, which is stable to train, and exhibits promising scaling properties. Our ViT-L backbone, CAPI, achieves 83.8% accuracy on ImageNet and 32.1% mIoU on ADE20K with simple linear probes, substantially outperforming previous MIM methods and approaching the performance of the current state-of-the-art, DINOv2. We release all our code and models.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. UniverSat: Resolution- and Modality-Agnostic Transformers for Earth Observation

    cs.CV 2026-06 unverdicted novelty 7.0

    UniverSat is a ViT-style model with a universal patch encoder enabling self-supervised training on heterogeneous multimodal Earth observation data from varying resolutions and sensors.

  2. Separating Representation from Reconstruction Enables Scalable Text Encoders

    cs.CL 2026-07 accept novelty 6.5

    Separating representation from token reconstruction via a bipartite CrossBERT architecture restores scalable frozen text embeddings and enables high-masking complementary training.

  3. Text-Conditional JEPA for Learning Semantically Rich Visual Representations

    cs.LG 2026-05 unverdicted novelty 6.0

    TC-JEPA conditions masked feature prediction on text captions via sparse cross-attention to produce more semantically rich visual representations and outperforms contrastive methods on fine-grained tasks.