Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

From Local Cues to Global Percepts: Emergent Gestalt Organization in Self-Supervised Vision Models

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper argues that Gestalt-like global perception in vision models is produced by the training objective, not the architecture, and presents evidence that masked-autoencoding models exceed human performance on a new spatial-oddity…

desk verdict DiSRT is a genuinely useful new probe for global spatial sensitivity, but the unvalidated 10-step texture-synthesis proxy leaves the headline SSL-vs-supervised claims weaker than they look. read the letter →

arxiv 2506.00718 v1 pith:NP2LDBUH submitted 2025-05-31 cs.CV cs.AI

Tianqin Li , Ziqi Wen , Leiran Song , Jun Liu , Zhi Jing , Tai Sing Lee This is my paper
classification cs.CVcs.AI
keywords Gestaltperceptionself-supervisedlearningmaskedautoencodingglobalspatialstructurefigure-groundsegregationillusorycontourstexturesynthesisactivationsparsity
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether machines can exhibit Gestalt perception—the human tendency to organize local visual cues into coherent wholes—and shows that the answer depends less on architecture and more on the training objective. It documents that vision transformers and convolutional networks trained by masked autoencoding (MAE), a self-supervised objective that reconstructs hidden image patches, develop internal representations that complete illusory contours, prefer convex figures, and separate figure from ground. To measure this, the paper introduces DiSRT, an oddity-detection benchmark in which texture synthesis destroys global spatial relationships while preserving local textures. On this benchmark, self-supervised models trained with MAE or CLIP outperform supervised baselines and sometimes exceed the human average, while classification finetuning erodes the ability and a biologically inspired Top-K activation sparsity operation restores much of it. If correct, the result identifies training objective as a primary driver of global structure sensitivity in vision models.

What carries the argument

The load-bearing machinery is DiSRT, a benchmark that generates global-relationship-distorted images by Gram-matrix texture synthesis and scores a model by whether its final-layer feature vectors single out the intact image via cosine-distance oddity detection. The paper couples this with a PCA projection of internal activations that assigns signed values to figure and ground, and with a Top-K activation sparsity operation that keeps only the most strongly activated neurons per channel to restore global sensitivity after finetuning. Together they convert a qualitative Gestalt claim into a quantitative, architecture-agnostic measurement.

What would settle it

A concrete falsifier would be to give a model only local texture statistics—for example, a readout trained on Gram matrices of local patches with no access to global layout—and show it still achieves high DiSRT accuracy; if it does, the benchmark is not isolating global spatial sensitivity. A second decisive check is to regenerate the DiSRT stimuli with the full 100-step texture-synthesis optimization and compare model rankings against the 10-step approximation the paper uses.

Watch

Extended reading notes

Core claim

The paper's central claim is that Gestalt-like perceptual organization is an emergent property of training objectives that require global context, and not a privilege of attention architectures. Its evidence is that ViT-MAE activation patterns align with closure, continuity, proximity, convexity, and figure-ground segregation, and that both ViT and ConvNeXt models trained with MAE or CLIP achieve high DiSRT scores, whereas supervised classifiers do poorly. DiSRT asks a model to pick the odd image out of three: one original and two versions whose global layout has been randomized by Gram-matrix texture synthesis while local texture statistics are preserved. On this benchmark, MAE and CLIP models exceed the human average in some configurations. The paper also states that classification finetuning degrades DiSRT performance, and that applying Top-K sparsity to activations recovers much of the lost sensitivity.

Load-bearing premise

The DiSRT score is meaningful only if VGG Gram-matrix texture synthesis destroys global spatial relationships while preserving local texture statistics, and the paper uses a 10-step approximation of that synthesis without showing the scores have converged.

Editorial extensions

If this is right

  • If training objective drives global perceptual sensitivity, DiSRT can serve as a standard diagnostic for whether a vision model is ready for tasks that require scene-level understanding rather than local texture matching.
  • MAE-style pretraining should be preferred over classification pretraining when human-like global organization is the goal, since classification finetuning measurably erodes this ability.
  • The Top-K sparsity result implies that even after supervised finetuning the information for global structure remains in the activations and can be unlocked without retraining.
  • Because ConvNeXt-MAE matches ViT-MAE, comparisons between architectures must control for training objective before attributing global sensitivity to attention.
  • The paper's intermediate-layer analyses suggest that supervised models discard global structure in later layers, so diagnosing sensitivity layer-by-layer may be more informative than only testing final features.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If DiSRT truly isolates global spatial relationships, a testable extension is that models ranking high on DiSRT should also show better generalization to geometric-perturbation benchmarks and compositional scene tasks; the paper does not report such correlations.
  • The Top-K restoration may indicate that sparse activations suppress texture-dominated nuisance signals rather than literally re-instating Gestalt grouping; an ablation that varies the sparsity ratio and checks activation geometry could separate these accounts.
  • The 10-step approximation of texture synthesis is the most direct threat to the benchmark's validity, so recomputing DiSRT with longer synthesis runs on a subset would show whether the model rankings are stable.
  • The PCA figure-ground readout trained on natural images might not transfer to synthetic illusory stimuli; a clean validation would be to compare PC1 sign assignments against human figure-ground judgments on the same Kanizsa and convexity images.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper investigates whether self-supervised vision models exhibit Gestalt-like global perceptual organization. It presents qualitative evidence from attention-head/PCA visualizations that ViT-MAE internal activations follow closure, continuity, proximity, convexity, and figure-ground segregation. It then introduces DiSRT, an oddity-detection benchmark in which distractors are generated by VGG Gram-matrix texture synthesis, and reports that MAE- and CLIP-trained models outperform supervised baselines and sometimes exceed human performance, that classification fine-tuning degrades this sensitivity, and that a Top-K activation sparsity mechanism partially restores it. The paper also reports human psychophysical data on DiSRT and per-layer DiSRT curves for several model families.

Significance. If the DiSRT benchmark is valid, the central claim is significant: training objective, rather than architecture alone, would be identified as a primary driver of global structural sensitivity in vision models, and DiSRT would serve as a useful diagnostic. The paper deserves credit for including a large human study (16,800 trials), for transparently disclosing the 10-step approximation, and for analyzing intermediate layers across many model families. However, the validity of the central claims depends on three load-bearing points that are not yet established: the convergence of the texture-synthesis proxy, the statistical reliability of the reported model scores, and the independence of the Top-K improvement from benchmark-specific tuning.

major comments (4)
  1. [Section 4] The validity of DiSRT rests on the assumption that the texture-synthesis distractors preserve local texture while destroying global spatial relationships, so that oddity detection isolates global-structure sensitivity. The paper states (Section 4) that 'we use 10-step optimization results to approximate the 100-step optimization result' and notes that 100 steps take over 55 seconds per image. No convergence or stability analysis is reported. Starting from Gaussian noise, 10 steps of Gram-matrix matching are generally far from the converged texture-synthesis solution, so the distractors may differ from the original in low-level statistics (frequency content, edge density, sharpness, color distribution) in addition to global layout. Since the model selects the odd image using cosine distance in final-layer features (Eq. 1), such low-level artifacts would provide a shortcut: a model could achieve a high DiSRT score by detecting these artifacts without encoding long-range spatial relations. This concern propagates to all headline claims (SSL superiority, finetuning erosion, Top-K recovery) and must be addressed, for example by reporting DiSRT accuracy as a function of optimization steps on a subset, quantifying low-level statistics of the distractors, and including a control model with restricted receptive field.
  2. [Section 5, Figures 7-9] The DiSRT scores for models in Figures 7-9 are reported as point values without error bars or significance tests. The claims that self-supervised models 'dramatically outperform' supervised baselines and 'sometimes even exceed human performance' require an estimate of trial-to-trial or seed-to-seed variability, especially because each distractor is generated from a random initialization. At minimum, the authors should report mean and standard error over multiple texture-synthesis seeds and over multiple image subsets, and perform a paired significance test for the key MAE/CLIP-versus-supervised comparisons. The human data have an associated sample size, but no comparable uncertainty is reported for the model scores.
  3. [Supplementary D.2] The Top-K activation sparsity result is load-bearing for the paper's third claim, but the per-block sparsity schedule is hand-picked: for ConvNeXt-v2-MAE-finetuned the schedule is {'1': 0.2, '2': 0.25, '5': 0.25, '6': 0.25, '23': 0.3, '24': 0.25}, and the main-text improvement is reported for these specific settings. If these thresholds were chosen after observing DiSRT scores, the improvement is partly circular and likely overfits to the benchmark. The authors should state whether the configuration was selected on a separate validation set, report sensitivity to the sparsity values and block choices, and show DiSRT scores for a range of simple uniform schedules.
  4. [Section 2 and Supplementary C] The qualitative figure-ground conclusions rely on a PCA basis computed on natural images being a valid readout for synthetic illusory stimuli. The validation in Supplementary C demonstrates that negative/positive PC1 projection values correlate with figure/ground in natural images, but the transfer of this axis to Kanizsa-type stimuli and cue-conflict synthetic images is assumed, not tested. The authors should validate the PCA readout on synthetic stimuli with known ground truth, or use an independent linear probe trained on natural-image labels and tested on the synthetic set. Additionally, the claims in Figures 1-3 are based on selected attention heads and hand-selected stimuli; the selection criteria should be stated, and a quantitative measure (e.g., agreement with human figure-ground annotations) would strengthen the case.
minor comments (6)
  1. [Abstract and throughout] There are numerous typos and grammatical errors, including 'acoss' (Abstract), 'Kanisza', 'illustory countour', 'seperation', 'relationshp', 'ConNeXt', 'neccessarily', and 'squeeze-excitation'. The manuscript would benefit from thorough proofreading.
  2. [Eq. (2)] Equation (2) computes a softmax over the dissimilarities and then takes the argmax; since softmax is monotonically increasing, this is equivalent to simply taking argmax_i D_i. The equation should be simplified or clarified if the exponentiation is intentional for some other purpose.
  3. [Section 4] The statement that increasing N 'will not increase the difficulty' of the task is made without supporting data. Since N=2 is used throughout, a small experiment varying N (e.g., N=2,3,4) would make the choice principled.
  4. [Figure 4] The histograms in Figure 4 are described as showing 'clear separability' between figure and ground pixels, but no quantitative separation measure is reported. An AUROC or similar metric computed from the PC1 projections would substantiate the claim.
  5. [Supplementary D.2] The text says 'To reproduce the Figure 2 in the main manuscript' but the Top-K results appear in Figure 9 of the main text; the figure reference is incorrect.
  6. [Section 5] The suggestion that the squeeze-excitation module in MobileNetV3 'may introduce global structural sensitivities' is speculative and not tested by a controlled comparison (e.g., with and without the SE module in the same architecture). It should be presented as a hypothesis, or tested.

Circularity Check

1 steps flagged · score 4.0 of 10

Central DiSRT comparison is self-contained, but the Top-K restoration result is in-sample fitted to the benchmark it is used to support.

  1. fitted input called prediction [Section 5, 'Top-K Sparsity to the Rescue'; Supplementary D.2]
    "Additionally, we also observe a significant increase in the model without pre-training (ConvNeXt-V1), from 62.1 DiSRT score to 88.1 DiSRT score when applied Top-K operation to activation. We refer to the supplementary for the implementation of Top-K and the parameter setup used to bring up the performance."

    The reported Top-K gains are obtained with a per-block sparsity schedule (Supplementary D.2) that the paper explicitly identifies as 'the parameter setup used to bring up the performance.' Selecting the schedule on the DiSRT score makes the subsequent DiSRT improvement an in-sample optimized outcome rather than an independent test of the claim that Top-K activation sparsity restores global sensitivity. Absent a held-out configuration or stability check, the 'restoration' result is fitted to the metric it is used to support.

full rationale

The main DiSRT comparison (self-supervised vs. supervised models) is not circular: the distorted stimuli are generated by VGG Gram-matrix texture synthesis, which is independent of the evaluated models' training objectives, and the oddity-detection score is defined on final-layer feature distances. The figure-ground PCA analyses are also not definitionally circular, because the PCA basis is calibrated on natural images with segmentation labels and then applied to synthetic illusions; this is a probe rather than a derivation from the conclusion. The one genuine circular step is the Top-K demonstration: the per-block sparsity schedule in Supplementary D.2 is described in the main text as 'the parameter setup used to bring up the performance,' indicating that the schedule was chosen to raise the DiSRT score. The reported improvements (69.8 to 94.6 and 62.1 to 88.1) are therefore in-sample optimization results, not out-of-sample predictions that Top-K restores global sensitivity. This is a fitted-input-called-prediction pattern, but it affects a secondary claim; the central benchmark comparison retains independent content. The 10-step texture-synthesis proxy is an internal-validity concern, not a circularity, because the proxy is independent of the evaluated models. Self-citations [16] and [17] motivate Top-K but are not load-bearing, as the mechanism is evaluated directly in this paper rather than assumed from those citations.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

No new physical entities are introduced. The central claims depend on unproven assumptions about the DiSRT stimuli and the PCA probe, plus one fitted TopK schedule.

free parameters (1)
  • TopK sparsity thresholds per ConvNeXt block = {'1':0.2,'2':0.25,'5':0.25,'6':0.25,'23':0.3,'24':0.25} for fcmae_ft_in22k_in1k; uniform 0.2 for ConvNeXt-V1
    These per-layer sparsity values are manually chosen and the paper says the configuration is 'the parameter setup used to bring up the performance,' so the reported TopK recovery is fit to the DiSRT benchmark rather than being a parameter-free prediction.
assumptions (4)
  • domain assumption Texture synthesis via Gram matrix matching preserves local textures and destroys global spatial relationships.
    Central premise of DiSRT; asserted in Section 4 and Figure 6 but not demonstrated with control experiments.
  • ad hoc to paper A 10-step optimization approximates the 100-step optimization result well enough for DiSRT.
    Stated in Section 4: 'Due to the high time cost... we use 10-step optimization results to approximate the 100-step optimization result'; no stability check is reported.
  • ad hoc to paper A PCA basis computed on natural image activations remains a valid figure-ground axis for synthetic illusory stimuli.
    Supplementary C calibrates positive/negative PCA values on natural images, then applies the same sign convention to Kanizsa and margin stimuli in Figure 3.
  • domain assumption Modeling global dependencies is necessary for Gestalt-like organization.
    The paper states this as a hypothesis in Section 3 and uses it to motivate DiSRT; it is not proven, and DiSRT measures only a proxy.

how reviews work

0 comments
Cite this review

Pith. "Pith review of From Local Cues to Global Percepts: Emergent Gestalt Organization in Self-Supervised Vision Models." pith.science (2026). https://pith.science/paper/NP2LDBUH

@misc{pith2026250600718,
  author       = {Pith},
  title        = {Pith review of: From Local Cues to Global Percepts: Emergent Gestalt Organization in Self-Supervised Vision Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NP2LDBUH}},
  note         = {Machine review of arXiv:2506.00718}
}
read the original abstract

Human vision organizes local cues into coherent global forms using Gestalt principles like closure, proximity, and figure-ground assignment -- functions reliant on global spatial structure. We investigate whether modern vision models show similar behaviors, and under what training conditions these emerge. We find that Vision Transformers (ViTs) trained with Masked Autoencoding (MAE) exhibit activation patterns consistent with Gestalt laws, including illusory contour completion, convexity preference, and dynamic figure-ground segregation. To probe the computational basis, we hypothesize that modeling global dependencies is necessary for Gestalt-like organization. We introduce the Distorted Spatial Relationship Testbench (DiSRT), which evaluates sensitivity to global spatial perturbations while preserving local textures. Using DiSRT, we show that self-supervised models (e.g., MAE, CLIP) outperform supervised baselines and sometimes even exceed human performance. ConvNeXt models trained with MAE also exhibit Gestalt-compatible representations, suggesting such sensitivity can arise without attention architectures. However, classification finetuning degrades this ability. Inspired by biological vision, we show that a Top-K activation sparsity mechanism can restore global sensitivity. Our findings identify training conditions that promote or suppress Gestalt-like perception and establish DiSRT as a diagnostic for global structure sensitivity across models.

Figures

Figures reproduced from arXiv: 2506.00718 by the authors.

Figure 1
Figure 1. Internal activation of ViT Mask Autoencoder on several illustory countour images. We [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Other Gestalt Perception Rules learned in ViT-MAE: Law of Continuity and Proximity [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Law of Convexity. Convex and Concave preference for figure ground identification. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (17 more)
Figure 4
Figure 4. Figure 4: We compute the first principal component (PC1) over all activation vectors collected from [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Distorted Spatial Relationship Testbench (DiSRT) [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Mechanism of computing global relationship distorted images. We implement approach [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: DiSRT score on Supervised Learning Models. [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 8
Figure 8. Figure 8: DiSRT score on Self-Supervised Learning Models: Various self-supervise learned ViTs. [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 9
Figure 9. Figure 9: Curse of Classification Finetuning and Top-K Activation Sparsity. Finetuning with [PITH_FULL_IMAGE:figures/full_fig_p009_9.png]
Figure 10
Figure 10. Figure 10: DiSRT score for the intermediate layer of supervised learning trained ViTs. The ViT-B/L/H [PITH_FULL_IMAGE:figures/full_fig_p013_10.png]
Figure 11
Figure 11. Figure 11: DiSRT score for another supervised learning variant, DeiT v3 [ [PITH_FULL_IMAGE:figures/full_fig_p014_11.png]
Figure 12
Figure 12. Figure 12: DiSRT score for different architectures trained on MAE [ [PITH_FULL_IMAGE:figures/full_fig_p014_12.png]
Figure 13
Figure 13. Figure 13: DiSRT score for different architectures trained on CLIP with LAION-2B dataset [ [PITH_FULL_IMAGE:figures/full_fig_p014_13.png]
Figure 14
Figure 14. Figure 14: DiSRT score on DINOv2 architectures. A.4 Impact of Supervised Finetuning in the Intermediate Layers [PITH_FULL_IMAGE:figures/full_fig_p015_14.png]
Figure 15
Figure 15. Figure 15: DiSRT score comparison before and after supervised finetuning on ViT-Large CLIP [PITH_FULL_IMAGE:figures/full_fig_p015_15.png]
Figure 16
Figure 16. Figure 16: DiSRT score on ConvNeXt-MAE, the ImageNet finetuned variant and Top-K improved version. The DiSRT score in the last few layers drops significantly in supervised finetuning models but with Top-K applied, the final DiSRT score can be improved significantly. B Methodolog…
Figure 17
Figure 17. Figure 17: PCA projection value indicating figure and ground assignment. Front is represented by [PITH_FULL_IMAGE:figures/full_fig_p016_17.png]
Figure 18
Figure 18. Figure 18: DiSRT score on Giant Models (Range from 600 M to 1.8 B). [PITH_FULL_IMAGE:figures/full_fig_p017_18.png]
Figure 19
Figure 19. Figure 19: Standard trial in the psychophysical experiment. Image in the red box is the correct [PITH_FULL_IMAGE:figures/full_fig_p018_19.png]
Figure 20
Figure 20. Figure 20: Catch trial in the psychophysical experiment. Image in the red box is the correct answer [PITH_FULL_IMAGE:figures/full_fig_p019_20.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Human-like Object Grouping in Self-supervised Vision Transformers

    cs.CV 2026-03 conditional novelty 6.5 of 10

    DINO self-supervised transformers best predict human same/different object RTs; object-centric patch affinity and Gram-matrix distillation explain and transfer the alignment.

Reference graph

Works this paper leans on

43 extracted references · 27 canonical work pages · cited by 1 Pith paper

  1. [1]

    A century of gestalt psychology in visual perception: I

    Johan Wagemans, James H Elder, Michael Kubovy, Stephen E Palmer, Mary A Peterson, Manish Singh, and Rüdiger von der Heydt. A century of gestalt psychology in visual perception: I. perceptual grouping and figure–ground organization.Psychological Bulletin, 138(6):1172–1217, 2012

    work page 2012

  2. [2]

    A century of gestalt psychology in visual perception: Ii

    Johan Wagemans, Jacob Feldman, Sergei Gepshtein, Ruth Kimchi, James R Pomerantz, Peter A van der Helm, and Cees van Leeuwen. A century of gestalt psychology in visual perception: Ii. conceptual and theoretical foundations.Psychological Bulletin, 138(6):1218–1252, 2012

    work page 2012

  3. [3]

    Untersuchungen zur lehre von der gestalt, ii.Psychologische Forschung, 4:301–350, 1923

    Max Wertheimer. Untersuchungen zur lehre von der gestalt, ii.Psychologische Forschung, 4:301–350, 1923

    work page 1923

  4. [4]

    Subjective contours.Scientific American, 234(4):48–52, 1976

    Gaetano Kanizsa. Subjective contours.Scientific American, 234(4):48–52, 1976

    work page 1976

  5. [5]

    Masked autoencoders are scalable vision learners

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16000–16009, 2022

    2022

  6. [6]

    Univ of California Press, 1972

    Rudolf Arnheim.Art and visual perception: A psychology of the creative eye. Univ of California Press, 1972

    work page 1972

  7. [7]

    Convexity and symmetry in figure-ground organization

    Gaetano Kanizsa, Rudolf ARNHEIM, Mary HENLE, and Walter GERBINO. Convexity and symmetry in figure-ground organization. InVision and artifact, 1976

    work page 1976

  8. [8]

    Inhibitory competition in figure-ground perception: Context and convexity.Journal of Vision, 8(16):4–4, 2008

    Mary A Peterson and Elizabeth Salvagio. Inhibitory competition in figure-ground perception: Context and convexity.Journal of Vision, 8(16):4–4, 2008

    work page 2008

Show all 43 references
  1. [9]

    Who owns the contour of a visual hole?Perception, 35(7):883–894, 2006

    Marco Bertamini. Who owns the contour of a visual hole?Perception, 35(7):883–894, 2006

    2006

  2. [10]

    A convnet for the 2020s

    Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. A convnet for the 2020s. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11976–11986, 2022

    2022

  3. [11]

    Convnext v2: Co-designing and scaling convnets with masked autoencoders

    Sanghyun Woo, Shoubhik Debnath, Ronghang Hu, Xinlei Chen, Zhuang Liu, In So Kweon, and Saining Xie. Convnext v2: Co-designing and scaling convnets with masked autoencoders. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16133–16142, 2023

    2023

  4. [12]

    Texture synthesis using convolutional neural networks.Advances in neural information processing systems, 28, 2015

    Leon Gatys, Alexander S Ecker, and Matthias Bethge. Texture synthesis using convolutional neural networks.Advances in neural information processing systems, 28, 2015

    2015

  5. [13]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. InInternational conference on machine learning, pag...

    2021

  6. [14]

    Dinov2: Learning robust visual features without supervision.arXiv preprint arXiv:2304.07193, 2023

    Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision.arXiv preprint arXiv:2304.07193, 2023

    2023 arXiv

  7. [15]

    When and why vision-language models behave like bags-of-words, and what to do about it?arXiv preprint arXiv:2210.01936, 2022

    Mert Yuksekgonul, Federico Bianchi, Pratyusha Kalluri, Dan Jurafsky, and James Zou. When and why vision-language models behave like bags-of-words, and what to do about it?arXiv preprint arXiv:2210.01936, 2022

    2022 arXiv

  8. [16]

    Emergence of shape bias in convolutional neural networks through activation sparsity.Advances in Neural Information Processing Systems, 36:71755–71766, 2023

    Tianqin Li, Ziqi Wen, Yangfan Li, and Tai Sing Lee. Emergence of shape bias in convolutional neural networks through activation sparsity.Advances in Neural Information Processing Systems, 36:71755–71766, 2023

    2023

  9. [17]

    Large-scale two-photon imaging revealed super-sparse population codes in the v1 superficial layer of awake monkeys.Elife, 7:e33370, 2018

    Shiming Tang, Yimeng Zhang, Zhihao Li, Ming Li, Fang Liu, Hongfei Jiang, and Tai Sing Lee. Large-scale two-photon imaging revealed super-sparse population codes in the v1 superficial layer of awake monkeys.Elife, 7:e33370, 2018. 10

    2018

  10. [18]

    Surfgen: Adversarial 3d shape synthesis with explicit surface discriminators

    Andrew Luo, Tianqin Li, Wen-Hao Zhang, and Tai Sing Lee. Surfgen: Adversarial 3d shape synthesis with explicit surface discriminators. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 16238–16248, 2021

    2021

  11. [19]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016

    2016

  12. [20]

    Language models are few-shot learners.Advances in neural information processing systems, 33:1877–1901, 2020

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners.Advances in neural information processing systems, 33:1877–1901, 2020

    1901

  13. [21]

    Robust category-level 6d pose estimation with coarse-to-fine rendering of neural features

    Wufei Ma, Angtian Wang, Alan Yuille, and Adam Kortylewski. Robust category-level 6d pose estimation with coarse-to-fine rendering of neural features. InEuropean Conference on Computer Vision, pages 492–508. Springer, 2022

    2022

  14. [22]

    Hallucination improves the performance of unsupervised visual representation learning

    Jing Wu, Jennifer Hobbs, and Naira Hovakimyan. Hallucination improves the performance of unsupervised visual representation learning. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 16132–16143, 2023

    2023

  15. [23]

    Integrating auxiliary information in self-supervised learning.arXiv preprint arXiv:2106.02869, 2021

    Yao-Hung Hubert Tsai, Tianqin Li, Weixin Liu, Peiyuan Liao, Ruslan Salakhutdinov, and Louis-Philippe Morency. Integrating auxiliary information in self-supervised learning.arXiv preprint arXiv:2106.02869, 2021

    2021 arXiv

  16. [24]

    Hierarchical interdisciplinary topic detection model for research proposal classification.IEEE Transactions on Knowledge and Data Engineering, 35(9):9685–9699, 2023

    Meng Xiao, Ziyue Qiao, Yanjie Fu, Hao Dong, Yi Du, Pengyang Wang, Hui Xiong, and Yuanchun Zhou. Hierarchical interdisciplinary topic detection model for research proposal classification.IEEE Transactions on Knowledge and Data Engineering, 35(9):9685–9699, 2023

    2023

  17. [25]

    A deep learning framework based on dynamic channel selection for early classification of left and right hand motor imagery tasks

    Jiazhen Hong, Foroogh Shamsi, and Laleh Najafizadeh. A deep learning framework based on dynamic channel selection for early classification of left and right hand motor imagery tasks. In 2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Soc...

    2022

  18. [26]

    Learning weakly-supervised contrastive representations.arXiv preprint arXiv:2202.06670, 2022

    Yao-Hung Hubert Tsai, Tianqin Li, Weixin Liu, Peiyuan Liao, Ruslan Salakhutdinov, and Louis- Philippe Morency. Learning weakly-supervised contrastive representations.arXiv preprint arXiv:2202.06670, 2022

    2022 arXiv

  19. [27]

    Prototype memory and attention mechanisms for few shot image generation

    Tianqin Li, Zijie Li, Harold Rockwell, Amir Farimani, and Tai Sing Lee. Prototype memory and attention mechanisms for few shot image generation. InProceedings of the eleventh international conference on learning representations, volume 18, 2022

    2022

  20. [28]

    Conditional contrastive learning with kernel.arXiv preprint arXiv:2202.05458, 2022

    Yao-Hung Hubert Tsai, Tianqin Li, Martin Q Ma, Han Zhao, Kun Zhang, Louis-Philippe Morency, and Ruslan Salakhutdinov. Conditional contrastive learning with kernel.arXiv preprint arXiv:2202.05458, 2022

    2022 arXiv

  21. [29]

    A systematic review of ro- bustness in deep learning for computer vision: Mind the gap?arXiv preprint arXiv:2112.00639, 2021

    Nathan Drenkow, Numair Sani, Ilya Shpitser, and Mathias Unberath. A systematic review of ro- bustness in deep learning for computer vision: Mind the gap?arXiv preprint arXiv:2112.00639, 2021

    2021 arXiv

  22. [30]

    Opening the black box: the promise and limitations of explainable machine learning in cardiology.Canadian Journal of Cardiology, 38(2):204–213, 2022

    Jeremy Petch, Shuang Di, and Walter Nelson. Opening the black box: the promise and limitations of explainable machine learning in cardiology.Canadian Journal of Cardiology, 38(2):204–213, 2022

    2022

  23. [31]

    Explaining explanations: An overview of interpretability of machine learning

    Leilani H Gilpin, David Bau, Ben Z Yuan, Ayesha Bajwa, Michael Specter, and Lalana Kagal. Explaining explanations: An overview of interpretability of machine learning. In2018 IEEE 5th International Conference on data science and advanced analytics (DSAA), pages 80–89. IEEE, 2018

    2018

  24. [32]

    Assessing neural network representations during training using data diffusion spectra.ICML workshop on TAG-ML, 2023

    Danqi Liao, Chen Liu, Alexander Tong, Guillaume Huguet, Guy Wolf, Maximilian Nickel, Ian Adelstein, and Smita Krishnaswamy. Assessing neural network representations during training using data diffusion spectra.ICML workshop on TAG-ML, 2023. 11

    2023

  25. [33]

    Neural networks trained on natural scenes exhibit gestalt closure.Computational Brain & Behavior, 4(3):251–263, 2021

    Been Kim, Emily Reif, Martin Wattenberg, Samy Bengio, and Michael C Mozer. Neural networks trained on natural scenes exhibit gestalt closure.Computational Brain & Behavior, 4(3):251–263, 2021

    2021

  26. [34]

    Evaluating machine learning models with nero: Non-equivariance revealed on orbits.arXiv preprint arXiv:2305.19889, 2023

    Zhuokai Zhao, Takumi Matsuzawa, William Irvine, Michael Maire, and Gordon L Kindlmann. Evaluating machine learning models with nero: Non-equivariance revealed on orbits.arXiv preprint arXiv:2305.19889, 2023

    2023 arXiv

  27. [35]

    Vision transformers with self-distilled registers.arXiv preprint arXiv:2505.21501, 2025

    Yinjie Chen, Zipeng Yan, Chong Zhou, Bo Dai, and Andrew F Luo. Vision transformers with self-distilled registers.arXiv preprint arXiv:2505.21501, 2025

    2025

  28. [36]

    Visualizing and understanding convolutional networks

    Matthew D Zeiler and Rob Fergus. Visualizing and understanding convolutional networks. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part I 13, pages 818–833. Springer, 2014

    2014

  29. [37]

    Imagenet-trained cnns are biased towards texture; increasing shape bias improves accuracy and robustness.arXiv preprint arXiv:1811.12231, 2018

    Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A Wichmann, and Wieland Brendel. Imagenet-trained cnns are biased towards texture; increasing shape bias improves accuracy and robustness.arXiv preprint arXiv:1811.12231, 2018

    2018 arXiv

  30. [38]

    Partial success in closing the gap between human and machine vision.Advances in Neural Information Processing Systems, 34:23885– 23899, 2021

    Robert Geirhos, Kantharaju Narayanappa, Benjamin Mitzkus, Tizian Thieringer, Matthias Bethge, Felix A Wichmann, and Wieland Brendel. Partial success in closing the gap between human and machine vision.Advances in Neural Information Processing Systems, 34:23885– 23899, 2021

    2021

  31. [39]

    An image is worth 16x16 words: Transformers for image recognition at scale.arXiv preprint arXiv:2010.11929, 2020

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale.arXiv preprint arXiv:...

    2010 arXiv

  32. [40]

    Vision transformers are robust learners

    Sayak Paul and Pin-Yu Chen. Vision transformers are robust learners. InProceedings of the AAAI conference on Artificial Intelligence, volume 36, pages 2071–2081, 2022

    2022

  33. [41]

    Imagenet: A large- scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large- scale hierarchical image database. In2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009

    2009

  34. [42]

    Deit iii: Revenge of the vit

    Hugo Touvron, Matthieu Cord, and Hervé Jégou. Deit iii: Revenge of the vit. InEuropean conference on computer vision, pages 516–533. Springer, 2022

    2022

  35. [43]

    given-up

    Romain Beaumont. Clip-vit-h-14-laion2b-s32b-b79k. https://huggingface.co/laion/ CLIP-ViT-H-14-laion2B-s32B-b79K , 2022. Model trained using OpenCLIP on the LAION-2B dataset. 12 Supplementary Materials A Intermediate Representation Analysis To better understand why certain meth...

    2022

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.