Pith. sign in

REVIEW 5 major objections 6 minor 73 references

On the rankability of visual embeddings

T0 review · 5 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Standard pretrained visual embeddings hide a linear 'rank axis' that orders images by continuous attributes such as age, crowd density, and aesthetics — and for several attributes, just two extreme example images are enough to find that…

desk verdict A solid empirical study showing linear rank axes in frozen embeddings; worth a serious referee, with caveats about the baseline framing and missing error bars. read the letter →

arxiv 2507.03683 v1 pith:KPFVHP27 submitted 2025-07-04 cs.CV

Ankit Sonthalia , Arnas Uselis , Seong Joon Oh This is my paper
classification cs.CV
keywords rankabilityvisualembeddingsrankaxisordinalattributeslinearprobingSpearmancorrelationzero-shotrankingvectordatabases
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Modern visual embeddings are built so that similar images sit close together, which supports retrieval; this paper asks whether they also support ranking, where all images must be ordered along a continuous attribute such as age, crowd density, head pose, aesthetics, or recency. It defines a representation as 'rankable' when a single linear direction, called a rank axis $v_A$, sorts the projected embeddings in the correct attribute order. Across seven encoders and nine datasets, the paper reports that pretrained embeddings are mostly rankable: linear regression on frozen embeddings reaches Spearman rank correlations close to the nonlinear and finetuning upper bounds and well above a randomly initialized encoder baseline. The paper further shows that a useful rank axis can sometimes be recovered from a handful of labeled samples — and on the Adience age dataset, from just two extreme images, one young and one old, with no labels at all. If the claim holds, global image ranking by arbitrary attributes becomes nearly a free operation in vector databases.

What carries the argument

The central object is the rank axis $v_A$: a unit vector in the embedding space $\mathbb{R}^d$ such that the projection $v_A^\top f(x)$ preserves the ordering of the continuous attribute $A(x)$. The paper measures rankability as the Spearman rank correlation coefficient $\rho$ between that projection and the true labels, computed after fitting $v_A$ by linear regression on training embeddings. Three reference points carry the argument: the no-train baseline, which fits the same linear probe to a randomly initialized encoder and quantifies how much of the correlation is not due to trained structure; the nonlinear (two-layer MLP) probe, which estimates the total ordinal information available in the frozen embeddings; and finetuning, which estimates what the encoder architecture itself can learn. A separate mechanism, borrowed from steering-vector constructions, builds a label-free rank axis as $v_A = (f(x_h) - f(x_l)) / \|f(x_h) - f(x_l)\|_2$, the normalized difference between the mean embeddings of high- and low-extreme image clusters.

What would settle it

Choose one attribute-dataset pair, such as KonIQ aesthetics, and measure the rankability gap between the pretrained encoder and its randomly initialized version while holding low-level statistics constant — for instance, by matching color histograms and texture between the low- and high-score image groups, or by randomly permuting the weights within each layer instead of sampling a fresh network. If the gap shrinks to near zero once low-level cues are controlled, then the linear rank axis is reading dataset statistics rather than a learned ordinal direction; if the gap persists, rankability is a genuine property of the trained embedding.

Watch

Extended reading notes

Core claim

The central discovery is that ordinal information in visual embeddings is largely linear and recoverable with minimal supervision. Concretely, for all but yaw and roll among the seven studied attributes, the average linear-rankability Spearman correlation sits closer to the two upper bounds (an MLP probe on frozen embeddings and finetuning the encoder) than to the no-train lower bound computed on a randomly initialized encoder; average values range from $\rho = 0.218$ for roll to $\rho = 0.869$ for crowd count on ShanghaiTech-B. The paper also establishes that the rank axis for an attribute can be estimated by the normalized difference between the mean embeddings of two extreme clusters, and reports that on Adience this two-sample construction already yields $\rho \approx 0.75$, beating few-shot regression at matching tiny training-set sizes. These results are taken as evidence that modern embeddings contain learnable, near-linear 'rank axes' for many continuous attributes — a property the authors name rankability and distinguish from retrieval.

Load-bearing premise

The claim's load-bearing premise is that a randomly initialized encoder is a true 'no-information' baseline; if the random network's features already share low-level statistics with the attribute (the paper reports no-train Spearman correlations up to 0.55 on KonIQ aesthetics), then the reported rankability gap over that baseline can overstate how much learned semantic structure is doing the work.

Editorial extensions

If this is right

  • Any vector database can support global ordering along the seven studied attributes by storing one projection per attribute, computed by linear regression on frozen embeddings — no finetuning or re-embedding needed.
  • A user who can fetch two extreme exemplars (e.g., a child and an elderly face) obtains a usable rank axis with zero labels; on Adience this two-image axis reaches $\rho \approx 0.75$, while the no-train baseline is about 0.27 and few-shot regression at that size is near zero.
  • Rank axes transfer across datasets of the same attribute (an Adience-trained age axis scores 0.68 SRCC on UTKFace), so an axis learned once on a large source dataset can label-free rank a smaller target collection.
  • Because the linear-regression rankability nearly matches the MLP probe on most attributes, the residual ordinal structure in these embeddings is mostly linear, which makes embedding spaces interpretable as collections of ordinal subspaces.
  • Zero-shot text prompting of CLIP trails the linear image-data rank axis (best prompt-based SRCC 0.782 vs 0.917 linear on Adience), so for ranking, image-derived axes remain the practical route over language prompts.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct extension the paper does not run: test rankability on attributes without strong low-level cues (e.g., shoes by formality, rooms by messiness) to see whether the two-extreme-sample axis keeps working when the extreme ends differ semantically but not photometrically.
  • The attribute-variety hypothesis the authors float suggests rankability is a diagnostic of pretraining data coverage; measuring the rankability of an unseen attribute could cheaply reveal whether the pretraining corpus contains that ordinal signal.
  • The unintended positive cosine similarity between the UTKFace age axis and the KonIQ aesthetics axis implies spurious dataset correlations leak into rank axes; a testable mitigation is to orthogonalize rank axes against known nuisance attributes before deployment.
  • If two-exemplar axes generalize across users' own mental attributes, the method could power interactive photo-album sorting where the 'low' and 'high' images are chosen by the user, effectively making ordinal supervision free.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. This paper introduces the notion of rankability for visual embeddings: a representation is rankable for an ordinal attribute if there exists a unit vector v_A such that projecting embeddings onto v_A preserves the attribute ordering. The authors measure rankability via the test Spearman rank correlation of a linear regressor fit on frozen embeddings, across seven encoders and nine datasets, and compare it with a randomly initialized-encoder lower bound, an MLP nonlinear upper bound, and end-to-end finetuning. They report that most attributes are highly rankable, that CLIP encoders tend to be more rankable, that a small number of labeled examples or even two extreme examples can recover useful rank axes, and that rank axes transfer across datasets. The paper also evaluates zero-shot prompt-based rank axes and finds them inferior to linear regression.

Significance. The study is a useful and timely empirical characterization of embedding geometry, complementary to retrieval-oriented analyses. Strengths include a clear formal definition, evaluation on held-out test splits, a broad model/dataset matrix, comparison with multiple reference points, honest reporting of weak cases (e.g., roll SRCC 0.218), and public code. If the main claim is sustained, the paper opens a practical direction for ranking in vector databases and motivates theoretical work on ordinal structure. However, the significance is currently tempered by the interpretation of the no-train baseline, lack of uncertainty estimates, and the absence of an operational threshold for "rankable."

major comments (5)
  1. [Section 3.1] The "No-train lower bound" is not a no-information baseline: it fits an optimal linear axis on the labels using randomly initialized encoder features, and random networks are known to carry low-level inductive biases (the paper itself cites Deep Image Prior [52] in this context). The paper's own tables show high baselines (e.g., KonIQ-10k 0.435, pitch 0.405, Adience 0.266 in Table 2), so the incremental contribution of pretraining is modest for several attributes. The headline claim that rankability is "non-trivially high" should be supported by additional controls such as a random axis in the pretrained embedding space, a raw-pixel linear probe, or a permuted-label probe, and by reporting the gain over the baseline rather than the absolute SRCC.
  2. [Tables 2 and 3, Section 3.3] All SRCC values in Tables 2 and 3 are point estimates, and the protocol selects the best of 30 random hyperparameter configurations by validation SRCC (Section 3.2.3). Without error bars across seeds or an explicit statement of how many runs contribute to the reported test numbers, the claims that rankability is "close to" the nonlinear upper bound and "much higher than" the lower bound are not quantitatively grounded. Please report standard errors or bootstrap confidence intervals, and clarify whether the best-validation run's test SRCC is reported.
  3. [Definition 1, Section 3.3] Definition 1 defines rankability as an exact order-preservation property, but the empirical claim "rankable" is never tied to a threshold on SRCC or to a statistical test against the baseline. For example, the average roll SRCC is 0.218 versus a no-train baseline of 0.151; the text calls some of these cases "struggle" but does not state whether they count as rankable. A pre-specified threshold or a test of whether the linear axis outperforms a distribution of random axes is needed to make the binary property operational.
  4. [Section 3.1, Tables B11 and B13] The "Nonlinear upper bound" is not always an upper bound: in several per-model results the MLP reference is below linear regression (e.g., Table B11, ViT-B/32: rankability 0.700 vs nonlinear 0.623; Table B13, ViT-B/32: 0.673 vs 0.505), so labeling the MLP as an "upper bound for embedding" is not justified by the reported optimization protocol. Because the linearity claim rests on the small gap between linear and MLP, this reference should be renamed a nonlinear baseline, and the conclusion should be qualified for cases where the MLP underperforms.
  5. [Section 4.2, Figure 2] The extreme-pair results in Figure 2 need more detail: it is not specified how the extreme samples are chosen from the training split, how many random trials are averaged, or whether the reported "just two extreme samples" SRCC of ~0.75 on Adience has variance. Since the selection of extremes uses ground-truth labels, the text should also clarify that the method is label-free only at the regression step, not at the sample-selection step. Without this information the central few-shot claim is hard to evaluate.
minor comments (6)
  1. [Table 1] The "Split" column for Adience says "Official 5-fold"; please clarify how the five folds are aggregated when computing the reported test SRCC.
  2. [Table 2 and Appendix B.5] The row "Recency (HCI)" uses the abbreviation HCI, but the dataset is introduced only in Appendix B.5 as Historical Color Images; please define the abbreviation where the dataset first appears.
  3. [Figure 1] In the aesthetics panel, the 100th percentile sample has a projection of 76 while the 75th percentile sample has 78, which is inconsistent with a strictly monotone rank axis; this should be explained or corrected.
  4. [Section 4.4] The prompt search protocol is not described: please state whether the GPT-generated prompts and prompt pairs are selected on a validation split or on the test split, and how many generated candidates were used per dataset.
  5. [Section 4.2] There is a typo: "consititute" should be "constitute," and in Section 4.3 "transfrability" should be "transferability."
  6. [Table C19] The comparison table uses CLIP-ConvNeXt-L embeddings, but the main text reports results for "CLIP-CNX" (OpenCLIP ConvNeXt-L, 320px); please clarify that these are the same model and checkpoint.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the rankability measurements are fit on training splits and evaluated on disjoint test splits, with external baselines and non-load-bearing self-citations.

full rationale

The paper's central claim is empirical and self-contained: rank axes are obtained by linear regression on a training split and evaluated on a disjoint test split (Section 3.1: "we measure rankability using the generalisation performance of the rank axis vA learned on a training split Xtrain and tested on a disjoint split Xtest"), and the extreme-pair axes are likewise evaluated on held-out data (Section 4.2). The no-train, nonlinear, and finetuned rows in Tables 2 and B8-B18 are reference points, not ingredients of the fitted rankability values. The no-train baseline is admittedly not a no-information control because it fits a linear probe to labels on random embeddings, but this is an interpretational caveat, not a circular reduction: the reported rankability SRCCs do not reduce to this baseline by construction. The zero-shot prompt-search upper bound is the separately measured linear-regression SRCC, which is an independent quantity, not an input. Self-citations [25, 53, 54] appear only in related-work discussion and do not carry the load-bearing argument; no uniqueness theorem or ansatz is imported from the authors' prior work. The paper also validates against external SOTA methods (Table C19), further confirming that the measurements are not defined in terms of the conclusions. No equation or fitted parameter is renamed as a prediction, and no step of the derivation is logically equivalent to its own input.

Assumptions & free parameters 2 free parameters · 5 assumptions · 1 invented entities

The paper's central claims rest on the quality and grain of the attribute labels, on the validity of the random-encoder lower bound, and on the MLP/finetuning proxies for 'upper bounds.' No physical or mathematical constants are fit; the only fitted quantities are the regression/MLP/finetune parameters and the model-selection hyperparameters, which is standard supervised evaluation. The rank axis is an operationally defined direction, not a postulated entity.

free parameters (2)
  • Model-selection hyperparameters (learning rate, weight decay, augmentation toggle) = best of 30 random configs per model-dataset pair
    Section 3.2.3; chosen by best validation SRCC, standard practice but introduces selection-based inflation risk since no seed variance is reported.
  • Zero-shot best prompt / prompt-pair selection = best among 500 single prompts and 100 prompt pairs
    Section 4.4; selection split (validation vs test) is not stated; if test selection was used, reported zero-shot SRCCs are upper-biased.
assumptions (5)
  • domain assumption Ground-truth attribute labels (age, MOS, Euler angles, crowd counts) are accurate enough orderings of the images they describe.
    Invoked throughout Section 3.2.1 and in every SRCC computation; label noise and coarse granularity (Adience age groups, A V A averaged MOS) directly cap the achievable SRCC and shape cross-attribute comparisons.
  • domain assumption The randomly-initialized encoder embeddings provide a valid no-information lower bound.
    Section 3.1 reference point (1); random encoders already give SRCCs of 0.08 to 0.55 (Table 2), so the rankability claim is the gap above this baseline, which may be influenced by low-level statistics shared by random and trained features. Cited [52].
  • domain assumption A two-layer MLP is a sufficient nonlinear upper bound for ordinal information extractable from a frozen embedding.
    Section 3.1 reference point (2); the claim that 'most ordinality is linearly encoded' rests on comparing linear SRCC against this MLP's SRCC.
  • domain assumption Finetuning the encoder is an upper bound for the architecture's capacity on the attribute.
    Section 3.1 reference point (3); finetuning results can be below linear regression in several rows (e.g., ST-A crowd, UTKFace CLIP-RN50), so the interpretation of the upper bound is loose.
  • domain assumption Learning the axis by least-squares regression on training labels generalizes to a disjoint test split, and SRCC on that split measures Definition 1's order preservation.
    Section 3.1; the operationalization replaces the exact-pairwise definition with approximate held-out rank correlation.
invented entities (1)
  • Rank axis (v_A) independent evidence
    purpose: A unit vector whose projection orders images by attribute A; the central object of study.
    Conceptually new term, but it reduces to a linear regression direction or a difference-of-cluster-means vector; the paper provides falsifiable handles in the form of held-out SRCC values per dataset and model, so the entity is operationally defined rather than postulated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of On the rankability of visual embeddings." pith.science (2026). https://pith.science/paper/KPFVHP27

@misc{pith2026250703683,
  author       = {Pith},
  title        = {Pith review of: On the rankability of visual embeddings},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KPFVHP27}},
  note         = {Machine review of arXiv:2507.03683}
}
read the original abstract

We study whether visual embedding models capture continuous, ordinal attributes along linear directions, which we term _rank axes_. We define a model as _rankable_ for an attribute if projecting embeddings onto such an axis preserves the attribute's order. Across 7 popular encoders and 9 datasets with attributes like age, crowd count, head pose, aesthetics, and recency, we find that many embeddings are inherently rankable. Surprisingly, a small number of samples, or even just two extreme examples, often suffice to recover meaningful rank axes, without full-scale supervision. These findings open up new use cases for image ranking in vector databases and motivate further study into the structure and learning of rankable embeddings. Our code is available at https://github.com/aktsonthalia/rankable-vision-embeddings.

Figures

Figures reproduced from arXiv: 2507.03683 by the authors.

Figure 1
Figure 1. Visualisation of rank axes. We show r th percentile samples along the rank axes found using linear regression over CLIP-ViT-B/32 embeddings from each respective dataset. Caveats. The current results are empirical, and our claims are based on the set of attributes considered in our study. Despite following choices established in the literature, we may sometimes not uncover the most optimal finetuned upper bounds. A b… view at source ↗
Figure 2
Figure 2. Few-shot learning with continuous labels vs extreme samples without continuous labels: extreme samples win out in the small-train-set regime. “Extreme” refers to training using samples from the extreme ends of the ranking axis, while "few" refers to few-shot learning on labeled samples. Full-dataset linear regression performance is given by the star-shaped marker. Observations. We report the results in [PITH_FULL_I… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

73 extracted references · 41 canonical work pages

  1. [52]

    Deep image prior

    Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky. “Deep image prior”. In:Proceedings of the IEEE conference on computer vision and pattern recognition. 2018, pp. 9446–9454

    work page 2018

  2. [1]

    Understanding intermediate layers using linear classifier probes

    Guillaume Alain and Yoshua Bengio. “Understanding intermediate layers using linear classifier probes”. In: arXiv preprint arXiv:1610.01644 (2016)

    arXiv 2016

  3. [2]

    Conditioned and composed image retrieval combining and partially fine-tuning CLIP-based features

    Alberto Baldrati et al. “Conditioned and composed image retrieval combining and partially fine-tuning CLIP-based features”. In: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) (2022), pp. 4955–4964. URL: https : / / api . semanticscholar.org/CorpusId:251034454

    work page 2022

  4. [3]

    Not Only Text: Exploring Compositionality of Visual Representations in Vision-Language Models

    Davide Berasi et al. “Not Only Text: Exploring Compositionality of Visual Representa- tions in Vision-Language Models”. In: ArXiv abs/2503.17142 (2025). URL: https://api. semanticscholar.org/CorpusId:277244112

    work page Pith review arXiv 2025

  5. [4]

    A Simple Framework for Contrastive Learning of Visual Representations

    Ting Chen et al. “A Simple Framework for Contrastive Learning of Visual Representations”. In: Proceedings of the 37th International Conference on Machine Learning . International Conference on Machine Learning. PMLR, Nov. 21, 2020, pp. 1597–1607. URL: https : //proceedings.mlr.press/v119/chen20j.html

    work page 2020

  6. [5]

    Deep Learning for Instance Retrieval: A Survey

    Wei Chen et al. “Deep Learning for Instance Retrieval: A Survey”. In: IEEE Transactions on Pattern Analysis and Machine Intelligence 45 (2021), pp. 7270–7292. URL: https://api. semanticscholar.org/CorpusId:245837930

    work page 2021

  7. [6]

    Deep learning for instance retrieval: A survey

    Wei Chen et al. “Deep learning for instance retrieval: A survey”. In: IEEE Transactions on Pattern Analysis and Machine Intelligence 45.6 (2022), pp. 7270–7292

    work page 2022

  8. [7]

    Composition Loss for Counting, Density Map Estimation and Localization in Dense Crowds

    “Composition Loss for Counting, Density Map Estimation and Localization in Dense Crowds”. In: Haroon Idrees et al. Lecture Notes in Computer Science . Cham: Springer International Publishing, 2018, pp. 544–559. ISBN : 978-3-030-01215-1 978-3-030-01216-8. DOI: 10.1007/ 978-3-030-01216-8_33 . URL: https://link.springer.com/10.1007/978-3-030- 01216-8_33

    doi:10.1007/978-3-030- 2018

Show all 73 references
  1. [8]

    Hyperbolic Image-Text Representations

    Karan Desai et al. “Hyperbolic Image-Text Representations”. In:ArXiv abs/2304.09172 (2023). URL: https://arxiv.org/pdf/2304.09172.pdf

    2023 arXiv

  2. [9]

    arXiv:2010.11929

    Alexey Dosovitskiy et al.An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. arXiv:2010.11929. June 2021. DOI: 10.48550/arXiv.2010.11929 . URL: http: //arxiv.org/abs/2010.11929

    2010 doi arXiv

  3. [10]

    Teach CLIP to Develop a Number Sense for Ordinal Regression

    Yao Du et al. Teach CLIP to Develop a Number Sense for Ordinal Regression. Aug. 7, 2024. DOI: 10.48550/arXiv.2408.03574 . arXiv: 2408.03574 [cs] . URL: http://arxiv. org/abs/2408.03574. Pre-published

    2024 doi arXiv

  4. [11]

    Age and Gender Estimation of Unfiltered Faces

    Eran Eidinger, Roee Enbar, and Tal Hassner. “Age and Gender Estimation of Unfiltered Faces”. In: IEEE Transactions on Information Forensics and Security 9.12 (Dec. 2014), pp. 2170–2179. ISSN : 1556-6021. DOI: 10 . 1109 / TIFS . 2014 . 2359646. URL: https : //ieeexplore.ieee.or...

    2014

  5. [12]

    Its Not a Modality Gap: Characterizing and Addressing the Contrastive Gap

    Abrar Fahim, Alex Murphy, and Alona Fyshe. “Its Not a Modality Gap: Characterizing and Addressing the Contrastive Gap”. In: ArXiv abs/2405.18570 (2024). URL: https://api. semanticscholar.org/CorpusId:270095104

    2024 arXiv

  6. [14]

    Heterogeneous face attribute estimation: A deep multi-task learning approach

    Hu Han et al. “Heterogeneous face attribute estimation: A deep multi-task learning approach”. In: IEEE transactions on pattern analysis and machine intelligence 40.11 (2017), pp. 2597– 2609

    2017

  7. [15]

    Deep Residual Learning for Image Recognition

    Kaiming He et al. “Deep Residual Learning for Image Recognition”. In:2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV , USA: IEEE, June 2016, pp. 770–778. ISBN : 978-1-...

    2016

  8. [16]

    Multilayer feedforward networks are universal approximators

    Kurt Hornik, Maxwell Stinchcombe, and Halbert White. “Multilayer feedforward networks are universal approximators”. In: Neural networks 2.5 (1989), pp. 359–366

    1989

  9. [17]

    KonIQ-10k: An Ecologically Valid Database for Deep Learning of Blind Image Quality Assessment

    Vlad Hosu et al. “KonIQ-10k: An Ecologically Valid Database for Deep Learning of Blind Image Quality Assessment”. In: IEEE Transactions on Image Processing 29 (2020), pp. 4041–

    2020

  10. [18]

    Lp++: A surprisingly strong linear probe for few-shot clip

    Yunshi Huang et al. “Lp++: A surprisingly strong linear probe for few-shot clip”. In: Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024, pp. 23773–23782

    2024

  11. [19]

    The platonic representation hypothesis

    Minyoung Huh et al. “The platonic representation hypothesis”. In: arXiv preprint arXiv:2405.07987 (2024)

    2024 arXiv

  12. [20]

    CLIP-Count: Towards Text-Guided Zero- Shot Object Counting

    Ruixiang Jiang, Lingbo Liu, and Changwen Chen. “CLIP-Count: Towards Text-Guided Zero- Shot Object Counting”. In: Proceedings of the 31st ACM International Conference on Multime- dia. Oct. 26, 2023, pp. 4535–4545. DOI: 10.1145/3581783.3611789. arXiv: 2305.07304 [cs]. URL: http:...

    2023

  13. [21]

    Hyperbolic Image Embeddings

    Valentin Khrulkov et al. “Hyperbolic Image Embeddings”. In:2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2019), pp. 6417–6427. URL: http: //ieeexplore.ieee.org/stamp/stamp.jsp?tp=%5C&arnumber=9156432

    2019

  14. [22]

    CompBench: A Comparative Reasoning Benchmark for Multimodal LLMs

    Jihyung Kil et al. CompBench: A Comparative Reasoning Benchmark for Multimodal LLMs. July 23, 2024. DOI: 10 . 48550 / arXiv . 2407 . 16837. arXiv: 2407 . 16837 [cs]. URL: http://arxiv.org/abs/2407.16837. Pre-published

    2024 doi arXiv

  15. [23]

    Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav)

    Been Kim et al. “Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav)”. In: International conference on machine learning. PMLR. 2018, pp. 2668–2677

    2018

  16. [24]

    CLIP Behaves like a Bag-of-Words Model Cross-modally but Not Uni-modally

    Darina Koishigarina, Arnas Uselis, and Seong Joon Oh. CLIP Behaves like a Bag-of-Words Model Cross-modally but Not Uni-modally . Feb. 8, 2025. DOI: 10.48550/arXiv.2502. 03566. arXiv: 2502 . 03566 [cs]. URL: http : / / arxiv . org / abs / 2502 . 03566. Pre- published

    2025 doi

  17. [25]

    CLIP Behaves like a Bag-of-Words Model Cross-modally but not Uni-modally

    Darina Koishigarina, Arnas Uselis, and Seong Joon Oh. “CLIP Behaves like a Bag-of-Words Model Cross-modally but not Uni-modally”. In: arXiv preprint arXiv:2502.03566 (2025)

    2025

  18. [26]

    Beyond a Pre-Trained Object Detector: Cross-Modal Textual and Visual Context for Image Captioning

    Chia-Wen Kuo and Z. Kira. “Beyond a Pre-Trained Object Detector: Cross-Modal Textual and Visual Context for Image Captioning”. In: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2022), pp. 17948–17958. URL: https : / / api . semanticscholar.org/Cor...

    2022

  19. [27]

    MiVOLO: Multi-input Transformer for Age and Gender Estimation

    Maksim Kuprashevich and Irina Tolstykh. MiVOLO: Multi-input Transformer for Age and Gender Estimation. Sept. 22, 2023. DOI: 10 . 48550 / arXiv . 2307 . 04616. arXiv: 2307 . 04616 [cs]. URL: http://arxiv.org/abs/2307.04616. Pre-published

    2023 doi arXiv

  20. [28]

    Meir Yossef Levi and Guy Gilboa.The Double-Ellipsoid Geometry of CLIP. Nov. 21, 2024. DOI: 10.48550/arXiv.2411.14517 . arXiv: 2411.14517 [cs] . URL: http://arxiv. org/abs/2411.14517. Pre-published

    2024 doi arXiv

  21. [29]

    Does clip bind concepts? probing compositionality in large image models

    Martha Lewis et al. “Does clip bind concepts? probing compositionality in large image models”. In: arXiv preprint arXiv:2212.10537 (2022)

    2022 arXiv

  22. [30]

    Align before fuse: Vision and language representation learning with mo- mentum distillation

    Junnan Li et al. “Align before fuse: Vision and language representation learning with mo- mentum distillation”. In: Advances in Neural Information Processing Systems. V ol. 34. 2021, pp. 9694–9705

    2021

  23. [31]

    BLIP: Bootstrapping Language-Image Pre-training for Unified Vision- Language Understanding and Generation

    Junnan Li et al. “BLIP: Bootstrapping Language-Image Pre-training for Unified Vision- Language Understanding and Generation”. In: Proceedings of the 39th International Confer- ence on Machine Learning. International Conference on Machine Learning. PMLR, June 28, 2022, pp. 1288...

    2022

  24. [32]

    OrdinalCLIP: Learning Rank Prompts for Language-Guided Ordinal Regres- sion

    Wanhua Li et al. OrdinalCLIP: Learning Rank Prompts for Language-Guided Ordinal Regres- sion. Oct. 1, 2022. DOI: 10.48550/arXiv.2206.02338 . arXiv: 2206.02338 [cs] . URL: http://arxiv.org/abs/2206.02338. Pre-published

    2022 doi arXiv

  25. [33]

    CrowdCLIP: Unsupervised Crowd Counting via Vision-Language Model

    Dingkang Liang et al. CrowdCLIP: Unsupervised Crowd Counting via Vision-Language Model. Apr. 9, 2023. DOI: 10.48550/arXiv.2304.04231. arXiv: 2304.04231 [cs]. URL: http://arxiv.org/abs/2304.04231. Pre-published. 11

    2023 doi arXiv

  26. [34]

    Beyond comparing image pairs: Setwise active learning for relative attributes

    Lucy Liang and Kristen Grauman. “Beyond comparing image pairs: Setwise active learning for relative attributes”. In: Proceedings of the IEEE conference on Computer Vision and Pattern Recognition. 2014, pp. 208–215

    2014

  27. [35]

    CLIP-EBC: CLIP Can Count Accurately through Enhanced Blockwise Classification

    Yiming Ma, Victor Sanchez, and Tanaya Guha. CLIP-EBC: CLIP Can Count Accurately through Enhanced Blockwise Classification. Version 3. Mar. 25, 2025.DOI: 10.48550/arXiv. 2403.09281 . arXiv: 2403.09281 [cs] . URL: http://arxiv.org/abs/2403.09281 . Pre-published

    2025 doi arXiv

  28. [36]

    Clipcap: Clip prefix for image captioning

    Ron Mokady, Amir Hertz, and Amit H Bermano. “Clipcap: Clip prefix for image captioning”. In: arXiv preprint arXiv:2111.09734 (2021)

    2021 arXiv

  29. [37]

    A V A: A Large-Scale Database for Aesthetic Visual Analysis

    N. Murray, L. Marchesotti, and F. Perronnin. “A V A: A Large-Scale Database for Aesthetic Visual Analysis”. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition. 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Providence, RI: IEEE, June 2...

    2012

  30. [38]

    A metric learning reality check

    Kevin Musgrave, Serge Belongie, and Ser-Nam Lim. “A metric learning reality check”. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXV 16. Springer. 2020, pp. 681–699

    2020

  31. [39]

    Parts of Speech-Grounded Subspaces in Vision-Language Models

    James Oldfield et al. “Parts of Speech-Grounded Subspaces in Vision-Language Models”. In: ArXiv abs/2305.14053 (2023). URL: https://api.semanticscholar.org/CorpusId: 258841517

    2023 arXiv

  32. [40]

    DINOv2: Learning Robust Visual Features without Supervision

    Maxime Oquab et al. DINOv2: Learning Robust Visual Features without Supervision . en. arXiv:2304.07193 [cs]. Feb. 2024. URL: http://arxiv.org/abs/2304.07193

    2024 arXiv

  33. [41]

    Teaching CLIP to Count to Ten

    Roni Paiss et al. Teaching CLIP to Count to Ten . Feb. 23, 2023. DOI: 10.48550/arXiv. 2302.12066 . arXiv: 2302.12066 [cs] . URL: http://arxiv.org/abs/2302.12066 . Pre-published

    2023 doi arXiv

  34. [42]

    Dating Historical Color Images

    Frank Palermo, James Hays, and Alexei A. Efros. “Dating Historical Color Images”. In: Computer Vision – ECCV 2012. Ed. by Andrew Fitzgibbon et al. Red. by David Hutchison et al. V ol. 7577. Berlin, Heidelberg: Springer Berlin Heidelberg, 2012, pp. 499–512. ISBN : 978-3-642-337...

    2012 doi

  35. [43]

    Relative Attributes

    Devi Parikh and Kristen Grauman. “Relative Attributes”. In:2011 International Conference on Computer Vision. 2011 IEEE International Conference on Computer Vision (ICCV). Barcelona, Spain: IEEE, Nov. 2011, pp. 503–510. ISBN : 978-1-4577-1102-2 978-1-4577-1101-5 978- 1-4577-110...

    2011

  36. [44]

    HYDEN: Hyperbolic Density Representations for Medical Images and Re- ports

    Zhi Qiao et al. “HYDEN: Hyperbolic Density Representations for Medical Images and Re- ports”. In: International Conference on Computational Linguistics. 2024. URL: https://api. semanticscholar.org/CorpusId:271903357

    2024

  37. [45]

    Learning Transferable Visual Models From Natural Language Supervi- sion

    Alec Radford et al. “Learning Transferable Visual Models From Natural Language Supervi- sion”. In: International Conference on Machine Learning. 2021. URL: https://arxiv.org/ pdf/2103.00020.pdf

    2021 arXiv

  38. [46]

    Learning Transferable Visual Models From Natural Language Super- vision

    Alec Radford et al. “Learning Transferable Visual Models From Natural Language Super- vision”. In: Proceedings of the 38th International Conference on Machine Learning . In- ternational Conference on Machine Learning. PMLR, July 1, 2021, pp. 8748–8763. URL: https://proceedings...

    2021

  39. [47]

    Steering Llama 2 via Contrastive Activation Addition

    Nina Rimsky et al. “Steering Llama 2 via Contrastive Activation Addition”. In:Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). ACL 2024. Ed. by Lun-Wei Ku, Andre Martins, and Vivek Srikumar. Bangkok, Thailand: Ass...

    2024 doi

  40. [48]

    Finetuning CLIP to Reason about Pairwise Differences

    Dylan Sam et al. Finetuning CLIP to Reason about Pairwise Differences. Sept. 15, 2024. DOI: 10.48550/arXiv.2409.09721 . arXiv: 2409.09721 [cs] . URL: http://arxiv.org/ abs/2409.09721. Pre-published. 12

    2024 doi arXiv

  41. [49]

    Improving Image Encoders for General-Purpose Nearest Neighbor Search and Classification

    Konstantin Schall et al. “Improving Image Encoders for General-Purpose Nearest Neighbor Search and Classification”. In: Proceedings of the 2023 ACM International Conference on Multimedia Retrieval. ICMR ’23. Thessaloniki, Greece: Association for Computing Machinery, 2023, pp. ...

    2023

  42. [50]

    Linear Representations of Sentiment in Large Language Models

    Curt Tigges et al. Linear Representations of Sentiment in Large Language Models. Oct. 23,

  43. [51]

    Linear Spaces of Meanings: Compositional Structures in Vision- Language Models

    Matthew Trager et al. “Linear Spaces of Meanings: Compositional Structures in Vision- Language Models”. In: 2023 IEEE/CVF International Conference on Computer Vision (ICCV) (2023), pp. 15349–15358. URL: https : / / api . semanticscholar . org / CorpusId : 257766294

    2023

  44. [53]

    BEYOND DECODABILITY: LIN- EAR FEATURE SPACES ENABLE VISUAL COMPOSITIONAL GENERALIZATION

    Arnas Uselis, Andrea Dittadi, and Seong Joon Oh. “BEYOND DECODABILITY: LIN- EAR FEATURE SPACES ENABLE VISUAL COMPOSITIONAL GENERALIZATION”. In: Workshop on Spurious Correlation and Shortcut Learning: Foundations and Solutions. Mar. 6, 2025. URL: https :/ / openreview. net /for...

    2025

  45. [54]

    Intermediate Layer Classifiers for OOD Generalization

    Arnas Uselis and Seong Joon Oh. “Intermediate Layer Classifiers for OOD Generalization”. In: International Conference on Learning Representations (ICLR). 2025

    2025

  46. [55]

    Order-Embeddings of Images and Language

    Ivan Vendrov et al. “Order-Embeddings of Images and Language”. In:CoRR abs/1511.06361 (2015). URL: https://arxiv.org/pdf/1511.06361.pdf

    2015 arXiv

  47. [56]

    Exploring CLIP for Assessing the Look and Feel of Images

    Jianyi Wang, Kelvin C.K. Chan, and Chen Change Loy. “Exploring CLIP for Assessing the Look and Feel of Images”. In: Proceedings of the Thirty-Seventh AAAI Conference on Artificial Intelligence and Thirty-Fifth Conference on Innovative Applications of Artificial Intelligence an...

    2023 doi

  48. [57]

    Learning-to-Rank Meets Language: Boosting Language-Driven Ordering Alignment for Ordinal Classification

    Rui Wang et al. “Learning-to-Rank Meets Language: Boosting Language-Driven Ordering Alignment for Ordinal Classification”. In: (2023)

    2023

  49. [58]

    Learning-to-rank meets language: Boosting language-driven ordering align- ment for ordinal classification

    Rui Wang et al. “Learning-to-rank meets language: Boosting language-driven ordering align- ment for ordinal classification”. In: Advances in Neural Information Processing Systems 36 (2023)

    2023

  50. [59]

    Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere

    Tongzhou Wang and Phillip Isola. “Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere”. In: ArXiv abs/2005.10242 (2020). URL: https://arxiv.org/pdf/2005.10242.pdf

    2020 arXiv

  51. [60]

    Disentangled representation learning

    Xin Wang et al. “Disentangled representation learning”. In: IEEE Transactions on Pattern Analysis and Machine Intelligence (2024)

    2024

  52. [61]

    ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoen- coders

    Sanghyun Woo et al. ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoen- coders. Jan. 2, 2023. DOI: 10.48550/arXiv.2301.00808. arXiv: 2301.00808 [cs]. URL: http://arxiv.org/abs/2301.00808. Pre-published

    2023 doi arXiv

  53. [62]

    Clip brings better features to visual aesthetics learners

    Liwu Xu et al. “Clip brings better features to visual aesthetics learners”. In: arXiv preprint arXiv:2307.15640 (2023)

    2023 arXiv

  54. [63]

    Filip: Fine-grained interactive language-image pre-training

    Lewei Yao et al. “Filip: Fine-grained interactive language-image pre-training”. In: arXiv preprint arXiv:2111.07783 (2021)

    2021 arXiv

  55. [64]

    Just noticeable differences in visual attributes

    Aron Yu and Kristen Grauman. “Just noticeable differences in visual attributes”. In:Proceed- ings of the IEEE International Conference on Computer Vision. 2015, pp. 2416–2424

    2015

  56. [65]

    CoCa: Contrastive Captioners are Image-Text Foundation Models

    Jiahui Yu et al. “CoCa: Contrastive Captioners are Image-Text Foundation Models”. In: Transactions on Machine Learning Research (2022). URL: https://openreview.net/ forum?id=M_Vb2v063oH

    2022

  57. [66]

    RANKING-AWARE ADAPTER FOR TEXT-DRIVEN IMAGE OR- DERING WITH CLIP

    Wei-Hsiang Yu et al. “RANKING-AWARE ADAPTER FOR TEXT-DRIVEN IMAGE OR- DERING WITH CLIP”. In: ICLR (2025). 13

    2025

  58. [67]

    Ranking-Aware Adapter for Text-Driven Image Ordering with CLIP

    Wei-Hsiang Yu et al. Ranking-Aware Adapter for Text-Driven Image Ordering with CLIP . Dec. 9, 2024. DOI: 10 . 48550 / arXiv . 2412 . 06760. arXiv: 2412 . 06760 [cs] . URL: http://arxiv.org/abs/2412.06760. Pre-published

    2024 doi arXiv

  59. [68]

    When and why vision-language models behave like bags-of-words, and what to do about it?

    Mert Yuksekgonul et al. “When and why vision-language models behave like bags-of-words, and what to do about it?” In: arXiv preprint arXiv:2210.01936 (2022)

    2022 arXiv

  60. [69]

    Single-Image Crowd Counting via Multi-Column Convolutional Neural Network

    Yingying Zhang et al. “Single-Image Crowd Counting via Multi-Column Convolutional Neural Network”. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV , USA: IEEE, Jun...

    2016 doi

  61. [70]

    URL: http://ieeexplore.ieee.org/document/7780439/

  62. [71]

    Learning Ordinal Relationships for Mid-Level Vision

    Daniel Zoran et al. “Learning Ordinal Relationships for Mid-Level Vision”. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV). Dec. 2015. 14 A Further Experimental Details Here, we list further details of the architectures considered in our study. T...

    2015

  63. [73]

    Age Progression/Regression by Conditional Adversarial Autoencoder

    Zhifei Zhang, Yang Song, and Hairong Qi. Age Progression/Regression by Conditional Adversarial Autoencoder. Mar. 28, 2017. DOI: 10 . 48550 / arXiv . 1702 . 08423. arXiv: 1702.08423 [cs]. URL: http://arxiv.org/abs/1702.08423. Pre-published

    2017 doi arXiv

  64. [2023]

    48550 / arXiv

    DOI: 10 . 48550 / arXiv . 2310 . 15154. arXiv: 2310 . 15154 [cs]. URL: http : / / arxiv.org/abs/2310.15154. Pre-published

    doi arXiv

  65. [4056]

    DOI: 10.1109/TIP.2020.2967829

    ISSN : 1057-7149, 1941-0042. DOI: 10.1109/TIP.2020.2967829. arXiv: 1910.06180 [cs]. URL: http://arxiv.org/abs/1910.06180

    1941

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.