Pith. sign in

REVIEW 4 major objections 4 minor 66 references

Seeing What Tastes Good: Revisiting Multimodal Distributional Semantics in the Billion Parameter Era

T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Image-trained encoders rival language models at predicting object attributes, even non-visual ones.

desk verdict Solid, useful probing study with a new dataset and a credible vision-only result; the abstract overstates multimodal wins and the GPT-4o gold labels need a precision check before the strongest claims are trusted. read the letter →

arxiv 2506.03994 v1 pith:OROIZC2A submitted 2025-06-04 cs.CL cs.CV

Dan Oneata , Desmond Elliott , Stella Frank This is my paper
classification cs.CLcs.CV
keywords semanticfeaturenormslinearprobingmultimodaldistributionalsemanticsvisionencoderslanguagemodelsconceptualattributesmodalityconvergenceMcRae×THINGS
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether modern billion-parameter vision models, trained on images alone or with language supervision, know the same things about concrete objects that humans list as semantic features—that a ROSE is red, smells sweet, and is a flower. Using linear probes on frozen representations, it tests 17 encoders against an extended, denser version of the McRae feature norms and the Binder attribute-rating norms. The central finding is that multimodal image encoders slightly outperform language-only models, while image-only encoders perform comparably to large language models even on attributes classified as encyclopedic or functional. The result matters because it challenges the assumption that language supervision or explicit labels are needed to acquire conceptual attribute knowledge, and it supports a picture of vision and language representations converging at scale.

What carries the argument

The load-bearing machinery is linear probing with selectivity control: one logistic-regression probe per McRae×THINGS attribute, one linear-regression probe per Binder rating, trained on frozen averaged representations (mean-pooled image embeddings, or averaged contextual word and sentence embeddings). F1 selectivity subtracts the expected score of a frequency-predicting probe, so reported results are above a random baseline; an untrained SigLIP encoder provides a second control. The new McRae×THINGS benchmark supplies dense binary labels for 278 attributes across 1,854 THINGS concepts by having GPT-4o fill in missing concept–attribute pairs, while Binder supplies human mean ratings on 155 overlapping concepts.

What would settle it

Ask human raters to judge a random sample of the new concept–attribute pairs in McRae×THINGS and compare their answers to the GPT-4o labels; then retrain the probes on a human-validated subset. If the labels show poor precision or the model rankings change on the human-validated subset, the conclusion that image-only encoders match language models on non-visual attributes is an artifact of annotator alignment rather than a fact about human-aligned conceptual knowledge.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that conceptual attributes are broadly linearly decodable from frozen image representations, and that the modality gap shrinks at scale. Image-only encoders (Swin-V2, DINOv2) reach F1-selectivity scores close to those of a 2B language model (Gemma) and a masked encoder (DeBERTa v3) on McRae×THINGS, and this closeness holds not just for visual-perceptual attributes but for function and encyclopedic types. Multimodal encoders (SigLIP, PaliGemma) are the top performers, and per-attribute probe scores are highly correlated across all competent models, which the paper reads as convergence rather than duplication. A surprising corollary is that label-supervised vision models (Max ViT) do visibly worse than self-supervised ones, suggesting the classification objective is not the right supervision for human-aligned attribute knowledge.

Load-bearing premise

The load-bearing premise is that the GPT-4o-generated labels for the new concept–attribute pairs are true human-like norms; the paper checks their recall against ten original McRae norms but never measures how often the new labels are wrong.

Editorial extensions

If this is right

  • If image-only encoders really encode function and encyclopedic attributes, then text is not a prerequisite for broad conceptual knowledge; visual self-supervision alone is a viable knowledge source.
  • Multimodal encoders still lead, so the practical recipe for attribute-rich representations is to keep combining vision and language rather than choose one modality.
  • Self-supervised objectives like masked image modeling appear better than object-label classification for human-aligned attribute knowledge, arguing for rethinking label-supervised pretraining.
  • The high across-model correlations on attribute probes give a new, probing-based line of evidence for convergence between vision and language representations.
  • The dense McRae×THINGS norms extend attribute probing from 541 to 1,854 concrete concepts, enabling finer-grained studies of what models know.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves open that 'non-visual' attributes are predictable from images because they correlate with visible proxies, such as foods looking edible or tools looking graspable; a testable extension is to construct concept pairs matched on appearance but differing on function and see whether image-only probes still separate them.
  • One could test the convergence claim causally by fine-tuning a vision encoder on a single semantic attribute and checking whether a paired language model improves on that attribute without ever seeing its text.
  • The GPT-4o-generated labels should be re-validated for precision; if human raters disagree with the new pairs, absolute probe scores would need recalibration even if relative rankings survive.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper probes frozen representations from vision-only, multimodal, and language-only models for conceptual attribute knowledge using linear classifiers and regressions. It introduces McRae×THINGS, a dense expansion of McRae feature norms to THINGS concepts annotated by GPT-4o, and also evaluates the human-collected Binder attribute ratings. Across 17 models, the paper reports F1 selectivity for McRae×THINGS and RMSE for Binder, and interprets the results as showing that multimodal image encoders slightly outperform language-only approaches and that image-only encoders are comparable to language models, even on non-visual attributes such as encyclopedic and function norms. The paper also analyzes per-attribute-type patterns, cross-model correlations, and best practices for extracting contextualized word representations.

Significance. If the findings hold, they speak directly to current debates about modality convergence and the role of linguistic supervision in grounding conceptual knowledge. The study has notable strengths: a broad model coverage, a transparent linear-probing protocol with a random encoder baseline, per-attribute and per-type analyses on two datasets, released code and data, and an unusually candid appendix of negative results for contextualized-representation extraction. However, the central interpretation is currently supported unevenly across the two datasets: the Binder results show text-only models ahead of multimodal and vision-only models, and the validity of the new McRae×THINGS dataset as a human semantic norm resource is not established. These issues are fixable within the scope of a revision, but they affect the paper's main claims as stated.

major comments (4)
  1. [Section 3 and Appendix A] The McRae×THINGS dataset is constructed from more than 515k GPT-4o binary judgments, and the only validation reported is a recall check on ten original McRae attributes, one per category, yielding 94–100% recall. Recall on original attributes does not measure precision on the many newly added concept–attribute pairs, such as the growth of 'tastes good' from 28 to 335 positive concepts. If GPT-4o over-predicts positive attributes, then all probe scores on this dataset measure model-to-GPT-4o alignment rather than model-to-human conceptual knowledge, which would directly undermine the claim about vision encoders knowing non-visual attributes. I ask the authors to provide a human-validated precision estimate, ideally with inter-annotator agreement, on a random sample of newly created positive and negative pairs, and to re-run the central comparison on the subset of attributes and pairs with high human agreement.
  2. [Abstract and §6.1 (Table 2)] The abstract's claim that multimodal image encoders 'slightly outperform language-only approaches' is not supported by the Binder half of Table 2: Gemma (RMSE 0.67) and DeBERTa v3 (0.68) outperform SigLIP (0.71) and PaliGemma (0.73). On McRae×THINGS, the best multimodal model (SigLIP, F1 selectivity 50.1) beats the best language-only model (Gemma, 49.8) by only 0.3 points. Likewise, the claim that image-only encoders perform comparably to language models even on non-visual attributes is dataset-dependent: on Binder's non-visual domains such as Social, Time, Emotion, and Drive, Figure 9 shows consistently lower RMSE for Gemma and DeBERTa than for the vision encoders. The abstract and §7 should be revised to state the per-dataset pattern rather than a general ordering across modalities.
  3. [Table 2 and §5] The headline comparisons rest on very small differences: 0.3 F1 selectivity on McRae×THINGS and about 0.04 RMSE on Binder between the best multimodal and language-only models. No confidence intervals or significance tests are reported for the aggregate results in Table 2, despite the repeated stratified cross-validation described in §5. Without bootstrap intervals or paired tests across attributes or folds, the conclusion that multimodal encoders 'slightly outperform' language-only models is not statistically supported. The authors should report uncertainty estimates for the aggregate metrics and, where possible, paired significance tests for the key model comparisons.
  4. [§6.2 and Table 3] The paper acknowledges that attributes such as 'tastes good' have extensions that are nearly subsets of a single supercategory, and Table 3 reports correlations up to 0.594 (CLIP image) between per-attribute probing performance and the proportion of the norm's extension belonging to one supercategory. This confound is especially relevant to the claim about encyclopedic and function attributes, because those norms may be predictable from category-level visual properties even if the specific attribute is not linearly encoded. The paper does not provide a control for this confound. I recommend a control condition, such as training the same probes to predict the THINGS supercategory label and comparing, or computing partial correlations of attribute-probe accuracy while controlling for the supercategory concentration. This would substantially strengthen the interpretation that vision encoders encode the attributes themselves rather than a proxy taxonomic signal.
minor comments (4)
  1. [Introduction] The sentence 'there remains an open the question ofhow' should read 'there remains an open question of how'.
  2. [Figure 4] The attribute label 'a gun used for music' in the left panel appears to combine two separate attributes ('a gun' and 'used for music') and should be corrected for readability.
  3. [Limitations and Appendix E] There are small typos: 'We forsee no risks' should be 'We foresee no risks', and in Appendix E 'extracting the representation from the last later' should likely read 'last layer'.
  4. [Table 1] The statement that the models were chosen to be approximately the same size should be qualified, since the encoders range from 86M (DeBERTa) to 669M (Qwen2.5-VL) parameters, and Gemma is a 2B model.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: the probing pipeline is self-contained; the only concern is a GPT-4o label-generation confound, which is a benchmark-validity issue rather than a by-construction reduction.

full rationale

The paper's central claims are empirical probe accuracies on two attribute datasets. The McRae x THINGS extension is generated by GPT-4o 'to annotate whether or not each attribute is a common trait of each concept' (Section 3), with a sanity check that measures recall on original McRae norms. This introduces a data-generation overlap: the labels and the sentence contexts both come from GPT-4o, which shares web-scale training data with the probed text encoders, so the numbers may partly measure alignment with GPT-4o's associations rather than with human norms. That is a legitimate validity threat, but it is not a circularity of the paper's derivation chain. Nothing in the evaluation is fitted to make the conclusion true: the linear probes are standard classifiers trained on frozen representations; the labels are not used to construct the representations; and the paper reports held-out splits. I found no load-bearing self-citation: the only self-citations (Abdou et al. 2021, Liu et al. 2021) support background or limitations and are not the basis of the main result. Thus there is no step where an output is equivalent to an input by definition, and no fitted parameter is renamed as a prediction. Score 0.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

No fitted theory parameters or invented physical entities. The load-bearing choices are procedural: GPT-4o-generated gold labels, attribute filtering thresholds, sentence contexts for LLMs, and the linear probing assumption. These are documented but only partially validated (recall-only check on ten attributes).

free parameters (3)
  • Attribute filtering thresholds = min 5 concepts; cosine > 0.9 merge
    Section 3: attributes with fewer than five concepts are discarded and similar attributes are merged using all-MiniLM-L6-v2 cosine similarity above 0.9; these choices shape the 278-attribute benchmark and affect which attribute types appear.
  • Number of context sentences for language models = 10
    Appendix E: performance peaks with 10 GPT-4o-generated sentences; 50 sentences did not improve, and the choice materially changes LLM probe scores.
  • Validation subset for GPT-4o annotations = 10 attributes (one per category)
    Section 3 and Appendix A: the sanity check covers only ten attributes and measures recall against original McRae pairs, not precision on new THINGS pairs.
assumptions (4)
  • ad hoc to paper GPT-4o labels are reliable gold annotations for the McRae x THINGS dataset
    Section 3 and Appendix A: 515k concept-attribute queries answered by GPT-4o with validation only on recall of ten original attributes; no human precision check.
  • domain assumption Semantic norms are a valid ground truth for conceptual attribute knowledge
    Section 3: the paper itself notes norms are incomplete and biased toward easily lexicalized attributes; the evaluation nevertheless treats them as targets.
  • domain assumption Linear probes are sufficient to measure whether an attribute is encoded
    Limitations: attributes might be encoded nonlinearly; MLPs were not used because datasets are too small.
  • domain assumption Vision models are compared fairly with language models despite different inputs (images vs words)
    Section 5: vision concepts average over many THINGS images while text models use surface labels or GPT-4o sentences, so the comparison assumes these are matched representations of the same concept.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Seeing What Tastes Good: Revisiting Multimodal Distributional Semantics in the Billion Parameter Era." pith.science (2026). https://pith.science/paper/OROIZC2A

@misc{pith2026250603994,
  author       = {Pith},
  title        = {Pith review of: Seeing What Tastes Good: Revisiting Multimodal Distributional Semantics in the Billion Parameter Era},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OROIZC2A}},
  note         = {Machine review of arXiv:2506.03994}
}
read the original abstract

Human learning and conceptual representation is grounded in sensorimotor experience, in contrast to state-of-the-art foundation models. In this paper, we investigate how well such large-scale models, trained on vast quantities of data, represent the semantic feature norms of concrete object concepts, e.g. a ROSE is red, smells sweet, and is a flower. More specifically, we use probing tasks to test which properties of objects these models are aware of. We evaluate image encoders trained on image data alone, as well as multimodally-trained image encoders and language-only models, on predicting an extended denser version of the classic McRae norms and the newer Binder dataset of attribute ratings. We find that multimodal image encoders slightly outperform language-only approaches, and that image-only encoders perform comparably to the language models, even on non-visual attributes that are classified as "encyclopedic" or "function". These results offer new insights into what can be learned from pure unimodal learning, and the complementarity of the modalities.

Figures

Figures reproduced from arXiv: 2506.03994 by the authors.

Figure 1
Figure 1. Given a dataset of concrete concepts, depicted [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Per-attribute Pearson correlation between models on McRae [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Relative rankings of models across the McRae×THINGS and Binder datasets (higher rank is better). The vision models are show in warm colours, language models in cool colours. Dataset differences. Text-only models (espe￾cially Gemma and DeBERTa v3) perform relatively better on the Binder attribute dimensions, as seen in the rankings ( [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Per feature comparison between pairs of models in terms of the F1 selectivity score. Left: Swin-V2 vs [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Results (F [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: The prompt used to collect the McRae×THINGS dataset. SYSTEM: “You are asked to write {num} short sentences about a word (to follow). Answer the request by returning a list of numbered sentences, 1–{num}.” USER: “Write {num} short sentences about {concept}. You must use…
Figure 7
Figure 7. Figure 7: The prompts used to collect sentence contexts for each concept in the [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]
Figure 8
Figure 8. Figure 8: Per-attribute RMSE on Binder attribute ratings, across models. Lower is better. [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]
Figure 9
Figure 9. Figure 9: Results (RMSE) aggregated over attribute domain on the Binder data (note: lower is better). The [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]
Figure 10
Figure 10. Figure 10: Five random predictions of linear probes trained on four attributes. Positive concepts are indicated by [PITH_FULL_IMAGE:figures/full_fig_p017_10.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

66 extracted references · 45 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Mostafa Abdou, Artur Kulmizev, Daniel Hershcovich, Stella Frank, Ellie Pavlick, and Anders S gaard. 2021. Can language models encode perceptual structure without grounding? A case study in color. In Proc. CoNLL

    work page 2021

  4. [4]

    Guillaume Alain and Yoshua Bengio. 2017. Understanding intermediate layers using linear classifier probes. In Proc. ICLR Workshop Track

    work page 2017

  5. [5]

    Aristotle. 4th c. BC / 1928. Categories (Translated by E . M . Edghill )

    work page 1928

  6. [6]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. 2025. Qwen2.5- VL technical report. arXiv preprint arXiv:2502.13923

    arXiv 2025

  7. [7]

    Marco Baroni and Alessandro Lenci. 2008. Concepts and properties in word spaces. Italian Journal of Linguistics, 20(1):55--88

    work page 2008

  8. [8]

    Yonatan Belinkov. 2022. Probing classifiers: Promises, shortcomings, and advances. Computational Linguistics, 48(1):207--219

    2022

Show all 66 references
  1. [9]

    Yael Benn, Anna A Ivanova, Oliver Clark, Zachary Mineroff, Chloe Seikus, Jack Santos Silva, Rosemary Varley, and Evelina Fedorenko. 2023. https://doi.org/10.1093/cercor/bhad289 The language network is not engaged in object categorization . Cerebral Cortex, 33(19):10380--10400

    2023 doi

  2. [10]

    Lucas Beyer, Andreas Steiner, Andr \'e Susano Pinto, Alexander Kolesnikov, Xiao Wang, Daniel Salz, Maxim Neumann, Ibrahim Alabdulmohsin, Michael Tschannen, Emanuele Bugliarello, et al. 2024. Pali G emma: A versatile 3 B VLM for transfer. arXiv preprint arXiv:2407.07726

    2024 arXiv

  3. [11]

    Sudeep Bhatia and Russell Richie. 2024. https://doi.org/10.1037/rev0000319 Transformer networks of human conceptual knowledge. Psychological Review, 131(1):271--306

    2024 doi

  4. [12]

    Binder, Lisa L

    Jeffrey R. Binder, Lisa L. Conant, Colin J. Humphries, Leonardo Fernandino, Stephen B. Simons, Mario Aguilar, and Rutvik H. Desai. 2016. https://doi.org/10.1080/02643294.2016.1147426 Toward a brain-based componential semantic representation . Cognitive Neuropsychology, 33(3-4)...

    2016

  5. [13]

    Rishi Bommasani, Kelly Davis, and Claire Cardie. 2020. https://doi.org/10.18653/v1/2020.acl-main.431 Interpreting pretrained contextualized representations via reductions to static embeddings . In Proc. ACL

    2020 doi

  6. [14]

    Bruni, N

    E. Bruni, N. K. Tran, and M. Baroni. 2014. https://doi.org/10.1613/jair.4135 Multimodal Distributional Semantics . Journal of Artificial Intelligence Research, 49:1--47

    2014 doi

  7. [15]

    Gabriella Chronis, Kyle Mahowald, and Katrin Erk. 2023. A method for studying semantic construal in grammatical constructions with interpretable contextual embedding spaces. In Proc. ACL

    2023

  8. [16]

    Guillem Collell and Marie-Francine Moens. 2016. Is an image worth more than a thousand words? O n the fine-grain semantic differences between visual and linguistic representations. In Proc. COLING

    2016

  9. [17]

    Cree and Ken McRae

    George S. Cree and Ken McRae. 2003. https://doi.org/10.1037/0096-3445.132.2.163 Analyzing the factors underlying the structure and computation of the meaning of chipmunk, cherry, chisel, cheese, and cello (and many other such concrete nouns). Journal of Experimental Psychology...

    2003 doi

  10. [18]

    Steven Derby. 2022. Interpretable Semantic Representations from Neural Language Models and Computer Vision. Ph.D. thesis, Queen's University, Belfast

    2022

  11. [19]

    Steven Derby, Paul Miller, Brian Murphy, and Barry Devereux. 2018. https://doi.org/10.18653/v1/K18-1026 Using sparse semantic embeddings learned from multimodal text and image data to model human conceptual knowledge . In Proc. CoNLL

    2018 doi

  12. [20]

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. 2020. An image is worth 16x16 words: Transformers for image recognition at scale. In Proc. ICLR

    2020

  13. [21]

    Luana Fagarasan, Eva Maria Vecchi, and Stephen Clark. 2015. From distributional semantics to feature norms: Grounding semantic models in human perceptual data. In Proc. IWCS

    2015

  14. [22]

    Maxwell Forbes, Ari Holtzman, and Yejin Choi. 2019. Do neural language representations learn physical commonsense? Proc. CogSci

    2019

  15. [23]

    Peter G \"a rdenfors. 2000. https://doi.org/10.7551/mitpress/2076.001.0001 Conceptual Spaces: The Geometry of Thought . The MIT Press

    2000 doi

  16. [24]

    Hannes Hansen and Martin N. Hebart. 2022. https://arxiv.org/abs/2202.03753 Semantic features of object concepts generated with GPT-3 . In Proc. CogSci

    2022 arXiv

  17. [25]

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Doll \'a r, and Ross Girshick. 2022. Masked autoencoders are scalable vision learners. In Proc. CVPR

    2022

  18. [26]

    Martin N Hebart, Oliver Contier, Lina Teichmann, Adam H Rockter, Charles Y Zheng, Alexis Kidder, Anna Corriveau, Maryam Vaziri-Pashkam , and Chris I Baker. 2023. https://doi.org/10.7554/eLife.82580 THINGS-data , a multimodal collection of large-scale datasets for investigating...

    2023 doi

  19. [27]

    Hebart, Adam H

    Martin N. Hebart, Adam H. Dickter, Alexis Kidder, Wan Y. Kwok, Anna Corriveau, Caitlin Van Wicklin, and Chris I. Baker. 2019. https://doi.org/10.1371/journal.pone.0223792 THINGS : A database of 1,854 object concepts and more than 26,000 naturalistic object images . PLOS ONE, 1...

    2019 doi

  20. [28]

    John Hewitt and Percy Liang. 2019. https://doi.org/10.18653/v1/D19-1275 Designing and interpreting probes with control tasks . In Proc. EMNLP-IJCNLP

    2019 doi

  21. [29]

    Felix Hill, Roi Reichart, and Anna Korhonen. 2015. https://doi.org/10.1162/COLI_a_00237 Simlex-999: Evaluating semantic models with (genuine) similarity estimation . Computational Linguistics, 41(4):665--695

    2015 doi

  22. [30]

    Minyoung Huh, Brian Cheung, Tongzhou Wang, and Phillip Isola. 2024. The P latonic representation hypothesis. In Proc. ICML

    2024

  23. [31]

    Dieuwke Hupkes, Sara Veldhoen, and Willem Zuidema. 2018. https://doi.org/10.1613/jair.1.11196 Visualisation and 'diagnostic classifiers' reveal how recurrent and recursive neural networks process hierarchical structure . Journal of Artificial Intelligence Research, 61:907--926

    2018 doi

  24. [32]

    Ivanova and Matthias Hofer

    Anna A. Ivanova and Matthias Hofer. 2020. Linguistic overhypotheses in category learning: Explaining the label advantage effect. In Proc. CogSci

    2020

  25. [33]

    Jiaang Li, Yova Kementchedjhieva, Constanza Fierro, and Anders S gaard. 2024. https://doi.org/10.1162/tacl_a_00698 Do vision and language models share concepts? A vector space alignment study . TACL, 12:1232--1249

    2024 doi

  26. [34]

    Fangyu Liu, Emanuele Bugliarello, Edoardo Maria Ponti, Siva Reddy, Nigel Collier, and Desmond Elliott. 2021. Visually grounded reasoning across languages and cultures. In Proc. EMNLP

    2021

  27. [35]

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. 2024. Improved baselines with visual instruction tuning. In Proc. CVPR

    2024

  28. [36]

    Ze Liu, Han Hu, Yutong Lin, Zhuliang Yao, Zhenda Xie, Yixuan Wei, Jia Ning, Yue Cao, Zheng Zhang, Li Dong, et al. 2022. Swin transformer v2: Scaling up capacity and resolution. In Proc. CVPR

    2022

  29. [37]

    Li Lucy and Jon Gauthier. 2017. https://doi.org/10.18653/v1/W17-2810 Are distributional representations ready for the real world? E valuating word vectors for grounded perceptual meaning . In Proceedings of the First Workshop on Language Grounding for Robotics

    2017 doi

  30. [38]

    Gary Lupyan. 2012. https://doi.org/10.3389/fpsyg.2012.00054 Linguistically modulated perception and cognition: The label-feedback hypothesis . Frontiers in Psychology, 3

    2012 arXiv

  31. [39]

    Florian P Mahner, Lukas Muttenthaler, Umut G \"u c l \"u , and Martin N Hebart. 2024. Dimensions underlying the representational alignment of deep neural networks with humans. arXiv preprint arXiv:2406.19087

    2024 arXiv

  32. [40]

    Cree, Mark S

    Ken McRae, George S. Cree, Mark S. Seidenberg, and Chris Mcnorgan. 2005. https://doi.org/10.3758/BF03192726 Semantic feature production norms for a large set of living and nonliving things . Behavior Research Methods, 37(4):547--559

    2005 doi

  33. [41]

    Jack Merullo, Louis Castricato, Carsten Eickhoff, and Ellie Pavlick. 2023. https://doi.org/10.48550/arXiv.2209.15162 Linearly mapping from image to text space . In Proc. ICLR

    2023 doi arXiv

  34. [42]

    Tomas Mikolov, Edouard Grave, Piotr Bojanowski, Christian Puhrsch, and Armand Joulin. 2018. Advances in pre-training distributed word representations. In Proc. LREC

    2018

  35. [43]

    Kanishka Misra, Julia Rayz, and Allyson Ettinger. 2023. COMPS : Conceptual minimal pair sentences for testing robust property knowledge and its inheritance in pre-trained language models. In Proc. EACL

    2023

  36. [44]

    Kanishka Misra, Julia Taylor Rayz, and Allyson Ettinger. 2022. A property induction framework for neural language models. In Proc. CogSci

    2022

  37. [45]

    Th \'e o Moutakanni, Maxime Oquab, Marc Szafraniec, Maria Vakalopoulou, and Piotr Bojanowski. 2024. You don't need domain-specific data augmentations when scaling self-supervised learning. In Proc. NeurIPS

    2024

  38. [46]

    Vandermeulen, and Simon Kornblith

    Lukas Muttenthaler, Jonas Dippel, Lorenz Linhardt, Robert A. Vandermeulen, and Simon Kornblith. 2023. Human alignment of neural network representations. In Proc. ICLR

    2023

  39. [47]

    Nosofsky, Craig A

    Robert M. Nosofsky, Craig A. Sanders, Brian J. Meagher, and Bruce J. Douglas. 2018. https://doi.org/10.3758/s13428-017-0884-8 Toward the development of a feature-space representation for a complex natural category domain . Behavior Research Methods, 50(2):530--556

    2018 doi

  40. [48]

    Maxime Oquab, Timoth \'e e Darcet, Th \'e o Moutakanni, Huy V Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. 2024. DINO v2: Learning robust visual features without supervision. Transactions on Machine Learning Research

    2024

  41. [49]

    Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014. Glove: Global vectors for word representation. In Proc. EMNLP

    2014

  42. [50]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In Proc. ICML

    2021

  43. [51]

    Eleanor Rosch and Carolyn B Mervis. 1975. https://doi.org/10.1016/0010-0285(75)90024-9 Family resemblances: Studies in the internal structure of categories . Cognitive Psychology, 7(4):573--605

    1975 doi

  44. [52]

    Dana Rubinstein, Effi Levi, Roy Schwartz, and Ari Rappoport. 2015. https://doi.org/10.3115/v1/P15-2119 How well do distributional models capture different types of semantic knowledge? In Proc. ACL-IJCNLP

    2015 doi

  45. [53]

    Piyush Sharma, Nan Ding, Sebastian Goodman, and Radu Soricut. 2018. Conceptual C aptions: A cleaned, hypernymed, image alt-text dataset for automatic image captioning. In Proc. ACL

    2018

  46. [54]

    Carina Silberer, Vittorio Ferrari, and Mirella Lapata. 2013. Models of semantic representation with visual attributes. In Proc. ACL

    2013

  47. [55]

    Pia Sommerauer and Antske Fokkens. 2018. Firearms and tigers are dangerous, kitchen knives and zebras are not: Testing whether word embeddings can tell. In Proc. EMNLP Workshop BlackboxNLP : Analyzing and Interpreting Neural Networks for NLP

    2018

  48. [56]

    Robyn Speer, Joshua Chin, and Catherine Havasi. 2017. ConceptNet 5.5: An open multilingual graph of general knowledge. In Proc. AAAI

    2017

  49. [57]

    Siddharth Suresh, Wei-Chun Huang, Kushin Mukherjee, and Timothy T Rogers. 2024. Categories vs semantic features: What shapes the similarities people discern in photographs of objects? In Proc. ICLR Workshop on Representational Alignment

    2024

  50. [58]

    Gemma Team, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivi \`e re, Mihir Sanjay Kale, Juliette Love, et al. 2024. Gemma: Open models based on G emini research and technology. arXiv preprint arXiv:2403.08295

    2024 arXiv

  51. [59]

    Zhengzhong Tu, Hossein Talebi, Han Zhang, Feng Yang, Peyman Milanfar, Alan Bovik, and Yinxiao Li. 2022. Max VIT : Multi-axis vision transformer. In Proc. ECCV

    2022

  52. [60]

    Jacob Turton, David Vinson, and Robert Smith. 2020. Extrapolating binder style word embeddings to new words. In Proc. Workshop on Linguistic and Neurocognitive Resources

    2020

  53. [61]

    Akira Utsumi. 2020. https://doi.org/10.1111/cogs.12844 Exploring what is encoded in distributional word vectors: A neurobiologically motivated analysis . Cognitive Science, 44(6):e12844

    2020 doi

  54. [62]

    Ivan Vuli \'c , Edoardo Maria Ponti, Robert Litschko, Goran Glava s , and Anna Korhonen. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.586 Probing pretrained language models for lexical semantics . In Proc. EMNLP

    2020 doi

  55. [63]

    Waxman and Dana B

    Sandra R. Waxman and Dana B. Markow. 1995. https://doi.org/10.1006/cogp.1995.1016 Words as invitations to form categories: Evidence from 12- to 13-month-old infants . Cognitive Psychology, 29(3):257--302

    1995

  56. [64]

    Zhenda Xie, Zheng Zhang, Yue Cao, Yutong Lin, Jianmin Bao, Zhuliang Yao, Qi Dai, and Han Hu. 2022. Sim MIM : A simple framework for masked image modeling. In Proc. CVPR

    2022

  57. [65]

    Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. 2023. Sigmoid loss for language image pre-training. In Proc. ICCV

    2023

  58. [66]

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. 2023. Judging LLM -as-a-judge with MT -bench and C hatbot A rena. In Proc. NeurIPS

    2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.