pith. sign in

arxiv: 2606.03871 · v1 · pith:QOEL57RKnew · submitted 2026-06-02 · 💻 cs.CV · cs.CL· cs.LG

Visual Instruction Tuning Aligns Modalities through Abstraction

Luis Palacios , Lorenzo Basile , Diego Doimo , Alberto Cazzaniga This is my paper

Pith reviewed 2026-06-28 10:23 UTC · model grok-4.3

classification 💻 cs.CV cs.CLcs.LG
keywords visual instruction tuningmultimodal integrationLLM layersabstractionvision-language modelsfine-tuning localizationsemantic alignment
0
0 comments X

The pith

Visual instruction tuning embeds image features into the intermediate semantic layers of LLMs, skipping the early unimodal layers.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper examines how visual instruction tuning integrates images into pre-trained language models across multiple architectures. It establishes that tuning acts mainly as a bridge, placing visual features straight into the middle layers that handle semantics instead of altering the initial layers used for single-modality work. These middle layers turn out to be the key drivers of success on many vision-language tasks. The work also shows that tuning extends an existing abstraction process to make visual and textual representations match more closely. Finally, the authors test that limiting changes to only those middle layers keeps benchmark performance while cutting training time.

Core claim

Visual instruction tuning serves as a bridge, embedding visual features directly into the intermediate semantic layers of the LLM, bypassing the early layers devoted to unimodal processing. These intermediate layers are the semantic core of vision-language processing and play a critical role in performance on a broad set of multimodal benchmarks. Fine-tuning extends and strengthens the existing abstraction phase, aligning visual features with pre-existing textual ones. Restricting fine-tuning to intermediate layers alone preserves the performance of full fine-tuning on vision-centric benchmarks while reducing training time. Multimodal integration is a localized phenomenon driven by the repur

What carries the argument

The intermediate semantic layers, which receive the embedded visual features and extend the model's existing abstraction phase to align modalities.

If this is right

  • Multimodal performance depends primarily on the intermediate layers rather than uniform changes across the model.
  • Training time can be reduced by updating only the intermediate layers without loss on vision-centric tasks.
  • Visual and textual representations become aligned by extending the abstraction already present in the pre-trained model.
  • Early layers continue to handle unimodal processing even after instruction tuning.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Similar localization of adaptation might occur when adding other data types such as audio or video to the same models.
  • The finding points toward modular designs where semantic alignment can be targeted without retraining the full network.
  • It raises the possibility that the abstraction engine in LLMs can be reused for new modalities with minimal changes.

Load-bearing premise

The probing analyses and causal interventions accurately isolate the contribution of each layer without interference from model architecture choices or dataset specifics.

What would settle it

An experiment in which restricting fine-tuning to early layers alone produces equivalent gains on multimodal benchmarks would falsify the claim that alignment occurs specifically in the intermediate layers.

Figures

Figures reproduced from arXiv: 2606.03871 by Alberto Cazzaniga, Diego Doimo, Lorenzo Basile, Luis Palacios.

Figure 1
Figure 1. Figure 1: Visual and textual abstractions align in the intermediate layers of VLMs. We compare two ways of asking the same question about the same visual content: a multimodal prompt, where the image is encoded and passed to the language model through the connector, and a text-only prompt, where natural-language captions replace the image. For both inputs, we track the hidden representation of the final contextualiz… view at source ↗
Figure 2
Figure 2. Figure 2: Intermediate layers drive vision–language performance. Average performance over six vision–language tasks (GQA, MMBench-cn, MMBench-en, MME, SeedBench, and VQAv2) for LLaVA-7B, OneVision-4B, and InternVL2-8B, normalized to the full model. Purple curves skip image-token layers and green curves knock out image-to-text attention. Solid lines show progressive interventions from early to late layers, while dash… view at source ↗
Figure 3
Figure 3. Figure 3: Probing semantic information across layers. Left: Label overlap of last-token residual￾stream representations on seven 10-option MCQ classification datasets. Right: Chance-adjusted MCQ accuracy from the logit lens (solid) and a linear probe (dashed) on vision–language benchmarks. Colors denote LLaVA-7B (orange), OneVision-4B (blue), and InternVL2-8B (green). network, the performance drops to a random level… view at source ↗
Figure 4
Figure 4. Figure 4: Cross-modal alignment measured by information imbalance (∆). For OneVision-4B (left) and InternVL2-8B (right), we compare hidden representations from multimodal VQA prompts (y-axis) with equivalent text-only prompts where the image is replaced by a caption (x-axis), before and after instruction tuning. Green squares show text-to-image predictability, and purple circles the opposite direction. Larger marker… view at source ↗
Figure 5
Figure 5. Figure 5: Localized fine-tuning helps most when the image matters. Each point is a benchmark. The x-axis measures image reliance: the normalized gap between image-conditioned and text-only performance, averaged over four reference VLMs. The y-axis measures the gain from fine-tuning the abstraction layers instead of their complement, for LLaVA-7B (left) and OneVision-4B (right). Benchmarks that rely more on visual in… view at source ↗
Figure 6
Figure 6. Figure 6: Multimodal ablation profiles for LLaVA-13B, OneVision-8B, and Cambrian-8B. Normalized performance is shown for visual layer-skipping (purple) and cross-modal attention knockout (green) using progressive (solid) and sliding-window (dashed) interventions. The sliding￾window ablation sweep is over 5 contiguous layers. Across all models, performance is resilient to early and late-layer perturbations but collap… view at source ↗
Figure 7
Figure 7. Figure 7: Text-only layer-skipping profiles for LLaVA-7B/13B, Cambrian-8B, OneVision-4B/8B, and InternVL2-8B. Normalized performance is shown for progressive (solid) and sliding-window (dashed, 5-layer window) ablation. While input-side progressive removal triggers immediate perfor￾mance collapse, sliding-window profiles identify specific localized layers critical for maintaining language capabilities. 21 [PITH_FUL… view at source ↗
Figure 8
Figure 8. Figure 8: Progressive layer-skipping on multimodal benchmarks. Normalized performance is plot￾ted against the cumulative depth of layers skipped from the input side for LLaVA-7B/13B, Cambrian￾8B, OneVision-4B/8B, and InternVL2-8B. Colors denote benchmarks: GQA (red), MMBench￾CN/EN (orange/yellow), MME (green), SEEDBench (teal), and VQAv2 (blue). Across all models and datasets, performance collapses once ablation rea… view at source ↗
Figure 9
Figure 9. Figure 9: Sliding-window layer-skipping on multimodal benchmarks. The same benchmark suite is evaluated while skipping a window-width of 5 contiguous layer window. The localized troughs show that performance loss is concentrated when the window covers intermediate layers, while early and late windows leave most multimodal behavior intact. Progressive attention knockout 0 4 8 12 16 20 24 28 32 Layers 0.0 0.2 0.4 0.6 … view at source ↗
Figure 10
Figure 10. Figure 10: Progressive cross-modal attention knockout on multimodal benchmarks. This intervention removes image–text attention connections cumulatively up to the layer on the x-axis. The curves fall at similar intermediate depths, showing that multimodal performance depends specifically on cross-modal communication within the same region, not only on generic layer capacity. 23 [PITH_FULL_IMAGE:figures/full_fig_p023… view at source ↗
Figure 11
Figure 11. Figure 11: Sliding-window cross-modal attention knockout on multimodal benchmarks. Cross￾modal attention is removed only within a moving window of 5 layers. The narrow performance dips show that image–text communication isn’t spread out but localized and exposes task-specific sensitivity to different intermediate depths across models and datasets. 24 [PITH_FULL_IMAGE:figures/full_fig_p024_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Progressive layer-skipping on text-only benchmarks. Normalized performance against unablated models on ARC-Easy (red), CommonsenseQA (orange), HellaSwag (yellow), MMLU (green), PIQA (teal), and WinoGrande (blue). Removing layers cumulatively from the input side collapses language performance almost immediately. Sliding window layer-skipping (all datasets) 0 4 8 12 16 20 24 28 32 Layers 0.0 0.2 0.4 0.6 0.8… view at source ↗
Figure 13
Figure 13. Figure 13: Sliding-window layer-skipping on text-only benchmarks. A window of 5 layers is ablated while the rest of the language pathway remains active. 25 [PITH_FULL_IMAGE:figures/full_fig_p025_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Average Label Overlap across models depth. We compare the nearest-neighbor structure of each layer’s last-token representation with the correct class labels in 10-option image-classification prompts. Each curve is one model, averaged over mini-ImageNet, Food101, SUN397, Caltech101, DTD, Flowers102, and Places365. The shared drop after the intermediate layers indicates that class-level semantic geometry is… view at source ↗
Figure 15
Figure 15. Figure 15: Label overlap profiles across models and datasets. The same label overlap setup as [PITH_FULL_IMAGE:figures/full_fig_p027_15.png] view at source ↗
Figure 16
Figure 16. Figure 16: Linear probing on multiple-choice VQA tasks. We report layer-wise performance values normalized against random-chance on ScienceQA (red), MMBench (yellow), MME (orange), and POPE (green). Accuracy rises from early to intermediate layers and saturates before the final layers, showing when answer-relevant information becomes linearly separable. F.4 Logit Lens on multiple-choice tasks 0 4 8 12 16 20 24 28 32… view at source ↗
Figure 17
Figure 17. Figure 17: Logit lens on the same multiple-choice VQA tasks. Accuracy stays low until later layers and then rises sharply, separating the emergence of linearly available answer information from its alignment to vocabulary logits. Values are normalized against random chance. 28 [PITH_FULL_IMAGE:figures/full_fig_p028_17.png] view at source ↗
Figure 18
Figure 18. Figure 18: Logit lens on open-ended VQA tasks. We measured the top 10-tokens accuracy of the logit lens on GQA (teal), VQAv2 (dark blue), COCO-QA (light blue), and CountBench (purple), where answers are not restricted to fixed multiple-choice options. Correct answers emerge mainly in the final layers, making this a stricter probe of when hidden states become generative outputs. 29 [PITH_FULL_IMAGE:figures/full_fig_… view at source ↗
Figure 19
Figure 19. Figure 19: COCO image-caption retrieval over averaged VLM representations. At each layer, we average the hidden representations from each COCO image and its paired caption to obtain image￾level and text-level embeddings, then run both image-to-text and text-to-image retrieval. The curves report the average over the two retrieval directions for mean percentile rank under Euclidean (orange) and cosine (blue) distances… view at source ↗
Figure 20
Figure 20. Figure 20: Semantic linguistic information peaks in intermediate hidden layers. We probe residual stream representations from LLaVA-7B/13B, LLaVA-OneVision-4B/8B, InternVL2, and Cambrian with the semantic SentEval tasks [75]: coordination inversion (CoordInv) and semantic odd man out (SOMO). For each layer, we train a logistic-regression classifier on the layer representation and report classification accuracy; the … view at source ↗
Figure 21
Figure 21. Figure 21: Information-imbalance planes for LLaVA-1.5. After fine-tuning, the bidirectional infor￾mativeness concentrate in the intermediate abstraction region and reduces the imbalance asymmetry. 1 5 9 13 17 21 25 29 33 Layer (text input) 1 5 9 13 17 21 25 29 33 Layer (multimodal input) Pretrained 1 5 9 13 17 21 25 29 33 Layer (text input) 1 5 9 13 17 21 25 29 33 Finetuned OneVision-4B text image image text < 0.040… view at source ↗
Figure 22
Figure 22. Figure 22: Information-imbalance planes for LLaVA-OneVision-1.5. The low-∆ region becomes denser and more bidirectional around intermediate layers. 1 5 9 13 17 21 25 29 Layer (text input) 1 5 9 13 17 21 25 29 Layer (multimodal input) Pretrained 1 5 9 13 17 21 25 29 Layer (text input) 1 5 9 13 17 21 25 29 Finetuned InternVL2-8B text image image text < 0.020 0.020 < 0.040 0.040 < 0.060 1 5 9 13 17 21 25 29 Layer (text… view at source ↗
Figure 23
Figure 23. Figure 23: Information-imbalance planes for InternVL2-8B and Cambrian-8B. InternVL2-8B shifts the directional imbalance asymmetry to a more symmetrical relative informativeness between modalities concentrated in the abstraction region. The pre-trained checkpoint of Cambrian-8B starts with very high and asymmetrical imbalance values (not shown) biased towards the purely textual inputs, which remain in the more symmet… view at source ↗
Figure 24
Figure 24. Figure 24: Linear CKA between text-only and multimodal representations for LLaVA-1.5. For the 7B and 13B variants, each heatmap compares text-input layers on the x-axis with multimodal-input layers on the y-axis, with darker cells indicating higher Linear CKA, which measures global linear subspace alignment. Values are averaged over the representation-analysis datasets. Fine-tuning turns broad cross-input similarity… view at source ↗
Figure 25
Figure 25. Figure 25: Linear CKA for LLaVA-OneVision-1.5. Visual instruction tuning strengthens the intermediate-to-late cross-input similarity block. 1 5 9 13 17 21 25 29 Layer (text input) 1 5 9 13 17 21 25 29 Layer (multimodal input) Pretrained 1 5 9 13 17 21 25 29 Layer (text input) 1 5 9 13 17 21 25 29 Finetuned 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 InternVL2-8B 1 5 9 13 17 21 25 29 Layer (text input) 1 5 9 13 17 21 25 29 L… view at source ↗
Figure 26
Figure 26. Figure 26: Linear CKA for InternVL2-8B and Cambrian-8B. Both models show stronger cross￾input similarity after instruction tuning, again centered on intermediate and later layers. 33 [PITH_FULL_IMAGE:figures/full_fig_p033_26.png] view at source ↗
Figure 27
Figure 27. Figure 27: Neighborhood Overlap between text-only and multimodal representations for LLaVA￾1.5. This repeats the layer-pair comparison from [PITH_FULL_IMAGE:figures/full_fig_p034_27.png] view at source ↗
Figure 28
Figure 28. Figure 28: Neighborhood Overlap for LLaVA-OneVision-1.5. The post-tuning heatmaps show stronger local-neighborhood agreement in the same intermediate-to-late region highlighted by Linear CKA and information imbalance. 1 5 9 13 17 21 25 29 Layer (text input) 1 5 9 13 17 21 25 29 Layer (multimodal input) Pretrained 1 5 9 13 17 21 25 29 Layer (text input) 1 5 9 13 17 21 25 29 Finetuned 0.1 0.2 0.3 0.4 0.5 InternVL2-8B … view at source ↗
Figure 29
Figure 29. Figure 29: Neighborhood Overlap for InternVL2-8B and Cambrian-8B. The darker post-tuning regions show that local geometry between text-only and multimodal inputs becomes more aligned around intermediate and later layers. 34 [PITH_FULL_IMAGE:figures/full_fig_p034_29.png] view at source ↗
Figure 30
Figure 30. Figure 30: Average performance of different LLaVA-7B fine-tuning strategies. The green curve reports the average performance of LLaVA models trained up to layer N-1, while the orange one reports the average performance of models trained from layer N to the last layer. All values are normalized with respect to the performance of fully trained LLaVA. Points at 0 and 32 represent the extreme values (only training the c… view at source ↗
Figure 31
Figure 31. Figure 31: Detailed performance analysis. Performance of localized fine-tuning strategies on LLaVA-7B and LLaVA-OneVision-1.5-4B across visual question answering, captioning, and com￾positional benchmarks, normalized against the full training baseline. Restricting updates to the intermediate layers proves beneficial for performance in the vast majority of cases. 35 [PITH_FULL_IMAGE:figures/full_fig_p035_31.png] view at source ↗
Figure 32
Figure 32. Figure 32: Cross-attention layer ablations in Llama-3.2-Vision-90B. Left: Average multimodal performance under ablation from the input side and from the output side across cross-attention layers. Center and right panels show the corresponding per-dataset curves for CV-Bench, GQA, MMBench, MME, SEED-Bench, and VQAv2. Performance drops identify the cross-attention depth ranges most important for visual information flo… view at source ↗
Figure 33
Figure 33. Figure 33: Information-imbalance plane for the instruction tuned Llama-3.2-90B-Vision￾Instruct. This instruction-tuned model, whose LLM backbone was not modified but integrates visual features using cross-attention, also depicts a large bidirectional informativeness in the early-to￾mid layers. 36 [PITH_FULL_IMAGE:figures/full_fig_p036_33.png] view at source ↗
Figure 34
Figure 34. Figure 34: Visual reliance predicts localized fine-tuning gains in Llama-3.2-Vision-Instruct￾90B. Each point is a benchmark; the x-axis measures image reliance, and the y-axis measures the performance advantage of fine-tuned cross-attention blocks 13–63 (50 out of 100 cross-attention layers used). The positive association reported in the plot (Spearman ρ = 0.73, p = 0.007) indicates that tasks that depend more on im… view at source ↗
read the original abstract

Visual instruction tuning effectively adapts a pre-trained Large Language Model (LLM) to process image information alongside text. Yet, it remains unclear how visual features are embedded into the layer-wise hierarchy of abstractions of the LLM backbone. Across a diverse set of vision-language architectures, we show that instruction tuning primarily serves as a bridge, embedding visual features directly into the intermediate semantic layers of the LLM, bypassing the early layers devoted to unimodal processing. With probing analyses and causal interventions, we show that these intermediate layers are the semantic core of vision-language processing and play a critical role in the performance on a broad set of multimodal benchmarks. In addition, by comparing the geometry of semantically equivalent visual and textual representations, we find that fine-tuning extends and strengthens the existing abstraction phase, aligning visual features with pre-existing textual ones. Finally, we confirm the functional role of this localized alignment by restricting fine-tuning to intermediate layers alone: this strategy preserves the performance of full fine-tuning on vision-centric benchmarks while reducing training time. Our results suggest that multimodal integration is a localized phenomenon driven by the repurposing of the internal abstraction engine of the LLM.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript claims that visual instruction tuning primarily serves as a bridge embedding visual features directly into the intermediate semantic layers of the LLM backbone (bypassing early unimodal processing layers) across diverse vision-language architectures. This localization is supported by probing analyses, causal interventions, and geometry comparisons of semantically equivalent representations; the intermediate layers are positioned as the semantic core driving performance on multimodal benchmarks. The work further shows that restricting fine-tuning to these layers alone preserves full fine-tuning performance on vision-centric tasks while reducing training time, suggesting multimodal integration is a localized repurposing of the LLM's abstraction engine.

Significance. If the central claims hold, the results would provide a mechanistic account of how instruction tuning achieves cross-modal alignment by extending pre-existing textual abstraction phases rather than creating new pathways. The multi-architecture scope, use of causal interventions, and practical demonstration of layer-restricted fine-tuning constitute strengths that could guide more efficient VLM training and layer-specific interpretability studies.

major comments (2)
  1. [Methods / Results (diverse architectures)] Diverse-architecture experiments (methods and results sections): the paper does not report controls that swap visual-token integration locations (e.g., cross-attention depth or prefix position) while holding other factors fixed. Without such controls, the observed routing of visual features to intermediate layers could be an artifact of the chosen injection mechanism rather than a general consequence of instruction tuning, which is load-bearing for the claim that tuning 'primarily serves as a bridge' bypassing early layers.
  2. [Results (probing and interventions)] Probing analyses and causal interventions (results section): the abstract states these methods support the localization and functional-role claims, yet the manuscript provides no quantitative metrics, error bars, statistical tests, or explicit exclusion criteria for layer selection or intervention targets. This prevents verification that the analyses isolate intermediate-layer contributions without architecture- or dataset-specific confounds.
minor comments (2)
  1. [Abstract] Abstract: no specific benchmark names, performance deltas, or layer indices are given to ground the statements about 'broad set of multimodal benchmarks' and 'preserves the performance of full fine-tuning.'
  2. [Methods] Notation: the term 'intermediate semantic layers' should be defined with explicit layer indices or fractional depth ranges relative to the LLM backbone for each architecture examined.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive comments, which help clarify the scope and presentation of our results. We respond to each major comment below.

read point-by-point responses

  1. Referee: [Methods / Results (diverse architectures)] Diverse-architecture experiments (methods and results sections): the paper does not report controls that swap visual-token integration locations (e.g., cross-attention depth or prefix position) while holding other factors fixed. Without such controls, the observed routing of visual features to intermediate layers could be an artifact of the chosen injection mechanism rather than a general consequence of instruction tuning, which is load-bearing for the claim that tuning 'primarily serves as a bridge' bypassing early layers.

    Authors: The concern is valid: our multi-architecture results compare models whose integration mechanisms differ by design, but do not include within-architecture ablations that isolate injection depth while holding all other factors fixed. Such controls would strengthen the generality claim. We will add a dedicated subsection discussing this limitation and, where feasible with existing model variants, report results from controlled injection-depth sweeps. revision: yes

  2. Referee: [Results (probing and interventions)] Probing analyses and causal interventions (results section): the abstract states these methods support the localization and functional-role claims, yet the manuscript provides no quantitative metrics, error bars, statistical tests, or explicit exclusion criteria for layer selection or intervention targets. This prevents verification that the analyses isolate intermediate-layer contributions without architecture- or dataset-specific confounds.

    Authors: We agree that the current presentation lacks explicit reporting of error bars, statistical tests, and layer-selection criteria. The underlying experiments contain multiple runs and performance thresholds, but these details are not fully documented. In revision we will expand the relevant results subsections to include standard deviations, significance tests where appropriate, and precise criteria used to designate intermediate layers. revision: yes

Circularity Check

0 steps flagged

No circularity: empirical claims rest on external benchmarks and interventions

full rationale

The paper advances an empirical claim about layer-wise effects of instruction tuning, supported by probing analyses, causal interventions, geometric comparisons, and ablation experiments across multiple vision-language architectures. No equations, fitted parameters, or self-referential definitions are present that would make any reported result equivalent to its inputs by construction. The central findings are falsifiable against held-out multimodal benchmarks and do not rely on load-bearing self-citations or ansatzes imported from prior author work. This is the standard case of an experimental study whose derivation chain is self-contained against external data.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The paper rests on standard assumptions about LLM layer hierarchies and the validity of probing techniques; no free parameters, invented entities, or ad-hoc axioms are introduced in the abstract.

axioms (1)
  • domain assumption Large language models organize processing into a layer-wise hierarchy of abstractions with early layers handling unimodal features and intermediate layers handling semantics
    Invoked to interpret where visual features are embedded after tuning

pith-pipeline@v0.9.1-grok · 5734 in / 1256 out tokens · 27368 ms · 2026-06-28T10:23:12.134805+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

75 extracted references · 8 canonical work pages · 8 internal anchors

  1. [1]

    Flamingo: a visual language model for few-shot learning.Advances in neural information processing systems, 35:23716–23736, 2022

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning.Advances in neural information processing systems, 35:23716–23736, 2022

    2022

  2. [2]

    Visual instruction tuning.Advances in neural information processing systems, 36:34892–34916, 2023

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning.Advances in neural information processing systems, 36:34892–34916, 2023

    2023

  3. [3]

    Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond

    Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond.arXiv preprint arXiv:2308.12966, 2023

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  4. [4]

    Gemma 3 Technical Report

    Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, et al. Gemma 3 technical report.arXiv preprint arXiv:2503.19786, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  5. [5]

    Learning transferable visual models from 10 natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from 10 natural language supervision. InInternational conference on machine learning, pages 8748–8763. PmLR, 2021

    2021

  6. [6]

    The geometry of hidden representations of large transformer models

    Lucrezia Valeriani, Diego Doimo, Francesca Cuturello, Alessandro Laio, Alessio Ansuini, and Alberto Cazzaniga. The geometry of hidden representations of large transformer models. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors,Advances in Neural Information Processing Systems, volume 36, pages 51234–51252. Curran Associates...

    2023

  7. [7]

    Emergence of a high-dimensional abstraction phase in language transformers

    Emily Cheng, Diego Doimo, Corentin Kervadec, Iuri Macocco, Lei Yu, Alessandro Laio, and Marco Baroni. Emergence of a high-dimensional abstraction phase in language transformers. InThe Thirteenth International Conference on Learning Representations, 2025

    2025

  8. [8]

    Layer by layer: Uncovering hidden representations in language models

    Oscar Skean, Md Rifat Arefin, Dan Zhao, Niket Nikul Patel, Jalal Naghiyev, Yann LeCun, and Ravid Shwartz-Ziv. Layer by layer: Uncovering hidden representations in language models. InForty-second International Conference on Machine Learning, 2025

    2025

  9. [9]

    Massive supervised fine-tuning experiments reveal how data, layer, and training factors shape LLM alignment quality

    Yuto Harada, Yusuke Yamauchi, Yusuke Oda, Yohei Oseki, Yusuke Miyao, and Yu Takagi. Massive supervised fine-tuning experiments reveal how data, layer, and training factors shape LLM alignment quality. In Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng, editors, Proceedings of the 2025 Conference on Empirical Methods in Natur...

    2025

  10. [10]

    Middle-layer representation alignment for cross-lingual transfer in fine-tuned LLMs

    Danni Liu and Jan Niehues. Middle-layer representation alignment for cross-lingual transfer in fine-tuned LLMs. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar, editors, Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15979–15996, Vienna, Austria, July 20...

    2025

  11. [11]

    Towards interpreting visual information processing in vision-language models

    Clement Neo, Luke Ong, Philip Torr, Mor Geva, David Krueger, and Fazl Barez. Towards interpreting visual information processing in vision-language models. InThe Thirteenth International Conference on Learning Representations, 2025

    2025

  12. [12]

    The semantic hub hypothesis: Language models share semantic representations across languages and modalities

    Zhaofeng Wu, Xinyan Velocity Yu, Dani Yogatama, Jiasen Lu, and Yoon Kim. The semantic hub hypothesis: Language models share semantic representations across languages and modalities. InThe Thirteenth International Conference on Learning Representations, 2025

    2025

  13. [13]

    Cross-modal information flow in multimodal large language models

    Zhi Zhang, Srishti Yadav, Fengze Han, and Ekaterina Shutova. Cross-modal information flow in multimodal large language models. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 19781–19791, 2025

    2025

  14. [14]

    Improved baselines with visual instruction tuning

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 26296–26306, 2024

    2024

  15. [15]

    LLaVA-OneVision-1.5: Fully Open Framework for Democratized Multimodal Training

    Xiang An, Yin Xie, Kaicheng Yang, Wenkang Zhang, Xiuwei Zhao, Zheng Cheng, Yirui Wang, Songcen Xu, Changrui Chen, Didi Zhu, et al. Llava-onevision-1.5: Fully open framework for democratized multimodal training.arXiv preprint arXiv:2509.23661, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  16. [16]

    Internvl2: Better than the best—expanding performance boundaries of open-source multimodal models with the progressive scaling strategy, Jul 2024

    OpenGVLab Team. Internvl2: Better than the best—expanding performance boundaries of open-source multimodal models with the progressive scaling strategy, Jul 2024

    2024

  17. [17]

    Cambrian-1: A fully open, vision-centric exploration of multimodal llms.Advances in Neural Information Processing Systems, 37:87310–87356, 2024

    Peter Tong, Ellis Brown, Penghao Wu, Sanghyun Woo, Adithya Jairam Vedagiri IYER, Sai Charitha Akula, Shusheng Yang, Jihan Yang, Manoj Middepogu, Ziteng Wang, et al. Cambrian-1: A fully open, vision-centric exploration of multimodal llms.Advances in Neural Information Processing Systems, 37:87310–87356, 2024

    2024

  18. [18]

    Llama 3.2: Revolutionizing edge ai and vision with open, customizable models.Meta AI Blog, Sep 25 2024

    Meta AI. Llama 3.2: Revolutionizing edge ai and vision with open, customizable models.Meta AI Blog, Sep 25 2024

    2024

  19. [19]

    Dissecting recall of factual associations in auto-regressive language models

    Mor Geva, Jasmijn Bastings, Katja Filippova, and Amir Globerson. Dissecting recall of factual associations in auto-regressive language models. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 12216–12235, 2023

    2023

  20. [20]

    Causal abstractions of neural networks

    Atticus Geiger, Hanson Lu, Thomas Icard, and Christopher Potts. Causal abstractions of neural networks. Advances in neural information processing systems, 34:9574–9586, 2021

    2021

  21. [21]

    Towards automated circuit discovery for mechanistic interpretability.Advances in Neural Information Processing Systems, 36:16318–16352, 2023

    Arthur Conmy, Augustine Mavor-Parker, Aengus Lynch, Stefan Heimersheim, and Adrià Garriga-Alonso. Towards automated circuit discovery for mechanistic interpretability.Advances in Neural Information Processing Systems, 36:16318–16352, 2023. 11

    2023

  22. [22]

    Svcca: Singular vector canonical correlation analysis for deep learning dynamics and interpretability.Advances in neural information processing systems, 30, 2017

    Maithra Raghu, Justin Gilmer, Jason Yosinski, and Jascha Sohl-Dickstein. Svcca: Singular vector canonical correlation analysis for deep learning dynamics and interpretability.Advances in neural information processing systems, 30, 2017

    2017

  23. [23]

    Similarity of neural network representations revisited

    Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey Hinton. Similarity of neural network representations revisited. InInternational Conference on Machine Learning, pages 3519–3529. PMLR, 2019

    2019

  24. [24]

    Ranking the informa- tion content of distance measures.PNAS nexus, 1(2):pgac039, 2022

    Aldo Glielmo, Claudio Zeni, Bingqing Cheng, Gábor Csányi, and Alessandro Laio. Ranking the informa- tion content of distance measures.PNAS nexus, 1(2):pgac039, 2022

    2022

  25. [25]

    Hierarchical nucleation in deep neural networks.Advances in Neural Information Processing Systems, 33, 2020

    Diego Doimo, Aldo Glielmo, Alessio Ansuini, and Alessandro Laio. Hierarchical nucleation in deep neural networks.Advances in Neural Information Processing Systems, 33, 2020

    2020

  26. [26]

    Llava-next: Improved reasoning, ocr, and world knowledge, January 2024

    Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. Llava-next: Improved reasoning, ocr, and world knowledge, January 2024

    2024

  27. [27]

    Mind the gap: Under- standing the modality gap in multi-modal contrastive representation learning

    Victor Weixin Liang, Yuhui Zhang, Yongchan Kwon, Serena Yeung, and James Zou. Mind the gap: Under- standing the modality gap in multi-modal contrastive representation learning. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors,Advances in Neural Information Processing Systems, volume 35, pages 17612–17625. Curran Associates, Inc., 2022

    2022

  28. [28]

    Representations of text and images align from layer one, 2026

    Evžen Wybitul, Javier Rando, Florian Tramèr, and Stanislav Fort. Representations of text and images align from layer one, 2026

    2026

  29. [29]

    Softmax linear units.Transformer Circuits Thread, 2022

    Nelson Elhage, Tristan Hume, Catherine Olsson, Neel Nanda, Tom Henighan, Scott Johnston, Sheer ElShowk, Nicholas Joseph, Nova DasSarma, Ben Mann, Danny Hernandez, Amanda Askell, Kamal Ndousse, Andy Jones, Dawn Drain, Anna Chen, Yuntao Bai, Deep Ganguli, Liane Lovitt, Zac Hatfield- Dodds, Jackson Kernion, Tom Conerly, Shauna Kravec, Stanislav Fort, Saurav ...

    2022

  30. [30]

    interpreting gpt: the logit lens.LessWrong, 2020

    nostalgebraist. interpreting gpt: the logit lens.LessWrong, 2020

    2020

  31. [31]

    Qwen3-vl technical report, 2025

    Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, Wenbin Ge, Zhifang Guo, Qidong Huang, Jie Huang, Fei Huang, Binyuan Hui, Shutong Jiang, Zhaohai Li, Mingsheng Li, Mei Li, Kaixin Li, Zicheng Lin, Junyang Lin, Xuejing Liu, Jiawei Liu, Chenglong Liu, Yang Liu, Dayiheng Liu, Shixuan ...

    2025

  32. [32]

    Pixtral 12b, 2024

    Pixtral Team. Pixtral 12b, 2024

    2024

  33. [33]

    Position: The platonic representation hypothesis

    Minyoung Huh, Brian Cheung, Tongzhou Wang, and Phillip Isola. Position: The platonic representation hypothesis. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp, editors,Proceedings of the 41st International Conference on Machine Learning, volume 235 ofProceedings of Machine Learn...

    2024

  34. [34]

    A quantitative analysis of semantic information in deep representations of text and images, 2025

    Santiago Acevedo, Andrea Mascaretti, Riccardo Rende, Matéo Mahaut, Marco Baroni, and Alessandro Laio. A quantitative analysis of semantic information in deep representations of text and images, 2025

    2025

  35. [35]

    How visual representa- tions map to language feature space in multimodal llms, 2025

    Constantin Venhoff, Ashkan Khakzar, Sonia Joseph, Philip Torr, and Neel Nanda. How visual representa- tions map to language feature space in multimodal llms, 2025

    2025

  36. [36]

    Same task, different circuits: Disentangling modality-specific mechanisms in VLMs

    Yaniv Nikankin, Dana Arad, Yossi Gandelsman, and Yonatan Belinkov. Same task, different circuits: Disentangling modality-specific mechanisms in VLMs. InThe Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025

    2025

  37. [37]

    Chameleon: Mixed-Modal Early-Fusion Foundation Models

    Chameleon Team. Chameleon: Mixed-modal early-fusion foundation models.arXiv preprint arXiv:2405.09818, 2024. 12

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  38. [38]

    Emu3: Next-token prediction is all you need, 2024

    Xinlong Wang, Xiaosong Zhang, Zhengxiong Luo, Quan Sun, Yufeng Cui, Jinsheng Wang, Fan Zhang, Yueze Wang, Zhen Li, Qiying Yu, Yingli Zhao, Yulong Ao, Xuebin Min, Tao Li, Boya Wu, Bo Zhao, Bowen Zhang, Liangdong Wang, Guang Liu, Zheqi He, Xi Yang, Jingjing Liu, Yonghua Lin, Tiejun Huang, and Zhongyuan Wang. Emu3: Next-token prediction is all you need, 2024

    2024

  39. [39]

    Region-based cluster discrimination for visual representation learning

    Yin Xie, Kaicheng Yang, Xiang An, Kun Wu, Yongle Zhao, Weimo Deng, Zimin Ran, Yumeng Wang, Ziyong Feng, Roy Miles, et al. Region-based cluster discrimination for visual representation learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1793–1803, 2025

    2025

  40. [40]

    The Llama 3 Herd of Models

    Abhimanyu Dubey et al. The llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  41. [41]

    Llama 3.2 vision model card

    Meta AI. Llama 3.2 vision model card. https://github.com/meta-llama/llama-models/blob/ main/models/llama3_2/MODEL_CARD_VISION.md, 2024. Official model card

    2024

  42. [42]

    Mmbench: Is your multi-modal model an all-around player? In European conference on computer vision, pages 216–233

    Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, et al. Mmbench: Is your multi-modal model an all-around player? In European conference on computer vision, pages 216–233. Springer, 2024

    2024

  43. [43]

    MME: A comprehensive evaluation benchmark for multimodal large language models

    Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Jinrui Yang, Xiawu Zheng, Ke Li, Xing Sun, Yunsheng Wu, Rongrong Ji, Caifeng Shan, and Ran He. MME: A comprehensive evaluation benchmark for multimodal large language models. InThe Thirty-ninth Annual Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2025

    2025

  44. [44]

    Seed-bench: Benchmarking multimodal large language models

    Bohao Li, Yuying Ge, Yixiao Ge, Guangzhi Wang, Rui Wang, Ruimao Zhang, and Ying Shan. Seed-bench: Benchmarking multimodal large language models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13299–13308, 2024

    2024

  45. [45]

    Are We on the Right Way for Evaluating Large Vision-Language Models?

    Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Jiaqi Wang, Yu Qiao, Dahua Lin, and Zhao Feng. Are we on the right way for evaluating large vision-language models? arXiv preprint arXiv:2403.20330, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  46. [46]

    Learn to explain: Multimodal reasoning via thought chains for science question answering.Advances in Neural Information Processing Systems, 35:2507–2521, 2022

    Pan Lu, Swaroop Mishra, Tanglin Xia, Liang Qiu, Kai-Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. Learn to explain: Multimodal reasoning via thought chains for science question answering.Advances in Neural Information Processing Systems, 35:2507–2521, 2022

    2022

  47. [47]

    MMMU: A massive multi-discipline multimodal understanding and reasoning benchmark for expert AGI

    Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, Cong Wei, Botao Yu, Ruibin Yuan, Renliang Sun, Ming Yin, Boyuan Zheng, Zhenzhu Yang, Yibo Liu, Wenhao Huang, Huan Sun, Yu Su, and Wenhu Chen. MMMU: A massive multi-discipline multimodal understanding and reasoning benchmark for exp...

    2024

  48. [48]

    MMMU-Pro: A more robust multi-discipline multimodal understanding benchmark

    Xiang Yue, Tianyu Zheng, Yuansheng Ni, Yubo Wang, Kai Zhang, Shengbang Tong, Yuxuan Sun, Botao Yu, Ge Zhang, Huan Sun, Yu Su, Wenhu Chen, and Graham Neubig. MMMU-Pro: A more robust multi-discipline multimodal understanding benchmark. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics, 2025

    2025

  49. [49]

    Evaluating object hallucination in large vision-language models

    Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen. Evaluating object hallucination in large vision-language models. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 292–305, 2023

    2023

  50. [50]

    Grok-1.5 vision preview.https://x.ai/news/grok-1.5v, 2024

    xAI. Grok-1.5 vision preview.https://x.ai/news/grok-1.5v, 2024

    2024

  51. [51]

    NaturalBench: Evaluating vision-language models on natural adversarial samples

    Baiqi Li, Zhiqiu Lin, Wenxuan Peng, Jean de Dieu Nyandwi, Daniel Jiang, Zixian Ma, Simran Khanuja, Ranjay Krishna, Graham Neubig, and Deva Ramanan. NaturalBench: Evaluating vision-language models on natural adversarial samples. InAdvances in Neural Information Processing Systems, 2024

    2024

  52. [52]

    Teaching CLIP to count to ten

    Roni Paiss, Ariel Ephrat, Omer Tov, Shiran Zada, Inbar Mosseri, Michal Irani, and Tali Dekel. Teaching CLIP to count to ten. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 3147–3157, 2023

    2023

  53. [53]

    Making the v in vqa matter: Elevating the role of image understanding in visual question answering

    Yash Goyal, Tejas Khot, Douglas Summers-Stay, Dhruv Batra, and Devi Parikh. Making the v in vqa matter: Elevating the role of image understanding in visual question answering. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 6904–6913, 2017

    2017

  54. [54]

    Gqa: A new dataset for real-world visual reasoning and compositional question answering

    Drew A Hudson and Christopher D Manning. Gqa: A new dataset for real-world visual reasoning and compositional question answering. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6700–6709, 2019. 13

    2019

  55. [55]

    Exploring models and data for image question answering

    Mengye Ren, Ryan Kiros, and Richard Zemel. Exploring models and data for image question answering. Advances in neural information processing systems, 28, 2015

    2015

  56. [56]

    Chartqa: A benchmark for question answering about charts with visual and logical reasoning

    Ahmed Masry, Xuan Long Do, Jia Qing Tan, Shafiq Joty, and Enamul Hoque. Chartqa: A benchmark for question answering about charts with visual and logical reasoning. InFindings of the association for computational linguistics: ACL 2022, pages 2263–2279, 2022

    2022

  57. [57]

    When and why vision-language models behave like bags-of-words, and what to do about it? InInternational Conference on Learning Representations, 2023

    Mert Yuksekgonul, Federico Bianchi, Pratyusha Kalluri, Dan Jurafsky, and James Zou. When and why vision-language models behave like bags-of-words, and what to do about it? InInternational Conference on Learning Representations, 2023

    2023

  58. [58]

    SugarCrepe: Fixing hackable benchmarks for vision-language compositionality

    Cheng-Yu Hsieh, Jieyu Zhang, Zixian Ma, Aniruddha Kembhavi, and Ranjay Krishna. SugarCrepe: Fixing hackable benchmarks for vision-language compositionality. InAdvances in Neural Information Processing Systems, 2023

    2023

  59. [59]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. InEuropean conference on computer vision, pages 740–755. Springer, 2014

    2014

  60. [60]

    Peter Young, Alice Lai, Micah Hodosh, and Julia Hockenmaier. From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions.Transactions of the association for computational linguistics, 2:67–78, 2014

    2014

  61. [61]

    Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge

    Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge.ArXiv, abs/1803.05457, 2018

    work page internal anchor Pith review Pith/arXiv arXiv 2018

  62. [62]

    CommonsenseQA: A question answering challenge targeting commonsense knowledge

    Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. CommonsenseQA: A question answering challenge targeting commonsense knowledge. InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4149–4158, Minneapolis, ...

    2019

  63. [63]

    Hellaswag: Can a machine really finish your sentence? InProceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019

    Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag: Can a machine really finish your sentence? InProceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019

    2019

  64. [64]

    Measuring massive multitask language understanding.Proceedings of the International Conference on Learning Representations (ICLR), 2021

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding.Proceedings of the International Conference on Learning Representations (ICLR), 2021

    2021

  65. [65]

    Piqa: Reasoning about physical commonsense in natural language

    Yonatan Bisk, Rowan Zellers, Ronan Le Bras, Jianfeng Gao, and Yejin Choi. Piqa: Reasoning about physical commonsense in natural language. InThirty-Fourth AAAI Conference on Artificial Intelligence, 2020

    2020

  66. [66]

    WinoGrande: An Adversarial Winograd Schema Challenge at Scale

    Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Winogrande: An adversarial winograd schema challenge at scale.arXiv preprint arXiv:1907.10641, 2019

    work page internal anchor Pith review Pith/arXiv arXiv 1907

  67. [67]

    Larochelle

    Sachin Ravi and H. Larochelle. Optimization as a model for few-shot learning. InInternational Conference on Learning Representations, 2016

    2016

  68. [68]

    Food-101 - mining discriminative components with random forests

    Lukas Bossard, Matthieu Guillaumin, and Luc Van Gool. Food-101 - mining discriminative components with random forests. InEuropean Conference on Computer Vision, 2014

    2014

  69. [69]

    Automated flower classification over a large number of classes.2008 Sixth Indian Conference on Computer Vision, Graphics & Image Processing, pages 722–729, 2008

    Maria-Elena Nilsback and Andrew Zisserman. Automated flower classification over a large number of classes.2008 Sixth Indian Conference on Computer Vision, Graphics & Image Processing, pages 722–729, 2008

    2008

  70. [70]

    Ehinger, Aude Oliva, and Antonio Torralba

    Jianxiong Xiao, James Hays, Krista A. Ehinger, Aude Oliva, and Antonio Torralba. Sun database: Large- scale scene recognition from abbey to zoo.2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pages 3485–3492, 2010

    2010

  71. [71]

    Places: A 10 million image database for scene recognition.IEEE Transactions on Pattern Analysis and Machine Intelligence, 40:1452–1464, 2018

    Bolei Zhou, Àgata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. Places: A 10 million image database for scene recognition.IEEE Transactions on Pattern Analysis and Machine Intelligence, 40:1452–1464, 2018

    2018

  72. [72]

    Li Fei-Fei, Rob Fergus, and Pietro Perona. Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories.2004 Conference on Computer Vision and Pattern Recognition Workshop, pages 178–178, 2004. 14

    2004

  73. [73]

    Describing textures in the wild.2014 IEEE Conference on Computer Vision and Pattern Recognition, pages 3606–3613, 2013

    Mircea Cimpoi, Subhransu Maji, Iasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. Describing textures in the wild.2014 IEEE Conference on Computer Vision and Pattern Recognition, pages 3606–3613, 2013

    2014

  74. [74]

    Lmms-eval: Reality check on the evaluation of large multimodal models

    Kaichen Zhang, Bo Li, Peiyuan Zhang, Fanyi Pu, Joshua Adrian Cahyono, Kairui Hu, Yuhao Dong, Shuai Liu, Yuanhan Zhang, Jingkang Yang, Chunyuan Li, and Ziwei Liu. Lmms-eval: Reality check on the evaluation of large multimodal models. InNorth American Chapter of the Association for Computational Linguistics, 2024

    2024

  75. [75]

    What you can cram into a single $&!#* vector: Probing sentence embeddings for linguistic properties

    Alexis Conneau, German Kruszewski, Guillaume Lample, Loïc Barrault, and Marco Baroni. What you can cram into a single $&!#* vector: Probing sentence embeddings for linguistic properties. In Iryna Gurevych and Yusuke Miyao, editors,Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2126–21...

    2018