Pith. sign in

REVIEW 4 major objections 5 minor 19 references

A small subset of attention heads in a vision-language model carries spatial-relation knowledge, and the extracted function vectors improve zero-shot accuracy, can be fine-tuned with frozen weights, and combine linearly to handle novel rela

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 12:38 UTC pith:OGZLB6OF

load-bearing objection Extends function vectors to an LMM with honorable mechanics but an inconsistent abstract and an unresolved positional-shortcut confound in the synthetic task. the 4 major comments →

arxiv 2510.02528 v2 pith:OGZLB6OF submitted 2025-10-02 cs.AI cs.LG

Multimodal Function Vectors for Visual Relations

Shuhao Fu , Esther Goldberg , Ying Nian Wu , Hongjing Lu This is my paper
classification cs.AI cs.LG
keywords function vectorsmechanistic interpretabilityvision-language modelsspatial relationscausal mediation analysisin-context learninglinear representation hypothesisrelational reasoning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper tries to establish that spatial relational knowledge in a vision-language model is localized in a small set of attention heads, and that the task representation can be extracted as a 'function vector' — a sum of averaged activations from those heads. The authors show that inserting such a vector into the model's intermediate layers improves zero-shot accuracy on above/below/left-of/right-of questions, that fine-tuning only the vector (while keeping the model frozen) outperforms in-context learning, and that linearly combining relation-specific vectors solves analogies for untrained relations such as above-left. If true, this provides evidence for modularity in multimodal reasoning and a practical way to steer LMMs without retraining.

Core claim

The central claim is that a sparse set of causally influential attention heads in a frozen vision-language model encodes visual spatial relations in a compact, manipulable form. Using causal mediation analysis, the paper identifies heads whose activations, when replaced by relation-conditional averages, most increase the probability of a correct relational prediction. Summing those average activations yields a function vector; adding it at the final token of a zero-shot prompt triggers the model to answer in the intended relation. Fine-tuning the vector on held-out examples, with all model parameters frozen, more than doubles zero-shot accuracy and beats few-shot in-context learning. The pap

What carries the argument

Function vector: for each relation, the sum of mean activations from the top attention heads selected by Average Indirect Effect (AIE), a causal-mediation score measuring how much replacing a head's activation with a relation-conditional average increases the probability of the correct answer. The vector is injected into the hidden state at the final token position of a chosen layer; fine-tuning optimizes only this vector, not the backbone; composite vectors are weighted sums of relation vectors, with weights set by the model's own probability of a source analogy.

Load-bearing premise

The experiments assume that the synthetic task isolates genuine relational reasoning, but because each answer object is placed at a fixed pixel offset (directly above, below, left, or right of the reference), the model could solve the task with a positional shortcut; the paper does not include a control that rules out this low-level confound.

What would settle it

A decisive control would keep all object labels and positions the same but randomize which object is designated the reference (so the correct answer relation changes while the pixels are unchanged). If the function vector still drives the original prediction despite the changed ground truth, the vector encodes a visual-relation meaning; if the prediction tracks absolute position instead of the designated reference, the claimed relational knowledge reduces to a coordinate heuristic.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • Function vector intervention raises zero-shot relational accuracy well above the unmodified model.
  • Fine-tuning the vector with frozen backbone outperforms 4-shot in-context learning on both synthetic and real-image datasets.
  • Relational knowledge is concentrated in a small set of heads (6–12); too few heads under-represent the task and too many add noise.
  • Composite function vectors transfer relational knowledge to untrained relations, nearly doubling one-shot analogy accuracy compared with one-shot ICL.
  • Injection at intermediate layers works best, suggesting the vectors trigger distributed nonlinear computation rather than acting as simple linear modifiers.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the localization result holds beyond the tested architecture and relation set, it suggests a general route to inspect and steer multimodal reasoning: find causally implicated heads, extract the vector, and add or subtract relational knowledge at inference time.
  • The linear-combination result implies that the space of spatial relations may have an additive geometry; a testable extension is whether vectors for fine-grained directions (e.g., 30° offsets) are predictable from interpolated weights, or whether they require separate extraction.
  • A natural follow-up is to test whether the same extraction works for non-spatial relations (physical, agentic, social), where the input features are less confounded with absolute pixel positions; success would strengthen the claim that the vectors encode relations rather than coordinates.
  • Because the authors rely on a frozen backbone, function-vector fine-tuning offers a lightweight adaptation strategy: updating a few thousand parameters instead of the whole model, which could scale to resource-constrained deployment.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper extends the function-vector framework to large multimodal models, focusing on spatial relations. Using a synthetic dataset (above/below/left/right) and a real-image GQA subset, it applies causal mediation (AIE) to select a small set of attention heads in OpenFlamingo-4B, defines multimodal function vectors from those heads, and shows that injecting a relation-specific vector into zero-shot prompts improves accuracy. It further fine-tunes these vectors with the backbone frozen and reports gains over zero-shot and 4-shot ICL baselines. Finally, it constructs composite function vectors by probability-weighted linear combinations of four cardinal relation vectors and evaluates them on one-shot analogies with untrained diagonal relations. The abstract claims results on two LMMs, OpenFlamingo and Qwen3-VL, but the body and supplement report only OpenFlamingo-4B.

Significance. If the central claims hold after methodological tightening, this is a useful contribution: it extends LLM function-vector analysis to vision-language models, uses causal interventions to localize relational knowledge, shows that task vectors can be fine-tuned while keeping the backbone frozen, and tests compositional transfer to novel relations. The reported code/data release and the inclusion of real-image GQA experiments are strengths. However, two issues currently undercut the headline claims: the synthetic task may be solvable by screen-coordinate heuristics, and the main hyperparameters (injection layer, head count) appear to be selected on the same test set used for the reported accuracies. The real-image experiments partially mitigate the first concern but do not resolve the coordinate-confounding of the central mechanistic and composition results.

major comments (4)
  1. [Supplementary A.1.1; §3.1] The synthetic task fixes the mapping between relation labels and screen coordinates: the reference object is placed in a central 400×400 region and the four relational objects are positioned 'directly above, below, left, and right' at deterministic offsets. A classifier on bounding-box centers or pixel offsets can solve this task without any view-invariant relational representation. The paper provides no control that varies the viewing frame, layout geometry, or label–position mapping, so the AIE maps and extracted FVs may encode coordinate-direction biases rather than spatial relational knowledge. This is load-bearing for the mechanism claim and for the diagonal composition result. Please add controls such as global image rotations/translations, shuffled label–position assignments, or varied scales, or explicitly re-scope the claims to coordinate-frame-relative direction.
  2. [§4.2; Supplementary A.3.1–A.3.2] The reported main results appear to use hyperparameters selected on the same zero-shot evaluation set. Figure 9 is used to choose the injection layer (ℓ=19 for synthetic, ℓ=8 for GQA) and Figure 10 to choose the head-count range, and these values then feed into the Figure 5 accuracy comparisons. Selecting ℓ and k on the test set inflates the reported gains and invalidates the claim that fine-tuned function vectors 'significantly outperform' ICL baselines. Use a separate validation split for selecting these hyperparameters, and report means with error bars and significance tests across random seeds and dataset splits.
  3. [Abstract; §5] The abstract claims experiments on two LMMs, 'including OpenFlamingo and Qwen3-VL,' but Qwen3-VL is never mentioned again in the main text or supplement, and the conclusion explicitly states that the study used a single architecture, OpenFlamingo-4B. This is not a trivial wording issue: the cross-architecture generality claim in the abstract is unsupported by any results in the manuscript. Either add the Qwen3-VL experiments with full methodological details or revise the abstract and conclusion consistently.
  4. [§3.3.5; Figure 6] The composite-function-vector (CFV) experiment is carried out only under the same generative layout used in training, where diagonal relations are deterministically recoverable from horizontal and vertical offsets. The weights w_t are obtained from the model's own probabilities under each component FV, but there is no comparison to simple baselines (equal weights, random weights, or the best single component), no oracle-weight upper bound, and no test of transfer across viewing conditions or to relations not decomposable into the four cardinal directions. As it stands, the analogy result is consistent with a coordinate-composition artifact. Please add these controls to support the claimed compositional generalization.
minor comments (5)
  1. [Eq. (2); §4.1] Equation (2) defines v_t as the sum of the selected head means, but §4.1 says the vector is obtained by averaging the activations of the top 10 heads. The scaling may be immaterial, but the two definitions should be harmonized for reproducibility.
  2. [§4.1; Figure 4 caption] The text says the two displayed relations are 'above and left of,' but the caption says the right panel is for 'right of.' Please correct the mismatch.
  3. [§3.3.2] The text introduces a 'causal indirect effect (CIE)' and then switches to 'average indirect effect (AIE).' Use consistent terminology throughout.
  4. [§3.1; Supplementary A.1.2] For GQA, 200 tasks per relation are sampled from only 100 test images, so the same images/object pairs recur across tasks and test examples are not independent. Report per-image or clustered accuracy, or subsample tasks to avoid this dependence.
  5. [Throughout] There are numerous typos: 'approch' (p.2), 'Followng' (p.3), 'rans' (p.4), 'relation-secific' (p.5), 'lable' (p.6). A careful proofreading pass is needed.

Circularity Check

0 steps flagged

No equation-level circularity; extraction and evaluation use disjoint splits; mild self-referentiality in composite weights is not construction-level.

full rationale

The paper's central chain is not circular. The relation-specific mean activations (Eq. 1) are computed from in-context prompts in an extraction split; the function vector (Eq. 2) is a sum of selected-head means; the zero-shot intervention (Eq. 3) is applied to disjoint prompts with no demonstrations. The held-out evaluation makes the accuracy gain an empirical transfer result rather than an identity. Fine-tuning (Eq. 4) uses a separate training set with a frozen backbone and is tested on held-out data, so it is standard supervised adaptation, not a fitted input renamed as a prediction. Composite vectors (Eqs. 5-6) derive weights from the model's own probabilities on the source analogy, but the target answer is never used to construct the composite, so the target prediction is not forced by construction. The strongest caveats are not circularity: the synthetic dataset (Supp. A.1.1) fixes objects at exact cardinal offsets, so a coordinate-direction shortcut could explain behavior without frame-invariant relational knowledge—a validity concern, not a derivation-level circularity; and the composite-weight procedure is mildly self-referential because it uses the same model being intervened on. No load-bearing self-citation or imported uniqueness theorem appears. The paper is self-contained against held-out benchmarks, so the overall circularity score is low.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 1 invented entities

The paper's central claims rest on several domain assumptions inherited from the function-vector literature and on hyperparameters (head count, injection layer) chosen post hoc. No new physical entities or forces are introduced; the 'function vector' is a representational construct.

free parameters (3)
  • Number of attention heads k = 10
    Selected because ablations showed a peak at 6–14 heads (Fig 10); a post-hoc hyperparameter, not derived.
  • Injection layer ℓ = 19 (synthetic), 8 (GQA)
    Chosen as the peak of the layer-ablation plot (Fig 9), which appears to be on the evaluation set; risks selection bias.
  • Fine-tuning hyperparameters = lr=0.001, 20 epochs, Adam, cosine schedule
    Standard settings, not extensively tuned; still free parameters of the fine-tuning procedure.
axioms (4)
  • domain assumption Average Indirect Effect (AIE) identifies causally influential attention heads for a task.
    Imported from Todd et al. (2024). Assumes that replacing uninformative-context activations with relation-conditioned means isolates task-relevant circuits.
  • domain assumption Linear representation hypothesis: high-level concepts are represented linearly in model internals.
    Invoked in §3.3.5 to justify linearly combining function vectors for novel relations; an empirical hypothesis, not proven.
  • domain assumption The synthetic spatial-relation task is representative of 'visual relational knowledge'.
    The paper generalizes from four rigid spatial relations to visual relations broadly, but the task is highly constrained and may admit positional shortcuts.
  • domain assumption Top-1 token accuracy is a valid measure of relational reasoning.
    The model is scored on the first token of the object label; no chance-level comparison or error bars are provided.
invented entities (1)
  • Multimodal function vector (FV) no independent evidence
    purpose: Compact task vector for a spatial relation, built as the sum of mean activations of the top AIE attention heads.
    It is an aggregate of the model's own activations; it has no falsifiable handle outside the paper other than the reported interventions, which are internal to the model.

pith-pipeline@v1.3.0-alltime-deepseek · 12758 in / 12189 out tokens · 122102 ms · 2026-08-04T12:38:11.266260+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of Multimodal Function Vectors for Visual Relations." pith.science (2026). https://pith.science/paper/OGZLB6OF

@misc{pith2026251002528,
  author       = {Pith},
  title        = {Pith review of: Multimodal Function Vectors for Visual Relations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OGZLB6OF}},
  note         = {Machine review of arXiv:2510.02528}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Large Multimodal Models (LMMs) demonstrate impressive in-context learning abilities from few multimodal demonstrations, yet the internal mechanisms supporting such task learning remain opaque. Building on prior work of Large Language Models, we show that a small subset of attention heads in Large Multimodal Models is responsible for transmitting representations of visual relations. The activations of these attention heads, termed function vectors, can be extracted and manipulated to alter an LMM's performance on relational tasks. First, using synthetic and real image datasets, we apply causal mediation analysis to identify attention heads that strongly influence relational predictions, and extract multimodal function vectors that improve zero-shot accuracy at inference time. We further demonstrate that these multimodal function vectors can be fine-tuned with a modest amount of training data, while keeping LMM parameters frozen, to significantly outperform in-context learning baselines. Finally, we show that relation-specific function vectors can be linearly combined to solve analogy problems involving novel and untrained visual relations, highlighting the strong generalization ability of this approach. Through experiments on two LMMs, including OpenFlamingo and Qwen3-VL, our results show that these models encode visual relational knowledge within localized internal structures, which can be systematically extracted and optimized, thereby advancing our understanding of model modularity and enhancing control over relational reasoning in LMMs.

Figures

Figures reproduced from arXiv: 2510.02528 by Esther Goldberg, Hongjing Lu, Shuhao Fu, Ying Nian Wu.

Figure 1
Figure 1. Figure 1: Relational rep￾resentations enrich per￾ception: rather than a disconnected list of ob￾jects, relations (e.g., the boy opening the fridge next to the cabinet) pro￾vide a structured, mean￾ingful description. Despite the promise of function vectors in LLMs, their extension to mul￾timodal settings remains at an early stage. LMMs such as Flamingo (Alayrac et al., 2022) or BLIP (Li et al., 2022) introduce additi… view at source ↗
Figure 2
Figure 2. Figure 2: Example 4-shot in-context learning (ICL) prompts for relation understanding. Each prompt includes four demonstrations followed by a query. We compare the model’s performance in a consistent relational setting (A) versus a perturbed setting (B) to isolate components responsible for relational inference. To quantify the overall contribution of an attention head in processing a specific relation, we compute i… view at source ↗
Figure 3
Figure 3. Figure 3: Illustration of the composite function vector approach for one-shot analogy tasks. In [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Average indirect effect (AIE) scores of attention heads for two spatial relations. Left panel for above relation, right panel for right of relation Each heatmap shows the AIE scores of attention heads indexed by layer and head position. Pink boxes mark the top 10 most causally influential attention heads. 4 EXPERIMENTS 4.1 IDENTIFYING CAUSALLY IMPORTANT ATTENTION HEADS FOR SPATIAL RELATIONS We first comput… view at source ↗
Figure 5
Figure 5. Figure 5: Top-1 prediction accuracy of zero-shot relation tasks for four models: zero-shot base￾line of LMM, 4-shot ICL of LMM, initial function vector, and fine-tuned function vector. Fine-tuned vectors significantly outperform all baselines on the held-out zero-shot test set. Layer effect. Zero-shot accuracy peaks when function vectors are injected at intermediate layers (e.g., around layer 19 for synthetic data),… view at source ↗
Figure 6
Figure 6. Figure 6: (right panel), the CFV model achieved substantial improvements, nearly doubling accu￾racy from 8.3% in one-shot ICL to 16.8% with CFV. Notably, CFVs also significantly outperformed in-context learning even when provided with four (8.1%) or ten demonstration examples (9.6%). Q: pencil A: bagel Q: spoon A: ? One-Shot Analogy Task [PITH_FULL_IMAGE:figures/full_fig_p009_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: AIE of attention heads for relations in the synthetic dataset. Each heatmap shows the average indirect effect (AIE) values of attention heads (indexed by layer and head position). Pink boxes mark the top 10 most causally influential heads. (a) Above relation. (b) Below relation. (c) Left-of relation. (d) Right-of relation. (e) Behind relation. (f) Next-to relation. (g) In-front-of relation [PITH_FULL_IMAG… view at source ↗
Figure 8
Figure 8. Figure 8: AIE of attention heads for relations in the real image dataset. Each heatmap shows the average indirect effect (AIE) values of attention heads (indexed by layer and head position). Pink boxes mark the top 10 most causally influential heads. A.3 ABLATION STUDIES A.3.1 EFFECTS OF INJECTION LAYER We examine how the effectiveness of function vector intervention varies across different injection layers. Zero-sh… view at source ↗
Figure 9
Figure 9. Figure 9: Effect of injection layer on zero-shot accuracy. Injecting the function vector at inter￾mediate layers yields the highest accuracy, indicating that these layers are optimal for triggering relation computations. This pattern highlights a trade-off: using too few attention heads underrepresent relational knowl￾edge, while using too many attention heads introduces idiosyncratic activations from those with low… view at source ↗
Figure 10
Figure 10. Figure 10: Zero-shot accuracy as a function of number of heads in function vector. Accuracy peaks when using 6 - 14 heads, suggesting that the function is distributed sparsely across a limited causal subnetwork. (a) below, 2-shot (b) below, 4-shot (c) below, 8-shot (d) left-of, 2-shot (e) left-of, 4-shot (f) left-of, 8-shot [PITH_FULL_IMAGE:figures/full_fig_p016_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Function vector accuracy across layers as a function of context size. Each subfigure shows accuracy when injecting function vectors extracted from prompts with 2, 4, or 8 in-context examples. Results are shown for the below relation (top row) and left-of relation (bottom row). B ETHICS STATEMENT This research complies with the ICLR Code of Ethics. The study did not involve human subjects, personally ident… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

19 extracted references · 14 linked inside Pith

  1. [4]

    Eliciting latent predictions from transformers with the tuned lens

    Nora Belrose, Zach Furman, Logan Smith, Danny Halawi, Igor Ostrovsky, Lev McKinney, Stella Biderman, and Jacob Steinhardt. Eliciting latent predictions from transformers with the tuned lens. arXiv preprint arXiv:2303.08112, March

    Pith/arXiv arXiv

  2. [5]

    Language models are few-shot learners.Advances in neural information processing systems, 33:1877–1901,

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners.Advances in neural information processing systems, 33:1877–1901,

    1901

  3. [8]

    In-context learning creates task vectors.arXiv preprint arXiv:2310.15916,

    Roee Hendel, Mor Geva, and Amir Globerson. In-context learning creates task vectors.arXiv preprint arXiv:2310.15916,

    Pith/arXiv arXiv

  4. [9]

    Multimodal task vectors enable many-shot multimodal in-context learning

    Brandon Huang, Chancharik Mitra, Assaf Arbelle, Leonid Karlinsky, Trevor Darrell, and Roei Herzig. Multimodal task vectors enable many-shot multimodal in-context learning. InAdvances in Neural Information Processing Systems 37 (NeurIPS 2024),

    2024

  5. [11]

    Linguistic regularities in continuous space word representations

    Tom´aˇs Mikolov, Wen-tau Yih, and Geoffrey Zweig. Linguistic regularities in continuous space word representations. InProceedings of the 2013 conference of the north american chapter of the association for computational linguistics: Human language technologies, pp. 746–751,

    2013

  6. [12]

    The linear representation hypothesis and the geometry of large language models.arXiv preprint arXiv:2311.03658,

    Kiho Park, Yo Joong Choe, and Victor Veitch. The linear representation hypothesis and the geometry of large language models.arXiv preprint arXiv:2311.03658,

    Pith/arXiv arXiv

  7. [13]

    Automatic discovery of visual circuits

    Achyuta Rajaram, Neil Chowdhury, Antonio Torralba, Jacob Andreas, and Sarah Schwettmann. Automatic discovery of visual circuits. arXiv preprint arXiv:2404.14349, April

    Pith/arXiv arXiv

  8. [14]

    Li, Arnab Sen Sharma, Aaron Mueller, Byron C

    Eric Todd, Millicent L. Li, Arnab Sen Sharma, Aaron Mueller, Byron C. Wallace, and David Bau. Function vectors in large language models. InProceedings of the 2024 International Conference on Learning Representations,

    2024

  9. [15]

    Together Computer

    arXiv:2310.15213. Together Computer. RedPajama-INCITE-Base-3B-v1: Programmable Base Model.https:// huggingface.co/togethercomputer/RedPajama-INCITE-Base-3B-v1,

    Pith/arXiv arXiv

  10. [16]

    Steering language models with activation engineering.arXiv preprint arXiv:2308.10248,

    Alexander Matt Turner, Lisa Thiergart, Gavin Leech, David Udell, Juan J Vazquez, Ulisse Mini, and Monte MacDiarmid. Steering language models with activation engineering.arXiv preprint arXiv:2308.10248,

    Pith/arXiv arXiv

  11. [17]

    Look before you leap: A universal emergent decomposition of retrieval tasks in language models

    11 Alexandre Variengien and Eric Winsor. Look before you leap: A universal emergent decomposition of retrieval tasks in language models. arXiv preprint arXiv:2312.10091, dec

    Pith/arXiv arXiv

  12. [18]

    In- terpretability in the wild: A circuit for indirect object identification in gpt-2 small

    Kevin Ro Wang, Alexandre Variengien, Arthur Conmy, Buck Shlegeris, and Jacob Steinhardt. In- terpretability in the wild: A circuit for indirect object identification in gpt-2 small. InICLR, 2022a. Lean Wang, Lei Li, Damai Dai, Deli Chen, Hao Zhou, Fandong Meng, Jie Zhou, and Xu Sun. Label words are anchors: An information flow perspective for understandin...

    Pith/arXiv arXiv 2022

  13. [19]

    From this dataset, we selected 32 diverse objects spanning various categories and size ranges, which were subsequently mapped to a relatively uniform scale

    12 A SUPPLEMENTALMATERIALS A.1 DATASETS A.1.1 SYNTHETIC IMAGE DATASET We constructed a synthetic image dataset using object cutouts from the Big and Small Objects dataset (Konkle & Oliva, 2012), which contains real-world objects annotated by their typical phys- ical size. From this dataset, we selected 32 diverse objects spanning various categories and si...

    2012

  14. [2019]

    Improving activation steering in language models with mean-centring.arXiv preprint arXiv:2312.03813,

    Ole Jorgensen, Dylan Cope, Nandi Schoots, and Murray Shanahan. Improving activation steering in language models with mean-centring.arXiv preprint arXiv:2312.03813,

    Pith/arXiv arXiv

  15. [2021]

    Explainable and interpretable multimodal large language models: A comprehensive survey

    Yunkai Dang, Kaichen Huang, Jiahao Huo, Yibo Yan, Sirui Huang, Dongrui Liu, Mengxi Gao, Jie Zhang, Chen Qian, Kun Wang, Yong Liu, Jing Shao, Hui Xiong, and Xuming Hu. Explainable and interpretable multimodal large language models: A comprehensive survey. arXiv preprint arXiv:2412.02104, December

    Pith/arXiv arXiv

  16. [2022]

    Anthropic

    arXiv:2204.14198. Anthropic. Mapping the mind of a large language model.https://www.anthropic.com/ research/mapping-mind-language-model, May

    Pith/arXiv arXiv

  17. [2023]

    arXiv preprint arXiv:2211.15661. Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, Roman Ring, Eliza Rutherford, Serkan Cabi, Tengda Han, Zhitao Gong, Sina Samangooei, Marianne Monteiro, Jacob Menick, Sebastian Borgeaud, Andrew Brock, Aida Nematzadeh, S...

    Pith/arXiv arXiv

  18. [2024]

    Toy models of superposi- tion.arXiv preprint arXiv:2209.10652,

    Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, Shauna Kravec, Zac Hatfield-Dodds, Robert Lasenby, Dawn Drain, Carol Chen, et al. Toy models of superposi- tion.arXiv preprint arXiv:2209.10652,

    Pith/arXiv arXiv

  19. [2025]

    Openflamingo: An open- source framework for training large autoregressive vision-language models

    Anas Awadalla, Irena Gao, Josh Gardner, Jack Hessel, Yusuf Hanafy, Wanrong Zhu, Kalyani Marathe, Yonatan Bitton, Samir Gadre, Shiori Sagawa, Jenia Jitsev, Simon Kornblith, Pang Wei Koh, Gabriel Ilharco, Mitchell Wortsman, and Ludwig Schmidt. Openflamingo: An open- source framework for training large autoregressive vision-language models. arXiv preprint ar...

    Pith/arXiv arXiv