Pith. sign in

REVIEW 2 major objections 6 minor 60 references

In-Context Collapse in Vision-Language Models and How to Mitigate it?

T0 review · 2 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that many-shot in-context learning in vision-language models can collapse below chance as demonstrations accumulate, and that the failure is a localized, repairable deficit of the vision–language integration pathway.

desk verdict A genuinely interesting causal localization of a real VLM in-context failure, with a cheap transferable fix, but the paper's own zero-shot remap numbers undermine its 'contamination-free' learning measure. read the letter →

arxiv 2608.02830 v1 pith:54LYQCD4 submitted 2026-08-03 cs.CV cs.AI

Mohammad Rostami This is my paper · ORCID
classification cs.CVcs.AI
keywords in-contextlearningvision-languagemodelsmany-shotpromptingmechanisticinterpretabilitylow-rankadaptationcatastrophicforgettingcontinualintegrationfailure
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that many-shot in-context learning in vision-language models is not the free lunch it is assumed to be: for a substantial subset of VLMs, accuracy falls as correctly labelled image–label demonstrations are added, sometimes below chance while the model still emits well-formed labels. It further claims that this 'in-context collapse' is a localized failure of the vision–language integration pathway, not of the readout, and that a small low-rank adapter placed at that locus restores genuine in-context learning and transfers to unseen tasks. If correct, the result would change where practitioners validate many-shot prompting and where they should spend adaptation capacity.

What carries the argument

The central machinery is a depth partition of the model into an integration locus I (connector plus early/mid language-model blocks) and a readout locus R (late blocks), together with a remap verbalizer that replaces class names with arbitrary tokens so above-chance accuracy can only come from binding the in-prompt mapping. The causal probe is a low-rank adapter attached to one region at a time, trained only on remap episodes with all original weights frozen, then re-measured on the learning curve. What this does is turn a behavioral failure into an editable, location-specific computation: the capacity-matched asymmetry (integration rescues, readout does not) is what localizes the collapse.

What would settle it

Run the remap probe with freshly generated random token strings per task instead of the reused 'alpha' through 'delta' set. If above-chance remap accuracy survives with novel tokens and the vaccine still transfers, the learning axis is clean; if accuracy drops to chance, the measured learning and transfer are inflated by token priors.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is the integration hypothesis (Definition 3): the in-context collapse is a deficit of the integration computation localized to the vision–language interface (connector plus early/mid language-model layers), not of the readout. As demonstrations accumulate, the integration pathway is overwhelmed and the model reverts to its prior or to copying the most recent label instead of binding the in-prompt mapping. The support is a parameter-matched lesion-and-rescue: a rank-8 adapter on the connector lifts contamination-free remap accuracy from 0.39 to 0.91 at 16 shots, while an equal-capacity late-readout adapter drives it to 0.08. A one-time vaccine trained on one synthetic remap task transfers this repaired capability to unseen CIFAR and Fashion task families (chance to 0.71 and 0.60).

Load-bearing premise

The whole learning axis rests on the assumption that the fixed arbitrary tokens 'alpha', 'beta', 'gamma', and 'delta' carry no usable pretraining association with the task classes or image features; if any token had even a weak prior correlation with a class, above-chance remap accuracy would partly reflect retrieval rather than in-prompt learning.

Editorial extensions

If this is right

  • Many-shot multimodal ICL is not monotonically helpful: more demonstrations can destroy competence a model already had, so deployment pipelines should test the full K-curve per model rather than assume more is better.
  • Robustness to demonstrations and the ability to learn a novel rule in context are separable axes; the three regimes give practitioners a diagnostic instead of a single 'does ICL work' answer.
  • The collapse is causally removable at the integration locus: a small adapter on the connector and early/mid layers restores in-context learning, so the fix is cheap and offline.
  • A one-time vaccine trained on one synthetic remap task confers transferable collapse-resistance to unseen task families, implying integration capacity is a general skill rather than a task-specific lookup.
  • The integration locus is not the consolidation locus: late readout layers give more durable weight-based consolidation with fewer parameters, so fast prompting and slow weight updates belong in different places.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: if the token-prior assumption holds, a natural next step is predicting collapse-proneness from weight statistics, such as the rank or conditioning of the connector-to-early-layer Jacobian, so models could be pre-vaccinated at release time; the paper sketches this direction but does not test it.
  • Editorial extension: the copy-rate diagnostic could be deployed online as a serving-time monitor that stops adding demonstrations when the model starts echoing the last label, even without ground-truth labels.
  • Editorial extension: because a training-free task-vector injection fails on collapse-prone models, the framework predicts that vaccine-then-inject ordering is necessary; this is testable on other architectures and connector families.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. The paper reports a phenomenon it calls in-context collapse in vision-language models: for several VLMs, accuracy on classification and VQA tasks falls as in-context demonstrations are added, sometimes below chance, even though outputs remain well-formed. It introduces a robustness-learning dissociation, measured via 'remap' tasks using supposedly arbitrary tokens, and claims to localize the collapse to the vision-language integration pathway (connector and early/mid transformer layers) using a parameter-matched lesion-and-rescue. It further proposes CircA, whose main component is a 'vaccine': a one-time adapter trained on a single synthetic remap task that, the authors claim, transfers to unseen task families. The paper presents a large experimental corpus across 11 open VLMs and several frontier API models.

Significance. If the claims hold, the paper offers three substantial contributions: a novel, reproducible, graded failure mode for many-shot multimodal ICL; a clean dissociation of robustness from genuine in-context learning; and an equal-capacity causal intervention suggesting the collapse is editable at a specific interface rather than a diffuse scale artifact. The cross-architecture replication and the integration-vs-consolidation dissociation are valuable. The code release is commendable. However, the central measurement premise—that the fixed token set (alpha/beta/gamma/delta) is contamination-free—is contradicted by the paper's own K=0 remap data, which threatens the learning-axis definition and the vaccine's transfer interpretation. The lesion-and-rescue asymmetry itself is less exposed, but the mechanistic interpretation depends on the same premise.

major comments (2)
  1. [Sec. 3.1, Definition 2; Table 1] The claim that A_rmp(0)≈1/C 'by construction' (Sec. 3.1) is contradicted by the paper's own Table 1, which reports K=0 remap accuracy of 0.50 (Qwen2-VL-7B, shapes), 0.48 (Qwen2.5-VL-7B, shapes), 0.44 (Qwen2.5-VL-3B, shapes), and 0.40 (LLaVA-OV-7B, CIFAR-4), all far above the 0.25 chance level. With no demonstrations, above-chance remap accuracy is impossible unless the fixed token set carries a pretraining association with the image classes—exactly the contamination the verbalizer was designed to exclude. Consequently, Definition 2's assertion that 'any above-chance remap accuracy can come only from the demonstrations' is empirically false, and the learning axis Lrn = A_rmp(K_max) − A_rmp(0) uses a contaminated baseline. This affects the three-regime classification and the labeling of models as 'genuine learners.' Please provide a token-prior control (e.g., novel random strings per task per model) and re-analyze the learning axis after removing the token-prior component, or explicitly quantify the contribution of token priors to the reported gains.
  2. [Sec. 4.3.1, Table 4] The vaccine transfer results are measured with the same fixed token set used in training. Given the K=0 contamination shown in Table 1, the improvements on held-out CIFAR-4 (0.34→0.71) and Fashion-4 (0.18→0.60) may partly reflect exploitation of token-class priors rather than acquisition of a general integration skill. This is load-bearing because the central claim of the vaccine is transfer to unseen task families. Please include a control in which the vaccine is trained and evaluated on different, task-specific random token sets, or otherwise demonstrate that the transfer gain cannot be explained by retrieval of fixed-token priors.
minor comments (6)
  1. [Abstract and Sec. 1] The phrases 'anin-context collapse' and 'thecollapse-prone' contain missing spaces; please fix these typos along with any similar spacing errors elsewhere in the text.
  2. [Table 1 header] The notation 'K= max' is unclear; please specify the shot-count maximum for each model and use a consistent arrow notation (e.g., '0→max') throughout the table.
  3. [Fig. 6 caption] The caption refers to a 'shaded band' in the top row being magnified in the bottom row, but the bottom row is a separate zoomed panel and no shaded band appears in the figure; please clarify or add the band to match the description.
  4. [Sec. 4.4.3 and Fig. 18] The y-axis label 'Forgetting (less negative = better)' is confusing because the plotted values are positive; please state explicitly that the figure plots −BWT (the negative of backward transfer) or adjust the label and sign convention accordingly.
  5. [Sec. 4.1.9] The continual-learning protocol says 'CIFAR-100 into five disjoint 4-class tasks,' which uses only 20 of the 100 classes; please state this explicitly to avoid the impression that the full CIFAR-100 benchmark is used.
  6. [References] Reference [36] is incompletely formatted as 'Olsson C, others, Olah C'; please supply the full author list.

Circularity Check

0 steps flagged · score 0.0 of 10

No constructional circularity: the localization and vaccine-transfer claims are measured against held-out queries and unseen task families, so no prediction reduces to a fitted input by construction.

full rationale

The paper's derivation chain is not circular. The collapse phenomenon and the robustness/learning dissociation are measured on held-out queries with a label-shuffle control, and the lesion-and-rescue compares equal-capacity adapters at predefined depth loci (Eq. 4) with evaluation queries disjoint from adapter training, so the integration-over-readout asymmetry is an empirical result rather than a consequence of how the loci are named. The vaccine (Eq. 7) is trained once on shapes remap, then frozen and evaluated on CIFAR-4 and Fashion-4 remap, task families outside its training distribution; its transfer scores therefore cannot reduce to the training objective by construction. The same-token remap verbalizer does create a validity concern: Table 1 reports zero-shot remap accuracy above chance for several models (e.g., Qwen2-VL-7B shapes 0.50, Qwen2.5-VL-7B shapes 0.48), contradicting the Sec. 3.1 claim that A_rmp(0) is at chance 'by construction' and weakening the contamination-free interpretation of the Lrn axis and of the vaccine's transfer. This is a correctness risk rather than a constructional circularity: no equation in the paper forces a predicted quantity to equal a fitted input, and the central localization and transfer claims remain measured against external held-out data and unseen task families. The paper also explicitly acknowledges boundary conditions, such as the asymmetric reverse intervention and validation only on small open VLMs, which constrain generality but do not introduce circularity. Self-citations (e.g., [6,7,8,18,40]) appear only in related-work and organizing-lens contexts and are not load-bearing for the main claims.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

The free parameters are the LoRA adapter hyperparameters and the gate threshold, plus the fixed remap token set reused across all tasks. The axioms are the contamination-free remap assumption, the integration and readout partition, and the attribution of rescue to adapter location. No invented entities are introduced.

free parameters (5)
  • LoRA rank r = 8
    Chosen by hand for all adapters; the magnitude of the rescue depends on it, and InternVL3 uses rank 32 for its directional replication.
  • LoRA scaling alpha = 16
    Chosen so that alpha over r equals 2; affects the adapter update size.
  • Adapter training steps, learning rate, batch size = 150 steps, lr 1e-4, batch size 1
    Fixed across regions; differences in optimization difficulty across depths could in principle confound the locus comparison.
  • Gate threshold tau = not reported
    Eq. (9) defines the gate with a tolerance tau, but the gate result in Table 6 does not state the value used, so the 0.21 gate row is not reproducible as specified.
  • Remap token set = alpha, beta, gamma, delta
    A fixed set of four tokens reused across all tasks and models; if any token carries a weak prior association with a class, remap accuracy would be partially contaminated.
assumptions (3)
  • domain assumption Above-chance remap accuracy can only arise from binding the in-prompt image-to-token mapping, never from retrieval of pretraining knowledge.
    Invoked in Sec. 3.1 to define contamination-free learning; the entire Lrn axis and the vaccine transfer results depend on it, but no token-prior control is provided.
  • ad hoc to paper A VLM factorizes into vision encoder, connector, transformer blocks, and unembedding, and the integration and readout split by contiguous thirds is a faithful decomposition of where demonstrations are bound.
    Used in Eq. (4) to define the integration locus I and readout R; the causal localization inherits the validity of this partition.
  • domain assumption Recovery in the lesion-and-rescue is attributable to adapter location rather than to optimizer behavior at different depths.
    Equal module counts and ranks are used, but learning rate and step count are not re-tuned per region; the separate continual-learning result partially supports that late layers are learnable.

how reviews work

0 comments
Cite this review

Pith. "Pith review of In-Context Collapse in Vision-Language Models and How to Mitigate it?." pith.science (2026). https://pith.science/paper/54LYQCD4

@misc{pith2026260802830,
  author       = {Pith},
  title        = {Pith review of: In-Context Collapse in Vision-Language Models and How to Mitigate it?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/54LYQCD4}},
  note         = {Machine review of arXiv:2608.02830}
}
abstract

Many-shot in-context learning (ICL) lets vision-language models (VLMs) adapt from image--label demonstrations without weight updates, and is widely assumed to improve as more demonstrations are supplied. We show the opposite: as demonstrations accumulate, a subset of VLMs undergo an \emph{in-context collapse}, a sharp, sometimes catastrophic accuracy drop spanning synthetic classification, natural-image classification, and VQA benchmarks, in some models falling below chance while outputs remain well-formed. Across an open VLM panel ($0.5$B--$11$B) and a frontier model (Claude Sonnet 4.5), the collapse is graded. Two capabilities turn out to be dissociable: robustness to accumulating demonstrations and the ability to learn a novel rule in context, their combinations yield three reproducible regimes. A parameter-matched lesion-and-rescue causally localizes the collapse to the vision-language integration pathway: an adapter on the connector and early/mid layers restores genuine learning (remap accuracy $0.39!\rightarrow!0.91$ at 16 shots), while an equal-capacity adapter on the late readout does not. We propose \textsc{CircA}, whose core is a one-time integration vaccine: trained once on one synthetic task, it transfers collapse-resistance to unseen task families (chance$\rightarrow$$0.71$/$0.60$ on CIFAR/Fashion). The layers best for in-context integration are not the layers best for weight-based consolidation, the late readout achieves higher accuracy and less forgetting at fewer parameters. The collapse is an integration failure at the vision--language interface, correctable by a lightweight, transferable intervention.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

60 extracted references · 11 canonical work pages

  1. [1]

    Advances in Neural Information Processing Systems (NeurIPS)

    Agarwal R, Singh A, Zhang LM, et al (2024) Many-shot in-context learning. Advances in Neural Information Processing Systems (NeurIPS)

    work page 2024

  2. [2]

    NeurIPS 2022 arXiv:2204.14198

    Alayrac JB, Donahue J, Luc P, et al (2022) Flamingo: a Visual Language Model for Few-Shot Learning. NeurIPS 2022 arXiv:2204.14198

    arXiv 2022

  3. [3]

    NeurIPS 2022 arXiv:2209.00647

    Bar A, Gandelsman Y, Darrell T, et al (2022) Visual Prompting via Image Inpainting. NeurIPS 2022 arXiv:2209.00647

    arXiv 2022

  4. [4]

    In: Advances in Neural Information Processing Systems (NeurIPS)

    Brown TB, Mann B, Ryder N, et al (2020) Language models are few-shot learners. In: Advances in Neural Information Processing Systems (NeurIPS)

    work page 2020

  5. [5]

    NeurIPS 2020 arXiv:2004.07211

    Buzzega P, Boschini M, Porrello A, et al (2020) Dark Experience for General Continual Learning: a Strong, Simple Baseline. NeurIPS 2020 arXiv:2004.07211

    arXiv 2020

  6. [6]

    Journal of Artificial Intelligence Research 83

    Cai Y, Rostami M (2025) CluMo: Cluster-based modality fusion prompt for con- tinual learning in visual question answering. Journal of Artificial Intelligence Research 83. https://doi.org/10.1613/jair.1.17940

    work page doi:10.1613/jair.1.17940 2025

  7. [7]

    Neurocomputing https://doi.org/10.1016/j.neucom.2025

    Cai Y, Rostami M (2026) Dynamic transformer architecture for continual learning of multimodal tasks. Neurocomputing https://doi.org/10.1016/j.neucom.2025. 129533, URL https://arxiv.org/abs/2401.15275

    arXiv 2026

  8. [8]

    In: Findings of the Association for Computational Linguistics: EMNLP 2023

    Cai Y, Thomason J, Rostami M (2023) Task-attentive transformer architecture for continual learning of vision-and-language tasks using knowledge distillation. In: Findings of the Association for Computational Linguistics: EMNLP 2023. 42 Association for Computational Linguistics, pp 6986–7000, https://doi.org/10. 18653/v1/2023.findings-emnlp.466

    work page 2023

Show all 60 references
  1. [9]

    ICLR 2019 arXiv:1812.00420

    Chaudhry A, Ranzato M, Rohrbach M, et al (2018) Efficient Lifelong Learning with A-GEM. ICLR 2019 arXiv:1812.00420

    2018 arXiv

  2. [10]

    NeurIPS 2024 arXiv:2403.08350

    Chen C, Zhu J, Luo X, et al (2024) CoIN: A Benchmark of Continual Instruction tuNing for Multimodel Large Language Model. NeurIPS 2024 arXiv:2403.08350

    2024 arXiv

  3. [11]

    CVPR 2024 arXiv:2312.14238

    Chen Z, Wu J, Wang W, et al (2023) InternVL: Scaling up Vision Foun- dation Models and Aligning for Generic Visual-Linguistic Tasks. CVPR 2024 arXiv:2312.14238

    2023 arXiv

  4. [12]

    arXiv 2024 arXiv:2412.14133

    Cohen I, Gottesman D, Geva M, et al (2024) Performance Gap in Entity Knowledge Extraction Across Modalities in Vision Language Models. arXiv 2024 arXiv:2412.14133

    2024

  5. [13]

    NeurIPS 2023 arXiv:2305.06500

    Dai W, Li J, Li D, et al (2023) InstructBLIP: Towards General-purpose Vision- Language Models with Instruction Tuning. NeurIPS 2023 arXiv:2305.06500

    2023 arXiv

  6. [14]

    arXiv preprint arXiv:240721783

    Dubey A, et al (2024) The llama 3 herd of models. arXiv preprint arXiv:240721783

    2024

  7. [15]

    NeurIPS 2024 arXiv:2406.16320

    Golovanevsky M, Rudman W, Palit V, et al (2024) What Do VLMs NOTICE? A Mechanistic Interpretability Pipeline for Gaussian-Noise-free Text-Image Cor- ruption and Evaluation. NeurIPS 2024 arXiv:2406.16320

    2024 arXiv

  8. [16]

    EMNLP 2023 arXiv:2310.15916

    Hendel R, Geva M, Globerson A (2023) In-Context Learning Creates Task Vectors. EMNLP 2023 arXiv:2310.15916

    2023 arXiv

  9. [17]

    ICLR 2022 arXiv:2106.09685

    Hu EJ, Shen Y, Wallis P, et al (2021) LoRA: Low-Rank Adaptation of Large Language Models. ICLR 2022 arXiv:2106.09685

    2021 arXiv

  10. [18]

    arXiv preprint arXiv:250508803 URL https://arxiv.org/abs/2505.08803

    Hu Z, Rostami M, Thomason J (2025) Multi-modal synthetic data training and model collapse: Insights from VLMs and diffusion models. arXiv preprint arXiv:250508803 URL https://arxiv.org/abs/2505.08803

    2025 arXiv

  11. [19]

    NeurIPS 2024 arXiv:2406.15334

    Huang B, Mitra C, Arbelle A, et al (2024) Multimodal Task Vectors Enable Many-Shot Multimodal In-Context Learning. NeurIPS 2024 arXiv:2406.15334

    2024 arXiv

  12. [20]

    arXiv 2025 arXiv:2506.07936

    Huang C, Zhu Y, Zhu S, et al (2025) Mimicking or Reasoning: Rethink- ing Multi-Modal In-Context Learning in Vision-Language Models. arXiv 2025 arXiv:2506.07936

    2025 arXiv

  13. [21]

    In: Advances in Neural Information Processing Systems (NeurIPS) 43

    Huang T, Hu S, Liu L (2024) Vaccine: Perturbation-aware alignment for large language models against harmful fine-tuning. In: Advances in Neural Information Processing Systems (NeurIPS) 43

    2024

  14. [22]

    arXiv 2024 arXiv:2405.09798

    Jiang Y, Irvin J, Wang JH, et al (2024) Many-Shot In-Context Learning in Multimodal Foundation Models. arXiv 2024 arXiv:2405.09798

    2024 arXiv

  15. [23]

    arXiv 2024 arXiv:2411.17491

    Kaduri O, Bagon S, Dekel T (2024) What’s in the Image? A Deep-Dive into the Vision of Vision Language Models. arXiv 2024 arXiv:2411.17491

    2024 arXiv

  16. [24]

    PNAS 2017 arXiv:1612.00796

    Kirkpatrick J, Pascanu R, Rabinowitz N, et al (2016) Overcoming catastrophic forgetting in neural networks. PNAS 2017 arXiv:1612.00796

    2016 arXiv

  17. [25]

    Trends in Cognitive Sciences https://doi.org/10.1016/j.tics.2016.05.004

    Kumaran D, Hassabis D, McClelland JL (2016) What Learning Systems do Intel- ligent Agents Need? Complementary Learning Systems Theory Updated. Trends in Cognitive Sciences https://doi.org/10.1016/j.tics.2016.05.004

    2016 doi

  18. [26]

    TPAMI 2022 arXiv:1909.08383

    Lange MD, Aljundi R, Masana M, et al (2019) A continual learning survey: Defying forgetting in classification tasks. TPAMI 2022 arXiv:1909.08383

    2019 arXiv

  19. [27]

    arXiv 2023 arXiv:2305.03726

    Li B, Zhang Y, Chen L, et al (2023) Otter: A Multi-Modal Model with In-Context Instruction Tuning. arXiv 2023 arXiv:2305.03726

    2023 arXiv

  20. [28]

    TMLR 2024 arXiv:2408.03326

    Li B, Zhang Y, Guo D, et al (2024) LLaVA-OneVision: Easy Visual Task Transfer. TMLR 2024 arXiv:2408.03326

    2024 arXiv

  21. [29]

    ICML 2023 arXiv:2301.12597

    Li J, Li D, Savarese S, et al (2023) BLIP-2: Bootstrapping Language-Image Pre- training with Frozen Image Encoders and Large Language Models. ICML 2023 arXiv:2301.12597

    2023 arXiv

  22. [30]

    arXiv 2025 arXiv:2505.17097

    Li Y, Yang J, Yang Z, et al (2025) Make LVLMs Focus: Context-Aware Attention Modulation for Better Multimodal In-Context Learning. arXiv 2025 arXiv:2505.17097

    2025

  23. [31]

    TPAMI 2018 arXiv:1606.09282

    Li Z, Hoiem D (2016) Learning without Forgetting. TPAMI 2018 arXiv:1606.09282

    2016 arXiv

  24. [32]

    NeurIPS 2023 arXiv:2304.08485

    Liu H, Li C, Wu Q, et al (2023) Visual Instruction Tuning. NeurIPS 2023 arXiv:2304.08485

    2023 arXiv

  25. [33]

    NeurIPS 2017 arXiv:1706.08840

    Lopez-Paz D, Ranzato M (2017) Gradient Episodic Memory for Continual Learning. NeurIPS 2017 arXiv:1706.08840

    2017 arXiv

  26. [34]

    Psychological Review https://doi.org/10.1037/0033-295X.102.3.419

    McClelland JL, McNaughton BL, O’Reilly RC (1995) Why there are complemen- tary learning systems in the hippocampus and neocortex. Psychological Review https://doi.org/10.1037/0033-295X.102.3.419

    1995 doi

  27. [35]

    arXiv 2024 arXiv:2410.07149

    Neo C, Ong L, Torr P, et al (2024) Towards Interpreting Visual Information Processing in Vision-Language Models. arXiv 2024 arXiv:2410.07149

    2024 arXiv

  28. [36]

    Transformer Circuits Thread 44

    Olsson C, others, Olah C (2022) In-Context Learning and Induction Heads. Transformer Circuits Thread 44

    2022

  29. [37]

    Cognitive Science https://doi.org/10.1111/j.1551-6709.2011.01214

    O’Reilly RC, Bhattacharyya R, Howard MD, et al (2014) Complementary Learn- ing Systems. Cognitive Science https://doi.org/10.1111/j.1551-6709.2011.01214. x

    2014

  30. [38]

    ICCV Workshop 2023 arXiv:2308.14179

    Palit V, Pandey R, Arora A, et al (2023) Towards Vision-Language Mecha- nistic Interpretability: A Causal Tracing Tool for BLIP. ICCV Workshop 2023 arXiv:2308.14179

    2023 arXiv

  31. [39]

    CVPR 2017 arXiv:1611.07725

    Rebuffi SA, Kolesnikov A, Sperl G, et al (2016) iCaRL: Incremental Classifier and Representation Learning. CVPR 2017 arXiv:1611.07725

    2016 arXiv

  32. [40]

    In: Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence (IJCAI-23), pp 3058–3066, https://doi.org/10.24963/ ijcai.2023/341

    Rostami M, Galstyan A (2023) Cognitively inspired learning of incremental drift- ing concepts. In: Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence (IJCAI-23), pp 3058–3066, https://doi.org/10.24963/ ijcai.2023/341

    2023

  33. [41]

    PNAS https://doi.org/ 10.1073/pnas.2502194122

    Russin J, Pavlick E, Frank MJ (2025) The dynamic interplay between in-context and in-weight learning in humans and neural networks. PNAS https://doi.org/ 10.1073/pnas.2502194122

    2025 doi

  34. [42]

    NeurIPS 2017 arXiv:1705.08690

    Shin H, Lee JK, Kim J, et al (2017) Continual Learning with Deep Generative Replay. NeurIPS 2017 arXiv:1705.08690

    2017 arXiv

  35. [43]

    In: Advances in Neural Information Processing Systems, pp 29440–29453, URL https://arxiv.org/abs/2206.09059

    Srinivasan T, Chang TY, Pinto Alva LL, et al (2022) CLiMB: A continual learning benchmark for vision-and-language tasks. In: Advances in Neural Information Processing Systems, pp 29440–29453, URL https://arxiv.org/abs/2206.09059

    2022 arXiv

  36. [44]

    ICLR 2020 arXiv:1909.03329

    Sun FK, Ho CH, Lee HY (2019) LAMOL: LAnguage MOdeling for Lifelong Language Learning. ICLR 2020 arXiv:1909.03329

    2019 arXiv

  37. [45]

    CVPR 2024 arXiv:2308.07891

    Tai Y, Fan W, Zhang Z, et al (2023) Link-Context Learning for Multimodal LLMs. CVPR 2024 arXiv:2308.07891

    2023 arXiv

  38. [46]

    ICLR 2024 arXiv:2310.15213

    Todd E, Li ML, Sharma AS, et al (2023) Function Vectors in Large Language Models. ICLR 2024 arXiv:2310.15213

    2023 arXiv

  39. [47]

    NeurIPS 2021 arXiv:2106.13884

    Tsimpoukelli M, Menick J, Cabi S, et al (2021) Multimodal Few-Shot Learning with Frozen Language Models. NeurIPS 2021 arXiv:2106.13884

    2021 arXiv

  40. [48]

    Nature Machine Intelligence https://doi.org/10.1038/s42256-022-00568-3

    van de Ven GM, Tuytelaars T, Tolias AS (2022) Three types of incremental learn- ing. Nature Machine Intelligence https://doi.org/10.1038/s42256-022-00568-3

    2022 doi

  41. [49]

    arXiv 2024 arXiv:2409.12191

    Wang P, Bai S, Tan S, et al (2024) Qwen2-VL: Enhancing Vision-Language Model’s Perception of the World at Any Resolution. arXiv 2024 arXiv:2409.12191

    2024 arXiv

  42. [50]

    CVPR 2023 arXiv:2212.02499 45

    Wang X, Wang W, Cao Y, et al (2022) Images Speak in Images: A Generalist Painter for In-Context Visual Learning. CVPR 2023 arXiv:2212.02499 45

    2022 arXiv

  43. [51]

    EMNLP 2023 arXiv:2310.14152

    Wang X, Chen T, Ge Q, et al (2023) Orthogonal Subspace Learning for Language Model Continual Learning. EMNLP 2023 arXiv:2310.14152

    2023 arXiv

  44. [52]

    ICCV 2023 arXiv:2304.03284

    Wang X, Zhang X, Cao Y, et al (2023) SegGPT: Segmenting Everything In Context. ICCV 2023 arXiv:2304.03284

    2023 arXiv

  45. [53]

    arXiv 2024 arXiv:2411.13949

    Wang Z, Che C, Wang Q, et al (2024) SMoLoRA: Exploring and Defying Dual Catastrophic Forgetting in Continual Visual Instruction Tuning. arXiv 2024 arXiv:2411.13949

    2024 arXiv

  46. [54]

    arXiv preprint arXiv:251005024

    Wichers N, et al (2025) Inoculation prompting: Eliciting undesirable behavior to suppress it. arXiv preprint arXiv:251005024

    2025

  47. [55]

    arXiv 2024 arXiv:2410.05849

    Zeng F, Zhu F, Guo H, et al (2024) ModalPrompt: Towards Efficient Multimodal Continual Instruction Tuning with Dual-Modality Guided Prompt. arXiv 2024 arXiv:2410.05849

    2024 arXiv

  48. [56]

    ICML 2017 arXiv:1703.04200

    Zenke F, Poole B, Ganguli S (2017) Continual Learning Through Synaptic Intelligence. ICML 2017 arXiv:1703.04200

    2017 arXiv

  49. [57]

    arXiv 2023 arXiv:2309.10313

    Zhai Y, Tong S, Li X, et al (2023) Investigating the Catastrophic Forgetting in Multimodal Large Language Models. arXiv 2023 arXiv:2309.10313

    2023 arXiv

  50. [58]

    Zhang Y, Zhou K, Liu Z (2023) What Makes Good Examples for Visual In- Context Learning? NeurIPS 2023 arXiv:2301.13670

    2023 arXiv

  51. [59]

    ICML 2024 arXiv:2402.12048

    Zhu D, Sun Z, Li Z, et al (2024) Model Tailor: Mitigating Catastrophic Forgetting in Multi-modal Large Language Models. ICML 2024 arXiv:2402.12048

    2024 arXiv

  52. [60]

    In: International Conference on Learning Representations (ICLR) 46

    Zong Y, Bohdal O, Yu T, et al (2025) VL-ICL bench: The devil in the details of multimodal in-context learning. In: International Conference on Learning Representations (ICLR) 46

    2025

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.