Pith. sign in

REVIEW 3 major objections 5 minor 31 references

When Is a Task Vector Enough? An Empirical Theory of Implicit Multimodal ICL

T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that whether a static task vector suffices for implicit multimodal in-context learning is determined by the structure of the demonstration-induced computation, and that four measurable diagnostics predict when…

desk verdict A solid empirical framework for choosing the cheapest sufficient implicit ICL intervention; the core relationship holds up, but the query-independent basis assumption needs a direct test. read the letter →

arxiv 2608.13385 v1 pith:HP2ACZVM submitted 2026-08-13 cs.CV

Jiaqian Li This is my paper · ORCID
classification cs.CV
keywords implicitmultimodalin-contextlearningtaskvectorsSelection–RealizationHypothesisactivationinterventionsrepresentationengineeringquery-conditionedattentionroutingcost-awaremethodselection
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks when a static task vector—a fixed shift in a model's internal state that encodes the demonstrations—is enough to reproduce in-context learning, and when the intervention must instead depend on the current query or act at several sites. It proposes the Selection–Realization Hypothesis: demonstrations induce a compact family of internal changes, the query selects a member of that family, and the model's computation determines how the selection can be implemented. Using controlled multimodal episodes where query dependence is varied without changing task primitives, the paper shows that the fraction of demonstration-induced change shared across queries predicts static-vector success, while coefficient predictability, support dispersion, and an additive fit–recovery gap predict when conditional, multi-site, or routing interventions are useful. A selector calibrated only on controlled tasks, using these diagnostics, picks interventions on natural VQA benchmarks within about 0.3 points of the post-hoc best method while reducing deployment cost. The contribution is an empirical theory that makes intervention complexity a measurable quantity rather than a post-hoc justification.

What carries the argument

The carrying object is the demonstration-induced transformation $T_D(x) \approx \sum_{k=1}^r c_k(x,D)B_k(D)$: a low-rank basis of internal changes, with query-dependent coefficients. Four diagnostics derived from this decomposition drive the argument: sharedness $S(D)$, the fraction of change energy that is query-invariant; the held-out coefficient-predictor $R^2$, which tests whether query-specific coefficients can be predicted from the zero-shot query; support dispersion $\kappa$, the normalized entropy of site-level recovery that distinguishes local from distributed causal support; and the additive fit–recovery gap $G_{\text{add}} = \text{Fit}_{\text{add}} - \text{Recovery}_{\text{add}}$, which flags when additive shifts reproduce representations but fail behavior. Each is measured on calibration traces and mapped to a minimal intervention family without seeing test labels.

What would settle it

On a task deliberately built so that each query's correct rule lives in a disjoint internal subspace, estimate held-out reconstruction $R^2$ of the fixed low-rank basis; if reconstruction stays high yet both static and conditional interventions fail while a per-query refit basis succeeds, the decomposition in Eq. (3) is falsified.

Watch

Extended reading notes

Core claim

The central claim is that whether a static task vector suffices is determined by the structure of the computation that explicit multimodal in-context learning induces, not by task difficulty or architecture. For a fixed demonstration set, the paper measures the change in internal activations relative to matched counterfactuals, $\Delta z_s(x;D)$, and decomposes it into a shared component $v_s(D)$ and a low-rank, query-dependent residual $U_s(D)c_s(x,D)$. When the shared component carries most of the energy (high sharedness $S(D)$), a static vector recovers the behavioral effect of explicit demonstrations. When the residual varies predictably with the query, conditional interventions are needed; when causal support is dispersed across sites, multi-site additive interventions help; and when additive shifts reconstruct the representations but not the behavior, attention routing helps. These four relationships are verified on controlled episodes with varying conditionality $\alpha$, then used with frozen thresholds to select minimal interventions on VQAv2, GQA, OK-VQA, and CVQA within about 0.3 points of the best post-hoc method.

Load-bearing premise

Everything rests on the assumption that the internal effect of the demonstrations can be written as a small set of reusable change patterns whose per-query weights vary; if the patterns themselves must change with the query, or if interactions between patterns are nonlinear, the four diagnostics will be miscalibrated.

Editorial extensions

If this is right

  • A practitioner can choose among static, conditional, multi-site, and routing interventions using only calibration traces, without running test evaluation.
  • Static task vectors are not universally limited: at $\alpha=0$ they recover 0.92 of the normalized behavioral gain, and the drop to 0.33 at $\alpha=1$ tracks the sharedness diagnostic rather than task difficulty.
  • Routing is a targeted remedy, not a default: it gives little gain on controlled tasks and VQAv2, but 0.04–0.06 on GQA, OK-VQA, and CVQA, and this gain is predicted by the additive fit–recovery gap.
  • The theory-selected intervention lands within 0.29 mean regret of the post-hoc best method at relative cost 0.58, cutting cost by 19% relative to always using M2IV and 42% relative to always using ICR.
  • Compactness holds across settings, with effective rank far below model width, so compressing demonstrations into reusable interventions is broadly feasible.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same selection–realization split likely applies to unimodal ICL and to prompt-routing architectures, so these diagnostics could index a reusable intervention library across LLMs rather than only multimodal models.
  • Equation (3) suggests a testable taxonomy: tasks whose computation is basis-stable (query-independent basis) versus tasks that require query-dependent basis elements; the latter would break the diagnostics and may need nonlinear or memory-augmented interventions.
  • The derangement counterfactuals assume shuffling output symbols isolates the demonstrated mapping; if marginal label statistics also drive behavior, sharedness would be overestimated. Comparing derangements with per-example shuffles and full rerandomizations would reveal such contamination.
  • Because diagnostics are cheap relative to re-encoding demonstrations, the selector could be run online and switch intervention families as the demonstration set changes, adapting cost to task drift.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes the Selection–Realization Hypothesis as an empirical account of when demonstrations in multimodal in-context learning can be compressed into a static task vector and when more expressive interventions are needed. It defines a decomposition of demonstration-induced activation changes into a query-independent basis with query-dependent coefficients (Eq. 3) and derives four diagnostics: sharedness, coefficient predictability, support dispersion, and the additive fit–recovery gap. Using a controlled multimodal task family where a gating attribute selects between a constant rule and a query-selected rule, the authors test hypotheses H1–H4 and report that the diagnostics predict static versus conditional selection and local versus multi-site versus routing realization. A family selector calibrated only on controlled tasks is then frozen and evaluated on natural VQA benchmarks, where it stays within 0.21–0.37 percentage points of the post-hoc best method at a relative cost of 0.58.

Significance. If the central decomposition assumption holds, the paper provides a valuable empirical theory and a practical, cost-aware method-selection procedure. The strengths are genuine: controlled episodes vary query dependence while holding task primitives fixed, intervention families are matched in rank, norm, and supervision, counterfactuals are designed to isolate the demonstrated mapping, cluster-bootstrap confidence intervals are used for correlations, and the decision rule is frozen before natural-task evaluation. The hypotheses are explicit and falsifiable, and the natural-task validation with a pre-specified selector is a strong design. However, the entire diagnostic machinery rests on the untested query-independence of the basis in Eq. (3), and part of the H2 claim is constructional rather than empirical. The absence of confidence intervals on the main recovery tables and the lack of released code or data further limit verification.

major comments (3)
  1. [§2, Eq. (3); §4.1; H1/Table 1] The decomposition T_D(x) ≈ Σ_k c_k(x,D) B_k(D) with a query-independent basis B_k(D) is load-bearing for all four diagnostics: sharedness (Eq. 6), coefficient predictability, support dispersion κ (Eq. 12), and the additive gap G_add (Eq. 8). The only support offered is H1/Table 1, which shows that a global low-rank basis reconstructs held-out changes and supports behavioral recovery. Reconstruction R² is necessary but not sufficient: a global approximation can have high R² while missing a small, behaviorally critical query-dependent subspace, and reconstruction quality does not guarantee that the recovered basis supports causal intervention. The paper should test query-independence directly using the gating split in §4.1: at α=1, fit separate per-branch bases for g(x)=0 and g(x)=1 and compare reconstruction and recovery against the global basis. If per-branch bases recover substantially better, Eq. (3) is violated and the sharedness, predictability, dispersion, and gap diagnostics are miscalibrated.
  2. [§3, H2; Eq. (5)–(6)] The relation between sharedness and static-vector sufficiency is partly true by construction at the representation level. v_s(D) is defined as the query mean of Δz_s, the static intervention inserts exactly v_s, and Shareds(D) is the fraction of demonstration-induced energy contained in v_s. Thus high sharedness mechanically implies high representational overlap between the static intervention and the average demonstration-induced change. The empirical content reduces to whether that representational overlap transfers to behavioral recovery, as measured in Tables 2–3 and Figure 3. The text should state this explicitly and separate the constructional part from the transfer part; for example, it could test whether recovery tracks sharedness after conditioning on the representation-level alignment, or compare episodes matched on Shareds but differing in behavioral transfer.
  3. [Tables 2 and 3; §5.3] The main quantitative claims that carry the theory—static recovery declining with α, multi-site gain of 0.04–0.09 over local interventions, routing gain of 0.04–0.06 on GQA/OK-VQA/CVQA, and the selector regret of 0.29 points in Table 4—are reported as point estimates without confidence intervals. Figure 3 and Figure 4 provide cluster-bootstrap intervals only for the correlation analyses, not for the recovery comparisons that motivate the family selector. The matched-capacity comparisons are the key evidence for H3 and H4, so confidence intervals on these recovery differences are necessary to judge whether the observed gains are reliable.
minor comments (5)
  1. [§4.2] The rank criterion (90% variance, cap 32) and the sufficiency threshold ρ=0.65 are free parameters; please add a sensitivity analysis over these choices to show that the controlled-to-natural transfer does not depend on their exact values.
  2. [Eq. (9), §4.4] The admissible set C(D) and the predicted-recovery mapping from diagnostics d(D) are described qualitatively; specify exactly how diagnostic thresholds and the recovery estimator produce C(D) so that the selector is fully reproducible.
  3. [Table 2, α=0.25 row] The random-subspace entry '23.2 / −0.0125.8' appears to be a formatting error and should be corrected.
  4. [§5.3, H1 discussion] The sentence 'compactness does not imply that one direction is shared by all queries' seems to contain a typo; 'direction' should likely be 'intervention' or 'basis direction'.
  5. [General] No code, data, or paired traces are released; given that the diagnostics require precise extraction and intervention pipelines, releasing the code is important for reproducibility and for independent verification of the decomposition assumption.

Circularity Check

2 steps flagged · score 6.0 of 10

H2's static-vector and coefficient-predictability diagnostics are partly re-statements of how the static intervention and the fitted predictor are defined; the behavioral transfer and natural-task selector provide the remaining independent content.

  1. self definitional [Section 3, H2; Eqs. (5)-(6); Section 4.3]
    "Let vs(D) denote the shared component defined in Section 2. We measure the fraction of demonstration-induced energy that is shared across queries as Shareds(D) = ∥vs(D)∥2F / (∥vs(D)∥2F + 1/N ∑_{i=1}^N ∥Δzs(xi;D)−vs(D)∥2F) . (6) A static intervention should recover more of the effect of explicit M-ICL when this quantity is high."

    Eq. (5) defines vs(D) as the mean of Δzs(xi;D), and Section 4.3 states 'the static condition uses the shared component for every query.' Thus the static intervention is exactly vs(D). Eq. (6) then defines sharedness as the fraction of the total demonstration-induced change energy contained in vs(D). H2's claim that static recovery rises with sharedness is therefore true by construction at the representation level: a static vector trivially 'recovers' more of the change when that change is, by definition, mostly the static vector. The only non-tautological content is whether this representational overlap transfers to behavioral accuracy, which the random-basis, oracle, and shuffled-coefficient controls address.

  2. fitted input called prediction [Section 4.2, Section 4.3, and Section 5.3 (Fig. 3b)]
    "For H2, sharedness and held-out coefficient-prediction R2 measure whether query-varying structure can be selected from the zero-shot query. ... The deployable conditional condition instead uses coefficients predicted from the zero-shot query representation. ... held-out coefficient-predictor R2 predicts the gain from conditional selection (ρs = 0.80)."

    The coefficient-predictor R2 is measured on the same two-layer MLP whose outputs are inserted in the predicted-conditional condition. Therefore high R2 is definitionally equivalent to the deployed coefficients being close to the oracle coefficients. The claimed conditional gain is the behavioral consequence of using those exact coefficients, so the correlation between R2 and gain is a property of the fitted model's own accuracy, not an independent theoretical law. The paper's shuffle control shows the coefficients matter, but the 'prediction' from R2 to gain is partially the same measurement appearing on both sides of the claimed relationship, with the only independent residue being how coefficient error maps to behavior.

full rationale

The derivation chain starts from Eq. (2)'s counterfactual change Δz_s and Eq. (3)'s low-rank decomposition. The shared component v_s is the query-mean of Δz_s (Eq. 5), and the static intervention is defined as v_s (Section 4.3). Consequently Shared_s (Eq. 6) measures, by construction, how much of the total change the static intervention already contains. H2's central association between sharedness and static recovery is therefore partly a definitional coupling: a static vector trivially accounts for more of the representation when the representation is mostly the static vector. The genuinely empirical residue is whether that representational overlap produces behavioral recovery, which the paper tests with random-basis, oracle, and shuffled-coefficient controls. Similarly, the coefficient-predictor R2 is measured on the same MLP whose outputs are deployed in the predicted-conditional arm, so the claim that predictability predicts conditional gain is partly a validation of the fitted predictor rather than an independent theoretical prediction. These are partial, not total, circularities: the behavioral measurements are out-of-sample, the random/shuffle controls are meaningful, and the natural-task selector is a frozen decision rule evaluated on held-out benchmarks. H1, H3, and H4 involve additional assumptions (notably Eq. (3)) but do not reduce to their definitions: reconstruction, support dispersion, and the additive fit-recovery gap are measured quantities with matched-budget controls. No load-bearing self-citation was found; the cited ICR method [15] is an external baseline, not a justification of the theory. Overall the central H2 'predictions' are partially constructed rather than fully independent, giving a moderate circularity burden.

Assumptions & free parameters 5 free parameters · 4 assumptions · 1 invented entities

The central claim rests on a low-rank additive decomposition of demonstration-induced activation changes, plus the assumption that counterfactual derangements isolate the demonstrated mapping. The main fitted numbers are thresholds and hyperparameters chosen on validation splits, not derived from first principles. No new physical entity is introduced; the invented construct is the representational decomposition itself, which is tested behaviorally.

free parameters (5)
  • Intervention rank (90% variance criterion, cap 32) = per episode or dataset, up to 32
    Rank is selected on validation traces by the 90% variance criterion and affects all intervention families and H1 calculations.
  • Sufficiency threshold rho = 0.65
    Fixed on the controlled validation split before natural-task evaluation; a manual threshold that determines whether the selector abstains or picks the minimal family.
  • Coefficient-predictor MLP hyperparameters = hidden 128, learning rate 1e-3, weight decay 1e-4, up to 1000 steps
    Chosen for the two-layer MLP that predicts coefficients from zero-shot query representations; conditional and routing families depend on this predictor.
  • Counterfactual count K = 5
    Number of deranged demonstration sets used in Eq. 2; chosen by hand and influences the baseline subtraction.
  • Conditional selector architecture width = two-layer GELU, width 256
    Used for all conditional families; fixed across comparisons but a design choice that affects the expressivity of the query-conditioned selector.
assumptions (4)
  • domain assumption Explicit M-ICL's behavioral effect is carried by activation changes at the final query token, as recorded at residual, MHA, MLP, and attention-logit sites.
    Section 4.2 records paired forward passes at the final query token; if the relevant computation occurs elsewhere, such as in earlier tokens or in cross-layer nonlinear dynamics, the diagnostics miss it.
  • domain assumption Matched counterfactual derangements isolate the demonstrated mapping while preserving all other prompt properties.
    Section 4.1 constructs D^-_j by deranging output symbols; any additional effect of the derangement on representations is attributed to the mapping, which could conflate other factors.
  • ad hoc to paper The demonstration-induced transformation is low-rank and additive in a query-independent basis (Eq. 3).
    This is the core modeling assumption of the Selection-Realization Hypothesis; H1 tests it empirically through reconstruction and recovery, but it is not proven.
  • standard math Standard statistical assumptions for held-out generalization and cluster bootstrap hold.
    Used for held-out R^2, correlations, and 95% confidence intervals; no formal distributional proof is given.
invented entities (1)
  • Low-rank Selection-Realization decomposition (basis B_k(D) and coefficients c_k(x,D)) independent evidence
    purpose: Models how demonstrations induce a compact family of internal changes from which the query selects; underpins all diagnostics in the paper.
    The paper provides falsifiable handles: held-out reconstruction, functional recovery, and shuffled-coefficient controls; it is not a physical entity but a postulated representational construct.

how reviews work

0 comments
Cite this review

Pith. "Pith review of When Is a Task Vector Enough? An Empirical Theory of Implicit Multimodal ICL." pith.science (2026). https://pith.science/paper/HP2ACZVM

@misc{pith2026260813385,
  author       = {Pith},
  title        = {Pith review of: When Is a Task Vector Enough? An Empirical Theory of Implicit Multimodal ICL},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HP2ACZVM}},
  note         = {Machine review of arXiv:2608.13385}
}
read the original abstract

Implicit multimodal in-context learning compresses demonstrations into internal interventions, ranging from static task vectors to query-conditioned transformations and attention routing. Despite their common goal, these methods differ substantially in how the intervention depends on the query and where it modifies the model, leaving unclear which additional complexity is necessary for a given task. We propose the Selection--Realization Hypothesis. It views demonstrations as inducing a compact family of internal changes from which the query selects, while the model's computation constrains how the selected change can be implemented. We evaluate this account using controlled multimodal tasks in which query dependence varies without changing the underlying task primitives or prompt format. By contrasting correct demonstrations with matched counterfactuals, we measure the structure of explicit M-ICL and test whether it predicts intervention behavior. We find that the success of a static task vector is closely tied to how much of the demonstration-induced change is shared across queries. Additional intervention complexity becomes useful when explicit M-ICL contains query-specific or distributed structure that a local additive shift cannot recover. These relationships extend to natural VQA benchmarks and support cost-aware method selection without access to test performance. Our results provide a unified empirical theory of when demonstrations can be compressed into a task vector and when a more expressive intervention is warranted.

Figures

Figures reproduced from arXiv: 2608.13385 by the authors.

Figure 1
Figure 1. Conceptual design space of implicit M-ICL methods. Existing approaches differ in whether intervention selection is shared or query￾conditioned and whether its realization uses local activation shifts, distributed MHA/MLP modifications, or attention routing. The dashed curve indicates increasing intervention expressiv￾ity rather than measured performance. Existing implicit M-ICL meth￾ods realize this idea in markedly… view at source ↗
Figure 2
Figure 2. Overview of the empirical framework. Controlled episodes yield paired compu￾tation changes ∆zs from correct and counterfactual demonstrations. H1 tests whether these changes admit compact compression; H2 determines whether selection can re￾main static or must depend on the query; and H3–H4 determine whether the selected transformation can be realized locally, across multiple sites, or through routing. Deci￾sion rule… view at source ↗
Figure 3
Figure 3. Testing H2 on held-out queries. Each point is one model–episode–condition observation. Panel (a) relates sharedness to static recovery; panel (b) relates coefficient￾predictor R 2 to the gain of predicted conditioning. Lines show fitted trends with cluster￾bootstrap 95% confidence intervals. benefit on the controlled tasks and VQAv2, but exceeds the strongest additive intervention by 0.04–0.06 on GQA, OK-VQA, and CV… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Partial-residual tests of H3–H4. Panel (a) relates support dispersion to multi￾site gain after controlling for strongest-local recovery; panel (b) relates the additive fit–recovery gap to routing gain after controlling for strongest-additive recovery. Lines show fitted…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

31 extracted references · 2 canonical work pages

  1. [1]

    Akyürek, E., Schuurmans, D., Andreas, J., Ma, T., Zhou, D.: What learning al- gorithm is in-context learning? investigations with linear models (2023),https: //arxiv.org/abs/2211.15661

    arXiv 2023

  2. [2]

    Awadalla, A., Gao, I., Gardner, J., Hessel, J., Hanafy, Y., Zhu, W., Marathe, K., Bitton, Y., Gadre, S., Sagawa, S., Jitsev, J., Kornblith, S., Koh, P.W., Ilharco, G., Wortsman, M., Schmidt, L.: Openflamingo: An open-source framework for training large autoregressive vision-language models (2023),https://arxiv.org/ abs/2308.01390

    arXiv 2023

  3. [3]

    Baldassini, F.B., Shukor, M., Cord, M., Soulier, L., Piwowarski, B.: What makes multimodal in-context learning work? (2024),https://arxiv.org/abs/2404. 15736

    work page 2024

  4. [4]

    Chen, S., Han, Z., He, B., Liu, J., Buckley, M., Qin, Y., Torr, P., Tresp, V., Gu, J.: Can multimodal large language models truly perform multimodal in-context learning? (2024),https://arxiv.org/abs/2311.18021

    arXiv 2024

  5. [5]

    Dai, D., Sun, Y., Dong, L., Hao, Y., Ma, S., Sui, Z., Wei, F.: Why can gpt learn in- context? language models implicitly perform gradient descent as meta-optimizers (2023),https://arxiv.org/abs/2212.10559

    arXiv 2023

  6. [6]

    Doveh, S., Perek, S., Mirza, M.J., Lin, W., Alfassy, A., Arbelle, A., Ullman, S., Kar- linsky, L.: Towards multimodal in-context learning for vision & language models (2024),https://arxiv.org/abs/2403.12736

    arXiv 2024

  7. [7]

    Goyal, Y., Khot, T., Summers-Stay, D., Batra, D., Parikh, D.: Making the v in vqa matter: Elevating the role of image understanding in visual question answering (2017),https://arxiv.org/abs/1612.00837

    arXiv 2017

  8. [8]

    Hendel, R., Geva, M., Globerson, A.: In-context learning creates task vectors (2023),https://arxiv.org/abs/2310.15916

    arXiv 2023

Show all 31 references
  1. [9]

    Huang,B.,Mitra,C.,Arbelle,A.,Karlinsky,L.,Darrell,T.,Herzig,R.:Multimodal task vectors enable many-shot multimodal in-context learning (2024),https:// arxiv.org/abs/2406.15334

    2024 arXiv

  2. [10]

    Huang, Y., Roth, K., Bouniot, Q., Xu, W., Akata, Z.: Dissecting multimodal in- context learning: Modality asymmetries and circuit dynamics in modern trans- formers (2026),https://arxiv.org/abs/2601.20796

    2026 arXiv

  3. [11]

    Hudson, D.A., Manning, C.D.: Gqa: A new dataset for real-world visual reason- ing and compositional question answering (2019),https://arxiv.org/abs/1902. 09506

    2019

  4. [12]

    Jiang, Y., Fu, J., Hao, C., Hu, X., Peng, Y., Geng, X., Yang, X.: Mimic in-context learning for multimodal tasks (2025),https://arxiv.org/abs/2504.08851

    2025 arXiv

  5. [13]

    Laurençon, H., Tronchon, L., Cord, M., Sanh, V.: What matters when building vision-language models? (2024),https://arxiv.org/abs/2405.02246

    2024 arXiv

  6. [14]

    Li, J., Hu, Q., Li, J., Wang, W.: Stare at the structure: Steering icl exemplar selection with structural alignment (2025),https://arxiv.org/abs/2508.20944

    2025 arXiv

  7. [15]

    Li, J., Li, Y., Han, L., Tang, R., Wang, W.: Train once, reuse everywhere: Generalizable implicit in-context learning by routing attention (2026),https: //arxiv.org/abs/2509.22854

    2026 arXiv

  8. [16]

    Li, J., Li, Y., Huang, K.H.: Steering vector fields for context-aware inference-time control in large language models (2026),https://arxiv.org/abs/2602.01654

    2026

  9. [17]

    Li, Y., Cao, Y., He, H., Cheng, Q., Fu, X., Xiao, X., Wang, T., Tang, R.: M2iv: To- wards efficient and fine-grained multimodal in-context learning via representation engineering (2025),https://arxiv.org/abs/2504.04633 Empirical Theory of Implicit Multimodal ICL 15

    2025 arXiv

  10. [18]

    Li, Z., Xu, Z., Han, L., Gao, Y., Wen, S., Liu, D., Wang, H., Metaxas, D.N.: Implicit in-context learning (2025),https://arxiv.org/abs/2405.14660

    2025 arXiv

  11. [19]

    Liu, H., Li, C., Li, Y., Lee, Y.J.: Improved baselines with visual instruction tuning (2024),https://arxiv.org/abs/2310.03744

    2024 arXiv

  12. [20]

    Liu, S., Ye, H., Xing, L., Zou, J.: In-context vectors: Making in context learning more effective and controllable through latent space steering (2024),https:// arxiv.org/abs/2311.06668

    2024 arXiv

  13. [21]

    Marino, K., Rastegari, M., Farhadi, A., Mottaghi, R.: Ok-vqa: A visual question answering benchmark requiring external knowledge (2019),https://arxiv.org/ abs/1906.00067

    2019 arXiv

  14. [22]

    Olsson, C., Elhage, N., Nanda, N., Joseph, N., DasSarma, N., Henighan, T., Mann, B., Askell, A., Bai, Y., Chen, A., Conerly, T., Drain, D., Ganguli, D., Hatfield- Dodds, Z., Hernandez, D., Johnston, S., Jones, A., Kernion, J., Lovitt, L., Ndousse, K., Amodei, D., Brown, T., Cl...

    2022 arXiv

  15. [23]

    von Oswald, J., Niklasson, E., Randazzo, E., Sacramento, J., Mordvintsev, A., Zhmoginov,A.,Vladymyrov,M.:Transformerslearnin-contextbygradientdescent (2023),https://arxiv.org/abs/2212.07677

    2023 arXiv

  16. [24]

    Peng, Y., Hao, C., Yang, X., Peng, J., Hu, X., Geng, X.: Live: Learnable in-context vector for visual question answering (2024),https://arxiv.org/abs/2406.13185

    2024 arXiv

  17. [25]

    Qin, L., Chen, Q., Fei, H., Chen, Z., Li, M., Che, W.: What factors affect multi- modal in-context learning? an in-depth exploration (2024),https://arxiv.org/ abs/2410.20482

    2024 arXiv

  18. [26]

    Romero, D., Lyu, C., Wibowo, H.A., Lynn, T., Hamed, I., Kishore, A.N., Man- dal, A., Dragonetti, A., Abzaliev, A., Tonja, A.L., Balcha, B.F., Whitehouse, C., Salamea, C., Velasco, D.J., Adelani, D.I., Meur, D.L., Villa-Cueva, E., Koto, F., Fa- rooqui, F., Belcavello, F., Batna...

    2024 arXiv

  19. [27]

    Singh, A.K., Moskovitz, T., Hill, F., Chan, S.C.Y., Saxe, A.M.: What needs to go right for an induction head? a mechanistic study of in-context learning circuits and their formation (2024),https://arxiv.org/abs/2404.07129

    2024 arXiv

  20. [28]

    Sun, Q., Cui, Y., Zhang, X., Zhang, F., Yu, Q., Luo, Z., Wang, Y., Rao, Y., Liu, J., Huang, T., Wang, X.: Generative multimodal models are in-context learners (2024),https://arxiv.org/abs/2312.13286

    2024 arXiv

  21. [29]

    Tai, Y., Fan, W., Zhang, Z., Zhu, F., Zhao, R., Liu, Z.: Link-context learning for multimodal llms (2023),https://arxiv.org/abs/2308.07891

    2023 arXiv

  22. [30]

    Todd, E., Li, M.L., Sharma, A.S., Mueller, A., Wallace, B.C., Bau, D.: Function vectors in large language models (2024),https://arxiv.org/abs/2310.15213

    2024 arXiv

  23. [31]

    Li Storyline 1.Why interesting? a

    Xie, S.M., Raghunathan, A., Liang, P., Ma, T.: An explanation of in-context learn- ing as implicit bayesian inference (2022),https://arxiv.org/abs/2111.02080 16 J. Li Storyline 1.Why interesting? a. Explicit M-ICL: demonstrations adapt a frozen model but are reprocessed for ev...

    2022 arXiv

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.