{"id":"51ceca25-61e6-4c0b-af8a-c7ac5fc48785","arxiv_id":"2501.03012","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Concept shift vectors, computed as mean activation differences, can partially recover fine-tuned multimodal LLM concepts and steer model outputs without additional training.","lead":"This paper shows how to track and steer the internal concepts of multimodal AI models after fine-tuning, using simple vector arithmetic instead of retraining. It offers a training-free way to debias or make image-text models safer, though several evaluations reuse the same data for both fitting and reporting.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Concept recovery is evaluated in-sample: at α=1, shifting an original k-means centroid by the mean original-to-finetuned delta of its own cluster equals the mean fine-tuned representation of those same images, so Fig. 6 may be near-tautological.","rationale":"The reader's weakest_assumption correctly flags that recovery is evaluated in-sample and that transfer to unseen images is untested, but it bundles this with reliance on the linear representation hypothesis. I see the in-sample evaluation as the more load-bearing concern because it can be made precise and mechanical: Eq. (4) with α=1 and k-means centroids makes the shifted concept equal to the mean fine-tuned representation of the original cluster's own images. This does not require the linear representation hypothesis to fail; it fails to establish generalization regardless of linearity. The paper does offer some independent support for the broader framework: steering is evaluated on held-out validation sets, multiple MLLMs are tested, and the safety/debiasing results are concrete downstream demonstrations. However, those applications do not rescue the specific claim that fine-tuned concepts can be recovered by additive shifts, because the recovery metric itself is computed on the same images used to define the shift. The appropriate outcome is to keep the reader's CONDITIONAL verdict: the central recovery claim should not be accepted until a held-out recovery experiment and a non-cluster random-shift control are reported. My recommendation is therefore UNCHANGED rather than ACCEPT or REJECT, since the concern is significant but the paper's other results and released code leave a conditional path to acceptance.","tokens_in":29758,"tokens_out":4466,"duration_ms":45326,"concrete_test":"Split the COCO/VG images into disjoint training and test sets with no image overlap. Compute shift vectors Δ_k on the training split only, and extract original concepts U_a and fine-tuned concepts U_b on the test split. Evaluate T-Overlap of u_a^k + Δ_k versus matched u_b^{m(k)} against the original-concept baseline, exactly as in Fig. 6. Additionally, include a control where Δ_k is replaced by the mean shift of an equal-sized random sample of training images, not specific to the concept's cluster. If the held-out gain collapses or matches the random control, the Fig. 6 recovery result is an in-sample artifact rather than evidence for transferable fine-tuned concept recovery.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central quantitative evidence for the recovery claim (Section 4.1, Fig. 6) is computed in-sample. The authors fix S(1)=S(2), extract both U_a and U_b from the same images, define A_k as the images for which u_a^k is the most active concept, and set Δ = mean_{m∈A_k}(b_m − a_m). Under the standard k-means assignment used, the centroid satisfies u_a^k = mean_{m∈A_k} a_m, so with the default α=1, Eq. (4) reduces to u_s^k = mean_{m∈A_k} b_m. Thus the 'shifted concept' is exactly the conditional mean of the fine-tuned representations of the original cluster's own images, and it is compared against fine-tuned concept centroids estimated from those same images. The observed T-Overlap gain therefore does not demonstrate that a shift vector computed from the original model plus a stored direction recovers fine-tuned concepts in general; it may only show that a within-sample conditional mean moves toward the fine-tuned cluster. The steering experiments provide some held-out support for shift vectors, but the concept-recovery claim—the main interpretability result—has no held-out evaluation.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes a concept-based framework for analyzing and steering multimodal LLMs (MLLMs). Concepts are extracted as k-means centroids of residual-stream activations and grounded in text and image spaces. For fine-tuned models, the authors define per-concept 'shift vectors' as the mean change of the representations of the images assigned to each original concept (Section 3.2, Eq. 4) and claim these recover fine-tuned concepts from the original model without accessing the fine-tuned model at inference. The same additive-shift mechanism is used for steering: coarse steering vectors between sets of answers/styles, and fine-grained steering vectors between concept pairs, with applications to VQA answer manipulation, caption style control, gender debiasing, and safety (refusal) alignment. Experiments cover LLaVA-1.5, Qwen2-VL-Instruct, and Idefics2, with fine-tuning on Visual Genome place/color/sentiment subsets.","tokens_in":30017,"tokens_out":9553,"duration_ms":78439,"significance":"The paper addresses a worthwhile and underexplored question — how fine-tuning changes the internal concept space of MLLMs — and offers a simple, training-free tool with plausible applications. Its strengths are the breadth of models and tasks (three MLLMs; VG fine-tuning subsets; VQAv2 and COCO evaluations), the extensive ablations (number of concepts, layer, steering strength, number of samples, token position), the public code release, and an honest limitations section acknowledging the linear representation hypothesis. The steering experiments are held-out on validation sets and, if confirmed with proper baselines, would be practically useful. The decisive weakness is that the central interpretability claim — concept recovery — currently rests on an in-sample evaluation, and the steering/safety results lack error bars and random-direction controls. With a held-out recovery evaluation and stronger baselines, the paper would make a solid contribution.","major_comments":[{"comment":"The concept-recovery evaluation is in-sample and, at the default α=1, is close to tautological. Since S(1)=S(2), both U^a and U^b are extracted from the same images, and Δ_k is the mean of (b_m − a_m) over A_k. For k-means centroids (up to the sign convention in the definition of A_k via |v_i^a(x_m)|), u_a^k equals the mean of the a_m in its cluster, so Eq. (4) reduces to u_s^k = mean_{m∈A_k} b_m: the 'shifted concept' is the conditional mean of the fine-tuned representations of the original cluster's own images, and it is compared against fine-tuned centroids estimated from those same images. The T-Overlap gain in Fig. 6 is therefore largely a consequence of the construction and does not demonstrate that a shift vector computed on one set recovers fine-tuned concepts on unseen images. Please add a held-out evaluation: compute Δ on a training split, extract U^a and U^b from a disjoint split (or evaluate on held-out images), and report T-Overlap there, alongside baselines such as a single global shift (mean of all b_m − a_m) and a random direction of matched norm. The claim in Section 3.2 that shifted concepts are computed 'without accessing the fine-tuned model' also needs this held-out evidence, since computing Δ already requires the fine-tuned model's representations on the paired set.","section":"Section 3.2, Eq. (4); Section 4.1, Fig. 6"},{"comment":"The steering experiments report no error bars and no random-direction baseline, and the reported numbers are partly selected on the validation set used for reporting. The steering layer is chosen by evaluation on the VQAv2 validation set (App. B.1), and the 'meaningful' fine-grained directions are selected by their measured impact on the same kind of set (App. B.3); the text does not clarify whether the 5k-sample subset underlying Table 2 is the same set used for these selections. A control in which a random direction of comparable norm (or a PCA direction) is added at the same layer would be needed to show that the answer-type shifts in Tables 1–3 are due to the semantic content of the steering vector rather than to any large perturbation. Bootstrap confidence intervals on the answer counts would also substantiate the 'targeted steering' claim, especially since the Yes→No row of Table 2 changes yes/no accuracy by 21.8 points while 'other' accuracy still drops by 2.1 points.","section":"Section 4.2, Tables 1–3, Fig. 9; App. B.1, B.3"},{"comment":"The safety result risks circularity because the steering vector is derived from samples on MM-SafetyBench using the same ASR metric on which it is then evaluated. The derivation set is selected as the MM-SafetyBench samples where Qwen2-VL-Instruct is unsafe with an image and safe without an image (judged by ASR), and the main paper reports ASR 45/100 → 5/100 on 'a portion' of MM-SafetyBench without stating that the 100 evaluation samples are disjoint from the derivation subset. Please specify the exact split, evaluate on a fixed held-out subset (or the full benchmark), and state the number of derivation samples used. The helpfulness check in Table 9 rests on only 100 samples with 3 borderline cases, so a larger evaluation of benign queries is needed to support the claim that the safety steering preserves utility.","section":"Table 5; App. D"}],"minor_comments":[{"comment":"The fine-grained steering vector is defined with a duplicated term, 'sf_ij = sf_ij = uj − ui'; the intended definition is sf_ij = uj − ui. Additionally, the definition of A_k in Section 3.2 uses the absolute activation |v_i^a(x_m)|, whereas standard k-means assignment uses signed projections; please specify the exact assignment rule so that A_k provably corresponds to the clusters whose centroids appear in Eq. (4).","section":"Section 3.3"},{"comment":"There is a typo, 'Throught', in the first sentence of Section 4.1. Also, Eq. (3) defines T-Overlap with |Twords(u)| in the denominator, making the metric asymmetric when the two concepts produce different grounding-word sets; please state the common value of Ngrounding used for both arguments.","section":"Section 4.1; Eq. (3)"},{"comment":"The reported p-values of '0.0000' should be expressed as, e.g., p < 10^-4, and the text should state the number of concept-level points pooled and whether the Pearson correlation is computed within or across the four tokens of interest.","section":"Fig. 7; App. A.5"},{"comment":"The gender-debiasing results report only counts of gendered-to-neutral conversions; without a caption-quality metric such as CIDEr (or human evaluation), it is unclear whether the conversions preserve caption fidelity, particularly given the small totals (794, 815, and 926 samples, respectively).","section":"Table 4; App. C"},{"comment":"The limitations paragraph appropriately acknowledges the linear representation hypothesis via Ref. [14], but it should also note that all quantitative evaluations (T-Overlap, answer-type counts, ASR string matching) are automatic proxies and that no human evaluation of concept interpretability or of steered caption quality is provided.","section":"Section 5"},{"comment":"There is a typo, 'safty', in App. D, and the relationship between the 733-sample steering-derivation subset of Table 8 and the 100-sample evaluation of Table 5 should be stated explicitly in the main text so that the reader can verify the train/evaluation split.","section":"App. D"}],"recommendation":"major_revision","confidential_remarks":"The paper is clearly written, the code is public, and the ablations are thorough; the main risk is the in-sample evaluation of the concept-recovery claim (Section 4.1, Fig. 6), which I believe is fixable within the manuscript's scope by adding a held-out split and proper baselines. The safety section needs a clean derivation/evaluation split. The authors should also position their steering contribution against multimodal task vectors (Ref. [15]) a bit more concretely, since the novelty of 'first MLLM steering' is undercut by that line of work. Overall, the contribution is promising but the central evidence needs strengthening before publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The concept shift vector is a genuinely useful idea: it gives a cheap way to characterize how fine-tuning moves a model's latent concepts, and the held-out steering results—VQA answer flipping, caption style control, debiasing, safety—show the approach actually moves outputs. I'd call the paper worth a serious referee. But the central interpretability claim, that shift vectors recover fine-tuned concepts, is not supported as stated.\n\nWhat's new: extending activation steering to multimodal LLMs and framing fine-tuning shifts as per-concept mean shifts in the residual stream. The paper is honest about borrowing concept extraction from CoX-LMM, and it ships code. The cross-model steering experiments (LLaVA, Qwen2-VL, Idefics2) are a real plus, and the shift-consistency versus recovery correlation is a nice diagnostic.\n\nThe soft spots are in proportion. The recovery evaluation (Fig. 6) is in-sample: shift vectors are computed and evaluated on the same image set. The stress test is right—with k-means, at α=1 the shifted concept is exactly the mean of the fine-tuned representations of the original cluster's own images. So the T-Overlap gain mostly shows that within a cluster, the average fine-tuned representation is more aligned with fine-tuned concepts than the original centroid is. That's not nothing, but it doesn't demonstrate recovery on unseen images. They need a held-out variant: compute shift vectors on one set, shift original concepts from a disjoint set, and compare to fine-tuned concepts from that disjoint set.\n\nSecond, the safety evaluation doesn't say whether the MM-SafetyBench subset used to derive the steering vector overlaps with the evaluation subset. If it does, the ASR drop from 45 to 5 is inflated. The authors should state the split and, if needed, re-run on a held-out portion.\n\nThird, the steering results have no error bars or random-baseline comparisons, and the steering layer is chosen on the same validation set used for reporting. Minor, but worth tightening.\n\nThe reliance on the linear representation hypothesis is acknowledged and the PCA separability plots are some evidence, but the held-out recovery test is the thing to demand.\n\nThis paper is for people working on MLLM interpretability or lightweight model control. It deserves a serious referee, but my advice is to ask for held-out recovery and disjoint safety splits before the interpretability claims are accepted.","headline":"A useful MLLM steering framework whose headline recovery claim is undercut by in-sample evaluation, but the held-out steering results and code make it worth refereeing.","tokens_in":30562,"tokens_out":3858,"would_cite":false,"duration_ms":35659,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Fine-tuned MLLM concepts can be recovered by adding simple mean shift vectors to the original model's concepts, and the same additive shifts steer outputs for debiasing and safety—all without retraining.","keywords":["concept-based explainability","multimodal LLM","representation shift","concept shift vectors","model steering","gender debiasing","safety alignment","linear representation hypothesis"],"falsifier":"Test concept recovery on a held-out set of images that were not used to compute the shift vectors: if the T-Overlap improvement of shifted over original concepts disappears on those images, then the shift vectors only memorize the training set. Alternatively, find a fine-tuning regime where per-sample shifts are consistent but the shifted concept fails to match the fine-tuned concept in T-Overlap, which would break the claimed link between consistency and recovery.","tokens_in":29559,"feed_emoji":"🧭","tokens_out":5849,"duration_ms":49569,"temperature":0.7,"pith_summary":"This paper tries to show that the changes a multimodal LLM undergoes during fine-tuning can be captured as simple additive shifts of its latent concepts, and that these same shifts can be used to steer the model's behavior without any additional training. The authors extract interpretable visual and textual concepts from the residual stream of an MLLM, compute for each concept the mean change in representations between the original and fine-tuned model, and add that shift vector to the original concept. On fine-tuning experiments for places, colors, and sentiments, the shifted concepts are textually closer to the fine-tuned ones than the original concepts are, indicating partial recovery. They then use the same additive mechanism to change answer types in VQA, alter caption style, convert gendered captions to neutral ones, and reduce the attack success rate on a multimodal safety benchmark. If correct, this gives a training-free way to inspect what fine-tuning changes and to control MLLM outputs cheaply.","feed_headline":"Additive shift vectors recover fine-tuned concepts, no retraining","feed_subtitle":"Simple mean differences reproduce fine-tuning's changes and steer MLLM outputs — no training required.","key_machinery":"Concept shift vectors: for each concept $u^a_k$ extracted from the original model as a K-means centroid in the residual stream, the shift vector $\\Delta^{a\\to b}_k$ is the average of the per-sample differences $b_m - a_m$ over the samples that most activate that concept; the shifted concept is $u^s_k = u^a_k + \\alpha \\Delta^{a\\to b}_k$. The same construction, computed either between sample sets or between concept pairs, yields steering vectors applied additively to hidden states. Concepts are grounded in text through the unembedding matrix (logit lens), and concept similarity is measured by text grounding overlap (T-Overlap), which is the evaluation metric for recovery.","core_discovery":"The paper's central claim is that fine-tuned concepts in a multimodal LLM are approximately recoverable from the original model by translating each original concept along a concept shift vector, defined as the mean of per-sample representation changes between the two models (Equation 4). The same shift-vector geometry, applied to activations rather than to concept dictionaries, provides a steering mechanism: coarse steering vectors are differences between mean representations of target and source sample sets, and fine-grained steering vectors are differences between paired concepts. The authors report that steering significantly increases the frequency of target answers, changes caption styles, converts gendered captions to neutral ones across three MLLMs, and drops the attack success rate from 45/100 to 5/100 on a MM-SafetyBench subset. They frame this as evidence that MLLM behavior can be monitored and controlled at a concept level without training.","pith_inferences":["The method's reliance on mean shifts suggests it will work best when fine-tuning changes are approximately linear and unimodal; fine-tunings that rotate or nonlinearly warp concept manifolds would be expected to show worse recovery, which the paper's own citation to non-linear features implies.","The steering vectors could in principle be transferred between models in the same family, since only hidden-state differences are used; this is an extension the paper does not test.","A natural testable extension is to use shift vectors computed on one set of images to steer on a held-out set, which would validate the implicit assumption that the shift direction generalizes beyond the samples used to compute it.","The safety steering result suggests a general recipe: contrasting a model's behavior on the same prompt with and without an image isolates the contribution of the image modality, which could be used more broadly for diagnosing and correcting modality-induced biases."],"forward_implications":["Fine-tuned concept recovery does not require access to the fine-tuned model after the shift vectors are computed: the vectors are learned once and then applied to the original model.","The same additive shift applied to activations, not just to concept centroids, can change the distribution of VQA answers toward a target type or a specific answer while leaving other answer types largely unchanged.","Steering can alter caption style (colors, places, sentiments) and can debias captions, with fine-grained concept-to-concept steering converting more gendered captions to neutral ones than coarse steering.","Safety behavior can be steered: applying a vector derived from safe text-only responses shifts unsafe multimodal responses toward refusal, reducing the attack success rate on a subset of MM-SafetyBench from 45% to 5%.","Recovery quality correlates with shift consistency: concepts whose individual sample shifts align with the mean shift vector are recovered better, a relationship the paper quantifies with a statistically significant Pearson correlation."],"supporting_citations":[{"why":"Supplies the K-means concept extraction and text/image grounding procedure the paper's analysis is built on.","marker":"[47]"},{"why":"The acknowledged evidence that some language model features are not linear, which bounds the method's validity.","marker":"[14]"},{"why":"Empirical support for the linear representation hypothesis in LLMs, the premise that makes additive shifts meaningful.","marker":"[48]"},{"why":"Cited as supporting evidence that LLM features are linearly separable, motivating later-layer steering.","marker":"[42]"},{"why":"The tuned lens technique for decoding hidden states through the unembedding matrix, used to ground concepts in text.","marker":"[5]"},{"why":"Provides the VQAv2 answer-type annotations and evaluation protocol for the steering experiments.","marker":"[24]"},{"why":"The MM-SafetyBench benchmark used to measure attack success rate in the safety steering experiment.","marker":"[40]"}],"fun_headline_variants":["Shift vectors recover fine-tuned MLLM concepts, no retraining","Additive shifts recover fine-tuned concepts and steer MLLMs","No-training MLLM steering via concept shift vectors","Fine-tune shifts mapped to concepts for MLLM steering","Concept shift vectors: cheap MLLM steering without training"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a concept is represented as a linear direction in the model's latent space, so that adding a mean difference vector moves the concept to its fine-tuned position; if concepts are non-linear or the shift vectors do not transfer to unseen images, both concept recovery and steering lose their foundation.","fun_headline_variants_meta":{"raw":{"variants":["Shift vectors recover fine-tuned MLLM concepts, no retraining","Additive shifts recover fine-tuned concepts and steer MLLMs","No-training MLLM steering via concept shift vectors","Fine-tune shifts mapped to concepts for MLLM steering","Concept shift vectors: cheap MLLM steering without training"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000197,"raw_usage":{"total_tokens":1351,"prompt_tokens":920,"completion_tokens":431,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":536,"completion_tokens_details":{"reasoning_tokens":345}},"tokens_in":536,"tokens_out":431,"duration_ms":4675,"temperature":1.0,"reasoning_tokens":345,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T21:58:38.201134+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Test concept recovery on a held-out set of images that were not used to compute the shift vectors: if the T-Overlap improvement of shifted over original concepts disappears on those images, then the shift vectors only memorize the training set. Alternatively, find a fine-tuning regime where per-sample shifts are consistent but the shifted concept fails to match the fine-tuned concept in T-Overlap, which would break the claimed link between consistency and recovery.","supporting_citations":[{"cited_title":"A concept-based explainability framework for large multimodal models","cited_arxiv_id":null,"evidence_quote":"Supplies the K-means concept extraction and text/image grounding procedure the paper's analysis is built on."},{"cited_title":"The linear representation hypothesis and the geometry of large language models","cited_arxiv_id":null,"evidence_quote":"Empirical support for the linear representation hypothesis in LLMs, the premise that makes additive shifts meaningful."},{"cited_title":"Actually, othello-gpt has a linear emergent world model, 2023","cited_arxiv_id":null,"evidence_quote":"Cited as supporting evidence that LLM features are linearly separable, motivating later-layer steering."},{"cited_title":"Making the v in vqa matter: Elevating the role of image understanding in visual question answering","cited_arxiv_id":null,"evidence_quote":"Provides the VQAv2 answer-type annotations and evaluation protocol for the steering experiments."},{"cited_title":"Mm-safetybench: A benchmark for safety evaluation of multimodal large language models","cited_arxiv_id":null,"evidence_quote":"The MM-SafetyBench benchmark used to measure attack success rate in the safety steering experiment."}],"review_version":1}