{"id":"84581f95-360f-4834-a081-c11ac82e3201","arxiv_id":"2507.18940","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"Selectively fine-tuning only 40% of a multimodal LLM's layers and neurons can match or slightly beat full fine-tuning on multilingual image-to-text translation benchmarks, though the measured gains are marginal.","lead":"This paper proposes a way to fine-tune a large multimodal translation model for many languages while updating only a fraction of its parameters, by selecting which layers and neurons to train for each language pair. The authors report slightly higher translation scores than full fine-tuning on two multilingual image-translation benchmarks.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The '40% of parameters' claim is not checkable: Eq. 3's reference activations X_B_l and the split used for selection are unspecified, so the method may quietly require a full fine-tuning pass, and the 40% label conflates layers with parameters.","rationale":"The reader's weakest assumption correctly identifies the underspecified reference model and the possibility of selection on test information. I agree with that diagnosis, but I would sharpen it: the most load-bearing issue is not only whether the gains are inflated, but whether the method's efficiency claim is well-defined at all. The paper never defines the actual parameter budget, never reports the origin of the fine-tuned activations used for importance scoring, and never states the split on which the selection hyperparameters are chosen. If the reference is a full fine-tuned model, the method's total training cost is at least that of full fine-tuning plus the selective run, making the 'fine-tuning only 40%' statement misleading even if the second stage is cheap. If the 40% label refers to layer fraction while the neuron masking updates a smaller or larger subset, the parameter count needs correction. The 80% variant shows larger gains and may still be a useful finding, but the central SOTA claim rests on a 0.1-0.3 BLEU average difference over a single run, which is not sufficient without protocol transparency or significance testing. I therefore keep the verdict conditional rather than unverified or rejected: the missing details are concrete and checkable, and the authors could resolve them by releasing code, parameter counts, and a validation-based selection protocol. My disagreement with the reader is only in emphasis: I consider the parameter-count definition and the total adaptation cost to be as central as the data-split leakage risk, which is why I mark agreement as partial.","tokens_in":11746,"tokens_out":5045,"duration_ms":54675,"concrete_test":"Ask the authors to release the training configuration and instrument the gradient mask in Eq. 9 to report (a) the exact checkpoint used as X_B_l in Eq. 3 (pretrained, full SFT, or LoRA warmup), (b) the split used to compute R_l and to set alpha, epsilon, and the 1:9 ratio, and (c) the total number of gradient-updated parameters for the 40% and 80% configurations. Then rerun the selection procedure using activations from a short LoRA warmup on the training split only, freeze the reference, and evaluate with three independent seeds. If the 40% variant no longer beats LLaVA-1.5-SFT, or if the reference turns out to be the full SFT checkpoint, the headline efficiency claim should be revised to report total adaptation cost rather than the second-stage parameter fraction.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is that LLaVA-NeuMT, while fine-tuning only 40% of parameters, surpasses full fine-tuning and reaches SOTA. The load-bearing precondition is that the layer and neuron selection (Eqs. 3-8) is obtained without paying full fine-tuning cost and without using test information. The paper never states which checkpoint supplies X_B_l in Eq. 3, which data split is used to compute R_l, I_k^n, sigma^2, and epsilon, or what fraction of parameters the 40% and 80% labels actually cover. If X_B_l is the fully fine-tuned LLaVA-1.5-SFT baseline reported in Tables 1-2, then the pipeline first performs complete SFT, uses it to select layers and neurons, and then fine-tunes a second model; total trainable-parameter updates would then exceed full fine-tuning, and 'fine-tuning only 40% of the model parameters' would describe only the second stage, not the whole adaptation procedure. The parameter-count ambiguity is also unresolved: Section 4.3 varies the fraction of selected layers, and the 1:9 specific-to-agnostic neuron ratio is described as applied within those layers. If neurons are masked within selected layers, the true updated parameter fraction is lower than the layer fraction; if Eqs. 7-8 partition all selected-layer neurons into specific or agnostic sets, then no neuron is actually frozen and the 40% figure is just the layer fraction. The empirical support is thin: the 40% variant beats the full-SFT baseline by 0.1-0.3 BLEU on average (Tables 1-2) while losing on several individual language pairs, and no error bars or repeated runs are reported. These gaps leave the efficiency and SOTA claims unverifiable as stated.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes LLaVA-NeuMT, a method for multilingual multimodal machine translation built on LLaVA-1.5-7B. The method has three components: (i) a layer selection step (Section 3.2) that ranks layers by the squared cosine similarity between activations of the pretrained model and a fine-tuned model, keeping the top alpha fraction; (ii) a neuron classification step (Section 3.3) that labels neurons as language-specific or language-agnostic using importance-score variance across language pairs; and (iii) a gradient masking strategy (Section 3.4) that updates only selected neurons. Experiments on M3-Multi30K and M3-AmbigCaps report BLEU scores for 40% and 80% layer-selection variants, with the abstract claiming that fine-tuning only 40% of parameters surpasses full fine-tuning and achieves state-of-the-art results. The paper also includes ablations on layer fraction, neuron ratio, and visualizations of layer and neuron importance.","tokens_in":12133,"tokens_out":3698,"duration_ms":39409,"significance":"If the central efficiency claim were verified, the paper would make a useful contribution to parameter-efficient multilingual multimodal adaptation: it proposes a concrete layer-and-neuron selection recipe, reports ablations over two hyperparameters, and provides some analysis of which modules matter. The paper also compares against several open- and closed-source multimodal baselines. However, the current evidence does not support the headline claims as stated. The selection protocol in Eq. (3) is underspecified in a way that directly affects whether the method actually saves training cost, the parameter-count accounting is ambiguous, the small BLEU gains over the full fine-tuning baseline lack statistical support, and key hyperparameters appear to be selected using test-set performance. These issues are load-bearing for the paper's central claim, so the manuscript needs substantial revision before the efficiency and SOTA conclusions can be accepted.","major_comments":[{"comment":"The definition of R_l requires activations X_B_l from a 'fine-tuned model', but the manuscript never states which checkpoint supplies X_B_l nor on which data split these activations are computed. If X_B_l comes from the fully fine-tuned LLaVA-1.5-SFT model used as the baseline in Tables 1 and 2, then layer selection requires a complete SFT pass before the '40% fine-tuning' stage begins, so the total computational cost and number of parameter updates over the whole pipeline can exceed full fine-tuning. This would directly invalidate the efficiency claim in the abstract. The paper must specify the reference checkpoint, the data split used for selection, and whether selection uses any information from the test sets; otherwise the method is not reproducible and the efficiency claim is not checkable.","section":"Section 3.2, Eq. (3)"},{"comment":"The layer fraction alpha and the 1:9 specific-to-agnostic neuron ratio appear to be selected by sweeping on BLEU scores reported in Figure 3a and Figure 3b. If these sweeps are performed on the test sets, then the BLEU numbers in Tables 1 and 2 for the selected configurations are not independent test-set measurements, and the 'SOTA' claim is circular. The paper should report a held-out validation split for hyperparameter selection, or clearly state that the reported test results are for configurations selected without test-set access. The Section 5 limitation statement that 'our current design relies on fixed thresholds' makes this concern more acute, because the thresholds are not derived from a principled criterion but appear to be tuned to the evaluation sets.","section":"Sections 4.3-4.4 and Figures 3a-3b"},{"comment":"The claim 'fine-tuning only 40% of the model parameters' is ambiguous. The 40% label is used for the top 40% of layers, but Eq. (9) also freezes neurons outside A ∪ S_k, so within selected layers some neurons should receive no gradient. If neurons are masked in this way, the true fraction of updated parameters is lower than 40%. If, instead, every neuron in a selected layer belongs to either some S_k or A, then no neuron is actually frozen and the 40% figure simply measures the layer fraction. The paper should report the exact number of trainable parameters for the 40% and 80% variants, including whether the vision encoder, connector, embeddings, and LM head are updated, and should reconcile this accounting with the claim that the method is more efficient than full fine-tuning.","section":"Abstract and Section 3.4, Eq. (9)"},{"comment":"The empirical support for 'surpasses full fine-tuning' is thin. The 40% variant beats the LLaVA-1.5-SFT baseline by 0.1 BLEU on average on M3-Multi30K (53.4 vs. 53.3) and by 0.3 on M3-AmbigCaps (58.3 vs. 58.0), while losing on several individual language pairs: on M3-Multi30K it is below baseline for Cs and Hi, and on M3-AmbigCaps it is below baseline for Cs, Lv, and Hi. No standard deviations, confidence intervals, significance tests, or multiple-seed results are reported. A single run with these margins cannot substantiate a claim of surpassing full fine-tuning. The authors should provide variance estimates, paired significance tests, or multiple seeds, and should temper the abstract if the differences are not significant.","section":"Tables 1-2"},{"comment":"The neuron classification definitions are underspecified in ways that affect the method. Eq. (5) aggregates importance scores over T training samples and K language pairs but does not state whether I_k^n is the mean over samples, the sum, or something else. Eq. (7) defines S_k as the set where I_k^n is the maximum over languages, but this condition can assign the same neuron to multiple language pairs when scores are tied, and it leaves unclassified any neuron that is never the maximum. Eq. (8) uses a variance threshold epsilon whose value is never given. The authors should define the aggregation precisely, state how ties are broken, specify epsilon or describe how it is set, and report the resulting sizes of S_k and A; otherwise the gradient masking in Eq. (9) cannot be implemented from the paper.","section":"Section 3.3, Eqs. (5)-(8)"}],"minor_comments":[{"comment":"The model name is spelled 'LLaV A' in the abstract and main text in several places; it should be 'LLaVA' consistently.","section":"Throughout"},{"comment":"The layer counts are inconsistent with the LLaVA-1.5-7B architecture. Section 4.5 refers to 'the first 250 layers (approximately 80% of the model depth)', and Section 4.6 refers to 'the top 40% of layers (108 layers)', but LLaVA-1.5-7B has 32 transformer layers. Please clarify what unit is being counted (e.g., parameter tensors, MLP blocks, or submodules) and make the layer counts consistent with the architecture.","section":"Section 4.5 and Section 4.6"},{"comment":"The paper says in Section 4.3 that selecting 80% of layers gives the best balance, and Tables 1 and 2 show the 80% variant outperforms the 40% variant on both datasets, while the abstract emphasizes the 40% variant. This tension should be addressed explicitly, including whether the 40% variant or the 80% variant is the recommended configuration and what 'fine-tuning only 40%' means in the abstract.","section":"Section 4.3 vs. Abstract"},{"comment":"The x-axis labels '0:1', '1:9', etc., are unclear without reading the caption; please clarify in the axis label that these are specific-to-agnostic neuron ratios.","section":"Figure 3b"},{"comment":"The tables do not report model parameter counts or training cost for the proposed variants, which makes it hard to evaluate the efficiency claim. Please include the number of trainable parameters and, if possible, training time or FLOPs.","section":"Tables 1-2"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a relevant problem and has a sensible experimental setup, but the central efficiency claim is not verifiable as written because the selection protocol in Eq. (3) is not fully specified, the parameter accounting is ambiguous, and the empirical gains over the full fine-tuning baseline are marginal with no significance testing. The authors should be asked to clarify the selection protocol and parameter counts, move hyperparameter selection to a validation split, and provide statistical evidence. If the clarified protocol reveals that a full fine-tuning pass is required before selection, the efficiency claim should be substantially revised. These are fixable within the manuscript's scope, so I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a reasonable incremental idea — pick important layers by activation change after SFT, pick important neurons by activation×gradient variance across languages, then fine-tune only those. The combination for multilingual multimodal MT is new relative to the cited literature, and the experiments cover two datasets and six language pairs, which is a solid amount of work. The paper is clearly written and does not oversell wildly; the limitation section acknowledges fixed thresholds.\n\nWhat worried me, in the order it bothered me. First, Eq. 3 never says which \"fine-tuned model\" X_B_l comes from. If that is the fully fine-tuned LLaVA-1.5-SFT baseline, then the pipeline pays for a full SFT pass before selection, and the \"40% of parameters\" describes only the second stage. That would make the efficiency claim misleading. This is the load-bearing question and it is simply not addressed.\n\nSecond, the hyperparameters look selected on test. Figure 3a sweeps the layer fraction and Figure 3b sweeps the neuron ratio; the paper describes no validation split for the M3 datasets. If those curves are computed on the test set, the reported numbers are inflated, and the 1:9 ratio is a fitted parameter, not a discovered regularity. The specific-agnostic distinction itself is also not crisply defined: Eq. 7 and Eq. 8 can overlap, and the paper never states whether the 1:9 ratio means 10% of neurons are updated or 10% are classified as specific within layers that are all updated. The 40% parameter figure is therefore ambiguous: layer count is not parameter count.\n\nThird, the headline BLEU gains are small: 0.1–0.3 on average for the 40% variant, with several language pairs losing to the baseline. No error bars, no repeated runs, no significance test. Given the tuning-on-test concern, I would not trust those numbers as evidence of superiority over full fine-tuning. The 80% variant looks better, but still within the same noise band.\n\nI want to give credit where it is due: the idea of separating language-specific and language-agnostic neurons is well motivated, and the visualizations in Figure 4 are a legitimate attempt to show that the selected components differ across languages. The paper does not fabricate anything; it just leaves too many procedural details unspecified. The citation pattern looks fine — LanderMT is credited as the closest neuron-selection prior work.\n\nBottom line: this deserves a serious referee, but not acceptance as is. The authors need to say which checkpoint provides X_B_l, run their selection on a validation split, report variance over seeds, and give exact parameter counts. If those are fixed, the method might well stand up. As written, the efficiency and SOTA claims are unverifiable.","headline":"Plausible parameter-efficient recipe, but the 40% claim and the selection protocol are underspecified enough that the efficiency story isn't checkable as written.","tokens_in":12684,"tokens_out":3450,"would_cite":false,"duration_ms":34222,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that selectively fine-tuning 40% of a vision-language model's parameters can match or beat full fine-tuning for multilingual multimodal translation.","keywords":["multilingual multimodal machine translation","parameter-efficient fine-tuning","layer selection","neuron selection","cross-lingual interference","vision-language models","gradient masking","multilingual translation"],"falsifier":"Recompute LLaVA-NeuMT with the layer importance scores obtained from a model fine-tuned only on a training split, with the layer fraction and variance cutoff fixed before any test-set evaluation; if the 40%-parameter variant then scores at or below the full-fine-tuning baseline on M3-Multi30K and M3-AmbigCaps, the central claim would be refuted.","tokens_in":11536,"feed_emoji":"🌐","tokens_out":11434,"duration_ms":101989,"temperature":0.7,"pith_summary":"This paper tries to establish that adapting a large vision-language model to many language pairs is better done by selecting which parameters to update than by fine-tuning everything. It proposes a two-level selection: first find which transformer layers carry the most task-relevant information, then find which neurons inside those layers are specific to one language pair and which are shared across all pairs. On the M3-Multi30K and M3-AmbigCaps benchmarks, the paper reports that fine-tuning only 40% of the 7-billion-parameter model's parameters slightly exceeds full fine-tuning in average BLEU, and fine-tuning 80% improves the average further. This matters because it points toward adapting one large multimodal system to many languages with a small parameter budget, while also showing which parts of the model absorb the adaptation.","feed_headline":"Tune just 40% of parameters to beat full fine-tuning","feed_subtitle":"A layer and neuron selection mask targets only the parameters each language pair needs, matching or beating retraining everything.","key_machinery":"The load-bearing object is the redundancy-based layer importance score $R_l = (X_l^A \\cdot X_l^B)^2 / (\\|X_l^A\\|^2 \\|X_l^B\\|^2 + \\epsilon)$, where $X_l^A$ and $X_l^B$ are the activations of layer $l$ in the pretrained and fine-tuned models; a low score means the layer changed substantially during fine-tuning and is therefore selected. On top of this, neuron importance $I_n = |A_n \\times G_n|$ is aggregated across language pairs, and its variance $\\sigma^2(n)$ classifies neurons as language-specific (the pair for which importance is maximal) or language-agnostic (variance below a threshold $\\epsilon$). A gradient mask $G'_n = G_n$ if $n \\in A \\cup S_k$, and $0$ otherwise, restricts updates to the union of agnostic and current-pair-specific neurons. This two-level mask is what lets the model keep most parameters frozen while still adapting per language pair.","core_discovery":"On the paper's own terms, the discovery is that multilingual interference can be mitigated by selective parameter sharing rather than by sharing everything. The method computes a redundancy-based importance score for each layer by comparing its activations in the pretrained model with its activations in a fine-tuned model: a layer whose activations change a lot is deemed important and kept trainable. Within those layers, each neuron receives an importance score from the product of its activation and gradient, and the variance of that score across language pairs separates language-specific neurons from language-agnostic ones. Only neurons that are either language-agnostic or specific to the current language pair receive gradient updates. The paper reports that with 40% of parameters trainable this selection reaches 53.4 average BLEU on M3-Multi30K versus 53.3 for full fine-tuning, and 58.3 versus 58.0 on M3-AmbigCaps; with 80% trainable it reaches 54.3 and 59.7. It further reports that a 1:9 ratio of specific to agnostic neurons and selection of the top 80% of layers give the best balance.","pith_inferences":["If the reference model behind Equation (3) is the fully fine-tuned model, then the 40% parameter budget is a saving for later adaptations, not for the initial fine-tuning: one full fine-tune must be completed first to measure layer importance.","A natural test the paper does not run is whether a layer and neuron mask computed on one set of language pairs transfers to unseen pairs; the reported stability of layer importance across languages suggests it might.","The same two-level selection could be applied to other multilingual multimodal tasks such as image captioning or visual question answering, since the mechanism only assumes that some neurons are shared across languages and some are not.","The parameter budget alone does not measure compute; a complete efficiency picture would include wall-clock training time and memory, because forward and backward passes still run through the full frozen model."],"forward_implications":["A 7-billion-parameter multimodal translation model can be adapted to a new language pair by updating only the layers and neurons identified as relevant, cutting the parameter update budget to 40% while keeping or slightly improving BLEU.","The final 20% of layers are not just unnecessary; including them reduces average BLEU, so redundancy removal is part of what improves translation, not merely a cost saving.","Language-agnostic neurons carry more of the multilingual adaptation load than language-specific neurons, since fine-tuning only agnostic neurons outperforms fine-tuning only specific neurons on average.","Layer importance is consistent across the six language pairs studied, so the same layer mask can be computed once and reused for other pairs without per-pair search.","Larger gains appear on the lower-resource pairs (Latvian, Hindi, Turkish), suggesting selective modulation is most valuable where cross-lingual interference is strongest."],"supporting_citations":[{"why":"This supplies the pretrained LLaVA-1.5-7B backbone and the full-fine-tuning baseline LLaVA-1.5-SFT that the method must beat.","marker":"[23]"},{"why":"This supplies the M3-Multi30K multilingual multimodal dataset and the LVP-M3 prior state of the art that the paper compares against.","marker":"[11]"},{"why":"This supplies the M3-AmbigCaps dataset and the Vision Matters gated-fusion and concatenation baselines.","marker":"[19]"},{"why":"This supplies the neuron-level selection idea of detecting and routing language-aware neurons for selective fine-tuning of LLMs to machine translation.","marker":"[50]"},{"why":"This supplies the text-only multilingual Transformer baseline that anchors the comparison between text-only and multimodal systems.","marker":"[8]"},{"why":"This supplies the layer-selection idea based on activation correlation that the paper adapts into its redundancy-based importance score.","marker":"[21]"}],"fun_headline_variants":["Selective tuning of 40% beats full fine-tuning","40% of parameters yield SOTA multilingual MMT","Layer and neuron masks: efficient multilingual translation","Language-specific neurons identified by variance, tuned selectively","Multilingual MMT: selective parameter sharing for SOTA"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole selection depends on a reference \"fine-tuned model\" whose activations define which layers are important, and on layer and neuron thresholds that the paper does not fully specify; if that reference is the full fine-tune the method is meant to beat, or if the thresholds were chosen by looking at test results, the reported gains are inflated.","fun_headline_variants_meta":{"raw":{"variants":["Selective tuning of 40% beats full fine-tuning","40% of parameters yield SOTA multilingual MMT","Layer and neuron masks: efficient multilingual translation","Language-specific neurons identified by variance, tuned selectively","Multilingual MMT: selective parameter sharing for SOTA"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001089,"raw_usage":{"total_tokens":4569,"prompt_tokens":985,"completion_tokens":3584,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":601,"completion_tokens_details":{"reasoning_tokens":3509}},"tokens_in":601,"tokens_out":3584,"duration_ms":25951,"temperature":1.0,"reasoning_tokens":3509,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T18:05:18.214662+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Recompute LLaVA-NeuMT with the layer importance scores obtained from a model fine-tuned only on a training split, with the layer fraction and variance cutoff fixed before any test-set evaluation; if the 40%-parameter variant then scores at or below the full-fine-tuning baseline on M3-Multi30K and M3-AmbigCaps, the central claim would be refuted.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"This supplies the pretrained LLaVA-1.5-7B backbone and the full-fine-tuning baseline LLaVA-1.5-SFT that the method must beat."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"This supplies the M3-Multi30K multilingual multimodal dataset and the LVP-M3 prior state of the art that the paper compares against."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"This supplies the M3-AmbigCaps dataset and the Vision Matters gated-fusion and concatenation baselines."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"This supplies the neuron-level selection idea of detecting and routing language-aware neurons for selective fine-tuning of LLMs to machine translation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"This supplies the text-only multilingual Transformer baseline that anchors the comparison between text-only and multimodal systems."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"This supplies the layer-selection idea based on activation correlation that the paper adapts into its redundancy-based importance score."}],"review_version":2}