{"id":"41aca3ad-a818-4143-b521-406cee0ff984","arxiv_id":"2411.12787","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Dual-LoRA plus Visual Cue Enhancement improves efficient visual instruction tuning over LoRA and LoRA-MoE baselines with near-vanilla-LoRA inference time.","lead":"The authors introduce Dual-LoRA, a two-space adapter for efficient visual instruction fine-tuning of multimodal LLMs, plus a Visual Cue Enhancement module that fuses multi-level image features. On UniFood, ScienceQA, Flickr30k, and general benchmarks, the combined method outperforms vanilla LoRA and LoRA-MoE baselines while running at only 1.16 times the inference cost of vanilla LoRA.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Corollary 2 is a static matrix identity, but Eq. (10) is a per-token gated map; the claimed ability of Dual-LoRA to simulate LoRA-MoE's local responses is asserted, not derived.","rationale":"The reader's analysis correctly isolates the weakest point: the theoretical bridge in Section 3.4.2. I agree; that is the load-bearing concern. The empirical sections are coherent: Tables 1-2 show consistent gains over four baselines on three datasets, the ablations in Tables 3-4 support both components, and Figure 8 reports the efficiency claim. However, I am not persuaded that the static Corollary 2 licenses the per-token gating in Eq. (10), and the paper itself provides no proof for the transition. This is not an accusation of dishonesty; it is a missing step in a key argument. The concern is addressable: a corrected derivation or a synthetic expressiveness experiment would settle it, and the empirical claims would remain even if the theoretical narrative were weakened. Therefore the verdict should stay CONDITIONAL rather than ACCEPT or REJECT, and I recommend the authors add either a formal statement of the class of per-token maps representable by Eq. (10) or remove the 'simulate any local response' wording.","tokens_in":13334,"tokens_out":7206,"duration_ms":69597,"concrete_test":"Independently re-derive Eq. (10) from Corollary 2, keeping the per-token dependence explicit. A decisive check: set r=1, d_in=2, S=[1,0], T=[0,1], B=1; the Corollary-2 static weight B(S⊙σ(T)) is identically zero, while D(x)=x1·ReLU(x2) is not. If no corrected derivation exists, run a synthetic 4-cluster task with per-cluster rank-1 target maps and compare rank-matched vanilla LoRA, 4-expert LoRA-MoE, and Dual-LoRA; if Dual-LoRA cannot match LoRA-MoE's fit, the local-response claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central conceptual claim is that Dual-LoRA can simulate the 'local response' of LoRA-MoE through skill-space rectification. Corollary 2 only states that a fixed weight matrix B(A⊙σ(T)) can be decomposed into LoRA groups. The actual module in Eq. (10) is D(x)=B(Sx ⊙ σ(Tx)), an input-dependent map. These are not the same object: (Sx)⊙(Tx) ≠ (S⊙T)x in general. A rank-1 counterexample makes this concrete: with S=[1,0], T=[0,1], B=1, the Corollary-2 fixed form is B((S⊙σ(T))x)=0, while Eq. (10) gives x1·ReLU(x2), a nonzero gated response that depends on x. Section 3.4.2 does not provide a bridge from the static decomposition to the per-token rectification; it simply asserts that T can be 'mapped through a non-linear activation to achieve rank rectification.' Thus the expressiveness justification for the holistic-to-localized design is unsupported. The empirical results may stand on their own, but the theoretical motivation is the load-bearing part of the paper's argument; if it fails, Dual-LoRA is an ad hoc gated adapter rather than a demonstrated simulation of MoE localization.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Dual Low-Rank Adaptation (Dual-LoRA) and Visual Cue Enhancement (VCE) for efficient visual instruction tuning of multimodal LLMs. Dual-LoRA replaces a LoRA-MoE router-expert ensemble with a single parameterized module D(x) = (r/alpha) B(Norm(Sx) ⊙ σ(Tx)), where S is a holistic 'skill space' and T is a 'task space'; the authors argue via Corollary 2 that this structure can simulate the local, task-conditioned responses of LoRA-MoE. VCE enhances high-level vision features by aggregating local multi-level features through deformable attention. Experiments on UniFood, ScienceQA, Flickr30k, and general MLLM benchmarks report consistent improvements over vanilla LoRA and LoRA-MoE baselines, with inference overhead of 1.16× over vanilla LoRA and 73% of a 4-expert LoRA-MoE.","tokens_in":13680,"tokens_out":2849,"duration_ms":26933,"significance":"If the claims hold, the paper offers a practically attractive alternative to LoRA-MoE: a structurally simpler adapter with competitive accuracy, lower memory, and substantially lower inference cost. The VCE module is lightweight and ablation results suggest it helps. The paper is also honest in reporting efficiency numbers. However, the central conceptual justification—that Dual-LoRA inherits the local-response capability of LoRA-MoE—rests on a proof gap, and the empirical gains are reported without variance or multiple seeds, so the strength of the empirical claim is not yet fully established.","major_comments":[{"comment":"Corollary 2 is a statement about a fixed matrix product B(A ⊙ σ(T)), where T is a constant matrix of the same shape as A. The actual Dual-LoRA module is D(x) = B(Norm(Sx) ⊙ σ(Tx)), an input-dependent, per-token gated map. These are not the same object: (Sx) ⊙ σ(Tx) is not equal to (S ⊙ σ(T))x in general. The paper asserts in Section 3.4.2 that the task space T can be 'mapped through a non-linear activation to achieve rank rectification' and thereby simulate local responses, but no proof or formal argument connects the static decomposition of Corollary 2 to the per-token rectification in Eq. (10). This is load-bearing because the paper's central claim is that Dual-LoRA is a unified, efficient simulation of LoRA-MoE's localized activation; without this bridge, Dual-LoRA is an ad hoc gated adapter whose empirical success must stand on its own.","section":"Section 3.4.2, Eq. (10)-(12)"},{"comment":"All experimental results are reported from a single run, without error bars, confidence intervals, or significance tests. Several head-to-head gaps are small: in Table 1, Dual-LoRA beats RoDE by only 0.6 IoU and 1.0 SacreBLEU, and in Table 3, adding VCE to vanilla LoRA improves IoU by only 0.1. For a paper whose main claim is consistent improvement across tasks, these differences could be within run-to-run variance. Please report at least three seeds with mean and standard deviation, or otherwise justify that the differences are reproducible.","section":"Tables 1, 2, 4, and 5"},{"comment":"The proposition and corollary state that a single higher-rank LoRA is at least as expressive as a sum of lower-rank LoRAs, which is true in a linear-algebra sense. However, the paper then argues that LoRA-MoE's advantage comes from 'differentiated local response capability' and that Corollary 2 provides the route to recovering this in a single module. Since the gap between Corollary 2 and the implemented input-dependent rectification is unresolved, the narrative from Proposition 1 through Corollary 2 to Eq. (10) does not yet constitute a rigorous motivation for the Dual-LoRA design. A precise statement of what class of per-token gating functions can be represented, or a constructive decomposition of a MoE forward pass into the Dual-LoRA form, would be needed.","section":"Section 3.4.1, Proposition 1 and Corollary 1"},{"comment":"The memory-efficiency comparison in Figure 7 varies the rank of vanilla LoRA and Dual-LoRA while freezing outside adapters, but the caption says 'only the adapters are set as trainable,' which contradicts the main experimental setup where the vision projector and VCE are also trainable. It should be clarified whether VCE is included in the Dual-LoRA curve and what 'adapter memory' includes; otherwise the comparison of parameter efficiency is not airtight.","section":"Section 4.5, Figure 7"}],"minor_comments":[{"comment":"The column header 'ScareBLEU' is a typo; it should be 'SacreBLEU'.","section":"Table 2"},{"comment":"The text says 'for ScienceQA, as illustrated in Fig 2', but the ScienceQA results appear in Table 2, not Figure 2; please correct the cross-reference.","section":"Section 4.2"},{"comment":"The phrase 'indicate the interest ares' appears to be a typo for 'indicate the interest areas'.","section":"Eq. (5)-(7)"},{"comment":"The caption uses 'it's rectified space'; this should be 'its rectified space'.","section":"Figure 6 caption"},{"comment":"The heading 'Disscusion on General Benchmarks' should be 'Discussion on General Benchmarks'.","section":"Section 4.6 heading"},{"comment":"The VCE module size is reported as 5.53 MB in the introduction and 5.52 MB in the hyperparameters paragraph; please make the numbers consistent.","section":"Section 3.3 vs Section 4.1"},{"comment":"The paper references Appendix B for implementation details, but the main text includes no appendix; please ensure the supplementary materials are available and referenced correctly at submission.","section":"Section 4.1"}],"recommendation":"major_revision","confidential_remarks":"The paper's empirical results may be publishable as an engineering contribution, but the theoretical motivation is currently overstated. The authors rely heavily on their own RoDE work [13] both as a baseline and as the source of the rectified-activation idea; this is not disqualifying, but it increases the need for the theoretical bridge from static decomposition to per-token gating to be explicit. I would also ask the editor to require the authors to share code or detailed reproducibility instructions, since the project page does not currently appear to contain a usable implementation."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a solid empirical paper wrapped in a shaky theoretical story. The Dual-LoRA module—skill space plus input-conditioned task space with ReLU rectification—is new in the LoRA-MoE literature, and the VCE module is a lightweight, sensible addition. The reported gains over vanilla LoRA and LoRA-MoE baselines on UniFood, ScienceQA, and Flickr30k are consistent, and the 1.16x inference time relative to vanilla LoRA is a real selling point. The ablations support the design choices.\n\nThe soft spot is exactly what the stress-test says. Corollary 2 is a static matrix identity about a fixed B(A⊙σ(T)) being decomposable into LoRA groups. The actual module in Eq. (10) is D(x)=B(Sx ⊙ σ(Tx)), which is per-token and input-dependent. These are not the same object. The rank-1 counterexample is concrete: with S=[1,0], T=[0,1], the static form gives 0 while Eq. (10) gives x1·ReLU(x2). Section 3.4.2 asserts the bridge rather than deriving it. So the central claim that Dual-LoRA simulates MoE local response is unsupported. The empirical results might still stand, but the paper needs to either prove the connection or drop the claim.\n\nOther weaknesses are more minor but real: single runs without error bars, small gains in places (IoU 24.2 vs 23.6, BLEU 14.8 vs 13.8), and no universal general-benchmark advantage (MMB 65.1 vs LLaVA-1.5-LoRA 66.1). The appendix proofs and code were not in the supplied manuscript, so I couldn't verify Corollary 2.\n\nWho this is for: people working on efficient fine-tuning of MLLMs, especially LoRA variants and data-conflict mitigation. The empirical design is worth testing and citing. Send it to peer review, but with major revisions requested: provide a real derivation or soften the theory, add error bars or multiple seeds, and release the code and proofs.\n\nVerdict: conditional, not reject.","headline":"A genuinely useful empirical adapter for EVIT, but the theoretical story oversells what the static Corollary 2 proves; worth a serious referee with major revisions.","tokens_in":14178,"tokens_out":3902,"would_cite":true,"duration_ms":32199,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Two-space LoRA beats LoRA-MoE at near-vanilla speed.","keywords":["Dual Low-Rank Adaptation","Visual Cue Enhancement","Efficient Visual Instruction Fine-Tuning","Mixture of Experts","Data Conflict","Multimodal Large Language Models","Rank Rectification","Parameter-Efficient Fine-Tuning"],"falsifier":"Train on a synthetic two-task dataset where the optimal task updates are rank-1 and have disjoint input supports; then check whether a single Dual-LoRA can match the loss of a two-expert LoRA-MoE, and if no $S$ and $T$ can achieve the per-token selection, the expressiveness claim is refuted. Alternatively, if removing the non-linear activation from the task space does not reduce task-specific activation (measured by entropy or gradient correlation), the rectification mechanism is not doing the claimed work.","tokens_in":13125,"feed_emoji":"🧠","tokens_out":4328,"duration_ms":38673,"temperature":0.7,"pith_summary":"Visual instruction tuning of multimodal LLMs with low-rank adapters suffers data conflicts as tasks diversify: knowledge learned for one task, like ingredient recognition, can contradict knowledge for another, like recipe generation. Mixture-of-experts variants of LoRA mitigate this by activating different expert adapters per task, but the router plus multiple experts adds inference time. This paper claims a single adapter can get the benefit of localized expert activation by splitting into two low-rank spaces: a skill space that stores holistic, stable knowledge, and a task space that rectifies, per token, which parts of that knowledge are locally activated. It also adds a lightweight visual-cue module that folds multi-level local image features into the vision projection. The result is a method that, the paper reports, beats vanilla LoRA and LoRA-MoE baselines on food, science QA, and captioning benchmarks while adding only 1.16x the inference time of vanilla LoRA.","feed_headline":"Two-space LoRA beats LoRA-MoE at near-vanilla speed","feed_subtitle":"Dual-LoRA plus visual cue enhancement resolves data conflicts with just 1.16x LoRA inference time.","key_machinery":"The central objects are the skill space $S$ and task space $T$: $S$ is a low-rank matrix that accumulates stable, holistic domain knowledge across tasks, while $T$ is a second low-rank matrix whose output is passed through ReLU and elementwise-multiplied with the normalized skill output to produce a rank-rectified, token-dependent gate on the holistic knowledge. The update is applied through $B$ and scaled by $r/\\alpha$. The other component, Visual Cue Enhancement (VCE), uses deformable attention to gather local features from selected intermediate ViT layers and adds them to the high-level anchor feature before projection. The key mechanism is the elementwise product of a holistic, normalized skill response with a per-token rectified task response, which the paper claims gives localized, task-specific activation without a router.","core_discovery":"The paper's central claim is that the Dual-LoRA module $D(x)=\\frac{r}{\\alpha}B(\\mathrm{Norm}(Sx)\\odot\\sigma(Tx))$ can replace the router-and-experts structure of LoRA-MoE: the skill matrix $S$ learns holistic cross-task knowledge, and the task matrix $T$, passed through ReLU, provides a per-token rectification that locally activates that knowledge. The paper argues via a corollary of a matrix decomposition result that a LoRA of the form $B(A\\odot \\sigma(T))$ is as expressive as a set of separate LoRA experts, and that this structure is more time-efficient because it requires no routing over multiple experts. On UniFood, ScienceQA, and Flickr30k the proposed method outperforms vanilla LoRA and three LoRA-MoE baselines, and the full Dual-LoRA+VCE system uses 1.16x vanilla LoRA inference time and 73% of the time of a 4-expert LoRA-MoE.","pith_inferences":["The paper's expressiveness argument moves from a static decomposition (fixed $T$) to a per-token $Tx$; whether the dynamic version truly spans the same function class is not proven, so the strongest theoretical reading rests on an unstated assumption.","A natural test is to measure how task-specific the learned task space actually is: if the ReLU gates collapse to a single pattern across tasks, the local-response explanation would be weakened even if accuracy stays high.","The same rectification idea might generalize to other parameter-efficient methods, for instance as a drop-in replacement for adapter MoEs in text-only LLMs where data conflicts also occur."],"forward_implications":["If the dual-space adapter indeed reproduces expert-level local response, LoRA-MoE systems could be replaced by a single adapter with comparable accuracy and lower latency.","The skill-space/task-space split offers a target for interpretability: the rectified skill space shows lower entropy and localized high-energy bands, suggesting a measurable, task-specific activation pattern.","The VCE module's use of multi-level features works with frozen vision encoders, so existing efficient visual instruction tuning pipelines can adopt it without full fine-tuning.","On general benchmarks, the method ranks among the top two in most settings, implying the benefits of localized adaptation transfer beyond specialized food and science tasks."],"supporting_citations":[{"why":"Defines the LoRA adapter formulation that the paper extends and whose data-conflict limitation motivates the whole approach.","marker":"[11]"},{"why":"Identifies data conflicts in LoRA instruction tuning and proposes sparse mixture of LoRA experts, supplying the problem statement and a baseline.","marker":"[5]"},{"why":"Provides a rectified-activation LoRA-MoE baseline and the UniFood dataset used for the main multi-task evaluation.","marker":"[13]"},{"why":"Supplies the dense and top-k LoRA-MoE baselines that Dual-LoRA is compared against on accuracy and inference time.","marker":"[33]"},{"why":"Defines LLaVA-1.5-7B, the base MLLM architecture on which all experiments are run.","marker":"[20]"},{"why":"Layer normalization is used in the Dual-LoRA module to normalize the skill space output before rectification.","marker":"[2]"},{"why":"Deformable attention is the mechanism VCE uses to gather local features from multi-level vision feature maps.","marker":"[34]"},{"why":"ReLU is the non-linear activation chosen for the task space to induce sparse, rectified local responses.","marker":"[17]"}],"fun_headline_variants":["Dual-LoRA: Two-space adapter resolves conflicts at 1.16x LoRA speed","Holistic-to-local LoRA: beats 4-expert MoE at near-vanilla cost","Skill and task spaces: LoRA without routing, faster than MoE","Local-enhanced LoRA: 1.16x speed, beats MoE baselines","Dual-LoRA: Holistic knowledge plus local activation, MoE-free"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper assumes that a static matrix factorization result about $B(A\\odot \\sigma(T))$ implies that the input-dependent update $B(Sx\\odot \\sigma(Tx))$ can simulate the per-token expert selection of LoRA-MoE, but it offers no proof for the dynamic, per-token version.","fun_headline_variants_meta":{"raw":{"variants":["Dual-LoRA: Two-space adapter resolves conflicts at 1.16x LoRA speed","Holistic-to-local LoRA: beats 4-expert MoE at near-vanilla cost","Skill and task spaces: LoRA without routing, faster than MoE","Local-enhanced LoRA: 1.16x speed, beats MoE baselines","Dual-LoRA: Holistic knowledge plus local activation, MoE-free"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000823,"raw_usage":{"total_tokens":3614,"prompt_tokens":972,"completion_tokens":2642,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":588,"completion_tokens_details":{"reasoning_tokens":2529}},"tokens_in":588,"tokens_out":2642,"duration_ms":16360,"temperature":1.0,"reasoning_tokens":2529,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T17:33:45.134256+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train on a synthetic two-task dataset where the optimal task updates are rank-1 and have disjoint input supports; then check whether a single Dual-LoRA can match the loss of a two-expert LoRA-MoE, and if no $S$ and $T$ can achieve the per-token selection, the expressiveness claim is refuted. Alternatively, if removing the non-linear activation from the task space does not reduce task-specific activation (measured by entropy or gradient correlation), the rectification mechanism is not doing the claimed work.","supporting_citations":[{"cited_title":"Visual instruction tuning, 2023","cited_arxiv_id":null,"evidence_quote":"Defines LLaVA-1.5-7B, the base MLLM architecture on which all experiments are run."},{"cited_title":"Vision transformer with deformable attention","cited_arxiv_id":null,"evidence_quote":"Deformable attention is the mechanism VCE uses to gather local features from multi-level vision feature maps."}],"review_version":1}