{"id":"650ae304-cb62-49a0-99f5-74dd20c631ba","arxiv_id":"2508.11673","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"MSLoRA-CR incrementally adds modality-specific LoRA branches to a frozen medical LVLM with contrastive regularization, reporting improved overall performance over separate per-modality models on nine biomedical datasets.","lead":"This paper proposes MSLoRA-CR, which adds a new low-rank adapter for each new biomedical imaging task and uses contrastive regularizers to share knowledge within a modality and separate it across modalities. It claims this beats training separate models per modality on multimodal biomedical image incremental learning benchmarks, making a single medical vision-language model cheaper to deploy.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Main 9-dataset result may be order-selected: Table 6 shows a 150-point SUM swing on a 6-task subset, yet the central claim rests on a single favorable task order with no variance reporting.","rationale":"The reader's weakest_assumption identifies task-order representativeness, and that is indeed the single most load-bearing concern. The paper's own Table 6 shows order changes produce a ~150-point swing on a six-task subset, which is roughly seven times the reported ~20-point advantage over the separate-model baseline. Since the main table provides no error bars or order-averaged numbers, the claimed superiority could be an artifact of selecting a favorable sequence rather than a genuine property of the method. I do not see an internal inconsistency in the method itself; the theoretical Proposition 1 is overstated as a stability-plasticity proof, but it does not threaten the empirical finding as directly as the order dependence does. The reader's CONDITIONAL verdict is appropriate: the concern is addressable by reporting order-robust comparisons, but until then the headline claim is not fully supported.","tokens_in":15594,"tokens_out":3147,"duration_ms":34847,"concrete_test":"Re-run the full Table 2 comparison using at least: (i) the reversed-modality order from Table 6, (ii) a report-generation-first order, and (iii) five random orders, for both MSLoRA-CR and Data-Specific FT, reporting SUM and per-task metrics. If MSLoRA-CR fails to exceed Data-Specific FT under any fair order, or the mean over random orders is not above, the central superiority claim is not order-robust.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim is that MSLoRA-CR (SUM 990.25, or 995.21 with ortho) beats the separate-model baseline Data-Specific FT (SUM 970.29). This comparison depends entirely on one task order for the nine datasets, but Section 4.3 and Table 6 show that on a six-dataset subset the SUM varies from 610.40 to 760.83—a range of ~150 points—depending only on task order. The order used in Table 6 for the best configuration (A1,B1,A2,B2,A3,B3, i.e., VQA-first, pathology-first) is also the one most favorable to the method. No error bars, seeds, or order-averaged results are reported. If the same order sensitivity holds on the full nine-dataset setup, a different but still reasonable order could erase or reverse the ~20-point advantage over Data-Specific FT. This is not a flaw in the method's internal logic; it is a load-bearing evaluation-setup assumption because the headline superiority claim would fail under a fair distribution of orders.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Multimodal Biomedical Image Incremental Learning (MBIIL), where a frozen large vision-language model is augmented with new LoRA branches for each incoming task, and an inference-time mask merges the task-specific branches. The proposed MSLoRA-CR adds a contrastive regularization loss that pulls the current task's LoRA parameters toward those of previous tasks in the same modality and pushes them away from those in different modalities, plus an orthogonality penalty on the LoRA factors. The authors report experiments on nine biomedical datasets spanning pathology, radiology, and dermatology, with VQA, classification, and report generation tasks. Their headline result is that MSLoRA-CR reaches an aggregate SUM of 990.25 (995.21 with the orthogonality term) versus 970.29 for a separate-model baseline, which they interpret as a 1.88% improvement. They also provide Proposition 1 claiming that fine-tuning the t-th LoRA delta is equivalent to fine-tuning the pretrained weight W and is independent of earlier LoRA deltas, which they use to argue for a balance between stability and plasticity.","tokens_in":15880,"tokens_out":6298,"duration_ms":76517,"significance":"If the empirical claim is robust, the paper addresses a useful and under-studied setting: incrementally adding biomedical imaging modalities/tasks to a single LVLM while avoiding separate models per task. The idea of using contrastive regularization over LoRA parameters based on modality membership is simple and plausible, and the public code release is a practical strength. However, the current evidence does not establish the claimed superiority. The main evaluation is a single run on one task order, the aggregate SUM metric adds heterogeneous scales without justification, and the paper's own task-order ablation shows very large performance swings. The theoretical stability-plasticity argument is also not rigorously proven as stated. These issues are fixable with additional experiments and a more careful analysis, so the underlying direction is worth pursuing.","major_comments":[{"comment":"The central empirical claim (Table 2: MSLoRA-CR 990.25 vs. Data-Specific FT 970.29) is based on a single task order and no repeated runs or error bars. Table 6 shows that on a six-dataset subset, changing only the task order changes SUM from 610.40 to 760.83, a swing of over 150 points, and the best order is arguably the one used in the main evaluation. A different but equally reasonable order could erase or reverse the reported ~20-point advantage. Please report order-averaged results over multiple permutations, at least on the six-task subset, and include variance over at least three seeds.","section":"Section 4.3, Table 6"},{"comment":"The aggregate 'SUM' adds accuracy, recall, F1, AUC, and BLEU scores directly. These are not commensurable metrics with comparable scales or distributions; summing them imposes an arbitrary equal weighting and makes the 1.88% improvement difficult to interpret. The paper should either justify the aggregation, use a normalized/rank-based aggregate, or present the per-dataset comparisons as the primary evidence. The per-task tables are useful, but the headline claim depends on the SUM definition.","section":"Section 4.1, Evaluation Metrics"},{"comment":"The proof that fine-tuning ΔW_t is equivalent to fine-tuning W treats the layer input h as fixed. In a deep LVLM, h for a given layer depends on both W and all previous LoRA modules in earlier layers, so the gradient of the full loss w.r.t. W is not generally equal to the gradient w.r.t. ΔW_t. Moreover, ΔW_t = A_tB_t is low-rank while W is not, so even a local gradient equality does not imply equivalent updates. Thus the claimed 'theoretically proven balance between plasticity and stability' is not established by the argument given.","section":"Section 3.3, Proposition 1 and Eqs. (2)-(4)"},{"comment":"There is a dimensional inconsistency in the orthogonality loss. If A_i ∈ R^{d_o×r} and B_i ∈ R^{r×d_i}, then A_i^T A_i is r×r and B_i^T B_i is d_i×d_i. The term ∥B_i^T B_i − I∥_F^2 cannot drive a rank-r matrix to a d_i-dimensional identity. The text correctly states the desired constraints as A_i^T A_i = I and B_iB_i^T = I, so Eq. (9) likely has a typo (B_i^T B_i should be B_iB_i^T). Since the ortho variant gives the best reported SUM, this needs correction and the experiments should be rerun or confirmed.","section":"Section 3.4, Eq. (9)"},{"comment":"The hyperparameters α and β were selected by 'trained for several steps to observe the scale differences' and picking values that yielded the best performance on the same six tasks used later in the evaluation (Table 4). This is a form of test-set selection and makes the reported improvements optimistic. Please use a held-out validation split for hyperparameter selection or report results as a sensitivity study without claiming the selected configuration is unbiased.","section":"Section 4.3, hyperparameter selection"}],"minor_comments":[{"comment":"The text calls the baseline 'LoRA-Each' but the table and caption use 'Data-Specific FT'; please reconcile the terminology and define what '⊤' marks.","section":"Section 4.2, Table 2"},{"comment":"The order labels (A1,A2,A3,B1,B2,B3 and A1,B1,A2,B2,A3,B3) are confusing: the first row uses all A's before all B's, while the last row interleaves them. A small schematic or legend would make the order effects easier to read.","section":"Section 4.3, Table 6"},{"comment":"The similarity sim(P,Q) = exp(-dis(P,Q)) relies on a matrix Manhattan distance, but the exact definition of dis(P,Q) for two LoRA matrices is not stated. Please specify whether this is element-wise L1 averaged or summed, and how A and B are combined.","section":"Section 3.4, Eq. (5)"},{"comment":"The t-SNE visualizations are qualitative. If used as evidence for the CR loss's effect, a quantitative clustering metric (e.g., silhouette score) would be more convincing.","section":"Section 4.4, Figure 3"},{"comment":"The phrase 'unconstrained incremental learning methods' is vague; the abstract's 1.88% improvement seems to be relative to a particular baseline but the text does not clearly define the denominator. Please state the comparison explicitly.","section":"Abstract / Conclusion"}],"recommendation":"major_revision","confidential_remarks":"The paper introduces a relevant new problem and a reasonable method, but the evaluation is not yet convincing. The task-order sensitivity in Table 6 is large enough to undermine the headline comparison, and the SUM metric needs justification. I would support a revised version that adds multiple orders/seeds, redefines or justifies the aggregate metric, fixes the orthogonality loss typo, and compares against at least one standard continual learning method adapted to this setting. The theoretical claim in Proposition 1 should be either substantially reworked or softened."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Read this one if you care about continual learning for medical LVLMs. What's genuinely new: the MBIIL framing as modality-incremental learning, and a simple mechanism: per-task LoRA branches with contrastive regularization that pulls same-modality LoRA parameters together and pushes cross-modality apart, plus an orthogonality term. Code is public. On nine datasets spanning pathology, radiology, dermatology and three task types, it beats separate-model fine-tuning and plain incremental LoRA. That's a real, useful result if it holds.\n\nThe experimental breadth is the best part: nine datasets, ablations on rank, alpha/beta, module placement, and task order. But the headline is under-supported. Table 2 has no seeds or error bars. The SUM metric adds accuracies, F1s, AUCs, and BLEUs directly, with no normalization or justification. Table 6 is the bigger issue: on a 6-task subset, SUM swings from 610 to 761 just by changing task order, and the order used in the main table is the one that produces the highest total. If the same sensitivity exists on the full 9-task setup, a different reasonable order could erase the ~20-point advantage over Data-Specific FT. That's a load-bearing evaluation-setup problem.\n\nProposition 1 is also overclaimed. The derivative identity is correct, but it just says the gradient of the linear output w.r.t. W and ΔW_t is h^T. Tuning ΔW_t is not 'equivalent to fine-tuning W' in any meaningful stability-plasticity sense, and the loss of task t still depends on previous branches through the forward pass. This should be presented as an observation, not a proof of balance.\n\nMissing comparisons: InfLoRA and MoE-adapter are cited but never run. Hyperparameters alpha and beta were selected after checking loss scales on the same tasks used in evaluation; that's a mild selection effect, not circularity. The abstract's 1.88% doesn't match Table 2 cleanly.\n\nOverall, this is a plausible method with public code and a useful problem framing. The central claim needs stronger evidence. I'd send it to peer review and ask for error bars, order-averaged results or a clear protocol, a defensible aggregate metric, and those missing baselines. If that comes back solid, I'd cite it.","headline":"A plausible, code-released method for modality-incremental learning on medical LVLMs, but the headline result rests on a single favorable task order and no error bars; worth a serious referee, not yet a citable result.","tokens_in":16355,"tokens_out":3047,"would_cite":false,"duration_ms":35157,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Contrastive LoRA branches beat separate models on biomedical imaging","keywords":["Continual Learning","Low-Rank Adaptation (LoRA)","Contrastive Regularization","Multimodal Biomedical Image Incremental Learning","Large Vision-Language Model","Catastrophic Forgetting","Modality-Specific LoRA","Orthogonality Constraint"],"falsifier":"Re-run the six-task comparison under the orders listed in Table 6 (or a random sample of orders) for both MSLoRA-CR and separately fine-tuned per-task models, and compare the distribution of SUM scores; if the median or mean advantage is not positive across orders, the reported 1.88% improvement is an artifact of a favorable order. A single decisive signal: the radiology-first order B1,B2,B3,A1,A2,A3 already drops to 682.25, below the best order by 78.58.","tokens_in":15503,"feed_emoji":"🩺","tokens_out":6223,"duration_ms":63248,"temperature":0.7,"pith_summary":"The paper tries to establish that a single large vision-language model can keep learning new biomedical imaging modalities and tasks without forgetting, by freezing the base model and adding one low-rank LoRA branch per task. The new ingredient is a contrastive regularizer that pulls a new branch's parameters toward branches of the same modality and pushes them away from branches of other modalities, so knowledge is shared within a modality and interference between modalities is reduced. An orthogonality constraint on each branch's low-rank factors reinforces this separation. If the claims hold, a unified model can replace the usual practice of training a separate model per modality, cutting inference cost while actually improving overall accuracy. The headline evidence is an overall score of 990.25 versus 970.29 for separately fine-tuned models, a 1.88% gain.","feed_headline":"LoRA branches with contrastive loss beat separate biomedical models","feed_subtitle":"One frozen vision-language model learns VQA, classification, and report generation across modalities, scoring 990 vs 970.","key_machinery":"MSLoRA-CR: a set of task-specific LoRA branches attached to a frozen LVLM, each branch written as a low-rank pair $A_i, B_i$, merged at inference through a task-selection mask. The carrying mechanism is the contrastive regularization loss, which defines similarity between parameter matrices by $\\mathrm{sim}(P,Q) = e^{-\\mathrm{dis}(P,Q)}$ with Manhattan distance and applies a convergence loss to same-modality pairs and a divergence loss to cross-modality pairs; the orthogonality constraint $A^\\top A = B B^\\top = I$ decorrelates the factors. Proposition 1, the gradient-equivalence argument, is the theoretical load-bearing piece that connects branch expansion to plasticity and stability.","core_discovery":"On its own terms, the paper's central claim is that modality-specific LoRA expansion, MSLoRA, achieves a balance between plasticity and stability: Proposition 1 states that when learning task t, updating the new LoRA delta equals updating the shared pretrained weight W directly and is independent of previous deltas, so new tasks do not build on or distort old task parameters. The contrastive regularization loss then structures the growing set of LoRA branches so that same-modality branches converge and cross-modality branches diverge, and the orthogonality loss $A_i^\\top A_i = B_i B_i^\\top = I$ removes redundant capacity. Together these let the frozen biomedical LVLM outperform both separate","pith_inferences":["Order sensitivity at the scale shown in Table 6 suggests the practical recipe must include curriculum choice; a natural extension would be to test whether the contrastive loss can be made order-robust, e.g., by asymmetric or temperature-scaled regularization.","The same 'pull same-domain, push cross-domain' logic applies to any grouping of continual tasks, not just biomedical modalities; natural-image domains such as sketches, clipart, and photos are an immediate test bed.","Proposition 1's equivalence is a gradient identity, not a guarantee about loss-landscape dynamics; whether the balance holds in practice depends on the optimizer, learning rate, and the scale of the CR and ortho terms, so the theoretical result is motivation rather than a full proof of no forgetting.","Because the regularization acts only on parameter matrices, it is agnostic to output format, so extending the same module to detection and segmentation tasks is plausible provided the LVLM can emit those outputs."],"forward_implications":["If correct, deployment of multi-modal biomedical assistants no longer requires one checkpoint per modality; a single frozen LVLM with mask-selected LoRA branches covers all tasks.","The method's gain rides on the foundation model's existing knowledge; weak zero-shot baselines improve only as far as the base LVLM can support, so the ceiling is tied to the base model.","Task order matters: the paper's Table 6 shows pathology-first, VQA-first curricula give the best totals, while radiology-first or report-generation-first orders drop sharply.","Orthogonality adds a few points (995.21 vs 990.25) and appears to help most on report generation (BLEU 14.12 vs 5.83), suggesting it mainly cleans up low-rank redundancy.","Because branches are merged by mask at inference, switching tasks is as cheap as changing the mask rather than loading a new model."],"supporting_citations":[{"why":"Supplies the LoRA low-rank delta parameterization used for each task branch.","marker":"[10]"},{"why":"Provides the LLaVA-Med-v1.5 base LVLM that stays frozen and serves as the baseline.","marker":"[22]"},{"why":"InfLoRA, the prior interference-free LoRA continual learning method this work contrasts with.","marker":"[25]"},{"why":"CLIP, the vision encoder of the base LVLM, whose image features the model consumes.","marker":"[36]"},{"why":"MoE adapters for continual vision-language learning, the prior incremental PEFT approach that motivates modality-specific routing.","marker":"[48]"},{"why":"AdaLoRA, source of the orthogonality regularization used to decorrelate A and B.","marker":"[51]"}],"fun_headline_variants":["LoRA contrastive regularization tops separate biomedical models","MSLoRA-CR: one frozen model, multimodal incremental win","Contrastive LoRA beats separate models on biomedical images","Modality-specific LoRA with contrastive loss outperforms separate training","Frozen LVLM with LoRA branches wins over per-modality models"],"cache_read_input_tokens":2816,"weakest_assumption_plain":"The headline result assumes the specific training order used in the main experiments is representative; the paper's own order study shows totals swinging from 610.40 to 760.83 on a six-task subset, so if the gain over separate models disappears under a fair sample of orders, the main empirical claim fails.","fun_headline_variants_meta":{"raw":{"variants":["LoRA contrastive regularization tops separate biomedical models","MSLoRA-CR: one frozen model, multimodal incremental win","Contrastive LoRA beats separate models on biomedical images","Modality-specific LoRA with contrastive loss outperforms separate training","Frozen LVLM with LoRA branches wins over per-modality models"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000169,"raw_usage":{"total_tokens":1124,"prompt_tokens":792,"completion_tokens":332,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":536,"completion_tokens_details":{"reasoning_tokens":246}},"tokens_in":536,"tokens_out":332,"duration_ms":3948,"temperature":1.0,"reasoning_tokens":246,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T23:03:03.973130+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the six-task comparison under the orders listed in Table 6 (or a random sample of orders) for both MSLoRA-CR and separately fine-tuned per-task models, and compare the distribution of SUM scores; if the median or mean advantage is not positive across orders, the reported 1.88% improvement is an artifact of a favorable order. A single decisive signal: the radiology-first order B1,B2,B3,A1,A2,A3 already drops to 682.25, below the best order by 78.58.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"InfLoRA, the prior interference-free LoRA continual learning method this work contrasts with."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"MoE adapters for continual vision-language learning, the prior incremental PEFT approach that motivates modality-specific routing."}],"review_version":1}