{"id":"8da9bd74-92e1-4f29-81cd-7d959f538149","arxiv_id":"2505.10088","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"MMRL++ inserts shared, learnable representation tokens into the upper layers of CLIP's image and text encoders and uses low-rank shared aligners, achieving state-of-the-art base-to-novel harmonic mean accuracy on 11 datasets with 0.813M trainable parameters.","lead":"A new adaptation method for vision-language models inserts learnable representation tokens into the upper layers of CLIP to adapt to new tasks while trying to preserve the original model's zero-shot ability. The authors report state-of-the-art few-shot accuracy across 11 to 15 benchmarks, with a parameter-efficient extension that cuts trainable parameters to under one million.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported SOTA average is per-dataset oracle-tuned: Table 2 matches the best lambda/beta per dataset in Supp. Tables A1/A2; fixing config drops average HM from 81.77 to ~80.2, erasing most of the claimed gap over MMA.","rationale":"The reader's formal weakest_assumption is the layer-specialization hypothesis, but the reader's rationale also flags hyperparameter tuning and missing error bars. My read agrees with the latter two concerns and sharpens the tuning issue into a concrete, checkable flaw: the headline Table 2 numbers are exactly the per-dataset best lambda and beta values from the supplementary ablation tables. The central claim, that MMRL++ achieves a new state-of-the-art harmonic mean of 81.77, is therefore not supported as a claim about a single reproducible method unless a fixed hyperparameter recipe is provided and evaluated. A fixed lambda=0.2 gives an average HM of about 80.21, and a fixed beta=0.9 gives about 80.28, both much closer to MMA's 79.87 and within plausible noise given the absent error bars. The layer-specialization concern is real but empirically defended by Figure 7; the oracle-tuning issue is not defended anywhere and directly targets the paper's main quantitative conclusion. I do not call the authors dishonest; the practice may stem from standard per-dataset tuning, but for a SOTA claim it must be controlled. The paper has strengths: thorough ablations, a plausible architectural rationale, and a striking stability improvement on EuroSAT in Table 9. Those strengths do not rescue the headline average. The conditional verdict remains appropriate: the paper should be accepted only with a fixed-configuration re-evaluation and significance estimates.","tokens_in":28143,"tokens_out":13413,"duration_ms":135533,"concrete_test":"Retrain MMRL++ on all 11 datasets with a single fixed (lambda=0.2, beta=0.9), chosen on ImageNet only or via leave-one-dataset-out, and recompute average HM over 3 seeds with 95% confidence intervals. If the fixed-configuration average is about 80.2 and the gap to MMA is below 0.5 HM with overlapping intervals, the state-of-the-art claim should be downgraded. If the gap remains at least 1.0 HM with non-overlapping intervals, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Table 2's MMRL++ row does not correspond to a single fixed configuration. Cross-referencing with Supplementary Tables A1 and A2 shows that the reported harmonic mean for each dataset is the best value over the lambda grid (and similarly over the beta grid): ImageNet 74.44 at lambda=0.2, Caltech101 96.75 at lambda=3, DTD 74.46 at lambda=7, EuroSAT 91.94 at lambda=0.01, and so on. Fixing lambda=0.2 across all 11 datasets gives average HM 80.21 (from Table A1); fixing beta=0.9 gives 80.28 (from Table A2). Thus the reported 81.77 average is an oracle-tuned, per-dataset selection on the evaluation metric, and the advertised +1.90 HM over MMA (79.87) largely reflects test-metric selection rather than the intrinsic advantage of a fixed method. Main Table 2 also reports no error bars or significance tests, so it is unclear whether a fixed-configuration MMRL++ beats MMA at all. Because the central claim is state-of-the-art base-to-novel generalization, this selection-on-the-evaluation-metric issue is the most load-bearing concern.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MMRL and MMRL++, parameter-efficient methods for adapting CLIP to few-shot downstream tasks. MMRL introduces a shared learnable representation space whose tokens are projected into upper layers of both the image and text encoders as representation tokens, while the class token is regularized to preserve zero-shot knowledge; inference uses both feature types for base classes and only class features for novel classes. MMRL++ adds a shared-residual representation aligner (SRRA) with low-rank factorization and a progressive representation composition (PRC) mechanism to reduce parameters and improve inter-layer information flow. Experiments cover base-to-novel generalization on 11 datasets, cross-dataset transfer, domain generalization, and few-shot learning, with ablations for each design choice.","tokens_in":28385,"tokens_out":6004,"duration_ms":53133,"significance":"If the central claim holds, the paper would provide a useful and practical contribution: a competitive parameter-efficient adapter that improves novel-class generalization over prior adapters (e.g., MMA) while using fewer trainable parameters than the earlier MMRL. The paper is thorough in its experimental coverage—four evaluation protocols, multiple ablations, training stability analysis, and a computational-cost table. The idea of decoupling class and representation features, and the layer-wise insertion strategy, are clearly motivated and supported by ablations. However, the headline state-of-the-art claim on base-to-novel generalization is undermined by a per-dataset hyperparameter selection issue in Table 2, which makes the magnitude of the reported improvement over prior methods questionable. Because the paper reports no formal derivation but relies on empirical evidence, the evaluation protocol is load-bearing for the central claim.","major_comments":[{"comment":"The reported MMRL++ harmonic mean of 81.77 in Table 2 is obtained by selecting the best λ and β per dataset on the evaluation metric. Cross-referencing Table 2 with the supplementary sweeps shows that each dataset's HM is the maximum over the λ-grid (Table A1) or β-grid (Table A2)—for example, ImageNet 74.44 at λ=0.2, Caltech101 96.75 at λ=3, DTD 74.46 at λ=7, EuroSAT 91.94 at λ=0.01, and similarly per-dataset best β values. Fixing λ=0.2 across all 11 datasets gives an average HM of approximately 80.21 (computed from Table A1), and fixing β=0.9 gives approximately 80.28 (from Table A2). Thus the advertised +1.90 HM over MMA (79.87) reduces to roughly +0.3–0.4 under a single fixed configuration, and the paper does not report a joint fixed (λ, β) configuration. Because the state-of-the-art claim is the central contribution, this test-metric selection is a load-bearing issue. Please report results under a single fixed configuration chosen on a validation split, or provide a comparison under an equivalent per-dataset tuning protocol applied fairly to all baselines.","section":"§4.2, §4.7"},{"comment":"The implementation details state that \"the configurations for λ and β are provided in the Supplementary Material,\" and Section 4.7 says the analysis of λ and β is performed on ImageNet with comprehensive results in the supplementary. The supplementary tables, however, report test harmonic means for each dataset across the λ and β grids, and Table 2 appears to use the per-dataset best value from these grids. This implies that the test split (or the combined base/novel evaluation metric) is used for hyperparameter selection, which can overstate performance relative to fixed-configuration baselines. Please clarify the exact selection protocol: which data split is used to choose λ and β, and whether the test base/novel accuracy is used for selection. If test-based selection is indeed used, this should be disclosed prominently and matched for all compared methods.","section":"§4.3, Table 2, Table 9"},{"comment":"Table 2 reports no standard deviations or significance tests for the average base, novel, and harmonic mean metrics, despite Section 4.2 stating that results are averaged over three independent runs. Given that the fixed-configuration gap over MMA is likely small (≈0.3–0.4 HM), the reader cannot assess whether the improvement is statistically meaningful. Please include per-seed standard deviations for the headline average metrics (and ideally per-dataset for MMRL and MMRL++), and consider reporting confidence intervals or a simple significance test for the comparison against the strongest baseline.","section":"Table 2"}],"minor_comments":[{"comment":"The baseline is spelled \"CoCoOp\" in the text but appears as \"CoOpOp\" in Fig. 1 and in the Table 2 header row; please correct the spelling for consistency.","section":"§4.5"},{"comment":"In Section 4.5, the sentence \"MMRL and MMRL++ attains top performance on 2 out of the 4 domain-shifted datasets\" contains a subject-verb agreement error and a mixed singular/plural construction; please revise.","section":"Related Work, §2.2"},{"comment":"In the last paragraph of Section 2.2, \"their substantial computational overhead may places them\" should be \"may place them.\"","section":"References"},{"comment":"Reference [55] is a duplicate of reference [51] (both cite the Nilsback and Zisserman flower paper) and is incorrectly used for UCF101; please replace with the correct UCF101 dataset citation.","section":"§3.2.2"},{"comment":"Equations (10)–(11) and (13)–(14) contain unclear notation: some lines have an empty placeholder (e.g., [ci, , E i]) and the difference between consecutive equations is not immediately obvious. Please rewrite these equations with explicit token sequences and clear subscripts so that the insertion of representation tokens across layers is unambiguous.","section":"§4.7 and Supplementary"},{"comment":"The main text says \"The analysis of λ and β is performed on ImageNet\" while the supplementary contains all-dataset sweeps; this wording is confusing because Table 6 (right) is indeed ImageNet-only but Table 2 appears to rely on the supplementary per-dataset best values. Please harmonize the description of which hyperparameters are fixed, which are tuned, and on which split.","section":"Conclusion"},{"comment":"The paper does not include an explicit limitations section. Given that the method depends on the empirically chosen insertion layer J and the assumption of layer-wise specialization (as acknowledged through the ablation in Fig. 7), a short limitations paragraph would help readers assess the scope of the method's applicability.","section":"Conclusion"}],"recommendation":"major_revision","confidential_remarks":"The per-dataset oracle-tuning issue is the most serious problem. The supplementary tables explicitly show that the reported per-dataset HM values in Table 2 are the maxima over the λ and β grids, which means the headline 81.77 average is not a fixed-method result. The paper should be required to report a fixed-configuration result and clarify the hyperparameter selection protocol. Note also that the paper is an extension of a CVPR 2025 paper (MMRL); the incremental contribution of MMRL++ (SRRA and PRC) is real but the evaluation protocol currently prevents a clean assessment of whether the claimed state-of-the-art performance is genuine."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nTwo things you should know before spending time on this. First, the headline SOTA claim is built on per-dataset oracle tuning. Second, the underlying architecture ideas are actually reasonable, but they need a cleaner evaluation.\n\nThe paper reports MMRL++ at 81.77 average harmonic mean on base-to-novel generalization across 11 datasets, 1.90 points above MMA. Checking the supplementary tables shows that Table 2's numbers are not produced by a single fixed configuration. For ImageNet they pick lambda=0.2 from Table A1; for Caltech101 lambda=3; for DTD lambda=7; for EuroSAT lambda=0.01; and similarly for beta from Table A2. Each dataset's entry is the best value over the grid. If you fix lambda=0.2 for all 11 datasets, the average HM drops to about 80.2; fixing beta=0.9 gives about 80.3. That puts the fixed-recipe advantage over MMA at roughly 0.4–0.5 points, with no error bars or significance test in the main table. The \"consistently outperform\" language is stronger than the evidence supports.\n\nWhat is genuinely new: the shared representation space with inserted tokens, the shared-residual aligner (SRRA), and progressive representation composition (PRC). SRRA+PRC cut trainable parameters from 4.992M (MMRL) to 0.813M while holding most of the performance, and Table 7's ablation is clean and informative. The experimental breadth is also real: 15 datasets, four protocols, 10+ baselines, and ablations for each design choice. The paper is an incremental extension of the authors' own CVPR 2025 MMRL work, but a competent one.\n\nThe soft spots in order: (1) oracle hyperparameter selection on the evaluation metric—load-bearing, as above; (2) no code or configuration files, so the fixed recipe can't be checked; (3) no variance reporting on the main tables; (4) the method has eight free parameters, with J also tuned on the same benchmarks, which inflates apparent gains. None of this kills the architectural idea, but it does mean the paper's central empirical claim is overstated.\n\nThis is for readers working on parameter-efficient CLIP adaptation. They should read the method and Table 7, but treat Table 2 as a max-over-hyperparameters bound, not a fixed-model result. It deserves a serious referee because the design is coherent and a cleaned-up version could be a legitimate modest SOTA. My recommendation: send it to review, but require a single fixed configuration, variance estimates, and code release before accepting any SOTA claim.","headline":"The headline MMRL++ base-to-novel gain is per-dataset oracle-tuned on lambda and beta; with a fixed configuration the SOTA margin over MMA shrinks to noise, though the architecture ideas merit a careful look.","tokens_in":28956,"tokens_out":5140,"would_cite":false,"duration_ms":44789,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Shared representation tokens lift CLIP few-shot generalization to a new state of the art.","keywords":["representation learning","vision-language models","parameter-efficient adaptation","few-shot learning","prompt learning","adapter","low-rank adapters","generalization"],"falsifier":"Train MMRL++ on a diverse set of new tasks where adaptation is known to require early-layer changes, such as datasets with heavy low-level distribution shift, and measure base-to-novel harmonic mean with token insertion starting at layers below 6: if inserting tokens at $J<6$ beats $J=6$, the layer-specialization premise collapses and the method's reported advantage would not transfer.","tokens_in":27852,"feed_emoji":"","tokens_out":3330,"duration_ms":35580,"temperature":0.7,"pith_summary":"This paper claims that a vision-language model like CLIP can be adapted with very few labeled examples by inserting learnable representation tokens into the upper layers of both the image and text encoders, while leaving the lower layers untouched and the class-token projection frozen. The authors argue this preserves the model's pre-trained generalization while still allowing task-specific adaptation. The follow-up method MMRL++ adds a shared low-rank aligner and progressive token composition, cutting trainable parameters from 4.992M to 0.813M while raising novel-class accuracy. On the 11-dataset base-to-novel benchmark, it reports an average harmonic mean of 81.77, 1.90 points above the previous best method. If the claim holds, parameter-efficient adapters can simultaneously improve both base-class adaptation and novel-class generalization.","feed_headline":"Shared token space lifts few-shot CLIP generalization","feed_subtitle":"MMRL++ beats prior vision-language adapters on novel classes while cutting trainable parameters to 0.8M.","key_machinery":"The central object is the shared representation space $\\mathcal{R}$, a set of learnable space tokens projected by aligners into both the image and text encoders starting from layer $J=6$. The two new mechanisms in MMRL++ are the Shared-Residual Representation Aligner, written as $W_i^m = W_{\\text{shared}}^m + A_i^m B_i^m$, which shares one aligner weight across layers and models layer-specific knowledge as low-rank residuals, and Progressive Representation Composition, which feeds $\\beta R_{i-1} + (1-\\beta) O_{i-1}$ into the next layer so representation tokens carry forward instance-level information. Together they reduce trainable parameters by roughly 84 percent while enabling gradient sharing across layers.","core_discovery":"MMRL++ proposes that a shared, modality-agnostic representation space, projected into both CLIP encoders as representation tokens in higher layers, can absorb task-specific knowledge while the frozen class token retains the CLIP's zero-shot capabilities. During training, the representation-token projection is learnable and the class-token projection is frozen, with a cosine-similarity regularization pulling class and text features toward the frozen zero-shot features. At inference, base classes use both class and representation features, while novel classes use only class features. MMRL++ then makes this scheme parameter-efficient with a Shared-Residual Representation Aligner (SRRA), which decomposes per-layer aligners into a shared weight plus low-rank residuals, and a Progressive Representation Composition (PRC) mechanism, which propagates representation-token outputs across layers so instance-level information flows through the network. The paper reports state-of-the-art harmonic means on base-to-novel generalization, cross-dataset transfer, and few-shot learning across 11-15 datasets.","pith_inferences":[],"forward_implications":["MMRL++ reports an average base-to-novel harmonic mean of 81.77 across 11 datasets, 1.90 points above the previous best MMA, with 0.813M trainable parameters.","Novel-class accuracy rises from 77.16 for MMRL to 78.32 for MMRL++, indicating that the shared aligner and progressive composition improve generalization to unseen classes.","The SRRA module alone cuts parameters from 4.992M to 0.813M while keeping harmonic mean near parity, so low-rank shared aligners are a viable route to parameter-efficient multimodal adaptation.","In few-shot learning from 1 to 16 shots, the methods rank first and second on average across all shot settings, with gains widening at higher shot counts.","On cross-dataset evaluation, MMRL++ achieves the highest average accuracy of 67.49 percent, suggesting the adapted features transfer to unseen datasets better than prior adapters.","On the EuroSAT dataset, MMRL++ raises novel-class harmonic mean from 87.21 to 91.94 and reduces novel-class standard deviation from 5.05 to 0.54, indicating more stable training.","The decoupling strategy implies that representation tokens are useful exactly where the training distribution matches, while class tokens carry the burden for out-of-distribution classes; this principle could transfer to other frozen backbones or modalities if the layer-specialization assumption holds.","Ablations show that removing the cosine regularization to frozen zero-shot features drops the harmonic mean from 74.44 to 74.02 on ImageNet, suggesting the regularization, rather than the token injection alone, is doing much of the work in preserving generalization."],"supporting_citations":[{"why":"CLIP is the frozen vision-language backbone on which all experiments are built.","marker":"[1]"},{"why":"CoOp defines the continuous prompt learning baseline whose overfitting to base classes this paper targets.","marker":"[19]"},{"why":"MaPLe is the multimodal prompt learning method whose biased text-centric interaction MMRL and MMRL++ compare against.","marker":"[21]"},{"why":"MMA is the previous state-of-the-art multimodal adapter whose shared projection bridges the two modalities and sets the numerical baseline.","marker":"[23]"},{"why":"This is the earlier MMRL paper that introduces the shared representation space and decoupling strategy, extended here.","marker":"[24]"},{"why":"LoRA provides the low-rank decomposition idea used by the Shared-Residual Representation Aligner.","marker":"[25]"},{"why":"CoCoOp supplies the cross-dataset evaluation protocol and an instance-specific prompting baseline.","marker":"[30]"}],"fun_headline_variants":["MMRL++: CLIP few-shot with shared tokens and 0.8M params","Shared token space: CLIP adapts in few-shot with 0.8M params","MMRL++: Interaction-aware representation tokens for CLIP adaptation","Efficient CLIP tuning: shared tokens, 0.8M params, stronger novel classes","MMRL++: Few-shot CLIP via shared projection and residual aligner"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The design assumes that CLIP's lower encoder layers store general knowledge that should stay frozen, while higher layers carry task-specific features that can be modified by inserted representation tokens, and the starting layer $J=6$ is fixed from ablations on the 11 benchmark datasets.","fun_headline_variants_meta":{"raw":{"variants":["MMRL++: CLIP few-shot with shared tokens and 0.8M params","Shared token space: CLIP adapts in few-shot with 0.8M params","MMRL++: Interaction-aware representation tokens for CLIP adaptation","Efficient CLIP tuning: shared tokens, 0.8M params, stronger novel classes","MMRL++: Few-shot CLIP via shared projection and residual aligner"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000863,"raw_usage":{"total_tokens":3789,"prompt_tokens":1038,"completion_tokens":2751,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":654,"completion_tokens_details":{"reasoning_tokens":2653}},"tokens_in":654,"tokens_out":2751,"duration_ms":18803,"temperature":1.0,"reasoning_tokens":2653,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T21:16:03.131614+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train MMRL++ on a diverse set of new tasks where adaptation is known to require early-layer changes, such as datasets with heavy low-level distribution shift, and measure base-to-novel harmonic mean with token insertion starting at layers below 6: if inserting tokens at $J<6$ beats $J=6$, the layer-specialization premise collapses and the method's reported advantage would not transfer.","supporting_citations":[{"cited_title":"International Journal of Computer Vision 130(9), 2337–2348 (2022) 20","cited_arxiv_id":null,"evidence_quote":"CoOp defines the continuous prompt learning baseline whose overfitting to base classes this paper targets."},{"cited_title":"In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp","cited_arxiv_id":null,"evidence_quote":"MaPLe is the multimodal prompt learning method whose biased text-centric interaction MMRL and MMRL++ compare against."},{"cited_title":"In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp","cited_arxiv_id":null,"evidence_quote":"MMA is the previous state-of-the-art multimodal adapter whose shared projection bridges the two modalities and sets the numerical baseline."},{"cited_title":"In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp","cited_arxiv_id":null,"evidence_quote":"CoCoOp supplies the cross-dataset evaluation protocol and an instance-specific prompting baseline."}],"review_version":1}