Pith. sign in

REVIEW 3 major objections 7 minor 61 references

MMRL++: Parameter-Efficient and Interaction-Aware Representation Learning for Vision-Language Models

T0 review · 3 major / 7 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Shared representation tokens lift CLIP few-shot generalization to a new state of the art.

desk verdict The headline MMRL++ base-to-novel gain is per-dataset oracle-tuned on lambda and beta; with a fixed configuration the SOTA margin over MMA shrinks to noise, though the architecture ideas merit a careful look. read the letter →

arxiv 2505.10088 v1 pith:RDF6FFPH submitted 2025-05-15 cs.CV

Yuncheng Guo , Xiaodong Gu This is my paper · ORCID
classification cs.CV
keywords representationlearningvision-languagemodelsparameter-efficientadaptationfew-shotpromptadapterlow-rankadaptersgeneralization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a vision-language model like CLIP can be adapted with very few labeled examples by inserting learnable representation tokens into the upper layers of both the image and text encoders, while leaving the lower layers untouched and the class-token projection frozen. The authors argue this preserves the model's pre-trained generalization while still allowing task-specific adaptation. The follow-up method MMRL++ adds a shared low-rank aligner and progressive token composition, cutting trainable parameters from 4.992M to 0.813M while raising novel-class accuracy. On the 11-dataset base-to-novel benchmark, it reports an average harmonic mean of 81.77, 1.90 points above the previous best method. If the claim holds, parameter-efficient adapters can simultaneously improve both base-class adaptation and novel-class generalization.

What carries the argument

The central object is the shared representation space $\mathcal{R}$, a set of learnable space tokens projected by aligners into both the image and text encoders starting from layer $J=6$. The two new mechanisms in MMRL++ are the Shared-Residual Representation Aligner, written as $W_i^m = W_{\text{shared}}^m + A_i^m B_i^m$, which shares one aligner weight across layers and models layer-specific knowledge as low-rank residuals, and Progressive Representation Composition, which feeds $\beta R_{i-1} + (1-\beta) O_{i-1}$ into the next layer so representation tokens carry forward instance-level information. Together they reduce trainable parameters by roughly 84 percent while enabling gradient sharing across layers.

What would settle it

Train MMRL++ on a diverse set of new tasks where adaptation is known to require early-layer changes, such as datasets with heavy low-level distribution shift, and measure base-to-novel harmonic mean with token insertion starting at layers below 6: if inserting tokens at $J<6$ beats $J=6$, the layer-specialization premise collapses and the method's reported advantage would not transfer.

Watch

Extended reading notes

Core claim

MMRL++ proposes that a shared, modality-agnostic representation space, projected into both CLIP encoders as representation tokens in higher layers, can absorb task-specific knowledge while the frozen class token retains the CLIP's zero-shot capabilities. During training, the representation-token projection is learnable and the class-token projection is frozen, with a cosine-similarity regularization pulling class and text features toward the frozen zero-shot features. At inference, base classes use both class and representation features, while novel classes use only class features. MMRL++ then makes this scheme parameter-efficient with a Shared-Residual Representation Aligner (SRRA), which decomposes per-layer aligners into a shared weight plus low-rank residuals, and a Progressive Representation Composition (PRC) mechanism, which propagates representation-token outputs across layers so instance-level information flows through the network. The paper reports state-of-the-art harmonic means on base-to-novel generalization, cross-dataset transfer, and few-shot learning across 11-15 datasets.

Load-bearing premise

The design assumes that CLIP's lower encoder layers store general knowledge that should stay frozen, while higher layers carry task-specific features that can be modified by inserted representation tokens, and the starting layer $J=6$ is fixed from ablations on the 11 benchmark datasets.

Editorial extensions

If this is right

  • MMRL++ reports an average base-to-novel harmonic mean of 81.77 across 11 datasets, 1.90 points above the previous best MMA, with 0.813M trainable parameters.
  • Novel-class accuracy rises from 77.16 for MMRL to 78.32 for MMRL++, indicating that the shared aligner and progressive composition improve generalization to unseen classes.
  • The SRRA module alone cuts parameters from 4.992M to 0.813M while keeping harmonic mean near parity, so low-rank shared aligners are a viable route to parameter-efficient multimodal adaptation.
  • In few-shot learning from 1 to 16 shots, the methods rank first and second on average across all shot settings, with gains widening at higher shot counts.
  • On cross-dataset evaluation, MMRL++ achieves the highest average accuracy of 67.49 percent, suggesting the adapted features transfer to unseen datasets better than prior adapters.
  • On the EuroSAT dataset, MMRL++ raises novel-class harmonic mean from 87.21 to 91.94 and reduces novel-class standard deviation from 5.05 to 0.54, indicating more stable training.
  • The decoupling strategy implies that representation tokens are useful exactly where the training distribution matches, while class tokens carry the burden for out-of-distribution classes; this principle could transfer to other frozen backbones or modalities if the layer-specialization assumption holds.
  • Ablations show that removing the cosine regularization to frozen zero-shot features drops the harmonic mean from 74.44 to 74.02 on ImageNet, suggesting the regularization, rather than the token injection alone, is doing much of the work in preserving generalization.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper proposes MMRL and MMRL++, parameter-efficient methods for adapting CLIP to few-shot downstream tasks. MMRL introduces a shared learnable representation space whose tokens are projected into upper layers of both the image and text encoders as representation tokens, while the class token is regularized to preserve zero-shot knowledge; inference uses both feature types for base classes and only class features for novel classes. MMRL++ adds a shared-residual representation aligner (SRRA) with low-rank factorization and a progressive representation composition (PRC) mechanism to reduce parameters and improve inter-layer information flow. Experiments cover base-to-novel generalization on 11 datasets, cross-dataset transfer, domain generalization, and few-shot learning, with ablations for each design choice.

Significance. If the central claim holds, the paper would provide a useful and practical contribution: a competitive parameter-efficient adapter that improves novel-class generalization over prior adapters (e.g., MMA) while using fewer trainable parameters than the earlier MMRL. The paper is thorough in its experimental coverage—four evaluation protocols, multiple ablations, training stability analysis, and a computational-cost table. The idea of decoupling class and representation features, and the layer-wise insertion strategy, are clearly motivated and supported by ablations. However, the headline state-of-the-art claim on base-to-novel generalization is undermined by a per-dataset hyperparameter selection issue in Table 2, which makes the magnitude of the reported improvement over prior methods questionable. Because the paper reports no formal derivation but relies on empirical evidence, the evaluation protocol is load-bearing for the central claim.

major comments (3)
  1. [§4.2, §4.7] The reported MMRL++ harmonic mean of 81.77 in Table 2 is obtained by selecting the best λ and β per dataset on the evaluation metric. Cross-referencing Table 2 with the supplementary sweeps shows that each dataset's HM is the maximum over the λ-grid (Table A1) or β-grid (Table A2)—for example, ImageNet 74.44 at λ=0.2, Caltech101 96.75 at λ=3, DTD 74.46 at λ=7, EuroSAT 91.94 at λ=0.01, and similarly per-dataset best β values. Fixing λ=0.2 across all 11 datasets gives an average HM of approximately 80.21 (computed from Table A1), and fixing β=0.9 gives approximately 80.28 (from Table A2). Thus the advertised +1.90 HM over MMA (79.87) reduces to roughly +0.3–0.4 under a single fixed configuration, and the paper does not report a joint fixed (λ, β) configuration. Because the state-of-the-art claim is the central contribution, this test-metric selection is a load-bearing issue. Please report results under a single fixed configuration chosen on a validation split, or provide a comparison under an equivalent per-dataset tuning protocol applied fairly to all baselines.
  2. [§4.3, Table 2, Table 9] The implementation details state that "the configurations for λ and β are provided in the Supplementary Material," and Section 4.7 says the analysis of λ and β is performed on ImageNet with comprehensive results in the supplementary. The supplementary tables, however, report test harmonic means for each dataset across the λ and β grids, and Table 2 appears to use the per-dataset best value from these grids. This implies that the test split (or the combined base/novel evaluation metric) is used for hyperparameter selection, which can overstate performance relative to fixed-configuration baselines. Please clarify the exact selection protocol: which data split is used to choose λ and β, and whether the test base/novel accuracy is used for selection. If test-based selection is indeed used, this should be disclosed prominently and matched for all compared methods.
  3. [Table 2] Table 2 reports no standard deviations or significance tests for the average base, novel, and harmonic mean metrics, despite Section 4.2 stating that results are averaged over three independent runs. Given that the fixed-configuration gap over MMA is likely small (≈0.3–0.4 HM), the reader cannot assess whether the improvement is statistically meaningful. Please include per-seed standard deviations for the headline average metrics (and ideally per-dataset for MMRL and MMRL++), and consider reporting confidence intervals or a simple significance test for the comparison against the strongest baseline.
minor comments (7)
  1. [§4.5] The baseline is spelled "CoCoOp" in the text but appears as "CoOpOp" in Fig. 1 and in the Table 2 header row; please correct the spelling for consistency.
  2. [Related Work, §2.2] In Section 4.5, the sentence "MMRL and MMRL++ attains top performance on 2 out of the 4 domain-shifted datasets" contains a subject-verb agreement error and a mixed singular/plural construction; please revise.
  3. [References] In the last paragraph of Section 2.2, "their substantial computational overhead may places them" should be "may place them."
  4. [§3.2.2] Reference [55] is a duplicate of reference [51] (both cite the Nilsback and Zisserman flower paper) and is incorrectly used for UCF101; please replace with the correct UCF101 dataset citation.
  5. [§4.7 and Supplementary] Equations (10)–(11) and (13)–(14) contain unclear notation: some lines have an empty placeholder (e.g., [ci, , E i]) and the difference between consecutive equations is not immediately obvious. Please rewrite these equations with explicit token sequences and clear subscripts so that the insertion of representation tokens across layers is unambiguous.
  6. [Conclusion] The main text says "The analysis of λ and β is performed on ImageNet" while the supplementary contains all-dataset sweeps; this wording is confusing because Table 6 (right) is indeed ImageNet-only but Table 2 appears to rely on the supplementary per-dataset best values. Please harmonize the description of which hyperparameters are fixed, which are tuned, and on which split.
  7. [Conclusion] The paper does not include an explicit limitations section. Given that the method depends on the empirically chosen insertion layer J and the assumption of layer-wise specialization (as acknowledged through the ablation in Fig. 7), a short limitations paragraph would help readers assess the scope of the method's applicability.

Circularity Check

1 steps flagged · score 6.0 of 10

Table 2's MMRL++ SOTA is per-dataset oracle-tuned: each reported HM matches the best λ/β in Supplementary Tables A1/A2, so the advertised 81.77 average is partly test-metric selection.

  1. fitted input called prediction [Sec. 4.2 and 4.7; Supplementary Tables A1-A2; Table 2]
    "All ablation experiments, except for the analysis of λ and β, are conducted under the base-to-novel generalization setting across 11 datasets, with averaged results reported. The analysis of λ and β is performed on ImageNet; please refer to the Supplementary Material for comprehensive results across all datasets. ... Table A1: Performance across different values of λ on 11 datasets, evaluated using the harmonic mean (HM) metric."

    Table 2's MMRL++ row is not a single fixed configuration. Cross-referencing Supplementary Table A1 shows that each reported HM is the best value over the λ grid for that dataset: ImageNet 74.44 at λ=0.2, Caltech101 96.75 at λ=3, DTD 74.46 at λ=7, EuroSAT 91.94 at λ=0.01, UCF101 83.81 at λ=3, etc.; similarly Table A2 selects β per dataset. Because λ and β are chosen to maximize the very HM reported in Table 2 on base/novel test splits, the advertised average HM 81.77 and the +1.90 gap over MMA are in part a maximum-over-grid artifact rather than the performance of a fixed MMRL++ configuration.

full rationale

The paper is an empirical adapter paper rather than a formal derivation: the architecture choices are ablated against external baselines (MMA, MaPLe, PromptSRC, etc.), the regularization target is the frozen CLIP zero-shot feature space, and the self-citation of the authors' own MMRL is disclosed and not load-bearing. There is no analytic circularity such as a uniqueness theorem imported from prior work or an ansatz smuggled in via citation. The one serious issue is that the headline base-to-novel result is obtained by selecting λ and β per dataset on the reported HM metric: Supplementary Tables A1/A2 give HM for each dataset over λ and β grids, and Table 2's MMRL++ numbers coincide with those per-dataset maxima. Consequently the central claim of a fixed MMRL++ state-of-the-art reduces in part to oracle hyperparameter selection, which is a fitted-input-called-prediction pattern. The remaining components (SRRA low-rank sharing, PRC composition, decoupled inference) are empirically tested against external methods and internal ablations, so the circularity is localized to the headline comparison rather than pervasive.

Assumptions & free parameters 7 free parameters · 5 assumptions · 1 invented entities

The central claim rests on several hand-chosen hyperparameters (alpha, lambda, beta, J, K, dr, r1, r2) that are tuned on the very benchmarks used to report final results. The architectural rationale also depends on untested assumptions about layer-wise specialization of CLIP features and the sufficiency of low-rank shared aligners. No invented entity predicts a measurable outside-the-benchmark phenomenon.

free parameters (7)
  • alpha (feature balance weight) = 0.7
    Chosen by sweeping alpha across 11 datasets and selecting the value with the highest average harmonic mean (Table 10).
  • lambda (regularization penalty) = per dataset, e.g., 0.01 to 10
    Tuned per dataset on the same benchmark used for final evaluation (Supplementary Table A1).
  • beta (PRC transfer coefficient) = per dataset, e.g., 0.0 to 1.0
    Tuned per dataset on the same benchmark used for final evaluation (Supplementary Table A2).
  • J (insertion layer) = 6
    Chosen by ablation on layer insertion position (Fig. 7 left) to maximize base-to-novel HM.
  • K (number of representation tokens) = 5
    Chosen by ablation on token count (Fig. 7 right).
  • dr (representation space dimension) = 512
    Chosen by ablation on representation space dimension (Fig. 7 middle).
  • r1, r2 (low-rank dimensions) = r1=4, r2=64
    Chosen by ablation on low-rank dimensions (Table 6) to maximize HM.
assumptions (5)
  • domain assumption CLIP's lower-layer representations are general and its higher-layer representations are task-specific, so inserting trainable tokens only from layer J onward preserves general knowledge.
    Adopted from MMA (ref [23]) and operationalized in Sec. 3.2.2; the paper validates it only empirically via Fig. 7.
  • domain assumption Frozen CLIP zero-shot features (f0, w0) are a reliable target for preserving generalization during adaptation.
    Used in the cosine regularization losses (Eq. 20-21); assumes the pre-trained zero-shot space is the correct reference for generalizable knowledge.
  • domain assumption Class token features generalize better than representation token features for unseen classes, justifying the decoupled inference rule.
    Used in Eq. 24; the paper provides ablation evidence (Table 5) but the assumption is not derived.
  • ad hoc to paper A single shared aligner Wshared plus per-layer low-rank residuals can approximate the expressive power of layer-specific aligners.
    Central to SRRA (Eq. 26-27); the paper demonstrates it empirically but provides no guarantee that the low-rank residual suffices for all tasks.
  • standard math The standard Transformer forward pass and attention masking rules apply when extra tokens are inserted into the text encoder sequence.
    Used throughout Sec. 3; the paper notes the attention mask must be modified but does not specify the exact mask construction.
invented entities (1)
  • Shared learnable representation space R with K space tokens
    purpose: Provides a modality-agnostic bridge from which representation tokens are projected into both image and text encoders, enabling cross-modal interaction and task-specific adaptation.
    The representation space is a learned internal construct. It makes no falsifiable prediction outside the evaluated benchmark tasks; its only evidence is improved accuracy on the reported datasets.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MMRL++: Parameter-Efficient and Interaction-Aware Representation Learning for Vision-Language Models." pith.science (2026). https://pith.science/paper/RDF6FFPH

@misc{pith2026250510088,
  author       = {Pith},
  title        = {Pith review of: MMRL++: Parameter-Efficient and Interaction-Aware Representation Learning for Vision-Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RDF6FFPH}},
  note         = {Machine review of arXiv:2505.10088}
}
read the original abstract

Large-scale pre-trained Vision-Language Models (VLMs) have significantly advanced transfer learning across diverse tasks. However, adapting these models with limited few-shot data often leads to overfitting, undermining their ability to generalize to new tasks. To address this, we propose Multi-Modal Representation Learning (MMRL), which introduces a shared, learnable, modality-agnostic representation space. MMRL generates space tokens projected into both text and image encoders as representation tokens, enabling more effective cross-modal interactions. Unlike prior methods that mainly optimize class token features, MMRL inserts representation tokens into higher encoder layers--where task-specific features are more prominent--while preserving general knowledge in the lower layers. During training, both class and representation features are jointly optimized: a trainable projection layer is applied to representation tokens for task adaptation, while the projection layer for class token remains frozen to retain pre-trained knowledge. To further promote generalization, we introduce a regularization term aligning class and text features with the frozen VLM's zero-shot features. At inference, a decoupling strategy uses both class and representation features for base tasks, but only class features for novel tasks due to their stronger generalization. Building upon this, we propose MMRL++, a parameter-efficient and interaction-aware extension that significantly reduces trainable parameters and enhances intra-modal interactions--particularly across the layers of representation tokens--allowing gradient sharing and instance-specific information to propagate more effectively through the network. Extensive experiments on 15 datasets demonstrate that MMRL and MMRL++ consistently outperform state-of-the-art methods, achieving a strong balance between task-specific adaptation and generalization.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

61 extracted references · 38 canonical work pages

  1. [1]

    : Learn- ing transferable visual models from natural language supervision

    Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. : Learn- ing transferable visual models from natural language supervision. In: International Con- ference on Machine Learning, pp. 8748–8763 (2021). PMLR

    2021

  2. [2]

    In: International Conference on Machine Learning, pp

    Jia, C., Yang, Y., Xia, Y., Chen, Y.-T., Parekh, Z., Pham, H., Le, Q., Sung, Y.-H., Li, Z., Duerig, T.: Scaling up visual and vision- language representation learning with noisy text supervision. In: International Conference on Machine Learning, pp. 4904–4916 (2021). PMLR

    2021

  3. [3]

    : Flamingo: a visual language model for few-shot learning

    Alayrac, J.-B., Donahue, J., Luc, P., Miech, A., Barr, I., Hasson, Y., Lenc, K., Mensch, A., Millican, K., Reynolds, M., et al. : Flamingo: a visual language model for few-shot learning. Advances in neural information processing systems 35, 23716–23736 (2022) 19

    work page 2022

  4. [4]

    arXiv preprint arXiv:2111.07783 (2021)

    Yao, L., Huang, R., Hou, L., Lu, G., Niu, M., Xu, H., Liang, X., Li, Z., Jiang, X., Xu, C.: Filip: Fine-grained interactive language-image pre-training. arXiv preprint arXiv:2111.07783 (2021)

    arXiv 2021

  5. [5]

    Advances in Neural Infor- mation Processing Systems 36, 72096–72109 (2023)

    Huang, S., Dong, L., Wang, W., Hao, Y., Singhal, S., Ma, S., Lv, T., Cui, L., Mohammed, O.K., Patra, B., et al.: Language is not all you need: Aligning perception with language models. Advances in Neural Infor- mation Processing Systems 36, 72096–72109 (2023)

    work page 2023

  6. [6]

    https://arxiv

    Peng, Z., Wang, W., Dong, L., Hao, Y., Huang, S., Ma, S., Wei, F.: Kosmos- 2: Grounding Multimodal Large Language Models to the World (2023). https://arxiv. org/abs/2306.14824

    arXiv 2023

  7. [7]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp

    Lin, J., Yin, H., Ping, W., Molchanov, P., Shoeybi, M., Han, S.: Vila: On pre-training for visual language models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 26689– 26699 (2024)

    work page 2024

  8. [8]

    https://arxiv.org/abs/1807.03748

    Oord, A., Li, Y., Vinyals, O.: Representation Learning with Contrastive Predictive Coding (2019). https://arxiv.org/abs/1807.03748

    arXiv 2019

Show all 61 references
  1. [9]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp

    Huang, C., Jiang, A., Feng, J., Zhang, Y., Wang, X., Wang, Y.: Adapting visual- language models for generalizable anomaly detection in medical images. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11375– 11385 (2024)

    2024

  2. [10]

    https://arxiv

    Zhao, Z., Liu, Y., Wu, H., Wang, M., Li, Y., Wang, S., Teng, L., Liu, D., Cui, Z., Wang, Q., Shen, D.: CLIP in Medical Imaging: A Comprehensive Survey (2024). https://arxiv. org/abs/2312.07353

    2024 arXiv

  3. [11]

    In: Gold- berg, Y., Kozareva, Z., Zhang, Y

    Wang, Z., Wu, Z., Agarwal, D., Sun, J.: MedCLIP: Contrastive learning from unpaired medical images and text. In: Gold- berg, Y., Kozareva, Z., Zhang, Y. (eds.) Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 3876–3887. Association fo...

    2022 doi

  4. [12]

    In: Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp

    Barraco, M., Cornia, M., Cascianelli, S., Baraldi, L., Cucchiara, R.: The unreason- able effectiveness of clip features for image captioning: an experimental analysis. In: Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4662–4670 (2022)

    2022

  5. [13]

    https://arxiv.org/abs/2111.09734

    Mokady, R., Hertz, A., Bermano, A.H.: Clip- Cap: CLIP Prefix for Image Captioning (2021). https://arxiv.org/abs/2111.09734

    2021 arXiv

  6. [14]

    In: Proceedings of the 29th ACM Interna- tional Conference on Multimedia, pp

    Tang, M., Wang, Z., Liu, Z., Rao, F., Li, D., Li, X.: Clip4caption: Clip for video caption. In: Proceedings of the 29th ACM Interna- tional Conference on Multimedia, pp. 4858– 4862 (2021)

    2021

  7. [15]

    In: Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp

    ¨Ozdemir, ¨O., Akag¨ und¨ uz, E.: Enhancing visual question answering through question- driven image captions as prompts. In: Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1562–1571 (2024)

    2024

  8. [16]

    In: 2023 IEEE Inter- national Conference on Image Processing (ICIP), pp

    Ye, S., Kong, W., Yao, C., Ren, J., Jiang, X.: Video question answering using clip-guided visual-text attention. In: 2023 IEEE Inter- national Conference on Image Processing (ICIP), pp. 81–85 (2023). IEEE

    2023

  9. [17]

    https://arxiv.org/abs/2305.12223

    Wang, G., Ge, Y., Ding, X., Kankanhalli, M., Shan, Y.: What Makes for Good Visual Tok- enizers for Large Language Models? (2023). https://arxiv.org/abs/2305.12223

    2023 arXiv

  10. [18]

    In: 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp

    Parkhi, O.M., Vedaldi, A., Zisserman, A., Jawahar, C.: Cats and dogs. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp. 3498–3505 (2012). IEEE

    2012

  11. [19]

    International Journal of Computer Vision 130(9), 2337–2348 (2022) 20

    Zhou, K., Yang, J., Loy, C.C., Liu, Z.: Learn- ing to prompt for vision-language models. International Journal of Computer Vision 130(9), 2337–2348 (2022) 20

    2022

  12. [20]

    In: Moens, M.-F., Huang, X., Specia, L., Yih, S.W.-t

    Lester, B., Al-Rfou, R., Constant, N.: The power of scale for parameter-efficient prompt tuning. In: Moens, M.-F., Huang, X., Specia, L., Yih, S.W.-t. (eds.) Proceed- ings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 3045–3059. Association fo...

    2021 doi

  13. [21]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp

    Khattak, M.U., Rasheed, H., Maaz, M., Khan, S., Khan, F.S.: Maple: Multi-modal prompt learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 19113–19122 (2023)

    2023

  14. [22]

    International Journal of Computer Vision 132(2), 581–595 (2024)

    Gao, P., Geng, S., Zhang, R., Ma, T., Fang, R., Zhang, Y., Li, H., Qiao, Y.: Clip-adapter: Better vision-language models with feature adapters. International Journal of Computer Vision 132(2), 581–595 (2024)

    2024

  15. [23]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp

    Yang, L., Zhang, R.-Y., Wang, Y., Xie, X.: Mma: Multi-modal adapter for vision- language models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 23826–23837 (2024)

    2024

  16. [24]

    arXiv preprint arXiv:2503.08497 (2025)

    Guo, Y., Gu, X.: Mmrl: Multi-modal repre- sentation learning for vision-language mod- els. arXiv preprint arXiv:2503.08497 (2025)

    2025 arXiv

  17. [25]

    : Lora: Low-rank adaptation of large language models

    Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W., et al. : Lora: Low-rank adaptation of large language models. ICLR 1(2), 3 (2022)

    2022

  18. [26]

    arXiv preprint arXiv:2404.13628 (2024)

    Wu, X., Huang, S., Wei, F.: Mixture of lora experts. arXiv preprint arXiv:2404.13628 (2024)

    2024 arXiv

  19. [27]

    arXiv preprint arXiv:2503.01359 (2025)

    Huang, Y., Ye, P., Huang, C., Cao, J., Zhang, L., Li, B., Yu, G., Chen, T.: Ders: Towards extremely efficient upcycled mixture-of-experts models. arXiv preprint arXiv:2503.01359 (2025)

    2025 arXiv

  20. [28]

    In: Zong, C., Xia, F., Li, W., Navigli, R

    Li, X.L., Liang, P.: Prefix-tuning: Opti- mizing continuous prompts for generation. In: Zong, C., Xia, F., Li, W., Navigli, R. (eds.) Proceedings of the 59th Annual Meeting of the Association for Computa- tional Linguistics and the 11th International Joint Conference on Natura...

    2021 doi

  21. [29]

    In: Muresan, S., Nakov, P., Villavicen- cio, A

    Liu, X., Ji, K., Fu, Y., Tam, W., Du, Z., Yang, Z., Tang, J.: P-tuning: Prompt tuning can be comparable to fine-tuning across scales and tasks. In: Muresan, S., Nakov, P., Villavicen- cio, A. (eds.) Proceedings of the 60th Annual Meeting of the Association for Computa- tional ...

    2022 doi

  22. [30]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp

    Zhou, K., Yang, J., Loy, C.C., Liu, Z.: Con- ditional prompt learning for vision-language models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16816–16825 (2022)

    2022

  23. [31]

    In: Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp

    Lu, Y., Liu, J., Zhang, Y., Liu, Y., Tian, X.: Prompt distribution learning. In: Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5206–5215 (2022)

    2022

  24. [32]

    https://arxiv.org/abs/2210

    Chen, G., Yao, W., Song, X., Li, X., Rao, Y., Zhang, K.: PLOT: Prompt Learning with Optimal Transport for Vision-Language Models (2023). https://arxiv.org/abs/2210. 01253

    2023

  25. [33]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp

    Yao, H., Zhang, R., Xu, C.: Visual-language prompt tuning with knowledge-guided con- text optimization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6757–6767 (2023)

    2023

  26. [34]

    21 In: Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pp

    Zhu, B., Niu, Y., Han, Y., Wu, Y., Zhang, H.: Prompt-aligned gradient for prompt tuning. 21 In: Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pp. 15659–15669 (2023)

    2023

  27. [35]

    In: Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pp

    Lee, D., Song, S., Suh, J., Choi, J., Lee, S., Kim, H.J.: Read-only prompt optimiza- tion for vision-language few-shot learning. In: Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pp. 1401–1411 (2023)

    2023

  28. [36]

    International Journal of Com- puter Vision, 1–16 (2024)

    Xu, C., Zhu, Y., Shen, H., Chen, B., Liao, Y., Chen, X., Wang, L.: Progressive visual prompt learning with contrastive feature re- formation. International Journal of Com- puter Vision, 1–16 (2024)

    2024

  29. [37]

    In: Proceed- ings of the IEEE/CVF International Confer- ence on Computer Vision, pp

    Khattak, M.U., Wasim, S.T., Naseer, M., Khan, S., Yang, M.-H., Khan, F.S.: Self- regulating prompts: Foundational model adaptation without forgetting. In: Proceed- ings of the IEEE/CVF International Confer- ence on Computer Vision, pp. 15190–15200 (2023)

    2023

  30. [38]

    IEEE Transactions on Image Processing (2024)

    Zhao, C., Wang, Y., Jiang, X., Shen, Y., Song, K., Li, D., Miao, D.: Learning domain invariant prompt for vision-language mod- els. IEEE Transactions on Image Processing (2024)

    2024

  31. [39]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp

    Yao, H., Zhang, R., Xu, C.: Tcp: Textual- based class-aware prompt tuning for visual- language model. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 23438–23448 (2024)

    2024

  32. [40]

    In: European Con- ference on Computer Vision, pp

    Zhang, R., Zhang, W., Fang, R., Gao, P., Li, K., Dai, J., Qiao, Y., Li, H.: Tip- adapter: Training-free adaption of clip for few-shot classification. In: European Con- ference on Computer Vision, pp. 493–510 (2022). Springer

    2022

  33. [41]

    Roy, S., Etemad, A.: Consistency-guided prompt learning for vision-language models. (2024)

    2024

  34. [42]

    In: Proceedings of the AAAI Conference on Artificial Intelligence, vol

    Wang, Y., Jiang, X., Cheng, D., Li, D., Zhao, C.: Learning hierarchical prompt with structured linguistic knowledge for vision- language models. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, pp. 5749–5757 (2024)

    2024

  35. [43]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp

    Tian, X., Zou, S., Yang, Z., Zhang, J.: Argue: Attribute-guided prompt tuning for vision- language models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 28578–28587 (2024)

    2024

  36. [44]

    International Jour- nal of Computer Vision, 1–16 (2025)

    Yang, L., Zhang, R.-Y., Chen, Q., Xie, X.: Learning with enriched inductive biases for vision-language models. International Jour- nal of Computer Vision, 1–16 (2025)

    2025

  37. [45]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp

    Li, Z., Li, X., Fu, X., Zhang, X., Wang, W., Chen, S., Yang, J.: Promptkd: Unsuper- vised prompt distillation for vision-language models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 26617–26626 (2024)

    2024

  38. [46]

    Advances in neural information processing systems 33, 1877–1901 (2020)

    Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J.D., Dhariwal, P., Neelakan- tan, A., Shyam, P., Sastry, G., Askell, A., et al.: Language models are few-shot learners. Advances in neural information processing systems 33, 1877–1901 (2020)

    2020

  39. [47]

    Advances in Neural Information Processing Systems (2017)

    Vaswani, A.: Attention is all you need. Advances in Neural Information Processing Systems (2017)

    2017

  40. [48]

    In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp

    Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee

    2009

  41. [49]

    In: 2004 Conference on Computer Vision and Pattern Recognition Workshop, pp

    Fei-Fei, L., Fergus, R., Perona, P.: Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories. In: 2004 Conference on Computer Vision and Pattern Recognition Workshop, pp. 178–178 (2004). IEEE

    2004

  42. [50]

    In: Proceedings of the IEEE Inter- national Conference on Computer Vision Workshops, pp

    Krause, J., Stark, M., Deng, J., Fei-Fei, L.: 3d 22 object representations for fine-grained cate- gorization. In: Proceedings of the IEEE Inter- national Conference on Computer Vision Workshops, pp. 554–561 (2013)

    2013

  43. [52]

    In: Computer vision– ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part VI 13, pp

    Bossard, L., Guillaumin, M., Van Gool, L.: Food-101–mining discriminative components with random forests. In: Computer vision– ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part VI 13, pp. 446–461 (2014). Springer

    2014

  44. [53]

    arXiv preprint arXiv:1306.5151 (2013)

    Maji, S., Rahtu, E., Kannala, J., Blaschko, M., Vedaldi, A.: Fine-grained visual classification of aircraft. arXiv preprint arXiv:1306.5151 (2013)

    2013 arXiv

  45. [54]

    In: 2010 IEEE Computer Society Conference on Com- puter Vision and Pattern Recognition, pp

    Xiao, J., Hays, J., Ehinger, K.A., Oliva, A., Torralba, A.: Sun database: Large-scale scene recognition from abbey to zoo. In: 2010 IEEE Computer Society Conference on Com- puter Vision and Pattern Recognition, pp. 3485–3492 (2010). IEEE

    2010

  46. [55]

    In: 2008 Sixth Indian Conference on Computer Vision, Graphics & Image Process- ing, pp

    Nilsback, M.-E., Zisserman, A.: Automated flower classification over a large number of classes. In: 2008 Sixth Indian Conference on Computer Vision, Graphics & Image Process- ing, pp. 722–729 (2008). IEEE

    2008

  47. [56]

    In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recogni- tion, pp

    Cimpoi, M., Maji, S., Kokkinos, I., Mohamed, S., Vedaldi, A.: Describing textures in the wild. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recogni- tion, pp. 3606–3613 (2014)

    2014

  48. [57]

    IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 12(7), 2217–2226 (2019)

    Helber, P., Bischke, B., Dengel, A., Borth, D.: Eurosat: A novel dataset and deep learning benchmark for land use and land cover clas- sification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 12(7), 2217–2226 (2019)

    2019

  49. [58]

    5389–5400 (2019)

    Recht, B., Roelofs, R., Schmidt, L., Shankar, V.: Do imagenet classifiers generalize to imagenet? In: International Conference on Machine Learning, pp. 5389–5400 (2019). PMLR

    2019

  50. [59]

    Advances in Neural Information Processing Systems 32 (2019)

    Wang, H., Ge, S., Lipton, Z., Xing, E.P.: Learning robust global representations by penalizing local predictive power. Advances in Neural Information Processing Systems 32 (2019)

    2019

  51. [60]

    In: Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pp

    Hendrycks, D., Zhao, K., Basart, S., Stein- hardt, J., Song, D.: Natural adversarial exam- ples. In: Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pp. 15262–15271 (2021)

    2021

  52. [61]

    : The many faces of robustness: A critical analysis of out-of- distribution generalization

    Hendrycks, D., Basart, S., Mu, N., Kadavath, S., Wang, F., Dorundo, E., Desai, R., Zhu, T., Parajuli, S., Guo, M., et al. : The many faces of robustness: A critical analysis of out-of- distribution generalization. In: Proceedings of the IEEE/CVF International Conference on Com...

    2021

  53. [62]

    Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., Houlsby, N.: An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale (2021). https: //arxiv.org/abs/201...

    2021 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.