Pith. sign in

REVIEW 8 cited by

A Good Prompt Is Worth Millions of Parameters: Low-resource Prompt-based Learning for Vision-Language Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2110.08484 v2 pith:QQEJ4SS7 submitted 2021-10-16 cs.CV cs.CL

Woojeong Jin , Yu Cheng , Yelong Shen , Weizhu Chen , Xiang Ren This is my paper · ORCID
classification cs.CVcs.CL
keywords fewvlmmodelspromptsfew-shotlargerlearningperformanceprompt-based
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Large pre-trained vision-language (VL) models can learn a new task with a handful of examples and generalize to a new task without fine-tuning. However, these VL models are hard to deploy for real-world applications due to their impractically huge sizes and slow inference speed. To solve this limitation, we study prompt-based low-resource learning of VL tasks with our proposed method, FewVLM, relatively smaller than recent few-shot learners. For FewVLM, we pre-train a sequence-to-sequence transformer model with prefix language modeling (PrefixLM) and masked language modeling (MaskedLM). Furthermore, we analyze the effect of diverse prompts for few-shot tasks. Experimental results on VQA show that FewVLM with prompt-based learning outperforms Frozen which is 31x larger than FewVLM by 18.2% point and achieves comparable results to a 246x larger model, PICa. In our analysis, we observe that (1) prompts significantly affect zero-shot performance but marginally affect few-shot performance, (2) models with noisy prompts learn as quickly as hand-crafted prompts given larger training data, and (3) MaskedLM helps VQA tasks while PrefixLM boosts captioning performance. Our code is publicly available at \url{https://github.com/woojeongjin/FewVLM}

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 8 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Leveraging Large Vision-Language Model as User Intent-aware Encoder for Composed Image Retrieval

    cs.IR 2024-12 conditional novelty 7.0 of 10

    CIR-LVLM fine-tunes Qwen-VL-Chat with LoRA and hybrid task and instance-specific prompts to produce query and target embeddings, achieving new state-of-the-art recall on Fashion-IQ, Shoes, and CIRR.

  2. Natural Language Supervision for Low-light Image Enhancement

    cs.CV 2025-01 conditional novelty 6.0 of 10

    A low-light enhancement network that conditions image features on text prompts through cross-modal attention reports the best PSNR and SSIM on LOLv1 and LOLv2-synthetic among the methods compared.

  3. DiMPLe -- Disentangled Multi-Modal Prompt Learning: Enhancing Out-Of-Distribution Alignment with Invariant and Spurious Feature Separation

    cs.CV 2025-06 reject novelty 5.0 of 10

    DiMPLe separates CLIP's image and text features into invariant and spurious components and claims large OOD accuracy gains through cross-modal alignment of the invariant parts.

  4. Honey, I Shrunk the Language Model: Impact of Knowledge Distillation Methods on Performance and Explainability

    cs.CL 2025-04 conditional novelty 5.0 of 10

    A small language model distilled with a combination of multitask and counterfactual training on critique-revised explanations produced human-rated higher-quality explanations, while multitask training alone gave the b...

  5. Integrated Structural Prompt Learning for Vision-Language Models

    cs.CV 2025-07 reject novelty 4.0 of 10

    An integrated structural prompt learning method for CLIP reports a state-of-the-art average harmonic mean of 80.70 on 11 base-to-new few-shot classification benchmarks using self- and cross-modal prompt-token interact...

  6. Few-Shot Optimization for Sensor Data Using Large Language Models: A Case Study on Fatigue Detection

    cs.CL 2025-05 reject novelty 4.0 of 10

    A hybrid Euclidean-distance and LLM-relevance example selector for few-shot sensor classification reports a small, statistically fragile gain over distance-only selection on a fatigue detection dataset.

  7. TSPE: Task-Specific Prompt Ensemble for Improved Zero-Shot Audio Classification

    cs.SD 2024-12 reject novelty 4.0 of 10

    Task-specific prompt ensembling with GPT-4-generated attributes and sources improves some zero-shot audio classification datasets while degrading others.

  8. Multimodal AI for Gastrointestinal Diagnostics: Tackling VQA in MEDVQA-GI 2025

    cs.CV 2025-07 conditional novelty 3.0 of 10

    Fine-tuning Florence-2 on a 1% subset of Kvasir-VQA with medical image augmentations yields moderate VQA performance on gastrointestinal endoscopy questions.

Pith tools