Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

Enhancing Multimodal In-Context Learning for Image Classification through Coreset Optimization

T0 review · 4 major / 6 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Discarded support images, folded into a small coreset, beat full-set in-context retrieval.

desk verdict Solid engineering paper with a credible central result, but the mechanism story is under-supported and there are reporting inconsistencies that need fixing before the claims are fully trustworthy. read the letter →

arxiv 2504.14200 v2 pith:G4YINJ2G submitted 2025-04-19 cs.CV cs.AI

Huiyi Chen , Jiawei Peng , Kaihua Tang , Xin Geng , Xu Yang This is my paper · ORCID
classification cs.CVcs.AI
keywords in-contextlearninglargevision-languagemodelscoresetoptimizationimageclassificationdemonstrationselectiondiversity-basedonlinevisualfeaturekeys
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes KeCO, a method for building a small demonstration set—a coreset—for in-context image classification with large vision-language models. Instead of exhaustively retrieving examples from the whole support set, KeCO randomly initializes a compact coreset and then updates each core sample's visual feature, called its key, by blending it with unused support images of the same class. The paper's central claim is that this feature-level update lets a coreset one-fifth the size of the support set outperform retrieval from the full support set, lifting OpenFlamingo-3B accuracy by 20% or more relative to the fixed coreset and IDEFICS-8B accuracy by about 10%. The update works best when each unused image is matched to the most dissimilar same-class core sample, because repeated averaging preserves shared class-typical features while blurring background, pose, and lighting. The same procedure also works when data arrives as a stream, which is the practically relevant setting for memory-constrained deployment.

What carries the argument

The central object is the key $k_t=\phi(I_t)$, the visual feature of a coreset image extracted by the model's vision encoder (CLIP/ViT-L-14 for OpenFlamingo, SigLIP for IDEFICS). The update rule is $k'_{t_j}=k_{t_j}-\alpha\,\frac{1}{|S'_{i,j}|}\sum_{s\in S'_{i,j}}(k_{t_j}-\phi(I_s))$, which moves each core key partway toward the mean feature of the untapped samples assigned to it. The argument's load-bearing pairing is this update with diversity-based target selection: choosing the least similar same-class core sample means averaging across dissimilar images, so the class-consistent signal survives while per-image background, pose, and illumination differences cancel. The paper's mechanism claim is that this makes keys more clustered by category, and that clustering is what lets retrieval from the updated coreset give the frozen LVLM cleaner demonstrations.

What would settle it

A concrete check: swap the backgrounds of same-class images and see whether diversity-updated keys still retrieve the correct class; if background similarity, not class, drives the updated key, the mechanism is refuted. Alternatively, rerun KeCO-DS with a vision encoder that is not aligned with the model's internal representation and check whether the 10–20% gains persist; vanishing gains would show the alignment premise, not the update rule, is doing the work.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that discarded support samples are not waste: their features can be folded into a much smaller coreset, and the choice of which core sample each discarded sample updates determines whether the fold helps or hurts. With diversity-based selection, each untapped image updates the least similar same-class core sample, and the updated keys become more tightly clustered by class, so top-k retrieval from the updated coreset supplies the frozen LVLM with more informative demonstrations. The strongest quantitative claim is that this updated coreset of about 1,000–1,200 samples beats the fixed full support set of 5,000–6,000 samples: on CUB-200, OpenFlamingo-3B rises from 61.04% to 74.20% in the 2-shot setting and from 58.61% to 76.99% in the 4-shot setting. Similarity-based selection, by contrast, tends to retain misleading background information and can underperform even the unrefined coreset. The paper also claims the same update transfers to a simulated online setting where each sample is seen once, including a gain of roughly 5 points for Qwen2-VL on CUB-200.

Load-bearing premise

The load-bearing premise is that a key updated by averaging a core image's feature with a dissimilar same-class image retains the class signal and erases the differences, and that this feature space is aligned well enough with the frozen LVLM's internal representation that the updated keys order demonstrations the model actually uses.

Editorial extensions

If this is right

  • A coreset of size 1,000–1,200 can beat full-support retrieval from 5,000–6,000 samples on the tested benchmarks, so the memory and search cost of ICL demonstration selection can be cut by roughly a factor of five without sacrificing accuracy.
  • Diversity-based selection is the rule that makes the update useful; random selection helps, but similarity-based selection can hurt, so the mechanism behind KeCO is the pairing of averaging with dissimilar partners.
  • The framework carries over to online and streaming data, where each sample is used once; this turns coreset construction into a continual update process rather than a one-time offline selection.
  • Fine-grained datasets benefit more than coarse-grained ones, consistent with the claim that updated keys supply class-discriminative visual knowledge the LVLM lacks.
  • The initialization strategy matters less than the update: even a random class-balanced coreset, after KeCO updates, outperforms coresets initialized by k-center or Infoscore in most compared settings.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: KeCO is effectively a feature-space data-distillation step, so the same update rule could be applied to other retrieval-based multimodal tasks, such as captioning or visual question answering, whenever the retriever's encoder is also the model's encoder; the diversity rule would need to be revalidated per task.
  • A testable extension the paper does not run: measure how the gain saturates as the untapped set grows; the paper's ratio study (1:2 to 1:6) shows continuing improvement, so finding the saturation point would tell practitioners how much unlabeled support data is worth collecting.
  • One risk not resolved by the paper's experiments: updated keys are interpolated feature vectors that may not correspond to any real image, so if the vision encoder and the language model are weakly aligned, the benefit could degrade; the paper's own comparison with Infoscore initialization hints that such misalignment exists.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes KeCO (Key-based Coreset Optimization), a method for multimodal in-context learning (ICL) in image classification. Starting from a randomly selected, class-balanced coreset from a support set, KeCO uses the remaining untapped support samples to update a stored visual feature (key) for each coreset image via linear interpolation, with the target coreset sample chosen by random (RS), similarity-based (SS), or diversity-based (DS) selection. At inference, demonstrations are retrieved by ranking the coreset images according to the updated keys, and the original images of the top-k coreset samples are fed to a frozen LVLM. Experiments on CUB-200, Stanford Dogs, and ImageNet-100 with OpenFlamingo-3B, IDEFICS-8B, and Qwen2-VL report that KeCO-RS and KeCO-DS outperform both a fixed coreset baseline (FS-IC) and a full-support-set similarity retrieval baseline (FS-IS), with KeCO-DS best and KeCO-SS worst. A simulated online variant is also evaluated. The central empirical claim is that updating keys with untapped data yields a compact coreset that beats five-times-larger support set retrieval.

Significance. If the empirical claims hold, KeCO is a simple, memory-efficient way to improve LVLM ICL for image classification, with a plausible practical advantage in streaming/online settings. The method is easy to implement, the code is provided, and the reported gains over FS-IS are large in several configurations (e.g., OF-3B 2-shot CUB-200: 74.20 vs 61.04). The paper also makes a useful conceptual point that discarded support-set samples need not be wasted. However, the strength of the contribution is currently tempered by the absence of any variance or statistical significance information across runs, and by the lack of direct evidence for the proposed retrieval-quality mechanism. The comparison ordering DS > RS > FS-IS is consistent across datasets and models, which lends some credibility, but single-run numbers and the absence of retrieval diagnostics leave the mechanism claim under-supported.

major comments (4)
  1. [Section 4.3, Table 1] All reported accuracies are single-run values with no error bars, no multiple random coreset initializations, and no significance tests. The central claim that KeCO-DS outperforms FS-IS relies on differences that, while large for OF-3B, are small for IDE-8B (e.g., CUB-200 2-shot: 87.38 vs 85.62; ImageNet-100 2-shot: 91.68 vs 90.48). Without repeated trials and a measure of variance, it is impossible to assess whether the apparent improvements are robust or partly due to the particular random initialization of the coreset. Please report mean and standard deviation over at least 3-5 seeds and, where feasible, a paired significance test.
  2. [Section 4.4, Figures 3-5] The paper claims that the improved performance is caused by updated keys retrieving more category-relevant demonstrations (Figure 4), but no retrieval-side evidence is provided. There is no measurement of precision@k, no class purity of the retrieved demonstrations, no analysis of how much the updated keys change the ranking, and no diagnostic for key collapse within a class. Since the LVLM only ever sees the original images of the coreset samples (Section 3.7), all untapped information enters through the reranking. If the five updated keys per class converge toward a class mean, the top-k selection may degenerate to a centroid-style class selector rather than the diversity-based mechanism claimed in Eq. (7). Please add quantitative retrieval diagnostics, such as the rank correlation between updated and original keys, the fraction of retrieved demonstrations matching the test label, and a dispersion measure of keys within each class before and after updating.
  3. [Abstract and Section 4.4] The abstract claims an 'average improvement of more than 20%.' This figure is not precisely defined: it appears to be a relative improvement over FS-IC for OF-3B on particular configurations, but the exact averaging set and whether it is relative or absolute is not stated. As written, the claim is ambiguous and could mislead readers. Please specify the exact baseline, the configuration subset, and whether the number is a relative or absolute improvement, or replace it with a table summary.
  4. [Section 4.3 and Appendix A.3] The hyperparameters alpha, epoch count e, and batch size b are tuned on CUB-200 (Tables 4-5) and the best values are then used for all datasets and models, but there is no validation split and no discussion of how selection bias might inflate the reported numbers. Because the same test sets are used for both hyperparameter selection and final evaluation, the magnitude of the reported gains may be optimistic. Please either perform hyperparameter selection on a held-out split or report sensitivity curves for all datasets, and state clearly whether the reported numbers used the best hyperparameters on each dataset or a single fixed setting.
minor comments (6)
  1. [Conclusion] Typo: 'everages' should be 'leverages'.
  2. [Section 4.4] Typo: 'optimzie' should be 'optimize'; also 'perforamce' in Table 4 header should be 'performance'.
  3. [Figure 3] The t-SNE visualization is qualitative. Adding a quantitative separability measure, such as silhouette score or mean intra-class to inter-class distance ratio, would strengthen the claim that keys become more clustered.
  4. [Table 2] The column header 'Online Δ' is unclear; specify whether the numbers are absolute accuracies of KeCO-DS in the online setting and what the delta refers to.
  5. [References] References [16] and [17] appear to be the same paper ('Obelics') with different years; please merge or disambiguate.
  6. [Section 3.3] The class-balance constraint '|C_c| = m/j' assumes m is divisible by j; when this is not the case, the text should specify how the remainder is assigned.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: KeCO's reported gains are measured on held-out test splits; hyperparameter tuning and post-hoc mechanism explanations are validation concerns, not definitional reductions.

full rationale

KeCO's derivation chain is empirical rather than analytical. The update rule (Eq. 10/11) is an exponential moving average of visual keys, and inference retrieves original coreset images by similarity to updated keys (Sec. 3.7); no test label or test image enters the update. The headline comparisons (Table 1) are accuracies of frozen LVLMs on held-out test images, not outputs of a fitted formula, so there is no fitted input renamed as a prediction. The diversity-based selection (Eq. 7) is defined independently of the measured accuracy, and its explanation in Sec. 4.4 is post-hoc, not a self-definitional premise. The paper's self-citations ([23], [43]) are related-work references and are not load-bearing; no uniqueness theorem or ansatz is imported from the authors' prior work. The choice of alpha=0.2, epoch=10, and batch=1000 via ablations on CUB-200 (Tables 4-5) before reporting CUB-200 numbers is a test-set selection concern, but selection-on-test is not circularity under the definition used here. Hence no circular step can be exhibited, and the score is 0.

Assumptions & free parameters 4 free parameters · 3 assumptions · 1 invented entities

The framework's free parameters are the update rate alpha (0.2, tuned), epochs (10, tuned), batch size (1000, tuned), and the coreset-to-support size ratios chosen per dataset. Structural assumptions: the frozen vision encoder's feature space supports averaging that preserves class-typical and suppresses instance-specific signal (Section 4.4), and the encoder's space is aligned enough with the LVLM to order good demonstrations (Section 4.1). No invented physical entities; the 'key' is an internal algorithmic device with no independent external evidence.

free parameters (4)
  • update rate alpha = 0.2
    Tuned on CUB-200 2-shot KeCO-DS accuracy, where Table 4 peaks at 0.2, and used for all headline numbers. The DS advantage persists from 0.1 to 0.5, so this is fitted but not load-bearing.
  • update epochs e = 10
    Chosen from {1, 5, 10, 15} in Table 5. At e = 10, batch-size sensitivity drops, but e = 15 is comparable, so the value is tuned within a plateau.
  • batch size b = 1000
    Table 5 shows weak dependence on b at e = 10; b = 1000 is the reported default and is used in the main tables.
  • coreset-to-support size ratio = 1:5 (1000:5000, 1200:6000)
    Fixed by design rather than fitted, but the headline claim that KeCO beats a five-times-larger support set depends on this ratio, and Table 6 shows the gain grows with more untapped data.
assumptions (3)
  • domain assumption Linear interpolation of visual features of two dissimilar images from the same class preserves category-relevant features and cancels category-irrelevant ones such as background, pose, and lighting.
    This is the central mechanism claim in Section 4.4 and Figure 4: the KeCO-DS advantage over SS is explained by this additive-geometry property of CLIP/SigLIP features. It is asserted with qualitative examples, not measured.
  • domain assumption The frozen vision encoder used for keys is aligned with the LVLM's internal representation well enough that keys updated in encoder space order demonstrations that help the LVLM.
    Section 4.1 states CLIP and SigLIP are chosen 'to align the representation space of LVLMs'. This alignment is assumed, and Section 5.1 itself notes that the vision encoder and LLM are poorly aligned in some respects.
  • domain assumption Random class-balanced initialization is a sufficient starting point because the key update erases differences in initialization quality.
    Section 5.1 and Table 3 show k-center and InfoScore initializations converge to similar performance after KeCO updates, but this is demonstrated on one dataset (Stanford Dogs) with one protocol, so the generality of the claim is assumed.
invented entities (1)
  • Coreset key, a stored visual feature vector attached to each coreset image
    purpose: Acts as the retrieval anchor and update target; the original image is displayed in demonstrations while its key is averaged with untapped samples.
    The key is an internal algorithmic object with no external falsifiable handle beyond the paper's own retrieval and accuracy metric.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Enhancing Multimodal In-Context Learning for Image Classification through Coreset Optimization." pith.science (2026). https://pith.science/paper/G4YINJ2G

@misc{pith2026250414200,
  author       = {Pith},
  title        = {Pith review of: Enhancing Multimodal In-Context Learning for Image Classification through Coreset Optimization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/G4YINJ2G}},
  note         = {Machine review of arXiv:2504.14200}
}
read the original abstract

In-context learning (ICL) enables Large Vision-Language Models (LVLMs) to adapt to new tasks without parameter updates, using a few demonstrations from a large support set. However, selecting informative demonstrations leads to high computational and memory costs. While some methods explore selecting a small and representative coreset in the text classification, evaluating all support set samples remains costly, and discarded samples lead to unnecessary information loss. These methods may also be less effective for image classification due to differences in feature spaces. Given these limitations, we propose Key-based Coreset Optimization (KeCO), a novel framework that leverages untapped data to construct a compact and informative coreset. We introduce visual features as keys within the coreset, which serve as the anchor for identifying samples to be updated through different selection strategies. By leveraging untapped samples from the support set, we update the keys of selected coreset samples, enabling the randomly initialized coreset to evolve into a more informative coreset under low computational cost. Through extensive experiments on coarse-grained and fine-grained image classification benchmarks, we demonstrate that KeCO effectively enhances ICL performance for image classification task, achieving an average improvement of more than 20\%. Notably, we evaluate KeCO under a simulated online scenario, and the strong performance in this scenario highlights the practical value of our framework for resource-constrained real-world scenarios.

Figures

Figures reproduced from arXiv: 2504.14200 by the authors.

Figure 1
Figure 1. (a) Strategy-Based Selection In-Context Learning: requires storing the full support set and computing similarities [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of the KeCO framework: First, a subset is randomly selected from the support set [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 4
Figure 4. When blending keys from dissimilar samples in (d) [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figures from the paper (1 more)
Figure 5
Figure 5. Figure 5: Case study on comparing FS-IC with three KeCO methods (RS, SS and DS) in selecting demonstrations and predicting [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. X-Factor: Quality Is a Dataset-Intrinsic Property

    cs.LG 2025-05 conditional novelty 5.0 of 10

    Across 2,500 class-balanced MNIST subsets and 10 model architectures, test-error Z-scores correlate strongly across models (mean R2=0.82 excluding GNB), supporting dataset quality as an intrinsic property.

Reference graph

Works this paper leans on

54 extracted references · 17 canonical work pages · cited by 1 Pith paper

  1. [1]

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al

  2. [2]

    Anas Awadalla, Irena Gao, Josh Gardner, Jack Hessel, Yusuf Hanafy, Wanrong Zhu, Kalyani Marathe, Yonatan Bitton, Samir Gadre, Shiori Sagawa, et al. 2023. Openflamingo: An open-source framework for training large autoregressive vision-language models. arXiv preprint arXiv:2308.01390 (2023)

    arXiv 2023

  3. [3]

    Folco Bertini Baldassini, Mustafa Shukor, Matthieu Cord, Laure Soulier, and Ben- jamin Piwowarski. 2024. What makes multimodal in-context learning work?. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion. 1539–1550

    work page 2024

  4. [4]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems 33 (2020), 1877–1901

    2020

  5. [5]

    Ting-Yun Chang and Robin Jia. 2022. Data curation alone can stabilize in-context learning. arXiv preprint arXiv:2212.10378 (2022)

    work page Pith review arXiv 2022

  6. [6]

    Anurag Das, Xinting Hu, Li Jiang, and Bernt Schiele. 2024. MTA-CLIP: Language- Guided Semantic Segmentation with Mask-Text Alignment. In Proceedings of the European Conference on Computer Vision (ECCV)

    work page 2024

  7. [7]

    Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Jingyuan Ma, Rui Li, Heming Xia, Jingjing Xu, Zhiyong Wu, Tianyu Liu, et al. 2022. A survey on in-context learning. arXiv preprint arXiv:2301.00234 (2022)

    arXiv 2022

  8. [8]

    Fu Feng, Yucheng Xie, Xu Yang, Jing Wang, and Xin Geng. 2025. Redefining< creative> in dictionary: Towards an enhanced semantic understanding of cre- ative generation. In Proceedings of the Computer Vision and Pattern Recognition Conference. 18444–18454

    work page 2025

Show all 54 references
  1. [9]

    Gregor Geigle, Radu Timofte, and Goran Glavaš. 2024. African or european swallow? benchmarking large vision-language models for fine-grained object classification. arXiv preprint arXiv:2406.14496 (2024)

    2024 arXiv

  2. [10]

    Hulingxiao He, Geng Li, Zijun Geng, Jinglin Xu, and Yuxin Peng. 2025. Analyzing and Boosting the Power of Fine-Grained Visual Recognition for Multi-modal Large Language Models. arXiv preprint arXiv:2501.15140 (2025)

    2025 arXiv

  3. [11]

    Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. 2020. Mo- mentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . 9729–9738

    2020

  4. [12]

    Xinting Hu, Li Jiang, and Bernt Schiele. 2024. Training Vision Transformers for Semi-Supervised Semantic Segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    2024

  5. [13]

    Jiaxing Huang, Jingyi Zhang, Kai Jiang, Han Qiu, and Shijian Lu. 2023. Visual instruction tuning towards general-purpose multimodal model: A survey. arXiv preprint arXiv:2312.16602 (2023)

    2023 arXiv

  6. [14]

    Yuchu Jiang, Jiale Fu, Chenduo Hao, Xinting Hu, Yingzhe Peng, Xin Geng, and Xu Yang. 2025. Mimic In-Context Learning for Multimodal Tasks. In Proceedings of the Computer Vision and Pattern Recognition Conference (CVPR) . 29825–29835

    2025

  7. [15]

    Aditya Khosla, Nityananda Jayadevaprakash, Bangpeng Yao, and Fei-Fei Li. 2011. Novel dataset for fine-grained image categorization: Stanford dogs. InProc. CVPR workshop on fine-grained visual categorization (FGVC) , Vol. 2

    2011

  8. [16]

    Hugo Laurençon, Lucile Saulnier, Léo Tronchon, Stas Bekman, Amanpreet Singh, Anton Lozhkov, Thomas Wang, Siddharth Karamcheti, Alexander Rush, Douwe Kiela, et al . 2023. Obelics: An open web-scale filtered dataset of interleaved image-text documents. Advances in Neural Informa...

    2023

  9. [17]

    Hugo Laurençon, Lucile Saulnier, Léo Tronchon, Stas Bekman, Amanpreet Singh, Anton Lozhkov, Thomas Wang, Siddharth Karamcheti, Alexander Rush, Douwe Kiela, et al . 2024. Obelics: An open web-scale filtered dataset of interleaved image-text documents. Advances in Neural Informa...

    2024

  10. [18]

    Itay Levy, Ben Bogin, and Jonathan Berant. 2023. Diverse Demonstrations Im- prove In-context Compositional Generalization. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) . 1401–1422

    2023

  11. [19]

    Bo Li, Peiyuan Zhang, Jingkang Yang, Yuanhan Zhang, Fanyi Pu, and Ziwei Liu. 2023. Otterhd: A high-resolution multi-modality model. arXiv preprint arXiv:2311.04219 (2023)

    2023 arXiv

  12. [20]

    Bo Li, Yuanhan Zhang, Liangyu Chen, Jinghao Wang, Fanyi Pu, Jingkang Yang, Chunyuan Li, and Ziwei Liu. 2023. Mimic-it: Multi-modal in-context instruction tuning. arXiv preprint arXiv:2306.05425 (2023)

    2023 arXiv

  13. [21]

    Bo Li, Yuanhan Zhang, Liangyu Chen, Jinghao Wang, Jingkang Yang, and Ziwei Liu. 2023. Otter: A Multi-Modal Model with In-Context Instruction Tuning.arXiv preprint arXiv:2305.03726 (2023)

    2023 arXiv

  14. [22]

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference on machine learning . PMLR, 19730–19742

    2023

  15. [23]

    Li Li, Jiawei Peng, Huiyi Chen, Chongyang Gao, and Xu Yang. 2024. How to configure good in-context sequence for visual question answering. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 26710– 26720

    2024

  16. [24]

    Lei Li, Yuwei Yin, Shicheng Li, Liang Chen, Peiyi Wang, Shuhuai Ren, Mukai Li, Yazheng Yang, Jingjing Xu, Xu Sun, et al . 2023. M 3IT: A Large-Scale Dataset towards Multi-Modal Multilingual Instruction Tuning. arXiv preprint arXiv:2306.04387 (2023)

    2023 arXiv

  17. [25]

    Xiaonan Li, Kai Lv, Hang Yan, Tianyang Lin, Wei Zhu, Yuan Ni, Guotong Xie, Xiaoling Wang, and Xipeng Qiu. 2023. Unified demonstration retriever for in- context learning. arXiv preprint arXiv:2305.04320 (2023)

    2023 arXiv

  18. [26]

    Xiaonan Li and Xipeng Qiu. 2023. Finding support examples for in-context learning. arXiv preprint arXiv:2302.13539 (2023)

    2023 arXiv

  19. [27]

    Zijing Liang, Yanjie Xu, Yifan Hong, Penghui Shang, Qi Wang, Qiang Fu, and Ke Liu. 2024. A Survey of Multimodel Large Language Models. In Proceedings of the 3rd International Conference on Computer, Artificial Intelligence and Control Engineering. 405–409

    2024

  20. [28]

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. Visual in- struction tuning. Advances in neural information processing systems 36 (2023), 34892–34916

    2023

  21. [29]

    Hanchao Liu, Wenyuan Xue, Yifei Chen, Dapeng Chen, Xiutian Zhao, Ke Wang, Liping Hou, Rongjun Li, and Wei Peng. 2024. A survey on hallucination in large vision-language models. arXiv preprint arXiv:2402.00253 (2024)

    2024 arXiv

  22. [30]

    Jiachang Liu, Dinghan Shen, Yizhe Zhang, William B Dolan, Lawrence Carin, and Weizhu Chen. 2022. What Makes Good In-Context Examples for GPT-3?. In Proceedings of Deep Learning Inside Out (DeeLIO 2022): The 3rd Workshop on Knowledge Extraction and Integration for Deep Learning...

    2022

  23. [31]

    Costas Mavromatis, Balasubramaniam Srinivasan, Zhengyuan Shen, Jiani Zhang, Huzefa Rangwala, Christos Faloutsos, and George Karypis. 2023. Which examples to annotate for in-context learning? towards effective and efficient selection. arXiv preprint arXiv:2310.20046 (2023)

    2023 arXiv

  24. [32]

    Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2022. Rethinking the role of demonstrations: What makes in-context learning work? arXiv preprint arXiv:2202.12837 (2022)

    2022 arXiv

  25. [33]

    Jane Pan. 2023. What in-context learning “learns” in-context: Disentangling task recognition and task learning . Master’s thesis. Princeton University

    2023

  26. [34]

    Yingzhe Peng, Gongrui Zhang, Miaosen Zhang, Zhiyuan You, Jie Liu, Qipeng Zhu, Kai Yang, Xingzhong Xu, Xin Geng, and Xu Yang. 2025. Lmm-r1: Empowering 3b lmms with strong reasoning abilities through two-stage rule-based rl. arXiv preprint arXiv:2503.07536 (2025)

    2025 arXiv

  27. [35]

    Jian Qian, Miao Sun, Sifan Zhou, Ziyu Zhao, Ruizhi Hun, and Patrick Chiang. 2024. Sub-SA: Strengthen In-context Learning via Submodular Selective Annotation. arXiv preprint arXiv:2407.05693 (2024)

    2024 arXiv

  28. [36]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learni...

    2021

  29. [37]

    Pengzhen Ren, Yun Xiao, Xiaojun Chang, Po-Yao Huang, Zhihui Li, Brij B Gupta, Xiaojiang Chen, and Xin Wang. 2021. A survey of deep active learning. ACM computing surveys (CSUR) 54, 9 (2021), 1–40

    2021

  30. [38]

    Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al

  31. [39]

    Ozan Sener and Silvio Savarese. 2017. Active learning for convolutional neural networks: A core-set approach. arXiv preprint arXiv:1708.00489 (2017)

    2017 arXiv

  32. [40]

    Maria Tsimpoukelli, Jacob L Menick, Serkan Cabi, SM Eslami, Oriol Vinyals, and Felix Hill. 2021. Multimodal few-shot learning with frozen language models. Advances in Neural Information Processing Systems 34 (2021), 200–212

    2021

  33. [41]

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. 2024. Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution.arXiv preprint arXiv:2409.12191 (2024)

    2024 arXiv

  34. [42]

    Peter Welinder, Steve Branson, Takeshi Mita, Catherine Wah, Florian Schroff, Serge Belongie, and Pietro Perona. 2010. Caltech-UCSD birds 200. (2010)

    2010

  35. [43]

    Xu Yang, Yingzhe Peng, Haoxuan Ma, Shuo Xu, Chi Zhang, Yucheng Han, and Hanwang Zhang. 2023. Lever LM: Configuring In-Context Sequence to Lever Large Vision Language Models. arXiv e-prints (2023), arXiv–2312

    2023

  36. [44]

    Xu Yang, Yongliang Wu, Mingzhuo Yang, Haokun Chen, and Xin Geng. 2023. Exploring diverse in-context configurations for image captioning. Advances in MM ’25, October 27–31, 2025, Dublin, Ireland Huiyi Chen, Jiawei Peng, Kaihua Tang, Xin Geng, and Xu Yang Algorithm 1 Modified K-...

    2023

  37. [45]

    Qinghao Ye, Haiyang Xu, Guohai Xu, Jiabo Ye, Ming Yan, Yiyang Zhou, Junyang Wang, Anwen Hu, Pengcheng Shi, Yaya Shi, et al. 2023. mplug-owl: Modular- ization empowers large language models with multimodality. arXiv preprint arXiv:2304.14178 (2023)

    2023 arXiv

  38. [46]

    Chenyu You, Nuo Chen, Fenglin Liu, Shen Ge, Xian Wu, and Yuexian Zou. 2022. End-to-end spoken conversational question answering: Task, dataset and model. arXiv preprint arXiv:2204.14272 (2022)

    2022 arXiv

  39. [47]

    Chenyu You, Nuo Chen, and Yuexian Zou. 2021. MRD-Net: Multi-Modal Residual Knowledge Distillation for Spoken Question Answering.. In IJCAI. 3985–3991

    2021

  40. [48]

    Chenyu You, Nuo Chen, and Yuexian Zou. 2021. Self-supervised Contrastive Cross-Modality Representation Learning for Spoken Question Answering. In Findings of the Association for Computational Linguistics: EMNLP

    2021

  41. [49]

    Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. 2023. Sig- moid loss for language image pre-training. In Proceedings of the IEEE/CVF inter- national conference on computer vision . 11975–11986

    2023

  42. [50]

    Yuhui Zhang, Alyssa Unell, Xiaohan Wang, Dhruba Ghosh, Yuchang Su, Ludwig Schmidt, and Serena Yeung-Levy. 2024. Why are visually-grounded language models bad at image classification? arXiv preprint arXiv:2405.18415 (2024)

    2024 arXiv

  43. [51]

    Yuanhan Zhang, Kaiyang Zhou, and Ziwei Liu. 2023. What makes good examples for visual in-context learning? Advances in Neural Information Processing Systems 36 (2023), 17773–17794

    2023

  44. [52]

    Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. 2023. Minigpt-4: Enhancing vision-language understanding with advanced large lan- guage models. arXiv preprint arXiv:2304.10592 (2023). A IMPLEMENTATION DETAILS A.1 Coreset Initialization K-center initializat...

    2023 arXiv

  45. [2015]

    International journal of computer vision 115 (2015), 211–252

    Imagenet large scale visual recognition challenge. International journal of computer vision 115 (2015), 211–252

    2015

  46. [2022]

    Advances in neural information processing systems 35 (2022), 23716–23736

    Flamingo: a visual language model for few-shot learning. Advances in neural information processing systems 35 (2022), 23716–23736

    2022

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.