Pith. sign in

REVIEW 3 major objections 4 minor 40 references

Embracing Collaboration Over Competition: Condensing Multiple Prompts for Visual In-Context Learning

T0 review · 3 major / 4 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Condensing multiple candidate prompts into one fine-grained prompt, rather than selecting a single 'ideal' prompt, improves visual in-context learning on segmentation, detection, and colorization while remaining faster than output…

desk verdict A solid, reproducible VICL paper that reframes prompt selection as prompt condensation and backs it with clean ablations, but whose patch-wise attention design quietly assumes spatial alignment between query and prompts. read the letter →

arxiv 2504.21263 v1 pith:VRTJ3XIH submitted 2025-04-30 cs.CV cs.LGcs.MM

Jinpeng Wang , Tianci Luo , Yaohua Zha , Yan Feng , Ruisheng Luo , Bin Chen , Tao Dai , Long Chen
show 2 more authors
Yaowei Wang Shu-Tao Xia
This is my paper · ORCID
classification cs.CVcs.LGcs.MM
keywords Visualin-contextlearningPromptcondensationselectionPatch-wisecross-attentionMAE-VQGANSegmentationMulti-promptintegration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes that visual in-context learning should stop treating candidate prompts as competitors and instead condense several good-enough prompts into one. Its Condenser plugin compresses the fine-grained context of multiple prompts at the input level, then feeds the single condensed prompt to a frozen MAE-VQGAN inpainting backbone that predicts the query label. On Pascal-5i foreground segmentation, 16 condensed prompts reach 46.63 mIoU against 43.14 for the strongest baseline, while costing 66.61 ms per query instead of 989.62 ms for an ensemble method with 16 prompts. If this holds, the paper's collaborative-condensation view replaces the field's reliance on retrieving one ideal prompt, and makes multi-prompt VICL practical.

What carries the argument

Condenser is a lightweight external module whose core operation is patch-wise cross-attention: for a query patch at position $(h,w)$, attention is restricted to prompt patches at that same $(h,w)$, with self-attention first applied to prompt images and labels so spatial correspondence is preserved. That same-position restriction is what lets the module fuse $K$ prompts into one resolution-preserving prompt while avoiding the drift full cross-attention introduces, and the authors show full attention actually hurts. The module is trained end-to-end with token prediction loss (cross-entropy over VQGAN codebook tokens for the masked query label) plus a cosine pre-alignment loss pulling the condensed image and label features toward the query's.

What would settle it

Take a trained Condenser and evaluate it on Pascal-5i after applying a random spatial shift of 16-32 pixels to the prompt images and labels at test time. A human can still map the shifted prompt to the query, but Condenser cannot see cross-position matches; a large accuracy drop relative to unshifted prompts would show the same-position assumption is load-bearing.

Watch

Extended reading notes

Core claim

The central claim is that combining multiple candidate prompts at the input level—rather than selecting the best one or averaging separate outputs—makes visual in-context learning both more accurate and more efficient. Condenser treats each prompt as an image-label pair, locates informative patches with self-attention, and uses patch-wise cross-attention to let each query patch gather evidence only from the same spatial position across all candidate prompts. The condensed image and label features are placed on the standard inpainting canvas with the query, and Condenser is trained end-to-end with token prediction and pre-alignment losses while the MAE-VQGAN backbone stays frozen. The authors report consistent gains over single-prompt selection, voting ensembles, and PEFT baselines on segmentation, detection, and colorization, with accuracy improving as the number of prompts grows from 1 to 32.

Load-bearing premise

The load-bearing premise is that the useful evidence in a prompt always sits at the same spatial position as the query patch it informs; prompts whose objects or annotations are shifted or scaled would break this assumption and cap Condenser's generalization.

Editorial extensions

If this is right

  • Accuracy keeps climbing with more prompts: Condenser segmentation mIoU rises from 44.14 at K=1 to 46.73 at K=32, with inference time growing only from 59.17 ms to 74.13 ms per query.
  • Condenser beats the strongest single-prompt baseline by 8.09% relative on segmentation and 3.31% on detection, and lowers colorization MSE to 0.54 from 0.58.
  • Output ensembling (voting) is harder to design and far slower: Prompt-SelF needs 989.62 ms per query for 16 prompts, while Condenser uses 66.61 ms.
  • Condenser narrows the gap between weak and strong retrievers, so even random or unsupervised prompt retrieval becomes more usable in multi-prompt mode.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension implied by the patch-wise restriction: translate or crop-shift a prompt's object within its frame and measure the mIoU drop; if it is large, Condenser's success is tied to spatial alignment rather than semantic context.
  • The same condensation layer could be plugged into other token-based vision backbones without retraining them, since it only exchanges patch features and consumes frozen-backbone feedback through the token prediction loss.
  • Because the pre-alignment loss is reported as a regularizer rather than a full learning signal, a backbone-agnostic zero-shot variant would need a different semantic supervision source, for example distilling from a teacher model.
  • The plateau from K=16 to K=32 (46.63 to 46.73) suggests returns diminish; a future method may need to select or prune prompts before condensation.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes Condenser, a lightweight external module for visual in-context learning (VICL). Instead of selecting one prompt, Condenser fuses K retrieved image-label prompt pairs into a single condensed prompt at patch-token level. The module applies shared self-attention to prompt images and labels and then patch-wise cross-attention, with the query image as the attention query (Eqs. (1)-(9)). It is trained end-to-end with a frozen MAE-VQGAN backbone using a token-prediction loss (Eq. (13)) and a pre-alignment cosine loss (Eq. (14)). Experiments compare Condenser with prompt-selection, voting, and PEFT baselines on Pascal-5i segmentation, Pascal VOC detection, and ImageNet colorization, plus cross-dataset and efficiency studies. The paper claims consistent gains, scalability with K, and lower inference cost than output ensembling.

Significance. If the headline numbers hold, collaborative prompt condensation is a plausible new direction for VICL and a useful alternative to output ensembling. The method is clearly specified, the ablation studies cover the main design choices (PCA, losses, output fusion, PEFT), and the code is open-sourced. I do not see circularity: the query label is used as a training target, which is standard supervision. However, the strength of the empirical evidence is currently below what the paper's broad claims require: the spatial-locking assumption is untested, the key comparisons lack error bars and partly reuse other papers' numbers, and the cross-dataset multi-prompt gain is negligible. The contribution is significant conditional on those points being addressed.

major comments (3)
  1. [Sec. 3.3, Eqs. (6)-(9)] Patch-wise cross-attention restricts the output for query patch (h,w) to attend only to prompt patches at the same (h,w), for both image and label streams. This means any task-relevant cue that appears at a shifted position in a candidate prompt (e.g., an object translated, a scale change, or an annotation offset) is discarded by construction. The ablation in Table 4, Variant (3), shows that full cross-attention is worse on the standard benchmarks, which supports the local-consistency rationale in-distribution, but it does not test the regime where the spatial-locking assumption is most brittle. Since the paper's central claim is that Condenser compresses relevant fine-grained context across multiple prompts from arbitrary candidates, please add an experiment with translated or scaled prompt content (or retrieve prompts with deliberately misaligned layout) and report whether the advantage over K=1 and over full cross-attention degrades; otherwise, the claim should be explicitly restricted to spatially aligned prompts.
  2. [Table 1, Sec. 4.1.3] The main empirical claims rest on a comparison in which no error bars or seeds are reported, and several baseline numbers appear to be copied from previous papers without shared reimplementation. The detection result illustrates the problem: Condenser with K=1 reaches 43.22 mIoU versus InMeMo's 43.21, i.e. a difference of 0.01, and the colorization gain is 0.02 MSE; such differences are within run-to-run noise. Please report mean and standard deviation over at least three random seeds for Condenser, state explicitly which baseline numbers were re-run under the same protocol and which were taken from the literature, and reconsider the wording 'consistently outperforms' for the cases where the margin is at noise level.
  3. [Sec. 4.2.3, Table 2] The cross-dataset experiment is presented as showing that condensing multiple prompts gives 'remarkable advantages' over single-prompting, but the numbers do not support that wording: K=16 gives 40.52 mIoU versus 40.37 for K=1, and on Fold-3 K=16 is worse (35.84 vs 36.33). This is a small single-run difference, and it is exactly the setting where the claimed generalization benefit of collaboration should appear. Please provide per-seed breakdowns and either temper the claim or show that the multi-prompt gain is consistent across folds and seeds.
minor comments (4)
  1. [Sec. 3.1] The word 'goad' should be 'goal' in the sentence 'To achieve this goad.'
  2. [Figure 1 and Table 1] The term 'impainting' in the Figure 1 caption should be 'inpainting', and the label 'V oting' in Table 1 should be 'Voting'.
  3. [References] References [4] and [5] are the same paper, and references [17] and [32] are also duplicates; these should be merged to avoid citation inflation.
  4. [Table 3, Sec. 4.2.4] The efficiency conclusion is driven by inference time, but the GPU cost for Condenser K=16 (1021.86 MB) is more than twice that of Prompt-SelF (446.71 MB) and InMeMo (497.13 MB); the text should qualify the efficiency claim as time-efficient but memory-heavier.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: Condenser is trained end-to-end on held-out benchmarks and its pre-alignment loss is normal supervised learning, not a prediction forced from fitted inputs.

full rationale

The paper's central claim is an empirical improvement from a learned plugin (Condenser) trained end-to-end with a frozen MAE-VQGAN backbone and evaluated on standard held-out splits (Pascal-5i, Pascal VOC 2012, ImageNet colorization). The two losses, LTP (Eq. 13) and LPA (Eq. 14), use ground-truth query labels as supervision targets during training only; the labeled query is explicitly removed at inference ('training-only and removed in inference' in Fig. 2). This is standard supervised learning, not a self-definitional prediction. No fitted parameter is renamed as a prediction: the reported mIoU/MSE numbers come from the frozen backbone on test queries after Condenser has been optimized. The patch-wise cross-attention design (Eqs. 6-9) is a stated modeling assumption with in-distribution ablation support (variant (3) with full cross-attention is worse); it may limit generalization to spatially shifted prompt cues, but that is a correctness/generalization risk, not circular reasoning. The paper does not rely on self-citations or imported uniqueness theorems to force its architecture or conclusions. The derivation chain is therefore self-contained with respect to circularity.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

No new physical or metaphysical entities are introduced. Condenser is a learned parameterized module and the condensed prompt is a representation; both have empirical justification from the ablation and benchmark results, so they do not fit the pattern of an entity pulled from a hat.

free parameters (2)
  • Pre-alignment loss weight lambda = 0.4
    Chosen once for all experiments; no sensitivity analysis or validation-based selection is reported in Sec. 4.1.3.
  • Learning rate = 0.03
    Set in Sec. 4.1.3 with a cosine schedule; no explicit link to the robustness of the central claim.
assumptions (3)
  • domain assumption MAE-VQGAN, frozen, is a sufficient backbone for visual in-context learning across segmentation, detection, and colorization.
    The method is trained on top of a fixed pre-trained MAE-VQGAN (Sec. 3.2 and Sec. 4.1.3); if this backbone cannot perform VICL well, Condenser cannot recover.
  • domain assumption The VQGAN encoder and decoder tokenization preserves the task-relevant information needed for the three tasks.
    Token prediction and pre-alignment losses operate in VQGAN token space (Eqs. 10-16).
  • domain assumption The standard VICL evaluation protocol, which retrieves prompts by image similarity and evaluates on fixed folds, is an appropriate measure of progress.
    Experiments follow previous works (Sec. 4.1.1); results inherit the protocol's limitations.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Embracing Collaboration Over Competition: Condensing Multiple Prompts for Visual In-Context Learning." pith.science (2026). https://pith.science/paper/VRTJ3XIH

@misc{pith2026250421263,
  author       = {Pith},
  title        = {Pith review of: Embracing Collaboration Over Competition: Condensing Multiple Prompts for Visual In-Context Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VRTJ3XIH}},
  note         = {Machine review of arXiv:2504.21263}
}
read the original abstract

Visual In-Context Learning (VICL) enables adaptively solving vision tasks by leveraging pixel demonstrations, mimicking human-like task completion through analogy. Prompt selection is critical in VICL, but current methods assume the existence of a single "ideal" prompt in a pool of candidates, which in practice may not hold true. Multiple suitable prompts may exist, but individually they often fall short, leading to difficulties in selection and the exclusion of useful context. To address this, we propose a new perspective: prompt condensation. Rather than relying on a single prompt, candidate prompts collaborate to efficiently integrate informative contexts without sacrificing resolution. We devise Condenser, a lightweight external plugin that compresses relevant fine-grained context across multiple prompts. Optimized end-to-end with the backbone, Condenser ensures accurate integration of contextual cues. Experiments demonstrate Condenser outperforms state-of-the-arts across benchmark tasks, showing superior context compression, scalability with more prompts, and enhanced computational efficiency compared to ensemble methods, positioning it as a highly competitive solution for VICL. Code is open-sourced at https://github.com/gimpong/CVPR25-Condenser.

Figures

Figures reproduced from arXiv: 2504.21263 by the authors.

Figure 1
Figure 1. (a) The impainting framework [2] is a typical prototype for Visual In-Context Learning (VICL), where a fixed-size input canvas is divided into regions. The prompt image and its label occupy the top half, while the query image is placed in the bottom left. The vision backbone is tasked with recovering the bottom right region, which represents the query label, by reasoning within the pixel context. Prompt selection pl… view at source ↗
Figure 2
Figure 2. Prompt condensation with CONDENSER for VICL. Given a query image Iq from the training set, we first retrieve a set of candidate prompts Sq from the prompt database P. Then, we assemble each prompt from Sq with the labeled query pair Pq = (Iq, Lq) and construct K contextualized labeled samples, namely X˜c1 , · · · , X˜cK . We encode each of them with a VQGAN encoder [7] h(·) and obtain a serial of query label tokens,… view at source ↗
Figure 4
Figure 4. CONDENSER design. The cross-prompt attention per￾forms patch-wise cross-attention to aggregate informative context among the K prompts. LN: layer normalization. P: permutation. to preserve their spatial correspondence. The process can be formulated as follows: F I(1) q = LN4(SA1(LN1(F I q )) + F I q ), (1) F I(1) c1:K = LN5(SA2(LN2(F I c1:K )) + F I c1:K ), (2) F L(1) c1:K = LN6(SA2(LN3(F L c1:K )) + F L c1:K ). (3)… view at source ↗
Figures from the paper (2 more)
Figure 3
Figure 3. Figure 3: MAE-VQGAN [2] neatly combines MAE [10] and VQ￾GAN [7] for inpainting. MAE subsequently acts as the backbone in VICL. (a) For pre-training, the MAE is trained to predict the masked tokens. (b) For inference, the VQGAN decoder decodes pixel results from MAE’s predicted t…
Figure 5
Figure 5. Figure 5: Impact of the prompt number K on foreground segmen￾tation, single object detection. with voting using a single query-prompt pair (see [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

40 extracted references · 16 canonical work pages

  1. [1]

    Flamingo: a visual language model for few-shot learning

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning. Advances in neural information processing systems, 35:23716–23736,

  2. [2]

    Visual prompting via image inpaint- ing

    Amir Bar, Yossi Gandelsman, Trevor Darrell, Amir Glober- son, and Alexei Efros. Visual prompting via image inpaint- ing. Advances in Neural Information Processing Systems, 35: 25005–25017, 2022. 1, 2, 3, 4, 5, 6

    work page 2022

  3. [3]

    Language models are few-shot learners

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Sub- biah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901, 2020. 1, 2

    work page 1901

  4. [4]

    Why can gpt learn in-context? language models secretly perform gradient descent as meta- optimizers

    Damai Dai, Yutao Sun, Li Dong, Yaru Hao, Shuming Ma, Zhifang Sui, and Furu Wei. Why can gpt learn in-context? language models secretly perform gradient descent as meta- optimizers. In Findings of the Association for Computational Linguistics: ACL 2023, pages 4005–4019, 2023. 1

    work page 2023

  5. [5]

    Why can GPT learn in-context? language models secretly perform gradient descent as meta- optimizers

    Damai Dai, Yutao Sun, Li Dong, Yaru Hao, Shuming Ma, Zhifang Sui, and Furu Wei. Why can GPT learn in-context? language models secretly perform gradient descent as meta- optimizers. In Findings of the Association for Computational Linguistics: ACL 2023, pages 4005–4019, Toronto, Canada,

    work page 2023

  6. [6]

    A survey on in-context learning

    Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Jingyuan Ma, Rui Li, Heming Xia, Jingjing Xu, Zhiyong Wu, Baobao Chang, et al. A survey on in-context learning. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 1107–1128, 2024. 1

    work page 2024

  7. [7]

    Taming transformers for high-resolution image synthesis

    Patrick Esser, Robin Rombach, and Bjorn Ommer. Taming transformers for high-resolution image synthesis. In Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12873–12883, 2021. 3, 4

    work page 2021

  8. [8]

    The pascal visual object classes challenge: A retrospective

    Mark Everingham, SM Ali Eslami, Luc Van Gool, Christo- pher KI Williams, John Winn, and Andrew Zisserman. The pascal visual object classes challenge: A retrospective. In- ternational journal of computer vision , 111:98–136, 2015. 5

    2015

Show all 40 references
  1. [9]

    Explore in-context learning for 3d point cloud understanding.Advances in Neural Information Processing Systems, 36, 2024

    Zhongbin Fang, Xiangtai Li, Xia Li, Joachim M Buhmann, Chen Change Loy, and Mengyuan Liu. Explore in-context learning for 3d point cloud understanding.Advances in Neural Information Processing Systems, 36, 2024. 1, 3

    2024

  2. [10]

    Masked autoencoders are scalable vision learners

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Doll´ar, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 16000– 16009, 2022. 4

    2022

  3. [11]

    In- context learning creates task vectors

    Roee Hendel, Mor Geva, and Amir Globerson. In- context learning creates task vectors. arXiv preprint arXiv:2310.15916, 2023. 1

    2023 arXiv

  4. [12]

    Lora: Low-rank adaptation of large language models.arXiv preprint arXiv:2106.09685, 2021

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models.arXiv preprint arXiv:2106.09685, 2021. 7, 8

    2021 arXiv

  5. [13]

    Mimic-it: Multi-modal in-context instruction tuning

    Bo Li, Yuanhan Zhang, Liangyu Chen, Jinghao Wang, Fanyi Pu, Jingkang Yang, Chunyuan Li, and Ziwei Liu. Mimic-it: Multi-modal in-context instruction tuning. arXiv preprint arXiv:2306.05425, 2023. 1

    2023 arXiv

  6. [14]

    Visual in-context prompting

    Feng Li, Qing Jiang, Hao Zhang, Tianhe Ren, Shilong Liu, Xueyan Zou, Huaizhe Xu, Hongyang Li, Jianwei Yang, Chun- yuan Li, et al. Visual in-context prompting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12861–12871, 2024. 1

    2024

  7. [15]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceeding...

    2014

  8. [16]

    P-tuning v2: Prompt tuning can be comparable to fine-tuning universally across scales and tasks

    Xiao Liu, Kaixuan Ji, Yicheng Fu, Weng Lam Tam, Zhengx- iao Du, Zhilin Yang, and Jie Tang. P-tuning v2: Prompt tuning can be comparable to fine-tuning universally across scales and tasks. arXiv preprint arXiv:2110.07602, 2021. 7, 8

    arXiv

  9. [17]

    Context diffusion: In-context aware image generation

    Ivona Najdenkoska, Animesh Sinha, Abhimanyu Dubey, Dhruv Mahajan, Vignesh Ramanathan, and Filip Radenovic. Context diffusion: In-context aware image generation. In European Conference on Computer Vision, pages 375–391. Springer, 2024. 2

    2024

  10. [18]

    Kosmos-2: Ground- ing multimodal large language models to the world

    Zhiliang Peng, Wenhui Wang, Li Dong, Yaru Hao, Shaohan Huang, Shuming Ma, and Furu Wei. Kosmos-2: Ground- ing multimodal large language models to the world. arXiv preprint arXiv:2306.14824, 2023. 2

    2023 arXiv

  11. [19]

    Measuring and narrowing the compositionality gap in language models

    Ofir Press, Muru Zhang, Sewon Min, Ludwig Schmidt, Noah A Smith, and Mike Lewis. Measuring and narrowing the compositionality gap in language models. arXiv preprint arXiv:2210.03350, 2022. 2

    2022 arXiv

  12. [20]

    Imagenet large scale visual recognition challenge

    Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, San- jeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International journal of computer vision, 115:211–252, 2015. 5

    2015

  13. [21]

    One-shot learning for semantic segmentation

    Amirreza Shaban, Shray Bansal, Zhen Liu, Irfan Essa, and Byron Boots. One-shot learning for semantic segmentation. arXiv preprint arXiv:1709.03410, 2017. 5

    2017 arXiv

  14. [22]

    On the effect of pretraining corpora on in-context learning by a large-scale language model

    Seongjin Shin, Sang-Woo Lee, Hwijeen Ahn, Sungdong Kim, HyoungSeok Kim, Boseop Kim, Kyunghyun Cho, Gichang Lee, Woomyoung Park, Jung-Woo Ha, et al. On the effect of pretraining corpora on in-context learning by a large-scale language model. arXiv preprint arXiv:2204.13509, 2022. 2

    2022 arXiv

  15. [23]

    Generative multimodal models are in- context learners

    Quan Sun, Yufeng Cui, Xiaosong Zhang, Fan Zhang, Qiying Yu, Yueze Wang, Yongming Rao, Jingjing Liu, Tiejun Huang, and Xinlong Wang. Generative multimodal models are in- context learners. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, page...

    2024

  16. [24]

    Exploring effective factors for improving visual in-context learning

    Yanpeng Sun, Qiang Chen, Jian Wang, Jingdong Wang, and Zechao Li. Exploring effective factors for improving visual in-context learning. arXiv preprint arXiv:2304.04748, 2023. 1, 2, 3, 4, 6, 7, 8

    2023

  17. [25]

    Rethinking and improving visual prompt selection for in-context learning segmentation

    Wei Suo, Lanqing Lai, Mengyang Sun, Hanwang Zhang, Peng Wang, and Yanning Zhang. Rethinking and improving visual prompt selection for in-context learning segmentation. In European Conference on Computer Vision, pages 18–35. Springer, 2024. 3

    2024

  18. [26]

    Gemini: a family of highly capable multimodal models

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805, 2023. 1, 2

    2023 arXiv

  19. [27]

    Llama: Open and efficient foundation language models.arXiv preprint arXiv:2302.13971, 2023

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Mar- tinet, Marie-Anne Lachaux, Timoth ´ee Lacroix, Baptiste Rozi`ere, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models.arXiv preprint arXiv:2302.13971, 2023. 1, 2

    2023 arXiv

  20. [28]

    Transformers learn in-context by gradient descent

    Johannes V on Oswald, Eyvind Niklasson, Ettore Randazzo, Jo˜ao Sacramento, Alexander Mordvintsev, Andrey Zhmogi- nov, and Max Vladymyrov. Transformers learn in-context by gradient descent. In International Conference on Machine Learning, pages 35151–35174. PMLR, 2023. 2

    2023

  21. [29]

    Images speak in images: A generalist painter for in-context visual learning

    Xinlong Wang, Wen Wang, Yue Cao, Chunhua Shen, and Tiejun Huang. Images speak in images: A generalist painter for in-context visual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6830–6839, 2023. 1

    2023

  22. [30]

    Seggpt: Segmenting everything in context

    Xinlong Wang, Xiaosong Zhang, Yue Cao, Wen Wang, Chun- hua Shen, and Tiejun Huang. Seggpt: Segmenting everything in context. arXiv preprint arXiv:2304.03284, 2023. 1, 3

    2023 arXiv

  23. [32]

    In- context learning unlocked for diffusion models

    Zhendong Wang, Yifan Jiang, Yadong Lu, Pengcheng He, Weizhu Chen, Zhangyang Wang, Mingyuan Zhou, et al. In- context learning unlocked for diffusion models. Advances in Neural Information Processing Systems, 36:8542–8562, 2023. 2

    2023

  24. [33]

    Larger language models do in-context learning differently

    Jerry Wei, Jason Wei, Yi Tay, Dustin Tran, Albert Webson, Yifeng Lu, Xinyun Chen, Hanxiao Liu, Da Huang, Denny Zhou, et al. Larger language models do in-context learning differently. arXiv preprint arXiv:2303.03846, 2023. 1

    2023 arXiv

  25. [34]

    Towards global optimal visual in-context learning prompt selection

    Chengming Xu, Chen Liu, Yikai Wang, Yuan Yao, and Yan- wei Fu. Towards global optimal visual in-context learning prompt selection. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. 1, 2, 3, 4, 5, 6

    2024

  26. [35]

    Ground-truth labels matter: A deeper look into input- label demonstrations

    Kang Min Yoo, Junyeob Kim, Hyuhng Joon Kim, Hyunsoo Cho, Hwiyeol Jo, Sang-Woo Lee, Sang-goo Lee, and Taeuk Kim. Ground-truth labels matter: A deeper look into input- label demonstrations. arXiv preprint arXiv:2205.12685, 2022. 2

    2022 arXiv

  27. [36]

    Instruct me more! random prompt- ing for visual in-context learning

    Jiahao Zhang, Bowen Wang, Liangzhi Li, Yuta Nakashima, and Hajime Nagahara. Instruct me more! random prompt- ing for visual in-context learning. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 2597–2606, 2024. 3, 4, 5, 6, 7, 8

    2024

  28. [37]

    What makes good examples for visual in-context learning? Advances in Neural Information Processing Systems, 36, 2023

    Yuanhan Zhang, Kaiyang Zhou, and Ziwei Liu. What makes good examples for visual in-context learning? Advances in Neural Information Processing Systems, 36, 2023. 1, 3, 4, 5, 6, 8

    2023

  29. [38]

    Mmicl: Empowering vision-language model with multi-modal in-context learning

    Haozhe Zhao, Zefan Cai, Shuzheng Si, Xiaojian Ma, Kaikai An, Liang Chen, Zixuan Liu, Sheng Wang, Wenjuan Han, and Baobao Chang. Mmicl: Empowering vision-language model with multi-modal in-context learning. In The Twelfth International Conference on Learning Representations, 2024. 1

    2024

  30. [39]

    Can we edit fac- tual knowledge by in-context learning? arXiv preprint arXiv:2305.12740, 2023

    Ce Zheng, Lei Li, Qingxiu Dong, Yuxuan Fan, Zhiyong Wu, Jingjing Xu, and Baobao Chang. Can we edit fac- tual knowledge by in-context learning? arXiv preprint arXiv:2305.12740, 2023. 2

    2023 arXiv

  31. [40]

    Visual in-context learning for large vision-language models

    Yucheng Zhou, Xiang Li, Qianning Wang, and Jianbing Shen. Visual in-context learning for large vision-language models. arXiv preprint arXiv:2402.11574, 2024. 1

    2024 arXiv

  32. [2023]

    Association for Computational Linguistics. 2

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.