Pith. sign in

REVIEW 4 major objections 5 minor 58 references

The paper argues that kNN-based demonstration selection remains optimal for subjective preference tasks, but a learned, diversity-aware policy is strictly necessary for superior performance on objective visual regression tasks.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 05:40 UTC pith:HYAG7P25

load-bearing objection The RL selection method is clever but the central comparison is confounded by an extra anchor demo and inconsistent tables; the dichotomy is unsupported as-is. the 4 major comments →

arxiv 2603.26775 v1 pith:HYAG7P25 submitted 2026-03-24 cs.LG cs.AIcs.CLcs.CV

Learning to Select Visual In-Context Demonstrations

Eugene Lee , Yu-Chi Lin , Jiajie Diao This is my paper
classification cs.LG cs.AIcs.CLcs.CV
keywords in-context learningdemonstration selectionmultimodal large language modelsvisual regressionreinforcement learningDueling DQNk-nearest neighbordiversity
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper asks when a multimodal vision-language model should learn to select its in-context examples rather than just retrieving the most similar images. It argues that for objective, factual regression tasks—such as predicting age or image quality—a reinforcement-learning agent that balances visual similarity with diversity builds better demonstration sets than the standard k-nearest-neighbor search. For subjective preference tasks like aesthetic rating, similarity alone remains the best strategy, and the learned diversity actually hurts. The authors train the LSD agent using downstream prediction error as the reward and report that it outperforms kNN on three objective benchmarks but not on two subjective ones. If this dichotomy holds, it gives practitioners a concrete rule: choose the selection strategy based on whether the task is objective or subjective, and be ready to train a policy for factual tasks.

Core claim

The central discovery is a task-dependent dichotomy in visual in-context learning. By formulating demonstration selection as a sequential decision-making problem and training a reinforcement-learning agent to maximize the MLLM's downstream accuracy, the authors show that a learned, diversity-aware policy is required to beat unsupervised kNN on objective regression tasks such as age and image-quality estimation. On those tasks, the best demonstrations are diverse boundary examples that span the output range. On subjective tasks such as aesthetic rating and facial beauty, simple visual similarity remains superior, and the learned diversity introduces unnecessary variance. The paper further sho

What carries the argument

The load-bearing mechanism is a Dueling DQN agent whose state is the query plus the set of previously selected demonstrations. The state is encoded by a query-centric Transformer Decoder that feeds the query embedding as the target sequence and demonstration embeddings as memory, forcing the policy to be query-specific. Q-values are computed as an inner product between a learned advantage query vector and precomputed SigLIP embeddings, and the enormous action space—the entire dataset—is handled by FAISS approximate nearest-neighbor search. The reward is the marginal improvement in the negative mean absolute error of the MLLM's prediction, which encourages the agent to select examples that pr

Load-bearing premise

The reported superiority of LSD over kNN rests on the assumption that the queries used to compute RL rewards during training are disjoint from the 1,000-query subset used for evaluation; if the same queries generated rewards, the MAE gains are fitted values rather than predictions.

What would settle it

Re-run the experiment with an explicit holdout: reserve 1,000 queries before any reward collection, train the LSD agent on rewards from the remaining pool, and evaluate on the reserved queries. If the MAE gap over kNN disappears or becomes negligible, the central claim is falsified; if it persists, the claim is supported.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • On objective visual regression tasks, LSD's learned diversity-aware selection policy consistently outperforms kNN across shot counts, and the gap widens as the number of demonstrations increases.
  • On subjective preference tasks, kNN remains the stronger selector; LSD's diversity actually hurts by introducing unnecessary variance.
  • A single LSD policy trained with one MLLM's reward signals transfers to other MLLMs, matching or beating kNN, indicating the learned heuristic is not tightly coupled to one model's architecture.
  • The agent implicitly learns label-aware selection without ever seeing labels, picking demonstrations closer in label space to the query—an emergent behavior that correlates with the task's underlying structure.
  • The learned selection order matters less than the set itself; shuffling the same demonstrations yields nearly identical performance, suggesting the agent's main skill is selecting a good set rather than ordering it.
  • The objective/subjective dichotomy likely generalizes to text-based in-context learning: factual tasks may benefit from diverse boundary examples, while stylistic tasks favor similarity. This is testable on LLM benchmarks.
  • The paper's evaluation uses MAE, which aligns with the reward; under other metrics such as correlation coefficients the same policy might behave differently, a question the paper does not address.
  • The dichotomy may depend on the MLLM's perception of task objectivity; a model that treats aesthetics as objective might benefit from the diversity policy, suggesting selection should condition on model uncertainty rather than the data's label source.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This paper proposes LSD, a reinforcement-learning approach to selecting in-context demonstrations for MLLM visual regression. Selection is cast as a finite-horizon MDP; a Dueling DQN with a query-centric Transformer decoder outputs an advantage query, and FAISS retrieves candidate demos from a large action space. The policy is trained on reward increments tied to downstream MAE. Experiments on UTKFace, AVA, SCUT-FBP5500, KonIQ-10k, and KADID-10k compare LSD against kNN and random selection and report a task-dependent dichotomy: LSD is claimed to be strictly necessary for objective regression tasks, while kNN remains best for subjective preference tasks.

Significance. If the dichotomy were established, the paper would be a useful practical contribution: it would show when learned sequential selection pays off in visual ICL and would demonstrate a scalable DQN/FAISS design for large action spaces. The authors provide a code/project page and include cross-model transfer experiments, which are strengths. However, the current evidence does not support the strong conclusion: the evaluation protocol appears to give LSD an extra demonstration, and key numbers are internally inconsistent. These issues must be resolved before the claimed dichotomy can be assessed.

major comments (4)
  1. [Sec. 3.1, Appendix C.5, Table 1] The MDP's initial state already includes an anchor nearest-neighbor demonstration, and each of the K actions adds one more. The final LSD prompt therefore contains K+1 demonstrations unless the anchor is counted within K. The paper never states how the anchor is handled in Table 1's LSD-K columns. If it is not counted, LSD-K is compared against K-shot kNN/Random, and the growing gap (e.g., UTKFace: 0.08 at K=1 to 1.00 at K=4) is consistent with an extra-shot advantage. If it is counted within K, LSD-K=1 should reduce to kNN-K=1 (both the nearest neighbor), yet Table 1 reports 5.90 vs 5.98. No consistent reading supports the headline comparison.
  2. [Table 7 vs. Table 1] The same evaluation setup reports sharply different numbers in Table 7 and Table 1 for identical conditions. For example, 0-shot MAE on KonIQ-10k is 0.78 in Table 7 but 0.42 in Table 1; SCUT-FBP5500 is 1.07 vs 0.59; KADID-10k is 1.13 vs 0.94. kNN at K=1 also differs (KonIQ 0.73 vs 0.40). No error bars or seeds are reported anywhere. These discrepancies are too large to be dismissed as sampling noise and mean the numerical superiority in Table 1 cannot be interpreted as a stable finding.
  3. [Appendix C.3, Sec. 4.4] The paper states that demonstrations are retrieved from an 80% pool and that 1,000 queries are sampled from the 20% query set for evaluation, but it never states that the RL agent's training rewards were computed on queries disjoint from that evaluation subset. If the same queries generated training rewards, the reported MAE gains are fitted values rather than predictions. This is load-bearing because the central claim is an empirical comparison. The authors must clarify the split and, if needed, retrain on a disjoint query set.
  4. [Table 6 / Figure 8] Appendix E contains a direct contradiction: the Figure 8 caption claims the Gemma-trained policy 'consistently beat[s] kNN on Qwen and InternVL,' but Table 6 shows LSD (Gemma) on InternVL2-8B with MAE 10.62 vs. kNN 7.38, i.e., substantially worse. This undermines the cross-model generalization narrative and suggests selective reporting.
minor comments (5)
  1. [Sec. 4.4] There is a typo: 'SCUT-FB5500' should be 'SCUT-FBP5500'. Also, a related-work paragraph uses 'IDCL' instead of 'ICL'.
  2. [Sec. 4.4.2, Fig. 3] The so-called 'emergent label-awareness' is not emergent: the reward is a function of label MAE, so selecting demos with close labels is a direct consequence of reward optimization rather than an unanticipated side effect.
  3. [Table 2] For UTKFace, shuffling the learned order improves MAE from 7.05 to 6.51; calling this 'nearly identical' is misleading and should be reworded.
  4. [Sec. 4.3 / C.3] No random seeds or repeated-run statistics are reported. Given the internal inconsistencies already noted, the authors should report mean and standard deviation over multiple seeds.
  5. [Sec. 3.1 / C.5] The counting of the anchor demonstration should be defined in the main text before any experiments. Currently Sec. 3.1 and C.5 conflict on whether the initial state is s0 or s1 and on whether the anchor is already a demonstration.

Circularity Check

1 steps flagged

LSD's reported 'K-shot' results likely include an extra anchor demonstration, making the central K-shot vs K-shot comparison confounded by shot count.

specific steps
  1. self definitional [Sec. 3.1 (MDP definition), Appendix C.5 (anchor initialization), Table 1]
    "The initial state s1 contains the query and one 'anchor' demonstration found via nearest-neighbor search... The episode terminates when K demonstrations have been selected (t=K). ... To address the cold-start problem, the environment employs an anchor initialization strategy: the initial state s0 always includes the query image and its nearest neighbor (retrieved via FAISS) as the first demonstration."

    The MDP defines each episode as starting with an anchor demo and then taking K actions, each adding a new demo, so the final prompt contains the anchor plus K selected demos = K+1 demonstrations. The kNN baseline selects exactly K samples. Table 1 reports 'LSD (Ours)' at K=1,4,8,16 against kNN at the same K, but the LSD numbers are, by construction, K+1-shot. At K=1, LSD would equal kNN if the anchor were counted within K, yet Table 1 reports different MAEs (5.90 vs 5.98), confirming that K in Table 1 is not the total number of demonstrations for LSD. Thus the claimed superiority on objective tasks is confounded with an extra demonstration, and the conclusion that the learned policy is 'strictly necessary' reduces to an asymmetric shot count.

full rationale

The paper contains no load-bearing self-citations and no uniqueness theorem imported from the authors' prior work; the cross-MLLM and cross-dataset transfer experiments provide some independent grounding. However, the central in-domain comparison in Table 1 is undermined by the anchor initialization: the MDP in Sec. 3.1 and the appendix's description of s0 imply that LSD final prompts contain the nearest-neighbor anchor plus K agent-selected demonstrations, i.e., K+1 demos, while the kNN and Random baselines use K demos. The inconsistency at K=1 (LSD ≠ kNN when both should be just the nearest neighbor if the anchor is counted within K) indicates that the reported LSD-K columns are not K-shot by the same definition as the baselines. This makes the paper's central dichotomy ('kNN optimal for subjective, LSD strictly necessary for objective') rest at least partly on an extra-shot effect rather than on the learned selection policy. In addition, the paper explicitly selects MAE as the evaluation metric because it is the training reward, which is a limitation but not itself circular; and the training/evaluation query split is not fully specified, leaving open the possibility of reward overfitting, though this is not demonstrated. Weighing the construction-level shot-count confound against the independent transfer evidence, the partial circularity score is 6.

Axiom & Free-Parameter Ledger

6 free parameters · 4 axioms · 0 invented entities

The paper introduces no new physical entities. Its central claims rest on hand-chosen reward scales, an ANN approximation parameter, and the assumption that SigLIP embeddings and single-MLLM rewards are sufficient signals for selecting demonstrations across models and datasets.

free parameters (6)
  • Reward scaling lambda = 10.0
    Section C.5 scales differential rewards by 1/lambda=0.1; hand-chosen, affects Q-value magnitudes and policy.
  • Task-specific reward multipliers = 10 (aesthetic), 20 (quality/beauty)
    Section C.5 Eq. 13; introduced ad hoc to amplify small errors for 0-5/0-10 scales.
  • FAISS candidate count N = 200
    Section 3.4 replaces true argmax over N actions with top-200 ANN; a free approximation parameter.
  • Anchor initialization count = 1 extra demo
    Section 3.1/C.5 prepends a kNN anchor to every LSD episode; if it counts beyond K, LSD uses more shots than baselines.
  • Dataset cap N_max = 25,000
    Section C.3.1 randomly downsamples larger datasets to 25k for FAISS memory; changes the candidate pool and thus the policy's reachable demonstrations.
  • Evaluation query sample size = 1,000
    Section C.3.2 randomly subsamples queries for evaluation; no seed provided, contributes to variance and Table 1 vs Table 7 mismatch.
axioms (4)
  • domain assumption SigLIP embeddings and top-200 FAISS retrieval cover all actions the policy can ever select.
    Sec 3.2/3.4: Q(s,a) is computed as inner product with embedding e_i and argmax is restricted to top-200 ANN candidates; if the optimal demo is outside this set, the agent cannot find it.
  • domain assumption The MLLM's MAE on individual queries is a stable, informative reward for sequential selection.
    Sec 3.1 defines r_t = R(s_{t+1}) - R(s_t) with R = -MAE; no averaging over queries or noise model is given.
  • domain assumption Training reward queries and evaluation queries are drawn from separate distributions.
    Sec C.3 separates demo pool and query set but does not state that RL training queries are disjoint from the 1,000-query evaluation subset.
  • domain assumption Dueling DQN with ANN-approximated max converges to a policy that maximizes prompt quality.
    Sec 3.4 uses approximate max over 200 candidates in the target; no convergence guarantee is provided.

pith-pipeline@v1.3.0-alltime-deepseek · 22178 in / 15788 out tokens · 156726 ms · 2026-08-04T05:40:06.737239+00:00 · methodology

0 comments
read the original abstract

Multimodal Large Language Models (MLLMs) adapt to visual tasks via in-context learning (ICL), which relies heavily on demonstration quality. The dominant demonstration selection strategy is unsupervised k-Nearest Neighbor (kNN) search. While simple, this similarity-first approach is sub-optimal for complex factual regression tasks; it selects redundant examples that fail to capture the task's full output range. We reframe selection as a sequential decision-making problem and introduce Learning to Select Demonstrations (LSD), training a Reinforcement Learning agent to construct optimal demonstration sets. Using a Dueling DQN with a query-centric Transformer Decoder, our agent learns a policy that maximizes MLLM downstream performance. Evaluating across five visual regression benchmarks, we uncover a crucial dichotomy: while kNN remains optimal for subjective preference tasks, LSD significantly outperforms baselines on objective, factual regression tasks. By balancing visual relevance with diversity, LSD better defines regression boundaries, illuminating when learned selection is strictly necessary for visual ICL.

Figures

Figures reproduced from arXiv: 2603.26775 by Eugene Lee, Jiajie Diao, Yu-Chi Lin.

Figure 1
Figure 1. Figure 1: An overview of our LSD (Learning to Select Demon￾strations) framework. The process is a training loop where the MLLM acts as the Environment. (1) The Agent (a Dueling DQN) receives the current state st, which contains the query embedding eq and the embeddings of all previously selected demonstrations {e1, . . . , et−1}. (2) The agent’s query-centric decoder outputs an advantage query as, which is used to r… view at source ↗
Figure 2
Figure 2. Figure 2: Performance vs. Number of Shots (K) on four datasets. We plot the MAE as K increases. The results are task-dependent: (a), (c), (d) Objective Tasks (UTKFace, KonIQ, KADID): Our LSD policy (blue) consistently outperforms the kNN baseline (orange). (b) Subjective Task (AVA): The kNN baseline, which is based on visual similarity, consistently outperforms LSD. LSD Agent. Our Dueling DQN agent’s state encoder i… view at source ↗
Figure 3
Figure 3. Figure 3: Demonstration Set Analysis on UTKFace, plotted against K shots. (a) MAE of Demo Labels vs. Query: The MAE between selected demo labels and the query’s true label. LSD finds demos with closer labels. (b) Pairwise Label MAE: The MAE computed over all pairwise label differences among the selected demos. (c) Demo-Query Feature Similarity: The cosine similarity between demo embeddings and the query embedding. L… view at source ↗
Figure 4
Figure 4. Figure 4: Qualitative Comparison of Selected Demonstrations (K = 12). (a) UTKFace: For an 8-year-old query, kNN selects only images with highly similar features (e.g., other young children). LSD selects a diverse spectrum of visual features (e.g., varied ages, genders, and lighting conditions) to build a richer context. (b) KADID-10k: For a motion-blurred query, kNN selects only other distorted versions of the same … view at source ↗
Figure 5
Figure 5. Figure 5: Cross-MLLM Generalization (MAE ↓) on UTKFace vs. Number of Shots (K). We use the single LSD policy (trained on Gemma 3 4B-it) to select demos for two unseen MLLMs. The plots show our policy (blue line) versus the kNN (orange line) and Random (green line) baselines. (a) On Qwen 2.5 7B, our policy consistently outperforms kNN. (b) On Phi-3.5-vision, our policy performs on par with kNN. Both LSD and kNN signi… view at source ↗
Figure 6
Figure 6. Figure 6: Extended Feature-Space Analysis (Relevance and Similarity). The plots in the right column demonstrate that on all five datasets, LSD (blue line) actively seeks low redundancy, maintaining the trend LSD ≪ kNN in pairwise similarity, which is the key behavioral difference [PITH_FULL_IMAGE:figures/full_fig_p016_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Extended Label-Space Analysis (Emergent Relevance and Consistency). The results show a critical, task-dependent pattern in minimizing the label difference (∆Label) between the query and selected demos. For Objective Tasks (UTKFace, KonIQ-10k, KADID￾10k), the LSD policy (blue) is the most effective implicit label retriever. Conversely, for Subjective Tasks (AVA, SCUT-FBP5500), the kNN Baseline (orange) cons… view at source ↗
Figure 8
Figure 8. Figure 8: Transfer Scaling for Source Policy: Gemma 3 4B-it. Performance of the Gemma-trained LSD policy evaluated across all four target models. The policy generalizes well, consistently beating kNN on Qwen and InternVL, and matching it on Phi. Source Policy: Qwen 2.5 7B 1 3 5 7 Number of Shots (K) 6 8 10 12 14 Mean Absolute Error (MAE) LSD kNN Random Zero-Shot (a) Target: Gemma 3 1 3 5 7 Number of Shots (K) 5 6 7 … view at source ↗
Figure 9
Figure 9. Figure 9: Transfer Scaling for Source Policy: Qwen 2.5 7B. Performance of the Qwen-trained LSD policy evaluated across all targets. Source Policy: InternVL2-8B 1 3 5 7 Number of Shots (K) 6 8 10 12 14 Mean Absolute Error (MAE) LSD kNN Random Zero-Shot (a) Target: Gemma 3 1 3 5 7 Number of Shots (K) 5 6 7 8 9 Mean Absolute Error (MAE) LSD kNN Random Zero-Shot (b) Target: Qwen 2.5 1 3 5 7 Number of Shots (K) 5.5 6.0 6… view at source ↗
Figure 10
Figure 10. Figure 10: Transfer Scaling for Source Policy: InternVL2-8B. Performance of the InternVL-trained LSD policy evaluated across all targets [PITH_FULL_IMAGE:figures/full_fig_p019_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Cross-Dataset Generalization Results. We compare LSD-Self (Blue, trained on target), LSD-Cross (Red, trained on Age), kNN (Orange), and Random (Green). (d) Successful Transfer: On KADID-10k, the Age policy (Red) transfers remarkably well, match￾ing the Self-trained agent and beating kNN/Random. (b) Negative Transfer: On SCUT-FBP5500, the Age policy hurts performance, performing worse than random. (a) Gene… view at source ↗
Figure 12
Figure 12. Figure 12: Extended Qualitative Comparison of Selected Demonstrations (K = 11) across benchmark datasets. Row 1: UTKFace (Age). For a query of a young child, the kNN baseline retrieves a homogeneous set of other children and babies. In contrast, LSD (Ours) retrieves a diverse timeline of faces, ranging from toddlers to adults and the elderly, providing the VLM with a complete regression scale. Row 2: KADID-10k (Qual… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

58 extracted references · 29 linked inside Pith

  1. [1]

    Phi-4 technical report.arXiv preprint arXiv:2412.08905, 2024

    Marah Abdin, Jyoti Aneja, Harkirat Behl, S ´ebastien Bubeck, Ronen Eldan, Suriya Gunasekar, Michael Harrison, Russell J Hewett, Mojan Javaheripi, Piero Kauffmann, et al. Phi-4 technical report.arXiv preprint arXiv:2412.08905, 2024. 5

    Pith/arXiv arXiv 2024

  2. [2]

    How do in- context examples affect compositional generalization?arXiv preprint arXiv:2305.04835, 2023

    Shengnan An, Zeqi Lin, Qiang Fu, Bei Chen, Nanning Zheng, Jian-Guang Lou, and Dongmei Zhang. How do in- context examples affect compositional generalization?arXiv preprint arXiv:2305.04835, 2023. 2

    Pith/arXiv arXiv 2023

  3. [3]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report.arXiv preprint arXiv:2502.13923, 2025. 5

    Pith/arXiv arXiv 2025

  4. [4]

    Editing factual knowledge in language models.arXiv preprint arXiv:2104.08164, 2021

    Nicola De Cao, Wilker Aziz, and Ivan Titov. Editing factual knowledge in language models.arXiv preprint arXiv:2104.08164, 2021. 1

    Pith/arXiv arXiv 2021

  5. [5]

    Is gpt-3 a good data annotator?arXiv preprint arXiv:2212.10450, 2022

    Bosheng Ding, Chengwei Qin, Linlin Liu, Yew Ken Chia, Shafiq Joty, Boyang Li, and Lidong Bing. Is gpt-3 a good data annotator?arXiv preprint arXiv:2212.10450, 2022. 1

    Pith/arXiv arXiv 2022

  6. [6]

    A survey on in-context learning.arXiv preprint arXiv:2301.00234, 2022

    Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Jingyuan Ma, Rui Li, Heming Xia, Jingjing Xu, Zhiyong Wu, Tianyu Liu, et al. A survey on in-context learning.arXiv preprint arXiv:2301.00234, 2022. 1

    Pith/arXiv arXiv 2022

  7. [7]

    The faiss library

    Matthijs Douze, Alexandr Guzhva, Chengqi Deng, Jeff Johnson, Gergely Szilvasy, Pierre-Emmanuel Mazar´e, Maria Lomeli, Lucas Hosseini, and Herv´e J´egou. The faiss library

  8. [8]

    Towards multimodal in-context learning for vi- sion and language models

    Sivan Doveh, Shaked Perek, M Jehanzeb Mirza, Wei Lin, Amit Alfassy, Assaf Arbelle, Shimon Ullman, and Leonid Karlinsky. Towards multimodal in-context learning for vi- sion and language models. InEuropean Conference on Com- puter Vision, pages 250–267. Springer, 2024. 3

    2024

  9. [9]

    Deep reinforcement learning in large discrete action spaces

    Gabriel Dulac-Arnold, Richard Evans, Hado van Hasselt, Peter Sunehag, Timothy Lillicrap, Jonathan Hunt, Timothy Mann, Theophane Weber, Thomas Degris, and Ben Coppin. Deep reinforcement learning in large discrete action spaces. arXiv preprint arXiv:1512.07679, 2015. 12

    Pith/arXiv arXiv 2015

  10. [10]

    In-context learning enables multimodal large language models to classify cancer pathology images.Na- ture Communications, 15(1):10104, 2024

    Dyke Ferber, Georg W ¨olflein, Isabella C Wiest, Marta Ligero, Srividhya Sainath, Narmin Ghaffari Laleh, Omar SM El Nahhas, Gustav M ¨uller-Franzes, Dirk J ¨ager, Daniel Truhn, et al. In-context learning enables multimodal large language models to classify cancer pathology images.Na- ture Communications, 15(1):10104, 2024. 2

    2024

  11. [11]

    What makes good few-shot examples for vision-language models?arXiv preprint arXiv:2405.13532,

    Zhaojun Guo, Jinghui Lu, Xuejing Liu, Rui Zhao, ZhenXing Qian, and Fei Tan. What makes good few-shot examples for vision-language models?arXiv preprint arXiv:2405.13532,

    Pith/arXiv arXiv

  12. [12]

    Structured prompting: Scaling in-context learning to 1,000 examples.arXiv preprint arXiv:2212.06713, 2022

    Yaru Hao, Yutao Sun, Li Dong, Zhixiong Han, Yux- ian Gu, and Furu Wei. Structured prompting: Scaling in-context learning to 1,000 examples.arXiv preprint arXiv:2212.06713, 2022. 2

    Pith/arXiv arXiv 2022

  13. [13]

    Koniq-10k: An ecologically valid database for deep learning of blind image quality assessment.IEEE Transactions on Image Processing, 29:4041–4056, 2020

    Vlad Hosu, Hanhe Lin, Tamas Sziranyi, and Dietmar Saupe. Koniq-10k: An ecologically valid database for deep learning of blind image quality assessment.IEEE Transactions on Image Processing, 29:4041–4056, 2020. 5

    2020

  14. [14]

    Explor- ing in-context learning capabilities of foundation models for generating knowledge graphs from text.arXiv preprint arXiv:2305.08804, 2023

    Hanieh Khorashadizadeh, Nandana Mihindukulasooriya, Sanju Tiwari, Jinghua Groppe, and Sven Groppe. Explor- ing in-context learning capabilities of foundation models for generating knowledge graphs from text.arXiv preprint arXiv:2305.08804, 2023. 1

    Pith/arXiv arXiv 2023

  15. [15]

    Self-generated in-context learning: Leveraging auto-regressive language models as a demonstration generator.arXiv preprint arXiv:2206.08082,

    Hyuhng Joon Kim, Hyunsoo Cho, Junyeob Kim, Taeuk Kim, Kang Min Yoo, and Sang-goo Lee. Self-generated in-context learning: Leveraging auto-regressive language models as a demonstration generator.arXiv preprint arXiv:2206.08082,

    Pith/arXiv arXiv

  16. [16]

    In-context compositional general- ization for large vision-language models

    Chuanhao Li, Chenchen Jing, Zhen Li, Mingliang Zhai, Yuwei Wu, and Yunde Jia. In-context compositional general- ization for large vision-language models. InProceedings of the 2024 Conference on Empirical Methods in Natural Lan- guage Processing, pages 17954–17966, 2024. 2

    2024

  17. [17]

    Deep instance-level hard negative min- ing model for histopathology images

    Meng Li, Lin Wu, Arnold Wiliem, Kun Zhao, Teng Zhang, and Brian Lovell. Deep instance-level hard negative min- ing model for histopathology images. InInternational con- ference on medical image computing and computer-assisted intervention, pages 514–522. Springer, 2019. 2

    2019

  18. [18]

    Advancing multimodal in-context learning in large vision-language models with task-aware demonstra- tions

    Yanshu Li. Advancing multimodal in-context learning in large vision-language models with task-aware demonstra- tions. InWorkshop on Reasoning and Planning for Large Language Models. 2

  19. [19]

    Scut-fbp5500: A diverse benchmark dataset for multi-paradigm facial beauty prediction

    Lingyu Liang, Luojun Lin, Lianwen Jin, Duorui Xie, and Mengru Li. Scut-fbp5500: A diverse benchmark dataset for multi-paradigm facial beauty prediction. In2018 24th In- ternational conference on pattern recognition (ICPR), pages 1598–1603. IEEE, 2018. 5

    2018

  20. [20]

    Kadid-10k: A large-scale artificially distorted iqa database

    Hanhe Lin, Vlad Hosu, and Dietmar Saupe. Kadid-10k: A large-scale artificially distorted iqa database. In2019 Eleventh International Conference on Quality of Multimedia Experience (QoMEX), pages 1–3. IEEE, 2019. 5

    2019

  21. [21]

    What makes good in-context examples for gpt-3?arXiv preprint arXiv:2101.06804, 2021

    Jiachang Liu, Dinghan Shen, Yizhe Zhang, Bill Dolan, Lawrence Carin, and Weizhu Chen. What makes good in-context examples for gpt-3?arXiv preprint arXiv:2101.06804, 2021. 1, 2, 3, 4, 5

    Pith/arXiv arXiv 2021

  22. [22]

    Let’s learn step by step: Enhancing in-context learning ability with curriculum learning.arXiv preprint arXiv:2402.10738, 2024

    Yinpeng Liu, Jiawei Liu, Xiang Shi, Qikai Cheng, Yong Huang, and Wei Lu. Let’s learn step by step: Enhancing in-context learning ability with curriculum learning.arXiv preprint arXiv:2402.10738, 2024. 1, 2, 4

    Pith/arXiv arXiv 2024

  23. [23]

    Active learning principles for in-context learning with large language models.arXiv preprint arXiv:2305.14264, 2023

    Katerina Margatina, Timo Schick, Nikolaos Aletras, and Jane Dwivedi-Yu. Active learning principles for in-context learning with large language models.arXiv preprint arXiv:2305.14264, 2023. 2

    Pith/arXiv arXiv 2023

  24. [24]

    Using in-context learning to improve dialogue safety.arXiv preprint arXiv:2302.00871, 2023

    Nicholas Meade, Spandana Gella, Devamanyu Hazarika, Prakhar Gupta, Di Jin, Siva Reddy, Yang Liu, and Dilek Hakkani-T¨ur. Using in-context learning to improve dialogue safety.arXiv preprint arXiv:2302.00871, 2023. 1

    Pith/arXiv arXiv 2023

  25. [25]

    Ava: A large-scale database for aesthetic visual analysis

    Naila Murray, Luca Marchesotti, and Florent Perronnin. Ava: A large-scale database for aesthetic visual analysis. In2012 IEEE conference on computer vision and pattern recognition, pages 2408–2415. IEEE, 2012. 5

    2012

  26. [26]

    Differentially private in-context learning

    Ashwinee Panda, Tong Wu, Jiachen Wang, and Prateek Mit- tal. Differentially private in-context learning. InThe 61st Annual Meeting Of The Association For Computational Lin- guistics, 2023. 1

    2023

  27. [27]

    A call to reflect on eval- uation practices for age estimation: Comparative analysis of the state-of-the-art and a unified benchmark

    Jakub Paplh ´am, V ojt Franc, et al. A call to reflect on eval- uation practices for age estimation: Comparative analysis of the state-of-the-art and a unified benchmark. InProceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 1196–1205, 2024. 6

    2024

  28. [28]

    Sample efficient demonstration selection for in-context learning

    Kiran Purohit, V Venktesh, Sourangshu Bhattacharya, and Avishek Anand. Sample efficient demonstration selection for in-context learning. InForty-second International Con- ference on Machine Learning. 2

  29. [29]

    In-context learning with iterative demon- stration selection.arXiv preprint arXiv:2310.09881, 2023

    Chengwei Qin, Aston Zhang, Chen Chen, Anirudh Dagar, and Wenming Ye. In-context learning with iterative demon- stration selection.arXiv preprint arXiv:2310.09881, 2023. 1, 2

    Pith/arXiv arXiv 2023

  30. [30]

    In- context retrieval-augmented language models.Transactions of the Association for Computational Linguistics, 11:1316– 1331, 2023

    Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Am- non Shashua, Kevin Leyton-Brown, and Yoav Shoham. In- context retrieval-augmented language models.Transactions of the Association for Computational Linguistics, 11:1316– 1331, 2023. 1

    2023

  31. [31]

    Learn- ing to retrieve prompts for in-context learning.arXiv preprint arXiv:2112.08633, 2021

    Ohad Rubin, Jonathan Herzig, and Jonathan Berant. Learn- ing to retrieve prompts for in-context learning.arXiv preprint arXiv:2112.08633, 2021. 2, 3

    Pith/arXiv arXiv 2021

  32. [32]

    Multilingual llms are better cross- lingual in-context learners with alignment.arXiv preprint arXiv:2305.05940, 2023

    Eshaan Tanwar, Subhabrata Dutta, Manish Borthakur, and Tanmoy Chakraborty. Multilingual llms are better cross- lingual in-context learners with alignment.arXiv preprint arXiv:2305.05940, 2023. 1

    Pith/arXiv arXiv 2023

  33. [33]

    Gemma 3 technical report.arXiv preprint arXiv:2503.19786, 2025

    Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Ta- tiana Matejovicova, Alexandre Ram ´e, Morgane Rivi `ere, et al. Gemma 3 technical report.arXiv preprint arXiv:2503.19786, 2025. 5

    Pith/arXiv arXiv 2025

  34. [34]

    Regres- sion or classification? new methods to evaluate no-reference picture and video quality models

    Zhengzhong Tu, Chia-Ju Chen, Li-Heng Chen, Yilin Wang, Neil Birkbeck, Balu Adsumilli, and Alan C Bovik. Regres- sion or classification? new methods to evaluate no-reference picture and video quality models. InICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 2085–2089. IEEE, 2021. 6

    2021

  35. [35]

    Attention is all you need.Advances in neural information processing systems, 30, 2017

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko- reit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need.Advances in neural information processing systems, 30, 2017. 4

    2017

  36. [36]

    In-context example retrieval from multi- perspectives for few-shot aspect-based sentiment analysis

    Qianlong Wang, Hongling Xu, Keyang Ding, Bin Liang, and Ruifeng Xu. In-context example retrieval from multi- perspectives for few-shot aspect-based sentiment analysis. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evalu- ation (LREC-COLING 2024), pages 8975–8985, 2024. 1

    2024

  37. [37]

    Want to reduce labeling cost? gpt-3 can help.arXiv preprint arXiv:2108.13487, 2021

    Shuohang Wang, Yang Liu, Yichong Xu, Chenguang Zhu, and Michael Zeng. Want to reduce labeling cost? gpt-3 can help.arXiv preprint arXiv:2108.13487, 2021. 1

    Pith/arXiv arXiv 2021

  38. [38]

    Large language models are im- plicitly topic models: Explaining and finding good demon- strations for in-context learning

    Xinyi Wang, Wanrong Zhu, Michael Saxon, Mark Steyvers, and William Yang Wang. Large language models are im- plicitly topic models: Explaining and finding good demon- strations for in-context learning. InWorkshop on efficient systems for foundation models@ icml2023, 2023. 1, 2, 3

    2023

  39. [39]

    Demonstration selection for in- context learning via reinforcement learning.arXiv preprint arXiv:2412.03966, 2024

    Xubin Wang, Jianfei Wu, Yichen Yuan, Deyu Cai, Mingzhe Li, and Weijia Jia. Demonstration selection for in- context learning via reinforcement learning.arXiv preprint arXiv:2412.03966, 2024. 2, 3

    Pith/arXiv arXiv 2024

  40. [40]

    Dueling network architectures for deep reinforcement learning

    Ziyu Wang, Tom Schaul, Matteo Hessel, Hado Hasselt, Marc Lanctot, and Nando Freitas. Dueling network architectures for deep reinforcement learning. InInternational conference on machine learning, pages 1995–2003. PMLR, 2016. 3

    1995

  41. [41]

    Chain-of-thought prompting elicits reasoning in large lan- guage models.Advances in neural information processing systems, 35:24824–24837, 2022

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large lan- guage models.Advances in neural information processing systems, 35:24824–24837, 2022. 1

    2022

  42. [42]

    To- wards reliable and holistic visual in-context learning prompt selection.arXiv preprint arXiv:2509.25989, 2025

    Wenxiao Wu, Jing-Hao Xue, Chengming Xu, Chen Liu, Xin- wei Sun, Changxin Gao, Nong Sang, and Yanwei Fu. To- wards reliable and holistic visual in-context learning prompt selection.arXiv preprint arXiv:2509.25989, 2025. 2

    arXiv 2025

  43. [43]

    The role of diversity in in-context learning for large language models

    Wenyang Xiao, Haoyu Zhao, and Lingxiao Huang. The role of diversity in in-context learning for large language models. arXiv preprint arXiv:2505.19426, 2025. 2

    Pith/arXiv arXiv 2025

  44. [44]

    Improving in-context learning with prediction feedback for sentiment analysis

    Hongling Xu, Qianlong Wang, Yice Zhang, Min Yang, Xi Zeng, Bing Qin, and Ruifeng Xu. Improving in-context learning with prediction feedback for sentiment analysis. arXiv preprint arXiv:2406.02911, 2024. 1

    Pith/arXiv arXiv 2024

  45. [45]

    Transferring rich deep features for facial beauty prediction.arXiv preprint arXiv:1803.07253, 2018

    Lu Xu, Jinhai Xiang, and Xiaohui Yuan. Transferring rich deep features for facial beauty prediction.arXiv preprint arXiv:1803.07253, 2018. 6

    Pith/arXiv arXiv 2018

  46. [46]

    Auto-icl: In- context learning without human supervision.arXiv preprint arXiv:2311.09263, 2023

    Jinghan Yang, Shuming Ma, and Furu Wei. Auto-icl: In- context learning without human supervision.arXiv preprint arXiv:2311.09263, 2023. 2

    Pith/arXiv arXiv 2023

  47. [47]

    Li Yang, Zengzhi Wang, Ziyan Li, Jin-Cheon Na, and Jian- fei Yu. An empirical study of multimodal entity-based sen- timent analysis with chatgpt: Improving in-context learning via entity-aware contrastive learning.Information Process- ing & Management, 61(4):103724, 2024. 1

    2024

  48. [48]

    Representative demonstration selection for in-context learning with two-stage determinantal point pro- cess

    Zhao Yang, Yuanzhe Zhang, Dianbo Sui, Cao Liu, Jun Zhao, and Kang Liu. Representative demonstration selection for in-context learning with two-stage determinantal point pro- cess. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 5443–5456,

    2023

  49. [49]

    Large language models are contrastive reasoners

    Liang Yao. Large language models are contrastive reasoners. arXiv preprint arXiv:2403.08211, 2024. 3

    arXiv 2024

  50. [50]

    Compositional exemplars for in-context learn- ing

    Jiacheng Ye, Zhiyong Wu, Jiangtao Feng, Tao Yu, and Ling- peng Kong. Compositional exemplars for in-context learn- ing. InInternational Conference on Machine Learning, pages 39818–39833. PMLR, 2023. 2

    2023

  51. [51]

    Sigmoid loss for language image pre-training

    Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. InProceedings of the IEEE/CVF international conference on computer vision, pages 11975–11986, 2023. 3

    2023

  52. [52]

    Sentiment analysis in the era of large language models: A reality check.arXiv preprint arXiv:2305.15005,

    Wenxuan Zhang, Yue Deng, Bing Liu, Sinno Jialin Pan, and Lidong Bing. Sentiment analysis in the era of large language models: A reality check.arXiv preprint arXiv:2305.15005,

    Pith/arXiv arXiv

  53. [53]

    Active ex- ample selection for in-context learning.arXiv preprint arXiv:2211.04486, 2022

    Yiming Zhang, Shi Feng, and Chenhao Tan. Active ex- ample selection for in-context learning.arXiv preprint arXiv:2211.04486, 2022. 2, 3

    Pith/arXiv arXiv 2022

  54. [54]

    What makes good examples for visual in-context learning?Advances in Neural Information Processing Systems, 36:17773–17794,

    Yuanhan Zhang, Kaiyang Zhou, and Ziwei Liu. What makes good examples for visual in-context learning?Advances in Neural Information Processing Systems, 36:17773–17794,

  55. [55]

    Age progres- sion/regression by conditional adversarial autoencoder

    Zhifei Zhang, Yang Song, and Hairong Qi. Age progres- sion/regression by conditional adversarial autoencoder. In IEEE Conference on Computer Vision and Pattern Recogni- tion (CVPR). IEEE, 2017. 5

    2017

  56. [56]

    Learning to select in-context demon- stration preferred by large language model.arXiv preprint arXiv:2505.19966, 2025

    Zheng Zhang, Shaocheng Lan, Lei Song, Jiang Bian, Yexin Li, and Kan Ren. Learning to select in-context demon- stration preferred by large language model.arXiv preprint arXiv:2505.19966, 2025. 2

    Pith/arXiv arXiv 2025

  57. [57]

    Visual in-context learning for large vision-language models.arXiv preprint arXiv:2402.11574, 2024

    Yucheng Zhou, Xiang Li, Qianning Wang, and Jianbing Shen. Visual in-context learning for large vision-language models.arXiv preprint arXiv:2402.11574, 2024. 2

    Pith/arXiv arXiv 2024

  58. [58]

    Age:< y i >

    Yan Zhu, Huan Ma, and Changqing Zhang. Exploring task- level optimal prompts for visual in-context learning. InPro- ceedings of the AAAI Conference on Artificial Intelligence, pages 11031–11039, 2025. 1 Learning to Select Visual In-Context Demonstrations Supplementary Material A. Prompt Construction and In-Context Learning We utilize a multimodal few-shot...

    2025