REVIEW 8 cited by
Self-Adaptive In-Context Learning: An Information Compression Perspective for In-Context Example Selection and Ordering
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Despite the surprising few-shot performance of in-context learning (ICL), it is still a common practice to randomly sample examples to serve as context. This paper advocates a new principle for ICL: self-adaptive in-context learning. The self-adaption mechanism is introduced to help each sample find an in-context example permutation (i.e., selection and ordering) that can derive the correct prediction, thus maximizing performance. To validate the effectiveness of self-adaptive ICL, we propose a general select-then-rank framework and instantiate it with new selection and ranking algorithms. Upon extensive evaluation on eight different NLP datasets, our self-adaptive ICL method achieves a 40% relative improvement over the common practice setting. Further analysis reveals the enormous potential of self-adaptive ICL that it might be able to close the gap between ICL and finetuning given more advanced algorithms. Our code is released to facilitate future research in this area: https://github.com/Shark-NLP/self-adaptive-ICL
Forward citations
Cited by 8 Pith papers
-
Train Once, Reuse Everywhere: Generalizable Implicit In-Context Learning by Routing Attention
ICR extracts shared attention directions from in-context learning and routes them at inference time, enabling zero-shot reuse across tasks.
-
STARE at the Structure: Steering ICL Exemplar Selection with Structural Alignment
A structure-aware exemplar retriever with a hidden-state syntactic injection module improves in-context semantic parsing across four benchmarks.
-
SV-TrustEval-C: Evaluating Structure and Semantic Reasoning in Large Language Models for Source Code Vulnerability Analysis
Large language models perform poorly on a new C-code vulnerability benchmark, indicating they rely on pattern matching rather than genuine reasoning.
-
DICE: Dynamic In-Context Example Selection in LLM Agents via Efficient Knowledge Transfer
DICE dynamically retrieves the most relevant in-context demonstrations at each agent step, and in this preprint it raises exact-match and success-rate scores on HotpotQA, ALFWorld, and Webshop across ReAct, Reflexion,...
-
Unveiling Effective In-Context Configurations for Image Captioning: An External & Internal Analysis
For Flamingo-style models, increasing the number of in-context examples improves language coherence but degrades visual-text alignment, and similarity-based image retrieval inflates CIDEr scores by encouraging caption...
-
How to Retrieve Examples in In-context Learning to Improve Conversational Emotion Recognition using Large Language Models?
Retrieving a semantically similar example and voting over paraphrased versions of it improves conversational emotion recognition macro F1 over random in-context examples.
-
A Hybrid Multi-Agent Prompting Approach for Simplifying Complex Sentences
A multi-agent GPT-4O pipeline with an internal semantic-lexical gate claims 70% success on simplifying 100 video game sentences, versus 48% for a single-agent version.
-
Mimicking or Reasoning: Rethinking Multi-Modal In-Context Learning in Vision-Language Models
Vision-language models improve little, often not at all, when given demonstrations, even when demonstrations contain explicit reasoning steps.
Discussion (0). Sign in to comment.