Pith. sign in

REVIEW 5 cited by

In-context Example Selection with Influences

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2302.11042 v2 pith:Q2NY5NJS submitted 2023-02-21 cs.CL cs.LG

Tai Nguyen , Eric Wong This is my paper
classification cs.CLcs.LG
keywords in-contextexamplesexampleperformancefew-shotinfluence-basedinfluencesnegative
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
abstract

In-context learning (ICL) is a powerful paradigm emerged from large language models (LLMs). Despite its promises, ICL performance is known to be highly sensitive to input examples. In this work, we use $\textit{in-context influences}$ to analyze few-shot ICL performance directly from the in-context examples. Our proposed influence-based example selection method can identify both positive and negative examples, outperforming several baselines when evaluated on 9 SuperGLUE tasks. Our analysis uncovers up to a $16.3\%$ performance gap between using the most negative in-context examples compared to the most positive. In a case study, we apply our influence-based framework to quantify the phenomena of recency bias in example ordering for few-shot ICL.

Discussion (0). Sign in to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. From Historical Patches to Repair Plans: Outcome-Conditioned Reasoning for Repository-Level Program Repair

    cs.SE 2026-01 conditional novelty 6.0 of 10

    Backward reasoning distilled from a repository's already-fixed issues, injected as prompts, lifts LLM repair success on SWE-Bench Lite by up to 10.4 Pass@1 points.

  2. DICE: Dynamic In-Context Example Selection in LLM Agents via Efficient Knowledge Transfer

    cs.AI 2025-07 conditional novelty 5.0 of 10

    DICE dynamically retrieves the most relevant in-context demonstrations at each agent step, and in this preprint it raises exact-match and success-rate scores on HotpotQA, ALFWorld, and Webshop across ReAct, Reflexion,...

  3. Demystifying Feature Requests: Leveraging LLMs to Refine Feature Requests in Open-Source Software

    cs.SE 2025-07 conditional novelty 5.0 of 10

    GPT-4o with in-context learning can flag ambiguity and incompleteness in GitHub feature requests and draft clarification questions, though moderate annotator agreement and a small sample limit the strength of the evidence.

  4. Improving Dialogue State Tracking through Combinatorial Search for In-Context Examples

    cs.CL 2025-05 conditional novelty 5.0 of 10

    A combinatorial scoring method for retriever training data improves few-shot dialogue state tracking by 20x in data efficiency and by 12% in oracle upper-bound JGA over prior methods.

  5. OpenReview Should be Protected and Leveraged as a Community Asset for Research in the Era of Large Language Models

    cs.CY 2025-05 conditional novelty 4.0 of 10

    The paper advocates protecting and leveraging OpenReview's peer review corpus as a community asset for LLM-based review assistance, benchmarks, and alignment.

Pith tools