Pith. sign in

REVIEW 4 major objections 5 minor 61 references

KITE: Kernelized and Information Theoretic Exemplars for In-Context Learning

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read KITE frames in-context example selection as a query-specific optimization problem, proves an approximate-submodularity guarantee for greedy selection, and reports the best accuracy in 13 of 15 dataset-model settings.

desk verdict KITE is a genuinely new ICL retrieval recipe with real empirical gains, but the paper's headline theoretical guarantee rests on a false lemma and should not be published as is. read the letter →

arxiv 2509.15676 v2 pith:SHKTRZNR submitted 2025-09-19 cs.LG cs.AIcs.CL

Vaibhav Singh , Soumya Suvra Ghosal , Kapu Nirmal Joshua , Soumyabrata Pal , Sayak Ray Chowdhury This is my paper · ORCID
classification cs.LGcs.AIcs.CL
keywords in-contextlearningexampleselectionexemplarretrievalsubmodularoptimizationkerneltrickD-optimaldesignfew-shotclassification
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper is trying to establish that the choice of in-context examples for an LLM can be made by a principled, fully unsupervised rule rather than by nearest-neighbor heuristics. It models the LLM as a linear function of the input embeddings and selects the subset of exemplars that minimizes a query-specific ridge prediction error, then adds a diversity regularizer. It proves the resulting objective is approximately submodular, so greedy selection carries a near-optimality guarantee. On five classification datasets and three models, the proposed selector, KITE, reports the highest accuracy in 13 of 15 settings.

What carries the argument

The central object is the design covariance matrix $V_S=\beta I+\sum_{i\in S}x_i x_i^\top$ and the greedy marginal gain it defines. For a candidate example $x$ given current set $S$, the relevance gain is $(z^\top V_S^{-1}x)^2/(1+x^\top V_S^{-1}x)$, and the Sherman-Morrison formula updates $V_S^{-1}$ in $O(d^2)$ per step. The kernel trick replaces every inner product with a kernel evaluation, turning the gain into $k_S(z,x)^2/(\beta+k_S(x,x))$, where $k_S$ is the residual kernel after projecting out the selected examples. Diversity enters through the D-optimal design gain $\log(1+x^\top V_S^{-1}x)$, and the algorithm maximizes the sum of the relevance and diversity gains with trade-off parameter $\lambda$. The proof machinery is the submodularity ratio $\gamma_k(f_z)$, lower-bounded by $1/(1+(k-1)\mu)$, which certifies near-optimality of the greedy selection.

What would settle it

A benchmark where the label depends on an XOR or parity function of embedding coordinates, with a fixed kernel chosen before seeing any validation labels, would settle the claim: if the linear version of KITE fails to beat dense kNN or DPP retrieval, or if kernel choices made without the evaluation split reverse the reported ranking, then the optimized quantity does not track actual prompt accuracy.

Watch

Extended reading notes

Core claim

Conditioned on a test query $z$, the paper claims that the LLM's response is generated as $y=\langle x,\theta\rangle+\eta$ with $\eta\sim N(0,1)$, so the best prompt subset is the one that minimizes the ridge prediction error at $z$. The paper bounds that error by $\|z\|_{V_S^{-1}}^2$ with $V_S=\beta I+\sum_{i\in S}x_i x_i^\top$ and proposes maximizing $f_z(S)=-z^\top V_S^{-1}z$. Because $f_z$ is only approximately submodular, the paper bounds its submodularity ratio below by $1/(1+(k-1)\mu)$ and proves greedy selection achieves a $1-e^{-1/(1+(k-1)\mu)}$ approximation factor. The kernel trick rewrites the greedy gain using only kernel evaluations, and the D-optimal term $\log\det(V_S)$ adds diversity. The empirical claim is that the combined algorithm, KITE, outperforms random, BM25, dense kNN, and DPP retrieval on most of the 15 dataset-model configurations.

Load-bearing premise

The load-bearing premise is that, conditioned on a test query, an LLM's in-context prediction is close enough to a linear function of the input embeddings that minimizing the ridge prediction error at that query also minimizes actual prompt error; if that approximation fails, the objective $f_z(S)$ no longer tracks the quantity the paper reports.

Editorial extensions

If this is right

  • KITE can be deployed as a frozen retriever: it needs only embeddings and a kernel, no labels at selection time, no fine-tuning, and each greedy step costs $O(d^2)$.
  • The $\log\det(V_S)$ diversity term is not decorative: on the MNLI ablation, pure relevance selection ($\lambda=0$) underperforms a balanced $\lambda$, so large redundant banks require diversity.
  • The best kernel is dataset-dependent; the reported gains come from picking among linear, polynomial, and Gaussian kernels, so kernel choice is part of the method's recipe.
  • The greedy guarantee degrades gracefully with the submodularity ratio, and increasing $\beta$ pushes the ratio toward 1, making the selected set closer to the optimal one in the theory.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural stress test would run KITE on tasks with known non-linear ICL behavior, such as parity or compositional reasoning; the theory predicts the linear version should lose its edge over nearest-neighbor retrieval there.
  • The reported protocol selects the best kernel per dataset on the evaluation split; a deployment-oriented version would choose kernels without validation labels, and that version is what a user would actually run.
  • The same query-specific ridge objective might double as a selection rule for other few-shot pipelines beyond LLMs, such as retrieval-augmented prediction, wherever the downstream predictor is approximately linear in the chosen representation.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes KITE, an unsupervised exemplar-selection method for in-context learning (ICL). It models the LLM as a linear function of input embeddings (Eq. (1)) and selects k examples to minimize a ridge-regression prediction-error bound for a query z, yielding the surrogate objective f_z(S) = -z^T V_S^{-1} z (Eq. (4)). A D-optimal-design diversity term log det(V_S) is added, and the method is kernelized so that computations are done through kernel evaluations. The central theoretical claim is Lemma 3, which lower-bounds the submodularity ratio of f_z by 1/(1+(k-1)μ), leading to a 1-exp(-1/(1+(k-1)μ)) greedy approximation guarantee in Theorem 4. Empirically, KITE is evaluated on five classification datasets with three LLMs against Random, BM25, Dense, and DPP baselines, with Table 1 reporting accuracy for each setting.

Significance. If the theoretical guarantee were valid, the paper would make a useful contribution: it offers a query-specific, computationally efficient selection rule with a clean diversity regularizer and a clear departure from kNN-style retrieval. The algorithmic machinery (Sherman-Morrison updates, kernel trick, D-optimal design) is presented cleanly, and the synthetic experiments provide some evidence that the score is sensible under the stylized linear model. However, the central guarantee does not hold: Lemma 3 is false, and the appendix proof explicitly truncates a Neumann-series expansion and then treats the truncated expression as exact. The empirical comparison is also compromised by per-dataset kernel selection on the evaluation split and by the absence of error bars. As a result, the advertised provable near-optimality is unsupported, and the empirical claims are not established at the reported level of confidence.

major comments (4)
  1. [Lemma 3 and Appendix: Proof of Lemma 3] The claimed lower bound γ_k(f_z) ≥ 1/(1+(k-1)μ) is false. Consider d=2, β=1, S=∅, k=2, x1=(1,0), x2=(0.8,0.6), z=(0.6,-1.8). Then V_S=I, μ=0.4, Δ1=Δ2=0.18, and f_z(S∪L)-f_z(S)=0.6, so the submodularity ratio is γ=0.6, which is strictly less than the claimed bound 1/1.4≈0.714. The appendix proof replaces (I+D^{-1}N)^{-1} by a two-term Neumann expansion and then treats the result as exact; the omitted terms are essential, as this counterexample shows. Consequently Theorem 4's approximation guarantee is not established, and the abstract's claim of a principled greedy algorithm with an approximation guarantee is unsupported.
  2. [Section 5 (Implementation Details) and Table 1] The headline empirical claim 'KITE consistently outperforms all baselines' is weakened by reporting the best result across the three kernel choices for each dataset on the same validation split used for evaluation. Table 3 shows that the kernel choice changes accuracy by up to 8 percentage points (e.g., MRPC: 75.27 for the linear kernel vs. 67.15 for the Gaussian RBF). Selecting the best kernel per dataset on the evaluation split is a form of test-set tuning, and the comparison to baselines, which do not receive this per-dataset kernel selection, is therefore not a fair test of the method. In addition, Tables 1, 3, and 4 report no error bars or multiple-seed variances, so the significance of differences of 0.1-1.7 points is unclear.
  3. [Section 5 (Empirical validation of submodularity) and Table 2] The empirical validation does not compute the quantity in Definition 2. It samples a random triplet (S,L,z) with L selected by farthest-point sampling and reports the minimum observed ratio, but Definition 2 requires a minimum over all disjoint S,L with |L|≤k, with no restriction on how L is formed. The reported γ_min therefore cannot certify an approximate-submodularity ratio for f_z. Moreover, because Lemma 3 is false, the empirical table does not repair the theoretical guarantee and should not be presented as evidence that the greedy approximation bound holds.
  4. [Section 3, Eq. (1)] The modeling premise—that LLM in-context responses are generated by a fixed linear function in BERT embedding space—is not validated by any experiment that compares the surrogate f_z(S) with actual LLM accuracy. The synthetic experiments verify the linear model only, not the transfer to LLMs. A concrete test would be to compare, on a held-out subset of queries, the ridge prediction error of selected sets against the LLM's prompt accuracy; without such evidence, the connection between the objective being optimized and the accuracy metric reported in Tables 1 and 3 remains an untested assumption.
minor comments (5)
  1. [Throughout] There are several typos: 'emty set' in Algorithm 1, 'embddings' in the experiments section, and 'diagonmal' in the appendix proof should be 'empty set', 'embeddings', and 'diagonal', respectively.
  2. [Table 4] The right block of Table 4 appears to have a formatting or unit error: the reported LITE errors for μ_test=0,...,5 are in the range 67.8-84.0, which is inconsistent with the left block's scale (0.7-4.2) and with the text describing mean absolute prediction error.
  3. [Appendix, Proof of Lemma 3] The proof invokes the condition ∥D^{-1}N∥≪1 without stating or verifying it; if Lemma 3 is to be salvaged, the statement should be revised to include an explicit small-coherence or small-norm assumption and the Neumann expansion should be carried out with a rigorous error bound.
  4. [Sections 3 and 4] The symbol β is used both as the ridge regularization parameter in Eq. (2) and as the noise variance in the mutual-information derivation in Section 4; the two uses should be distinguished notationally.
  5. [Definition 2 and Table 2] The notation for the submodularity ratio is inconsistent: Definition 2 uses γ_k(f), the main text later uses γ, and Table 2 uses γ_min; these should be unified.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: KITE's derivation is self-contained, with the surrogate objective grounded in external ridge-regression concentration bounds rather than in the empirical LLM accuracy it targets.

full rationale

The paper's central derivation chain is not circular. The query-specific objective f_z(S) = -z^T V_S^{-1} z is obtained from a standard concentration bound on ridge-regression prediction error (Eq. 3), credited to Lattimore and Szepesvári (2020), and the greedy selection rule is exactly the Sherman-Morrison marginal-gain decomposition of that objective. The approximate-submodularity guarantee (Theorem 4) is imported from Das and Kempe (2011) with the paper's claimed lower bound on the submodularity ratio; whether Lemma 3's bound is actually valid is a correctness question, not a circularity, because the bound is not assumed from the empirical LLM results. The diversity term is the classical D-optimal design objective, whose submodularity is proved in Lemma 5 from the matrix-determinant lemma. The kernelized selection rule follows from a stated operator identity (Lemma 6) and standard RKHS facts. The self-citations (Ghosal et al. 2025a,b) appear only in the related-work survey and are not load-bearing for any derivation. The one methodological weakness—reporting 'the best result across the three kernel choices' on the same evaluation split used to report accuracy—is test-set selection bias, not circularity: the validation accuracy is not used to define the selection objective and does not make the reported numbers equal to the objective by construction. Therefore the paper's theoretical and algorithmic claims are not circular, though the kernel-tuning protocol should be corrected for a fair comparison.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central contribution rests on two fitted hyperparameters, a per-dataset kernel selection step, and the stylized linear-response model of LLMs. No new physical entities are postulated.

free parameters (3)
  • Diversity weight lambda = 0.5 (ablation suggests lambda near 1 on MNLI)
    Hand-set trade-off between relevance and diversity; the MNLI ablation in Figure 2 shows sensitivity to lambda.
  • Ridge regularization beta = 0.02
    Fixed by hand; Remark 1 says larger beta improves submodularity, so the choice matters for the theoretical guarantee.
  • Kernel choice and kernel hyperparameters = Linear, Polynomial degree 3, Gaussian RBF sigma=1.0; best per dataset reported
    Table 1 reports the best result across three kernels, effectively selecting the kernel on the evaluation split.
assumptions (5)
  • standard math Woodbury and Sherman-Morrison identities, plus the Das-Kempe greedy approximation theorem for approximately submodular functions
    Used for efficient inverse updates and for the approximation guarantee in Theorem 4.
  • standard math Chernoff concentration bound for ridge regression prediction error from Lattimore and Szepesvari, Chapter 20
    Basis for Eq. (3) and for the surrogate objective f_z(S) = -z^T V_S^{-1} z.
  • domain assumption LLM response is linear in input embeddings with a query-conditioned parameter theta
    Eq. (1) asserts y = <x, theta> + eta; the paper does not validate this against LLM behavior.
  • domain assumption Gaussian prior theta ~ N(0, beta I) and Gaussian noise for the D-optimal design derivation
    Used to derive log det(V_S) as the mutual information between theta and the selected responses.
  • ad hoc to paper Approximate submodularity ratio lower bound in Lemma 3
    Theorem 4's guarantee depends on gamma >= 1/(1+(k-1)mu); the appendix proof truncates a Neumann series without controlled error, and the bound appears false for k=2 with mu=0.5.

how reviews work

0 comments
Cite this review

Pith. "Pith review of KITE: Kernelized and Information Theoretic Exemplars for In-Context Learning." pith.science (2026). https://pith.science/paper/SHKTRZNR

@misc{pith2026250915676,
  author       = {Pith},
  title        = {Pith review of: KITE: Kernelized and Information Theoretic Exemplars for In-Context Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SHKTRZNR}},
  note         = {Machine review of arXiv:2509.15676}
}
read the original abstract

In-context learning (ICL) has emerged as a powerful paradigm for adapting large language models (LLMs) to new and data-scarce tasks using only a few carefully selected task-specific examples presented in the prompt. However, given the limited context size of LLMs, a fundamental question arises: Which examples should be selected to maximize performance on a given user query? While nearest-neighbor-based methods like KATE have been widely adopted for this purpose, they suffer from well-known drawbacks in high-dimensional embedding spaces, including poor generalization and a lack of diversity. In this work, we study this problem of example selection in ICL from a principled, information theory-driven perspective. We first model an LLM as a linear function over input embeddings and frame the example selection task as a query-specific optimization problem: selecting a subset of exemplars from a larger example bank that minimizes the prediction error on a specific query. This formulation departs from traditional generalization-focused learning theoretic approaches by targeting accurate prediction for a specific query instance. We derive a principled surrogate objective that is approximately submodular, enabling the use of a greedy algorithm with an approximation guarantee. We further enhance our method by (i) incorporating the kernel trick to operate in high-dimensional feature spaces without explicit mappings, and (ii) introducing an optimal design-based regularizer to encourage diversity in the selected examples. Empirically, we demonstrate significant improvements over standard retrieval methods across a suite of classification tasks, highlighting the benefits of structure-aware, diverse example selection for ICL in real-world, label-scarce scenarios.

Figures

Figures reproduced from arXiv: 2509.15676 by the authors.

Figure 1
Figure 1. This figure is divided into two halves, illustrating the K [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Ablation study on the MNLI dataset illustrating the [PITH_FULL_IMAGE:figures/full_fig_p011_2.png] view at source ↗

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

61 extracted references · 22 canonical work pages

  1. [1]

    R.; Hall, J.; Fiedel, N.; Thoppilan, R.; Yang, Z.; Kulshreshtha, A.; Nemade, G.; Lu, Y.; et al

    Adiwardana, D.; Luong, M.-T.; So, D. R.; Hall, J.; Fiedel, N.; Thoppilan, R.; Yang, Z.; Kulshreshtha, A.; Nemade, G.; Lu, Y.; et al. 2020. Towards a human-like open-domain chatbot. arXiv preprint arXiv:2001.09977

    arXiv 2020

  2. [2]

    Black, S.; Gao, L.; Wang, P.; Leahy, C.; and Biderman, S. 2021. GPT-Neo: Large Scale Autoregressive Language Modeling with Mesh-Tensorflow

    work page 2021

  3. [3]

    B.; Mann, B.; Ryder, N.; Subbiah, M.; Kaplan, J.; Dhariwal, P.; Neelakantan, A.; Shyam, P.; Sastry, G.; and Amanda Askell, e

    Brown, T. B.; Mann, B.; Ryder, N.; Subbiah, M.; Kaplan, J.; Dhariwal, P.; Neelakantan, A.; Shyam, P.; Sastry, G.; and Amanda Askell, e. a. 2020 a . Language Models are Few-Shot Learners. arXiv:2005.14165

    arXiv 2020

  4. [4]

    B.; Mann, B.; Ryder, N.; Subbiah, M.; Kaplan, J.; Dhariwal, P.; Neelakantan, A.; Shyam, P.; Sastry, G.; Askell, A.; and et al

    Brown, T. B.; Mann, B.; Ryder, N.; Subbiah, M.; Kaplan, J.; Dhariwal, P.; Neelakantan, A.; Shyam, P.; Sastry, G.; Askell, A.; and et al. 2020 b . Language Models are Few-Shot Learners. In Advances in Neural Information Processing Systems 33, 1877--1901

    work page 2020

  5. [5]

    Chen, M.; Du, J.; Pasunuru, R.; Mihaylov, T.; Iyer, S.; Stoyanov, V.; and Kozareva, Z. 2022. Improving in-context few-shot learning via self-supervised training. arXiv preprint arXiv:2205.01703

    arXiv 2022

  6. [6]

    Cover, T. M. 1999. Elements of information theory. John Wiley & Sons

    1999

  7. [7]

    Dai, D.; Sun, Y.; Dong, L.; Hao, Y.; Ma, S.; Sui, Z.; and Wei, F. 2022. Why can gpt learn in-context? language models implicitly perform gradient descent as meta-optimizers. arXiv preprint arXiv:2212.10559

    arXiv 2022

  8. [8]

    Das, A.; and Kempe, D. 2011. Submodular meets spectral: Greedy algorithms for subset selection, sparse approximation and dictionary selection. arXiv preprint arXiv:1102.3975

    arXiv 2011

Show all 61 references
  1. [9]

    Devlin, J.; Chang, M.-W.; Lee, K.; and Toutanova, K. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv:1810.04805

    2019 arXiv

  2. [10]

    Dolan, B.; Quirk, C.; and Brockett, C. 2004. Unsupervised Construction of Large Paraphrase Corpora: Exploiting Massively Parallel News Sources. In COLING 2004: Proceedings of the 20th International Conference on Computational Linguistics , 350--356. Geneva, Switzerland: COLING

    2004

  3. [11]

    Dong, Q.; Li, L.; Dai, D.; Zheng, C.; Ma, J.; Li, R.; Xia, H.; Xu, J.; Wu, Z.; Liu, T.; et al. 2022. A survey on in-context learning. arXiv preprint arXiv:2301.00234

    2022 arXiv

  4. [12]

    Gehman, S.; Gururangan, S.; Sap, M.; Choi, Y.; and Smith, N. A. 2020. Realtoxicityprompts: Evaluating neural toxic degeneration in language models. arXiv preprint arXiv:2009.11462

    2020 arXiv

  5. [13]

    S.; Pal, S.; Mukherjee, K.; and Manocha, D

    Ghosal, S. S.; Pal, S.; Mukherjee, K.; and Manocha, D. 2025 a . PromptRefine: Enhancing Few-Shot Performance on Low-Resource Indic Languages with Example Selection from related Example Banks. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the A...

    2025

  6. [14]

    S.; Singh, V.; Ghosh, A.; Pal, S.; Baidya, S.; Saha, S.; and Manocha, D

    Ghosal, S. S.; Singh, V.; Ghosh, A.; Pal, S.; Baidya, S.; Saha, S.; and Manocha, D. 2025 b . Relic: Enhancing Reward Model Generalization for Low-Resource Indic Languages with Few-Shot Examples. arXiv preprint arXiv:2506.16502

    2025 arXiv

  7. [15]

    Gonen, H.; Hosking, J.; and Augenstein, I. 2023. Perplexity-Based Prompt Selection for Large Language Models. In Proceedings of the 2023 Conference of the Association for Computational Linguistics

    2023

  8. [16]

    Grattafiori, A.; Dubey, A.; Jauhri, A.; Pandey, A.; Kadian, A.; Al-Dahle, A.; Letman, A.; Mathur, A.; Schelten, A.; and Alex Vaughan, e. a. 2024. The Llama 3 Herd of Models. arXiv:2407.21783

    2024 arXiv

  9. [17]

    Hui, B.; Yang, J.; Cui, Z.; Yang, J.; Liu, D.; Zhang, L.; Liu, T.; Zhang, J.; Yu, B.; Lu, K.; Dang, K.; Fan, Y.; Zhang, Y.; Yang, A.; Men, R.; Huang, F.; Zheng, B.; Miao, Y.; Quan, S.; Feng, Y.; Ren, X.; Ren, X.; Zhou, J.; and Lin, J. 2024. Qwen2.5-Coder Technical Report. arXi...

    2024 arXiv

  10. [18]

    K \"o ppen, M. 2000. The curse of dimensionality. In 5th online world conference on soft computing in industrial applications (WSC5), volume 1, 4--8

    2000

  11. [19]

    J.; Clarke, C.; Lee, A.; Hill, P.; Kummerfeld, J

    Larson, S.; Mahendran, A.; Peper, J. J.; Clarke, C.; Lee, A.; Hill, P.; Kummerfeld, J. K.; Leach, K.; Laurenzano, M. A.; Tang, L.; et al. 2019. An evaluation dataset for intent classification and out-of-scope prediction. arXiv preprint arXiv:1909.02027

    2019 arXiv

  12. [20]

    Lattimore, T.; and Szepesv \'a ri, C. 2020. Bandit algorithms. Cambridge University Press

    2020

  13. [21]

    Li, L.; and Qiu, M. 2023. In-Context Learning Demonstration Selection via Influence Analysis. arXiv:2402.11750

    2023 arXiv

  14. [22]

    Li, X.; Lv, K.; Yan, H.; Lin, T.; Zhu, W.; Ni, Y.; Xie, G.; Wang, X.; and Qiu, X. 2023 a . Unified demonstration retriever for in-context learning. arXiv preprint arXiv:2305.04320

    2023 arXiv

  15. [23]

    E.; Papailiopoulos, D.; and Oymak, S

    Li, Y.; Ildiz, M. E.; Papailiopoulos, D.; and Oymak, S. 2023 b . Transformers as algorithms: Generalization and stability in in-context learning. In International Conference on Machine Learning, 19565--19594. PMLR

    2023

  16. [24]

    Liu, J.; Shen, D.; Zhang, Y.; Dolan, B.; Carin, L.; and Chen, W. 2021. What Makes Good In-Context Examples for GPT- 3 ? arXiv preprint arXiv:2101.06804

    2021 arXiv

  17. [25]

    Liu, J.; Shen, D.; Zhang, Y.; Dolan, B.; Carin, L.; and Chen, W. 2022. What Makes Good In-Context Examples for GPT-3? In Proceedings of the 3rd Workshop on Deep Learning for Low-Resource Natural Language Processing (DeepLo) at ACL, 100--114

    2022

  18. [26]

    Luo, M.; Xu, X.; Dai, Z.; Pasupat, P.; Kazemi, M.; Baral, C.; Imbrasaite, V.; and Zhao, V. Y. 2023. Dr. icl: Demonstration-retrieved in-context learning. arXiv preprint arXiv:2305.14128

    2023 arXiv

  19. [27]

    Luo, M.; Xu, X.; Liu, Y.; Pasupat, P.; and Kazemi, M. 2024. In-context learning with retrieved demonstrations for language models: A survey. arXiv preprint arXiv:2401.11624

    2024 arXiv

  20. [28]

    Min, S.; Lewis, M.; Zettlemoyer, L.; and Hajishirzi, H. 2021. Metaicl: Learning to learn in context. arXiv preprint arXiv:2110.15943

    2021 arXiv

  21. [29]

    Min, S.; Lyu, X.; Holtzman, A.; Artetxe, M.; Lewis, M.; Hajishirzi, H.; and Zettlemoyer, L. 2022. Rethinking the Role of Demonstrations: What Makes In-Context Learning Work? In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 11048--11064

    2022

  22. [30]

    Olsson, C.; Elhage, N.; Nanda, N.; Joseph, N.; DasSarma, N.; Henighan, T.; Mann, B.; Askell, A.; Bai, Y.; Chen, A.; et al. 2022. In-context learning and induction heads. arXiv preprint arXiv:2209.11895

    2022 arXiv

  23. [31]

    Pan, J. 2023. What in-context learning “learns” in-context: Disentangling task recognition and task learning. Master's thesis, Princeton University

    2023

  24. [32]

    Pukelsheim, F. 2006. Optimal design of experiments. SIAM

    2006

  25. [33]

    Robertson, S.; Zaragoza, H.; et al. 2009. The probabilistic relevance framework: BM25 and beyond. Foundations and Trends in Information Retrieval , 3(4): 333--389

    2009

  26. [34]

    Rubin, O.; Herzig, J.; and Berant, J. 2022. Learning to Retrieve Prompts for In-Context Learning. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics, 2655--2671

    2022

  27. [35]

    Saunshi, N.; Malladi, S.; and Arora, S. 2020. A mathematical exploration of why language models help solve downstream tasks. arXiv preprint arXiv:2010.03648

    2020 arXiv

  28. [36]

    Sch \"o lkopf, B.; and Smola, A. J. 2002. Learning with kernels: support vector machines, regularization, optimization, and beyond. MIT press

    2002

  29. [37]

    Shin, S.; Lee, S.-W.; Ahn, H.; Kim, S.; Kim, H.; Kim, B.; Cho, K.; Lee, G.; Park, W.; Ha, J.-W.; et al. 2022. On the effect of pretraining corpora on in-context learning by a large-scale language model. arXiv preprint arXiv:2204.13509

    2022 arXiv

  30. [38]

    D.; Ng, A.; and Potts, C

    Socher, R.; Perelygin, A.; Wu, J.; Chuang, J.; Manning, C. D.; Ng, A.; and Potts, C. 2013. Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank. In Yarowsky, D.; Baldwin, T.; Korhonen, A.; Livescu, K.; and Bethard, S., eds., Proceedings of the 2013 Con...

    2013

  31. [39]

    Sorensen, M.; Ding, N.; and Chang, M.-W. 2022. Information-Theoretic Demonstration Selection for Few-Shot Prompting. In Proceedings of the Findings of EMNLP

    2022

  32. [40]

    Talmor, A.; Herzig, J.; Lourie, N.; and Berant, J. 2019. C ommonsense QA : A Question Answering Challenge Targeting Commonsense Knowledge. In Burstein, J.; Doran, C.; and Solorio, T., eds., Proceedings of the 2019 Conference of the North A merican Chapter of the Association fo...

    2019

  33. [41]

    Wang, A.; Pruksachatkun, Y.; Nangia, N.; Singh, A.; Michael, J.; Hill, F.; Levy, O.; and Bowman, S. 2019. Superglue: A stickier benchmark for general-purpose language understanding systems. Advances in neural information processing systems, 32

    2019

  34. [42]

    Wang, A.; Singh, A.; Michael, J.; Hill, F.; Levy, O.; and Bowman, S. 2018. GLUE : A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding. In Linzen, T.; Chrupa a, G.; and Alishahi, A., eds., Proceedings of the 2018 EMNLP Workshop B lackbox NLP : Analyz...

    2018

  35. [43]

    Wang, D.; Dou, L.; and Che, W. 2022. A survey on table-and-text hybridqa: Concepts, methods, challenges and future directions. arXiv preprint arXiv:2212.13465

    2022 arXiv

  36. [44]

    Wang, L.; Chen, S.; Jiang, L.; Pan, S.; Cai, R.; Yang, S.; and Yang, F. 2025. Parameter-efficient fine-tuning in large language models: a survey of methodologies. Artificial Intelligence Review, 58(8): 227

    2025

  37. [45]

    Wang, X.; Zhu, W.; Saxon, M.; Steyvers, M.; and Wang, W. Y. 2023. Large Language Models are Latent Variable Models: Explaining and Finding Good Demonstrations for In-Context Learning. In Advances in Neural Information Processing Systems 36

    2023

  38. [46]

    Wei, J.; Hou, L.; Lampinen, A.; Chen, X.; Huang, D.; Tay, Y.; Chen, X.; Lu, Y.; Zhou, D.; Ma, T.; et al. 2023 a . Symbol tuning improves in-context learning in language models. arXiv preprint arXiv:2305.08298

    2023 arXiv

  39. [47]

    Wei, J.; Tay, Y.; Bommasani, R.; Raffel, C.; Zoph, B.; Borgeaud, S.; Yogatama, D.; Bosma, M.; Zhou, D.; Metzler, D.; et al. 2022. Emergent abilities of large language models. arXiv preprint arXiv:2206.07682

    2022 arXiv

  40. [48]

    Wei, J.; Wei, J.; Tay, Y.; Tran, D.; Webson, A.; Lu, Y.; Chen, X.; Liu, H.; Huang, D.; Zhou, D.; et al. 2023 b . Larger language models do in-context learning differently. arXiv preprint arXiv:2303.03846

    2023 arXiv

  41. [49]

    Williams, A.; Nangia, N.; and Bowman, S. R. 2017. A broad-coverage challenge corpus for sentence understanding through inference. arXiv preprint arXiv:1704.05426

    2017 arXiv

  42. [50]

    Wu, Z.; Wang, Y.; Ye, J.; and Kong, L. 2022. Self-adaptive in-context learning: An information compression perspective for in-context example selection and ordering. arXiv preprint arXiv:2212.10375

    2022 arXiv

  43. [51]

    M.; Raghunathan, A.; Liang, P.; and Ma, T

    Xie, S. M.; Raghunathan, A.; Liang, P.; and Ma, T. 2021. An explanation of in-context learning as implicit bayesian inference. arXiv preprint arXiv:2111.02080

    2021 arXiv

  44. [52]

    Yang, Z.; Zhang, Y.; Sui, D.; Liu, C.; Zhao, J.; and Liu, K. 2023. Representative Demonstration Selection for In-Context Learning with Two-Stage Determinantal Point Process. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 5443--5456

    2023

  45. [53]

    Ye, J.; Wu, Z.; Feng, J.; Yu, T.; and Kong, L. 2023 a . Compositional Exemplars for In-Context Learning. In Proceedings of the 40th International Conference on Machine Learning. ICML

    2023

  46. [54]

    Ye, J.; Wu, Z.; Feng, J.; Yu, T.; and Kong, L. 2023 b . Compositional exemplars for in-context learning. In International Conference on Machine Learning, 39818--39833. PMLR

    2023

  47. [55]

    M.; Kim, J.; Kim, H

    Yoo, K. M.; Kim, J.; Kim, H. J.; Cho, H.; Jo, H.; Lee, S.-W.; Lee, S.-g.; and Kim, T. 2022. Ground-truth labels matter: A deeper look into input-label demonstrations. arXiv preprint arXiv:2205.12685

    2022 arXiv

  48. [56]

    Zellers, R.; Holtzman, A.; Bisk, Y.; Farhadi, A.; and Choi, Y. 2019. H ella S wag: Can a Machine Really Finish Your Sentence? In Korhonen, A.; Traum, D.; and M \`a rquez, L., eds., Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 4791--4...

    2019

  49. [57]

    Zhang, Z.; Rudra, K.; and Anand, A. 2021. Explain and predict, and then predict again. In Proceedings of the 14th ACM international conference on web search and data mining, 418--426

    2021

  50. [58]

    Zhao, Z.; Wallace, E.; Feng, S.; Klein, D.; and Singh, S. 2021. Calibrate before use: Improving few-shot performance of language models. In International conference on machine learning, 12697--12706. PMLR

    2021

  51. [59]

    Zheng, L.; Chiang, W.-L.; Sheng, Y.; Zhuang, S.; Wu, Z.; Zhuang, Y.; Lin, Z.; Li, Z.; Li, D.; Xing, E.; et al. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in neural information processing systems, 36: 46595--46623

    2023

  52. [60]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all...

  53. [61]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.