Pith. sign in

REVIEW 3 major objections 5 minor 45 references

PICLe: Pseudo-Annotations for In-Context Learning in Low-Resource Named Entity Detection

T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read For in-context named entity detection, partially correct demonstration annotations transfer as effectively as gold ones, and a zero-human pipeline exploiting this beats few-shot ICL with 100 gold examples.

desk verdict The perturbation study is the real contribution; PICLe works on average, but the paper never proves its pseudo-annotations sit in the regime where partial correctness helps. read the letter →

arxiv 2412.11923 v2 pith:AHCV2UE7 submitted 2024-12-16 cs.CL cs.AI

Sepideh Mamooler , Syrielle Montariol , Alexander Mathis , Antoine Bosselut This is my paper · ORCID
classification cs.CLcs.AI
keywords in-contextlearningnamedentitydetectionpseudo-annotationself-verificationlow-resourceNLPbiomedicaldemonstrationselectionLLMprompting
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks what makes in-context demonstrations work for low-resource named entity detection, and reports a surprising answer: demonstrations whose entity annotations are only partially correct can transfer as effectively as fully correct gold annotations, as long as the label sets keep enough correct mentions and a semantic link to the input text. It establishes this with controlled perturbation studies that vary the type and degree of label corruption. Building on that finding, it introduces PICLe, a fully automatic pipeline that pseudo-annotates unlabeled sentences with a zero-shot LLM pass, self-verifies those annotations, clusters them, and samples cluster-specific demonstration sets for inference. On five biomedical NED datasets, PICLe with zero human annotation matches or outperforms standard ICL that uses up to 100 human-annotated demonstrations, suggesting that annotation effort can be redirected or eliminated in low-resource structured-output tasks.

What carries the argument

The central object is the demonstration perturbation curve of Section 4.2, which maps the precision, recall, and F1 of demonstration label sets to downstream prediction F1 and shows that performance is governed more by the number of surviving correct entity mentions and the text-label semantic correspondence than by the label set F1 itself. On the method side, the machinery is PICLe's pipeline: zero-shot pseudo-annotation, a self-verification step that filters wrong-type entities, k-means clustering of pseudo-annotated samples, and cluster-specific demonstration sampling (Sp-k-means) that produces diverse, specialized demonstration sets whose independent predictions are merged and self-verified at inference time.

What would settle it

Compute the mean number of correct entity mentions in the actual demonstration sets PICLe samples via Sp-k-means on a new dataset, and compare it to the threshold in the perturbation study at which downstream F1 falls to zero-shot levels for deletion-type noise; if the sampled sets fall below that threshold yet PICLe still claims gains, the central claim is falsified. Concretely, take a held-out dataset, run PICLe, and also run ICL with deletion-corrupted gold annotations matched to PICLe's demonstration F1; if deletion-corrupted ICL outperforms PICLe, the partial-correctness mechanism is not the operative one.

Watch

Extended reading notes

Core claim

In-context learning for named entity detection is robust to noise in the demonstration labels, provided the demonstrations preserve a semantic correspondence between text and labels and retain a sufficient number of correct entity mentions. Through perturbation experiments that delete, substitute, or add entities with varying probabilities, the authors show that deletion-heavy noise degrades downstream F1 most sharply, while substitution and addition schemes keep prediction F1 close to gold-label ICL even at high perturbation rates; fully incorrect labels (random in-distribution, out-of-distribution, or swapped) drop performance below zero-shot. The paper's central claim is that this regime—partial correctness with adequate coverage—is exactly what zero-shot pseudo-annotations occupy, making it possible to replace gold demonstrations with automatically generated ones.

Load-bearing premise

The method assumes that the noise in zero-shot pseudo-annotations (after self-verification) resembles the substitution or addition perturbations that keep enough correct mentions, not deletion-heavy noise; if a new domain yields pseudo-annotations with very few correct mentions, the transfer benefit should disappear.

Editorial extensions

If this is right

  • If partial correctness suffices, low-resource NED no longer requires gold annotations for ICL; a practitioner can bootstrap from unlabeled text with zero human effort.
  • The perturbation results indicate that demonstration selection for structured-output tasks should maximize the number of correct entity mentions and preserve text-label correspondence, not maximize demonstration F1.
  • The finding that fully incorrect labels hurt NED (contrary to classification) implies that pseudo-annotation quality control should target the precision of surviving mentions, not just label-space coverage.
  • PICLe's gains hold across three LLMs and five biomedical datasets, suggesting the approach transfers across base models and entity types.
  • The comparison with fine-tuning suggests that when gold samples exceed roughly 50, supervised fine-tuning becomes more effective, so PICLe's niche is the truly annotation-scarce regime.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The deletion-sensitivity result implies that a practical improvement to PICLe would be to generate pseudo-annotations with higher recall (e.g., multiple zero-shot passes with diverse temperatures) before self-verification, since the perturbation study shows that the recall of the demonstration pool is what sustains downstream F1.
  • Because word-order corruption in demonstrations barely hurt performance, the paper's data are consistent with the hypothesis that NED demonstrations act primarily as format and entity-type priors rather than syntactic templates; this could be tested by shuffling word order in real PICLe demonstrations and measuring the F1 change.
  • The saturation of downstream F1 when switching from Mistral to GPT-3.5-Turbo pseudo-annotations suggests the inference-time aggregation (Sp-k-means plus self-verification) is the current bottleneck; varying the cluster count or the number of sampled demonstrations per cluster would directly test that.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This paper studies which attributes of in-context demonstrations matter for low-resource named entity detection (NED) with LLMs. Through a perturbation study on gold demonstrations, the authors find that semantic input-label correspondence is essential but that partially correct labels can be as effective as fully correct labels, provided a sufficient number of entity mentions remain. Based on this, they propose PICLe, which builds a pool of pseudo-annotated samples via zero-shot LLM prediction, self-verification, and k-means clustering, then samples cluster-specific demonstrations (Sp-k-means) to predict entities and self-verifies the aggregated output. They evaluate on five biomedical NED datasets with three LLMs, reporting that PICLe outperforms zero-shot and gold-demonstration ICL with up to 100 gold examples. The paper includes ablations of each pipeline component and a model generalization study.

Significance. If the claimed result holds, PICLe would provide a practical way to reduce annotation cost for structured extraction tasks, and the perturbation analysis offers a useful counterpoint to prior work emphasizing the necessity of correct labels in ICL. The paper includes a component ablation (Table 3), an analysis of pseudo-annotation quality (Table 5), and multi-model/multi-dataset evaluation, which strengthen the empirical contribution. However, the central claim's support is currently incomplete: error bars are missing for PICLe, the connection between the perturbation curves and the actual pseudo-annotations is not quantified, and the comparison with gold ICL is confounded by the size of the demonstration pool. These issues are addressable but require additional experiments.

major comments (3)
  1. [§5 PICLe performance, Table 5 vs. §4.2 Figure 2] The paper never locates the actual pseudo-annotated demonstration sets on the perturbation curves of Figure 2. Table 5 reports that after self-verification the pseudo-annotation pool has an average recall of only 37.5%, which corresponds to a deletion-heavy regime in Figure 2 where downstream F1 decays most rapidly. Since Sp-k-means samples from this pool, the paper should report the precision, recall, and average number of correct gold mentions in the K demonstration sets actually used for inference and compare these values with the curves in Figure 2. Without such a comparison, the stated explanation of PICLe's benefit (that partially correct labels help ICL) is not verified for the actual operating point.
  2. [§5 Figure 4 and Table 4] PICLe results are presented without error bars or significance tests, despite the claim that PICLe 'significantly outperforms' zero-shot and gold ICL. The gold baselines show error bars over 5 seeds, and Section 5 states that PICLe experiments were also repeated over 5 seeds, so the variance should be reported for PICLe as well. The headline improvement of 57.1% vs. 52.8% is a macro-average over five datasets; per-seed results or a paired significance test are needed to support a 'significant outperformance' claim.
  3. [§5 Experimental setup and Figure 4] The comparison between PICLe and gold ICL is confounded by the size of the demonstration pool: gold baselines sample from pools of 10, 50, or 100 labeled examples, while PICLe pools 1000 unlabeled (pseudo-annotated) training sentences. A control that uses the same 1000 sentences with gold labels (an oracle upper bound) would isolate the effect of label noise from the number and coverage of demonstration texts. Without this control, the observed advantage could come from larger demonstration diversity rather than from the method's tolerance to partial correctness, which would materially weaken the paper's central claim.
minor comments (5)
  1. [Appendix A Reproducibility statement] The statement says 'We plan to share the code for PICLe and all of our experiments'; for a reproducibility statement in a methods-heavy paper, actual code release at submission would substantially strengthen the contribution.
  2. [§5, clustering description] The text says pseudo-annotated samples are clustered 'based off the embedding of their text and pseudo-annotations' but does not specify how the pseudo-annotations are embedded (e.g., as a concatenated string, or via a separate encoder). This should be clarified for reproducibility.
  3. [Table 1 caption] The column 'Ratio null samples (%)' is defined in the table note as samples with no labeled entities, but this definition should also appear in the caption for a self-contained table.
  4. [Throughout] The phrase 'based off our findings' appears several times; the conventional phrasing is 'based on our findings'.
  5. [Figure 4 caption] The caption states that error bars show variance only for sampling subsets of gold examples; please state explicitly that PICLe results are reported without error bars (or add them) so readers can assess the comparison fairly.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: PICLe's claims are evaluated against independent gold test labels, and no load-bearing step reduces to its own inputs.

full rationale

The paper's derivation chain is empirical rather than formal. The perturbation study in Section 4.2 measures downstream F1 against independently held gold test labels while varying hand-perturbed gold demonstrations; the finding that partially correct labels can suffice is therefore an externally measured relation, not an identity. PICLe then applies zero-shot pseudo-annotations, self-verification, and Sp-k-means sampling, and its 57.1 vs. 52.8 result (Table 4) is scored against the same gold test labels, with no parameter fitted to those labels and no prediction constructed from its own evaluation targets. The author-overlapping citations (e.g., Fei et al. 2023; Ye et al. 2023, 2024) are background references and are not invoked to justify a load-bearing premise or to forbid alternatives. The abstract's claim that partially correct annotations can be as effective as fully correct demonstrations is not assumed into PICLe's design; it is tested separately in Section 4.2 before PICLe is introduced. A legitimate scientific weakness exists, namely that the paper does not locate PICLe's actual pseudo-annotation operating point (average recall 37.5%, Table 5) on the Section 4.2 perturbation curves, but that is a quantification gap, not a circular step.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The method introduces no new theoretical entities, forces, or conserved quantities; it reuses existing LLM capabilities, standard clustering, and self-verification prompts. The central claim rests on hand-chosen hyperparameters and on empirical assumptions about LLM behavior that are tested only on biomedical NED.

free parameters (5)
  • K (number of clusters) = 10 (inferred)
    Sp-k-means uses K clusters; the main text does not specify K, but Appendix efficiency analysis implies 10 rounds.
  • N_pseudo (pseudo-annotation pool size) = 1000
    Number of unlabeled training samples pseudo-annotated; chosen by hand (Section 5).
  • demonstrations per cluster = not reported
    The number of in-context examples sampled from each cluster is not stated in the method or experiments.
  • embedding model for clustering = S-PubMedBert-MS-MARCO
    Sentence transformer used for k-means; no comparison or sensitivity analysis reported.
  • decoding temperature = greedy for pseudo-annotation; T=0.7/0.8 in ablations
    Greedy decoding is used for PICLe's pseudo-annotation; the inference decoding temperature is not explicitly specified.
assumptions (4)
  • domain assumption LLMs possess sufficient zero-shot NED ability and respond to in-context demonstrations
    The entire pipeline depends on the model's existing NED skill (Section 5).
  • domain assumption Exact-match span F1 on five biomedical datasets is a valid NED evaluation
    Evaluation methodology (Section 3).
  • ad hoc to paper Embedding-space clusters correspond to useful demonstration regimes
    Sp-k-means assumes each cluster yields coherent, diverse demonstrations; this is not independently validated.
  • domain assumption Self-verification prompts produce accurate entity-type judgments
    The model filters its own predictions with yes/no prompts (Section 5, Figure 9).

how reviews work

0 comments
Cite this review

Pith. "Pith review of PICLe: Pseudo-Annotations for In-Context Learning in Low-Resource Named Entity Detection." pith.science (2026). https://pith.science/paper/AHCV2UE7

@misc{pith2026241211923,
  author       = {Pith},
  title        = {Pith review of: PICLe: Pseudo-Annotations for In-Context Learning in Low-Resource Named Entity Detection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AHCV2UE7}},
  note         = {Machine review of arXiv:2412.11923}
}
read the original abstract

In-context learning (ICL) enables Large Language Models (LLMs) to perform tasks using few demonstrations, facilitating task adaptation when labeled examples are hard to obtain. However, ICL is sensitive to the choice of demonstrations, and it remains unclear which demonstration attributes enable in-context generalization. In this work, we conduct a perturbation study of in-context demonstrations for low-resource Named Entity Detection (NED). Our surprising finding is that in-context demonstrations with partially correct annotated entity mentions can be as effective for task transfer as fully correct demonstrations. Based off our findings, we propose Pseudo-annotated In-Context Learning (PICLe), a framework for in-context learning with noisy, pseudo-annotated demonstrations. PICLe leverages LLMs to annotate many demonstrations in a zero-shot first pass. We then cluster these synthetic demonstrations, sample specific sets of in-context demonstrations from each cluster, and predict entity mentions using each set independently. Finally, we use self-verification to select the final set of entity mentions. We evaluate PICLe on five biomedical NED datasets and show that, with zero human annotation, PICLe outperforms ICL in low-resource settings where limited gold examples can be used as in-context demonstrations.

Figures

Figures reproduced from arXiv: 2412.11923 by the authors.

Figure 1
Figure 1. 10-shot ICL performance using various demonstration corruption schemes, with Mistral and kNN demonstration retrieval. We compare to zero-shot and 10-shot with gold demonstrations, averaging over all datasets. ( [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. 10-shot ICL performance with perturbed demonstrations with different perturbation schemes and using Mistral and kNN demonstration retrieval. We report the prediction F1 as a function of the precision, recall, and F1 of the perturbed demonstration label sets (relative to the gold demonstrations) averaged over all datasets. The size of the points shows the average number of entities in the label sets of the perturbed … view at source ↗
Figure 3
Figure 3. PICLe pipeline. Unlabeled samples are pseudo-annotated through a zero-shot prediction and self￾verification pass. Subsequently, they are clustered, and cluster-specific sets of ICL demonstrations are chosen at random from each group. Each set is independently used to find entity mentions in the query, and the final set of entity mentions is obtained by aggregating these independent sets and asking the model to verif… view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Performance of PICLe, zero-shot, and 10-shot ICL with gold demonstrations selected from 10, 50, 100 gold examples using Mistral. The error bars show the variance across 5 seeds for sampling subsets of gold examples. All methods are followed by self-verification. PICLe …
Figure 5
Figure 5. Figure 5: 10-shot ICL performance using various demonstration corruption schemes, compared with zero￾shot and ICL with gold annotations, for each dataset. Experiments performed using Mistral (top) and GPT-3.5- Turbo (bottom) and kNN demonstration retrieval. (Best viewed in color…
Figure 6
Figure 6. Figure 6: 10-shot ICL performance with partially correct demonstrations with different perturbation schemes using Mistral and kNN demonstration retrieval. We observe the impact on prediction F1 score of the perturbation factor and the number of entities in the demonstrations for…
Figure 7
Figure 7. Figure 7: 10-shot ICL performance with partially correct demonstrations with different perturbation schemes using Mistral and kNN demonstration retrieval. We observe the impact of the demonstration precision and recall on the downstream prediction precision and recall, respectiv…
Figure 8
Figure 8. Figure 8: 10-shot NED with gold annotations with Mistral using different demonstration retrieval methods: k-means, kNN, and random [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]
Figure 9
Figure 9. Figure 9: Example of prompt used for self-verification. Dataset: BC5-Chem [PITH_FULL_IMAGE:figures/full_fig_p017_9.png]
Figure 10
Figure 10. Figure 10: Example of prompt used for NED. Dataset: BC5-Chem [PITH_FULL_IMAGE:figures/full_fig_p018_10.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

45 extracted references · 13 canonical work pages

  1. [1]

    O'Reilly Media, Inc

    Steven Bird, Ewan Klein, and Edward Loper. 2009. Natural language processing with Python: analyzing text with the natural language toolkit. " O'Reilly Media, Inc."

    2009

  2. [2]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877--1901

    2020

  3. [3]

    Jiawei Chen, Yaojie Lu, Hongyu Lin, Jie Lou, Wei Jia, Dai Dai, Hua Wu, Boxi Cao, Xianpei Han, and Le Sun. 2023 a . https://doi.org/10.18653/v1/2023.acl-long.764 Learning in-context learning for named entity recognition . In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 13661--13675, ...

    doi:10.18653/v1/2023.acl-long.764 2023

  4. [4]

    Wei-Lin Chen, Cheng-Kuang Wu, Yun-Nung Chen, and Hsin-Hsi Chen. 2023 b . https://doi.org/10.18653/v1/2023.emnlp-main.968 Self- ICL : Zero-shot in-context learning with self-generated demonstrations . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 15651--15662, Singapore. Association for Computational Linguistics

    doi:10.18653/v1/2023.emnlp-main.968 2023

  5. [5]

    Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Zhiyong Wu, Baobao Chang, Xu Sun, Jingjing Xu, and Zhifang Sui. 2022. A survey for in-context learning. arXiv, abs/2301.00234

    arXiv 2022

  6. [6]

    Yu Fei, Yifan Hou, Zeming Chen, and Antoine Bosselut. 2023. https://doi.org/10.18653/v1/2023.acl-long.783 Mitigating label biases for in-context learning . In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 14014--14031, Toronto, Canada. Association for Computational Linguistics

    doi:10.18653/v1/2023.acl-long.783 2023

  7. [7]

    Tianyu Gao, Adam Fisch, and Danqi Chen. 2021. https://doi.org/10.18653/v1/2021.acl-long.295 Making pre-trained language models better few-shot learners . In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 3816--...

    doi:10.18653/v1/2021.acl-long.295 2021

  8. [8]

    Or Honovich, Thomas Scialom, Omer Levy, and Timo Schick. 2023. https://doi.org/10.18653/v1/2023.acl-long.806 Unnatural instructions: Tuning language models with (almost) no human labor . In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 14409--14428, Toronto, Canada. Association for C...

    doi:10.18653/v1/2023.acl-long.806 2023

Show all 45 references
  1. [9]

    Jiaxin Huang, Shixiang Gu, Le Hou, Yuexin Wu, Xuezhi Wang, Hongkun Yu, and Jiawei Han. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.67 Large language models can self-improve . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages...

    2023 doi

  2. [10]

    Albert Qiaochu Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de Las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, L'elio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, T...

    2023 arXiv

  3. [11]

    Zhongtao Jiang, Yuanzhe Zhang, Cao Liu, Jun Zhao, and Kang Liu. 2023 b . https://doi.org/10.18653/v1/2023.findings-emnlp.152 Generative calibration for in-context learning . In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 2312--2333, Singapore. ...

    2023 doi

  4. [12]

    Dong-Ho Lee, Akshen Kadakia, Kangmin Tan, Mahak Agarwal, Xinyu Feng, Takashi Shibuya, Ryosuke Mitani, Toshiyuki Sekiya, Jay Pujara, and Xiang Ren. 2022. https://doi.org/10.18653/v1/2022.acl-long.192 Good examples make a faster learner: Simple demonstration-based learning for l...

    2022 doi

  5. [13]

    Johnson, Daniela Sciaky, Chih-Hsuan Wei, Robert Leaman, Allan Peter Davis, Carolyn J

    Jiao Li, Yueping Sun, Robin J. Johnson, Daniela Sciaky, Chih-Hsuan Wei, Robert Leaman, Allan Peter Davis, Carolyn J. Mattingly, Thomas C. Wiegers, and Zhiyong Lu. 2016. Biocreative V CDR task corpus: a resource for chemical disease relation extraction. Database: The Journal of...

    2016

  6. [14]

    Xiaonan Li and Xipeng Qiu. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.392 M o T : Memory-of-thought enables C hat GPT to self-improve . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 6354--6374, Singapore. Association fo...

    2023 doi

  7. [15]

    Jiachang Liu, Dinghan Shen, Yizhe Zhang, Bill Dolan, Lawrence Carin, and Weizhu Chen. 2021. What makes good in-context examples for gpt-3? In Workshop on Knowledge Extraction and Integration for Deep Learning Architectures; Deep Learning Inside Out

    2021

  8. [16]

    Jiachang Liu, Dinghan Shen, Yizhe Zhang, Bill Dolan, Lawrence Carin, and Weizhu Chen. 2022. https://doi.org/10.18653/v1/2022.deelio-1.10 What makes good in-context examples for GPT -3? In Proceedings of Deep Learning Inside Out (DeeLIO 2022): The 3rd Workshop on Knowledge Extr...

    2022 doi

  9. [17]

    Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel, and Pontus Stenetorp. 2022. https://doi.org/10.18653/v1/2022.acl-long.556 Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity . In Proceedings of the 60th Annual Meeting of th...

    2022 doi

  10. [18]

    Xinxi Lyu, Sewon Min, Iz Beltagy, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2023. https://doi.org/10.18653/v1/2023.acl-long.129 Z - ICL : Zero-shot in-context learning with pseudo-demonstrations . In Proceedings of the 61st Annual Meeting of the Association for Computational ...

    2023 doi

  11. [19]

    Yubo Ma, Yixin Cao, Yong Hong, and Aixin Sun. 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.710 Large language model is not a good few-shot information extractor, but a good reranker for hard samples! In Findings of the Association for Computational Linguistics: EMNLP ...

    2023 doi

  12. [20]

    Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2022. https://doi.org/10.18653/v1/2022.emnlp-main.759 Rethinking the role of demonstrations: What makes in-context learning work? In Proceedings of the 2022 Conference on ...

    2022 doi

  13. [21]

    Swaroop Mishra, Daniel Khashabi, Chitta Baral, Yejin Choi, and Hannaneh Hajishirzi. 2022. https://doi.org/10.18653/v1/2022.findings-acl.50 Reframing instructional prompts to GPT k ' s language . In Findings of the Association for Computational Linguistics: ACL 2022, pages 589-...

    2022 doi

  14. [22]

    David Fraile Navarro, Kiran Ijaz, Dana Rezazadegan, Hania Rahimi-Ardabili, Mark Dras, Enrico Coiera, and Shlomo Berkovsky. 2023. Clinical named entity recognition and relation extraction using natural language processing of medical free text: A systematic review. International...

    2023

  15. [23]

    Yasaman Razeghi, Robert L Logan IV, Matt Gardner, and Sameer Singh. 2022. https://doi.org/10.18653/v1/2022.findings-emnlp.59 Impact of pretraining term frequencies on few-shot numerical reasoning . In Findings of the Association for Computational Linguistics: EMNLP 2022, pages...

    2022 doi

  16. [24]

    Yongliang Shen, Zeqi Tan, Shuhui Wu, Wenqi Zhang, Rongsheng Zhang, Yadong Xi, Weiming Lu, and Yueting Zhuang. 2023. https://doi.org/10.18653/v1/2023.acl-long.698 P rompt NER : Prompt locating and typing for named entity recognition . In Proceedings of the 61st Annual Meeting o...

    2023 doi

  17. [25]

    Stavroula Skylaki, Ali Oskooei, Omar Bari, Nadja Herger, and Zac Kriegman. 2020. Named entity recognition in the legal domain using a pointer generator network. arXiv preprint arXiv:2012.09936

    2020 arXiv

  18. [26]

    Smith, Lorraine K

    Larry L. Smith, Lorraine K. Tanabe, Rie Ando, Cheng-Ju Kuo, I-Fang Chung, Chun-Nan Hsu, Yu-Shi Lin, Roman Klinger, C. Friedrich, Kuzman Ganchev, Manabu Torii, Hongfang Liu, Barry Haddow, Craig A. Struble, Richard J. Povinelli, Andreas Vlachos, William A. Baumgartner, Lawrence ...

    2008

  19. [27]

    Taylor Sorensen, Joshua Robinson, Christopher Rytting, Alexander Shaw, Kyle Rogers, Alexia Delorey, Mahmoud Khalil, Nancy Fulda, and David Wingate. 2022. https://doi.org/10.18653/v1/2022.acl-long.60 An information-theoretic approach to prompt engineering without ground truth l...

    2022 doi

  20. [28]

    Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R Brown, Adam Santoro, Aditya Gupta, Adri \`a Garriga-Alonso, et al. 2023. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. Transac...

    2023

  21. [29]

    Simeng Sun, Kalpesh Krishna, Andrew Mattarella-Micke, and Mohit Iyyer. 2021. https://doi.org/10.18653/v1/2021.emnlp-main.62 Do long-range language models actually use long-range context? In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing,...

    2021 doi

  22. [30]

    Roque, Irene Kouskoumvekaki, Alina Bora, Ramona Curpan, Thomas Sk t Jensen, S ren Brunak, and Tudor I

    Olivier Taboureau, Sonny Kim Nielsen, Karine Audouze, Nils Weinhold, Daniel Edsg \"a rd, Francisco S. Roque, Irene Kouskoumvekaki, Alina Bora, Ramona Curpan, Thomas Sk t Jensen, S ren Brunak, and Tudor I. Oprea. 2010. Chemprot: a disease chemical biology database. Nucleic Acid...

    2010

  23. [31]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288

    2023 arXiv

  24. [32]

    Asahi Ushio, Francesco Barbieri, Vitor Sousa, Leonardo Neves, and Jose Camacho-Collados. 2022. https://aclanthology.org/2022.aacl-main.25 Named entity recognition in T witter: A dataset and analysis on short-term temporal shifts . In Proceedings of the 2nd Conference of the As...

    2022

  25. [33]

    Xingchen Wan, Ruoxi Sun, Hanjun Dai, Sercan Arik, and Tomas Pfister. 2023 a . https://doi.org/10.18653/v1/2023.findings-acl.216 Better zero-shot reasoning with self-adaptive prompting . In Findings of the Association for Computational Linguistics: ACL 2023, pages 3493--3514, T...

    2023 doi

  26. [34]

    Xingchen Wan, Ruoxi Sun, Hootan Nakhost, Hanjun Dai, Julian Eisenschlos, Sercan Arik, and Tomas Pfister. 2023 b . https://doi.org/10.18653/v1/2023.emnlp-main.461 Universal self-adaptive prompting . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language ...

    2023 doi

  27. [35]

    Boshi Wang, Sewon Min, Xiang Deng, Jiaming Shen, You Wu, Luke Zettlemoyer, and Huan Sun. 2023 a . https://doi.org/10.18653/v1/2023.acl-long.153 Towards understanding chain-of-thought prompting: An empirical study of what matters . In Proceedings of the 61st Annual Meeting of t...

    2023 doi

  28. [36]

    Xinyi Wang, Wanrong Zhu, Michael Saxon, Mark Steyvers, and William Yang Wang. 2024. Large language models are latent variable models: Explaining and finding good demonstrations for in-context learning. Advances in Neural Information Processing Systems, 36

    2024

  29. [37]

    Smith, Daniel Khashabi, and Hannaneh Hajishirzi

    Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. 2023 b . https://doi.org/10.18653/v1/2023.acl-long.754 Self-instruct: Aligning language models with self-generated instructions . In Proceedings of the 61st Annual ...

    2023 doi

  30. [38]

    Wei, Jason Wei, Yi Tay, Dustin Tran, Albert Webson, Yifeng Lu, Xinyun Chen, Hanxiao Liu, Da Huang, Denny Zhou, and Tengyu Ma

    Jerry W. Wei, Jason Wei, Yi Tay, Dustin Tran, Albert Webson, Yifeng Lu, Xinyun Chen, Hanxiao Liu, Da Huang, Denny Zhou, and Tengyu Ma. 2023. https://api.semanticscholar.org/CorpusID:257378479 Larger language models do in-context learning differently . ArXiv, abs/2303.03846

    2023 arXiv

  31. [39]

    Yixuan Weng, Minjun Zhu, Fei Xia, Bin Li, Shizhu He, Shengping Liu, Bin Sun, Kang Liu, and Jun Zhao. 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.167 Large language models are better reasoners with self-verification . In Findings of the Association for Computational L...

    2023 doi

  32. [40]

    Xiangli Yang, Zixing Song, Irwin King, and Zenglin Xu. 2022. A survey on deep semi-supervised learning. IEEE Transactions on Knowledge and Data Engineering, 35(9):8934--8954

    2022

  33. [41]

    Shaokai Ye, Anastasiia Filippova, Jessy Lauer, Steffen Schneider, Maxime Vidal, Tian Qiu, Alexander Mathis, and Mackenzie Weygandt Mathis. 2024. Superanimal pretrained pose estimation models for behavioral analysis. Nature Communications, 15(1):5165

    2024

  34. [42]

    Shaokai Ye, Jessy Lauer, Mu Zhou, Alexander Mathis, and Mackenzie W Mathis. 2023. Amadeusgpt: a natural language interface for interactive animal behavioral analysis. In Thirty-seventh Conference on Neural Information Processing Systems

    2023

  35. [43]

    Kang Min Yoo, Junyeob Kim, Hyuhng Joon Kim, Hyunsoo Cho, Hwiyeol Jo, Sang-Woo Lee, Sang-goo Lee, and Taeuk Kim. 2022. https://doi.org/10.18653/v1/2022.emnlp-main.155 Ground-truth labels matter: A deeper look into input-label demonstrations . In Proceedings of the 2022 Conferen...

    2022 doi

  36. [44]

    Zhuosheng Zhang, Aston Zhang, Mu Li, and Alex Smola. 2022. Automatic chain of thought prompting in large language models. In The Eleventh International Conference on Learning Representations

    2022

  37. [45]

    Tony Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. 2021. Calibrate before use: Improving few-shot performance of language models. International Conference on Machine Learning

    2021

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.