Pith. sign in

REVIEW 5 cited by

Large Language Models as Annotators: Enhancing Generalization of NLP Models at Minimal Cost

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2306.15766 v1 pith:W6V6376T submitted 2023-06-27 cs.CL cs.LG

Parikshit Bansal , Amit Sharma This is my paper
classification cs.CLcs.LG
keywords modelsmodelsamplinginputsaccuracybasedomainsfinetuned
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

State-of-the-art supervised NLP models achieve high accuracy but are also susceptible to failures on inputs from low-data regimes, such as domains that are not represented in training data. As an approximation to collecting ground-truth labels for the specific domain, we study the use of large language models (LLMs) for annotating inputs and improving the generalization of NLP models. Specifically, given a budget for LLM annotations, we present an algorithm for sampling the most informative inputs to annotate and retrain the NLP model. We find that popular active learning strategies such as uncertainty-based sampling do not work well. Instead, we propose a sampling strategy based on the difference in prediction scores between the base model and the finetuned NLP model, utilizing the fact that most NLP models are finetuned from a base model. Experiments with classification (semantic similarity) and ranking (semantic search) tasks show that our sampling strategy leads to significant gains in accuracy for both the training and target domains.

Discussion (0). Sign in to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Evaluating Large Language Models as Expert Annotators

    cs.CL 2025-08 unverdicted novelty 6.0 of 10

    Material Fingerprinting recovers the form and parameters of hyperelastic material models by nearest-neighbor matching of test data against a simulated fingerprint database: exact at zero noise, degrading under 5% noise.

  2. Contrastive Learning on LLM Back Generation Treebank for Cross-domain Constituency Parsing

    cs.CL 2025-05 conditional novelty 6.0 of 10

    An LLM fills masked constituency trees to create a synthetic domain treebank, and span-level contrastive pre-training on it gives a new SOTA average F1 of 88.52 on cross-domain parsing.

  3. ACD-CLIP: Decoupling Representation and Dynamic Fusion for Zero-Shot Anomaly Detection

    cs.CV 2025-08 unverdicted novelty 5.0 of 10

    ACD-CLIP improves zero-shot anomaly detection by co-designing a convolutional low-rank adapter with a dynamic fusion gateway that modulates text prompts from visual context.

  4. QUEST: Query Optimization in Unstructured Document Analysis

    cs.DB 2025-07 reject novelty 5.0 of 10

    QUEST reduces LLM extraction cost in unstructured document analytics by retrieving only relevant segments via a two-level index and by generating per-document filter and join execution plans during query execution.

  5. Revisiting Active Learning under (Human) Label Variation

    cs.CL 2025-07 accept novelty 4.0 of 10

    A position paper that surveys and systematizes how active learning should change when human label variation is treated as a signal rather than noise.

Pith tools