REVIEW 8 cited by
UniversalNER: Targeted Distillation from Large Language Models for Open Named Entity Recognition
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
UniversalNER: Targeted Distillation from Large Language Models for Open Named Entity Recognition
read the original abstract
Large language models (LLMs) have demonstrated remarkable generalizability, such as understanding arbitrary entities and relations. Instruction tuning has proven effective for distilling LLMs into more cost-efficient models such as Alpaca and Vicuna. Yet such student models still trail the original LLMs by large margins in downstream applications. In this paper, we explore targeted distillation with mission-focused instruction tuning to train student models that can excel in a broad application class such as open information extraction. Using named entity recognition (NER) for case study, we show how ChatGPT can be distilled into much smaller UniversalNER models for open NER. For evaluation, we assemble the largest NER benchmark to date, comprising 43 datasets across 9 diverse domains such as biomedicine, programming, social media, law, finance. Without using any direct supervision, UniversalNER attains remarkable NER accuracy across tens of thousands of entity types, outperforming general instruction-tuned models such as Alpaca and Vicuna by over 30 absolute F1 points in average. With a tiny fraction of parameters, UniversalNER not only acquires ChatGPT's capability in recognizing arbitrary entity types, but also outperforms its NER accuracy by 7-9 absolute F1 points in average. Remarkably, UniversalNER even outperforms by a large margin state-of-the-art multi-task instruction-tuned systems such as InstructUIE, which uses supervised NER examples. We also conduct thorough ablation studies to assess the impact of various components in our distillation approach. We release the distillation recipe, data, and UniversalNER models to facilitate future research on targeted distillation.
Forward citations
Cited by 8 Pith papers
-
Decoding Text Spans for Efficient and Accurate Named-Entity Recognition
SpanDec achieves competitive NER accuracy with improved efficiency by using a final-stage lightweight decoder for span representations and early candidate filtering to reduce redundant computation.
-
Different Teachers, Different Capabilities: Sub-1B On-Device Distillation for Structured Text Enrichment
Distilling an 8B reasoning teacher into a 0.6B student recovers most summary quality at ~50× speed, but teacher type—not scale alone—determines which capabilities transfer.
-
RuleChef: Grounding LLM Task Knowledge in Human-Editable Rules
RuleChef generates and refines executable rules for NLP tasks using LLMs only at learning time to produce fast, deterministic, and inspectable rule systems.
-
RedactionBench
Introduces a 200-document benchmark and character-level R-Score for contextual PII redaction, with model evaluations and human agreement data showing the task remains unsolved.
-
IRC-Bench: Recognizing Entities from Contextual Cues in First-Person Reminiscences
IRC-Bench is a new dataset and evaluation framework for implicit entity recognition in reminiscence narratives, where entities must be inferred from non-local contextual cues across 1,994 transcripts linked to 12,337 ...
-
SAM-NER: Semantic Archetype Mediation for Zero-Shot Named Entity Recognition
SAM-NER improves cross-domain zero-shot NER by discovering entities, projecting them into domain-invariant semantic archetypes, and then calibrating those archetypes to target labels with a frozen LLM.
-
Quantifying Geospatial in the Common Crawl Corpus
Analysis estimates 18.7% of Common Crawl documents contain geospatial information like coordinates and addresses, with little difference by language.
-
FMI@SU ToxHabits: Evaluating LLMs Performance on Toxic Habit Extraction in Spanish Clinical Texts
Few-shot prompting with GPT-4.1 achieves an F1 score of 0.65 for extracting and classifying substance use mentions in Spanish clinical texts as part of the ToxHabits shared task.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.