REVIEW 13 cited by
UniversalNER: Targeted Distillation from Large Language Models for Open Named Entity Recognition
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Large language models (LLMs) have demonstrated remarkable generalizability, such as understanding arbitrary entities and relations. Instruction tuning has proven effective for distilling LLMs into more cost-efficient models such as Alpaca and Vicuna. Yet such student models still trail the original LLMs by large margins in downstream applications. In this paper, we explore targeted distillation with mission-focused instruction tuning to train student models that can excel in a broad application class such as open information extraction. Using named entity recognition (NER) for case study, we show how ChatGPT can be distilled into much smaller UniversalNER models for open NER. For evaluation, we assemble the largest NER benchmark to date, comprising 43 datasets across 9 diverse domains such as biomedicine, programming, social media, law, finance. Without using any direct supervision, UniversalNER attains remarkable NER accuracy across tens of thousands of entity types, outperforming general instruction-tuned models such as Alpaca and Vicuna by over 30 absolute F1 points in average. With a tiny fraction of parameters, UniversalNER not only acquires ChatGPT's capability in recognizing arbitrary entity types, but also outperforms its NER accuracy by 7-9 absolute F1 points in average. Remarkably, UniversalNER even outperforms by a large margin state-of-the-art multi-task instruction-tuned systems such as InstructUIE, which uses supervised NER examples. We also conduct thorough ablation studies to assess the impact of various components in our distillation approach. We release the distillation recipe, data, and UniversalNER models to facilitate future research on targeted distillation.
Forward citations
Cited by 13 Pith papers
-
Different Teachers, Different Capabilities: Sub-1B On-Device Distillation for Structured Text Enrichment
Distilling an 8B reasoning teacher into a 0.6B student recovers most summary quality at ~50× speed, but teacher type—not scale alone—determines which capabilities transfer.
-
What Matters When Building Universal Multilingual Named Entity Recognition Models?
Multilingual training data and per-language thresholds matter most for universal NER; plain binary cross-entropy and an architecture-matched backbone suffice, yielding OTTER, which outperforms similar-size models by 5...
-
Inference Gap in Domain Expertise and Machine Intelligence in Named Entity Recognition: Creation of and Insights from a Substance Use-related Dataset
Fine-tuned DeBERTa-large outperforms LLMs on extracting clinical and social impacts from opioid-use Reddit posts (relaxed token F1 0.61 vs 0.44), yet remains below human agreement (kappa 0.81).
-
GuideX: Guided Synthetic Data Generation for Zero-Shot Information Extraction
A fully automated pipeline for generating annotation schemas, guidelines, and synthetic labeled examples from documents improves zero-shot NER after fine-tuning.
-
GLiREL -- Generalist Model for Zero-Shot Relation Extraction
A single-pass encoder-scorer model with synthetic LLM pretraining matches or beats prior zero-shot relation classification methods on FewRel and on Wiki-ZSL with 10 or 15 unseen relations, while running far faster.
-
GliLem: Leveraging GliNER for Contextualized Lemmatization in Estonian
A hybrid system combining the rule-based analyzer Vabamorf with the GliNER span-labeling model reaches 97.7% lemmatization accuracy in Estonian and improves BM25 recall slightly over the default disambiguator.
-
Selective Shot Learning for Code Explanation
SSL_ner, a named-entity-based few-shot example selector, improves code explanation over token-based selection but not significantly over embedding-based selection across open-source Code-LLMs.
-
A Benchmark and Robustness Study of In-Context-Learning with Large Language Models in Music Entity Detection
Large language models with in-context learning outperform fine-tuned BERT and RoBERTa for music entity detection in user-generated content, but their edge shrinks for entities not memorized during pre-training.
-
Can AI Extract Antecedent Factors of Human Trust in AI? An Application of Information Extraction for Scientific Literature in Behavioural and Computer Sciences
The paper introduces the first English annotated corpus for extracting factors that influence human trust in AI from scientific text, and shows supervised NER and RE models outperform prompt-based LLMs.
-
Skill-based Explanations for Serendipitous Course Recommendation
A user study of skill-based explanations in a course recommender found no significant overall effect on interest, unexpectedness, or serendipity, but a significant reduction in neutral responses among undeclared students.
-
CORE-KG: An LLM-Driven Knowledge Graph Construction Framework for Human Smuggling Networks
CORE-KG reduces node duplication by 33.28% and legal noise by 38.37% versus a GraphRAG baseline on 20 human smuggling court cases, through type-aware LLM coreference resolution and domain-filtered extraction prompts.
-
KnowCoder-V2: Deep Knowledge Analysis
KnowCoder-V2 augments deep research with offline knowledge organization and code-based knowledge computation, reporting gains on information extraction, KBQA, and LLM-judged report generation.
-
Deploying Foundation Model Powered Agent Services: A Survey
This survey proposes a layered framework (execution, resource, model, agent, application) for deploying foundation-model-powered agent services across edge-cloud environments, and reviews optimization techniques at ea...
Discussion (0). Continue with ORCID to comment.