A Multi-Task Semantic Decomposition Framework with Task-specific Pre-training for Few-Shot NER

Chen Zeng; Daichi Guo; Dayuan Fu; Gang Zhao; Guanting Dong; Jinxu Zhao; Keqing He; Liwen Wang; Tingfeng Hui; Weiran Xu

RECORD

A Multi-Task Semantic Decomposition Framework with Task-specific Pre-training for Few-Shot NER

Guanting Dong, Zechen Wang, Jinxu Zhao, Gang Zhao, Daichi Guo, Dayuan Fu, Tingfeng Hui, Chen Zeng

show 5 more authors

Keqing He, Xuefeng Li, Liwen Wang, Xinyue Cui, Weiran Xu

Computation and Language · Submitted 28 Aug 2023

abstract compiled · 2026-09-23 · no language-model review

The authors' main claim Experimental results of two few-shot NER benchmarks demonstrate that MSDP consistently outperforms strong baselines by a large margin.

how it is stated · asserted 1.00 read the record →

Comment (0) Read on arXiv ↗

Cite ▾

arXiv ID2308.14533 (version 1)

Pith ID6E4SOE7H, Pith's permanent record of this paper and its review

CitationChen Zeng, Daichi Guo, Dayuan Fu, Gang Zhao, Guanting Dong, Jinxu Zhao, Keqing He, Liwen Wang, Tingfeng Hui, Weiran Xu, Xinyue Cui, Xuefeng Li, Zechen Wang. "A Multi-Task Semantic Decomposition Framework with Task-specific Pre-training for Few-Shot NER." arXiv:2308.14533 (2023). https://pith.science/paper/6E4SOE7H

Claim paper

The compiled record

The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.

Open the compiled record

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-09-23: 1 request, 6,190 input tokens.

What the abstract itself states, as probabilities: claims first 0.07 · states limitation 0.05 · human participants 0.03 · controlled comparison 0.87 · randomized assignment 0.01 · states quantitative result 0.06 · compares with prior results 0.96 · mentions code or data availability 0.02.

The abstract, sentence by sentence

arXiv listing abstract v1. Results and the main claim are underlined; show every role.

The objective of few-shot named entity recognition is to identify named entities with limited labeled instances. Previous works have primarily focused on optimizing the traditional token-wise classification framework, while neglecting the exploration of information based on NER data characteristics. To address this issue, we propose a Multi-Task Semantic Decomposition Framework via Joint Task-specific Pre-training (MSDP) for few-shot NER. Drawing inspiration from demonstration-based and contrastive learning, we introduce two novel pre-training tasks: Demonstration-based Masked Language Modeling (MLM) and Class Contrastive Discrimination. These tasks effectively incorporate entity boundary information and enhance entity representation in Pre-trained Language Models (PLMs). In the downstream main task, we introduce a multi-task joint optimization framework with the semantic decomposing method, which facilitates the model to integrate two different semantic information for entity classification. Experimental results of two few-shot NER benchmarks demonstrate that MSDP consistently outperforms strong baselines by a large margin. Extensive analyses validate the effectiveness and generalization of MSDP.

background contribution method result interpretation main claim

What the classifier reports for each sentence. The number is the probability it gave the label.
#	sentence	role	the authors' own work	states the main conclusion	strength
s0	The objective of few-shot named entity recognition is to identify named entities with limited labeled instances.	background 1.00	0.04	0.02	asserted 1.00
s1	Previous works have primarily focused on optimizing the traditional token-wise classification framework, while neglecting the exploration of information based on NER data characteristics.	gap 0.94	0.05	0.02	asserted 1.00
s2	To address this issue, we propose a Multi-Task Semantic Decomposition Framework via Joint Task-specific Pre-training (MSDP) for few-shot NER.	contribution 0.93	0.84	0.06	asserted 0.97
s3	Drawing inspiration from demonstration-based and contrastive learning, we introduce two novel pre-training tasks: Demonstration-based Masked Language Modeling (MLM) and Class Contrastive Discrimination.	method 0.51	0.90	0.03	asserted 0.94
s4	These tasks effectively incorporate entity boundary information and enhance entity representation in Pre-trained Language Models (PLMs).	method 0.52	0.67	0.05	asserted 1.00
s5	In the downstream main task, we introduce a multi-task joint optimization framework with the semantic decomposing method, which facilitates the model to integrate two different semantic information for entity classification.	method 0.94	0.89	0.05	asserted 0.98
s6	Experimental results of two few-shot NER benchmarks demonstrate that MSDP consistently outperforms strong baselines by a large margin.	result 1.00	0.96	0.82	asserted 1.00
s7	Extensive analyses validate the effectiveness and generalization of MSDP.	result 0.84	0.89	0.16	asserted 1.00

What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.99.

What was examined, and what was not

sources read

arXiv listing abstract v1

not examined

full text
figures and tables
references and citing sentences
relations to other papers

levels run

P abstract

not run

S sections
T body sentences
C citations
R cross-paper relations

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. Every number in the measurements table is the authors', checked by code. Record as JSON

The paper

Original abstract

The objective of few-shot named entity recognition is to identify named entities with limited labeled instances. Previous works have primarily focused on optimizing the traditional token-wise classification framework, while neglecting the exploration of information based on NER data characteristics. To address this issue, we propose a Multi-Task Semantic Decomposition Framework via Joint Task-specific Pre-training (MSDP) for few-shot NER. Drawing inspiration from demonstration-based and contrastive learning, we introduce two novel pre-training tasks: Demonstration-based Masked Language Modeling (MLM) and Class Contrastive Discrimination. These tasks effectively incorporate entity boundary information and enhance entity representation in Pre-trained Language Models (PLMs). In the downstream main task, we introduce a multi-task joint optimization framework with the semantic decomposing method, which facilitates the model to integrate two different semantic information for entity classification. Experimental results of two few-shot NER benchmarks demonstrate that MSDP consistently outperforms strong baselines by a large margin. Extensive analyses validate the effectiveness and generalization of MSDP.

Human review

How Pith checks papers → · Full audit of this paper →

How the literature uses it

3 arXiv papers cite this in 3 sentences

Each citing sentence is taken from the citing paper's LaTeX source and typed by Pith's classifier. A label appears at probability 0.8 or more (0.95 for support), a contest at 0.7 or more; the quoted words are the citing authors'. How each label was checked · JSON

Papers citing it · 2 sentences without a confident label

Cites it as background 1 paper

Recent data-driven supervised learning methods, based on sequence labeling framework, have been applied to slot filling and achieved excellent results [ref] .

Towards Robust and Generalizable Training: An Empirical Study of Noisy Slot Filling for Input Perturbations · Introduction · 0.94

What it cites · 29 sentences without a confident label

Uses the method or tool of 4 papers

Following SpanProto [ref] , we only optimize the $L_span$ objective in the first $T$ steps, and jointly optimize both $L_span$ and $L_cls$ after $T$ steps.

SpanProto: A Two-stage Span-based Prototypical Network for Few-shot Named Entity Recognition · Training and Inference of MSDP · 0.99
For downstream few-shot NER, we follow the two-stage framework [ref] including span extraction and entity classification, and initialize them with the pre-trained parameters.

Decomposed Meta-Learning for Few-Shot Named Entity Recognition · Introduction · 0.94
We follow the design of masked language modeling (MLM) in BERT [ref] and integrate the idea of demonstration-based learning on this basis.

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding · Task-Specific Pre-training · 0.89

Uses data or values from 2 papers

We evaluate our method on two widely used few-shot benchmarks Few-NERD [ref] and CrossNER [ref] .

Few-NERD: A Few-Shot Named Entity Recognition Dataset · Datasets · 0.96
We conduct extensive experiments over two widely-used benchmarks, including Few-NERD [ref] and CrossNER [ref] .

Few-shot Slot Tagging with Collapsed Dependency Transfer and Label-enhanced Task-adaptive Projection Network · Introduction · 0.88

Cites as background 29 papers

Previous few-shot NER methods [ref] generally formulate the task as a sequence labeling task based on prototypical networks [ref] .

Label Semantics for Few Shot Named Entity Recognition · Introduction · 0.95
Previous few-shot NER methods [ref] generally formulate the task as a sequence labeling task based on prototypical networks [ref] .

Simple and Effective Few-Shot Named Entity Recognition with Structured Nearest Neighbor Learning · Introduction · 0.95
Previous few-shot NER methods [ref] generally formulate the task as a sequence labeling task based on prototypical networks [ref] .

Example-Based Named Entity Recognition · Introduction · 0.95

Discussion

Know something about this paper? Leave a comment, or highlight any passage above to comment on it.