Pith. sign in

REVIEW 3 major objections 5 minor 47 references

Towards DS-NER: Unveiling and Addressing Latent Noise in Distant Annotations

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper argues that distant-supervision noise in named entity recognition splits into two problems with different remedies, and that solving each separately beats state-of-the-art baselines on eight datasets.

desk verdict Useful framework and reusable LLM-supervised datasets, but the headline empirical claim is undercut by the paper's own tables and missing significance tests. read the letter →

arxiv 2505.12454 v1 pith:IGNP5SMK submitted 2025-05-18 cs.CL cs.LG

Yuyang Ding , Dan Qiao , Juntao Li , Jiajie Xu , Pingfu Chao , Xiaofang Zhou , Min Zhang This is my paper · ORCID
classification cs.CLcs.LG
keywords distantlysupervisednamedentityrecognitionunlabeledproblemnoisyspan-basednegativesamplingconfidentlearninglargelanguagemodelannotationnoisetransitionmatrixpositive-unlabeled
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Distantly supervised named entity recognition builds training data by matching text against dictionaries or by asking large language models to label text, and both routes inject label noise. The paper's central claim is that this noise is not a single problem: entities that are missed entirely (the unlabeled-entity problem) differ in origin and cure from entities that are tagged with the wrong type (the noisy-entity problem), and the mix differs between rule-based and LLM annotators. It proposes a two-stage span-selection framework that warms up the model on a provably clean set of negative spans and then filters both negatives and positives by the model's own confidence, with no extra hyper-parameters. On eight real-world datasets spanning three data sources and four annotation techniques, the method reports higher F1 than current baselines. If the claim holds, DS-NER can be made substantially more robust regardless of whether the distant labels come from dictionaries or from large language models.

What carries the argument

Two constructions carry the argument. The first is the cross-entity negative set $N_{ce}$, the set of negative spans that overlap some observed entity span; Theorem 1 states that under the span-based unlabeled-entity setting $N_{ce} \cap N_{fal} = \emptyset$, i.e., the warm-up negatives are free of false negatives. The second is the class-wise self-confidence threshold $t_l$, the average predicted probability of the observed class over all positive spans of that class; it prunes noisy positives without introducing hyper-parameters, and the paper shows its estimate stays within 0.05 of the optimal threshold on the news dataset.

What would settle it

Count the fraction of semi-unlabeled spans—true entities that overlap but are only partially covered by an observed positive span—in the Webpage, BC5CDR, mit-movie, and mit-restaurant distant annotations. If that fraction is far above the 0.1% reported for the news dataset, the Theorem 1 guarantee that cross-entity negatives contain no false negatives would fail precisely in the regimes where the method claims generality.

Watch

Extended reading notes

Core claim

The central discovery is that the two noise types can be separated and handled by different mechanisms within one training loop. Under span-based settings, the paper proves (Theorem 1) that cross-entity negatives—spans that overlap an observed entity span—are guaranteed to contain no false negatives, because a false negative is itself a true entity and true entities are assumed not to overlap. That makes the cross-entity set a safe warm-up set. After the warm-up, the model's own predictions define a confident-negative set, and a class-wise average self-confidence threshold defines a confident-positive set; noisy positives are pruned whenever their predicted probability for the observed class falls below the class average. The paper reports that this combined procedure outperforms existing baselines on all tested settings, and that the two components are flexible enough to be used selectively when the dominant noise type is known.

Load-bearing premise

The reliable warm-up negatives are truly negative only if every observed positive span is a genuine entity and no genuine entity overlaps another; when an annotator labels only part of an entity, a false negative can slip into the reliable set, and the paper measures that slippage on only one dataset.

Editorial extensions

If this is right

  • Because the noisy-positive threshold is computed from the model's own predictions, the framework can be applied to rule-based and LLM-generated annotations without per-dataset threshold tuning.
  • Synthetic experiments masking up to 90% of entities show the warm-up on clean negatives keeps F1 high where ordinary training collapses, so the UES component addresses severe unlabeled-entity noise directly.
  • The two components can be used independently: the paper observes that on dictionary-annotated biomedical data, where wrong-type noise is almost absent, applying noisy-positive elimination can hurt, so a practitioner can deploy UES alone when NEP is known to be rare.
  • Training time per epoch remains modest compared with heavier denoising baselines, since only one model is trained and negative sampling is capped.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct extension would measure the semi-unlabeled overlap rate on every dataset used here; the paper reports 0.1% on one dataset, and the UES guarantee would need revision if the rate is materially higher elsewhere.
  • The same UEP/NEP decomposition could be applied to other span-level information extraction tasks under distant supervision, such as relation or event argument extraction, where dictionary and LLM annotators also produce both missed mentions and wrong-type mentions.
  • The noise transition matrices in the paper suggest a practical decision rule: inspect the matrix to estimate which noise type dominates, then switch on UES, NPE, or both, converting the framework into a configurable denoising pipeline.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper formalizes two sources of noise in distantly supervised named entity recognition—the unlabeled-entity problem (UEP) and the noisy-entity problem (NEP)—and proposes a two-stage span-selection framework. In the first stage, the model is warmed up using cross-entity negatives Nce, which Theorem 1 shows are free of false negatives under a pure span-based UEP with non-overlapping true entities. In the second stage, training uses confident negatives selected by the model and confident positives selected by a class-confidence threshold (NPE). Experiments compare the method with several baselines on CoNLL03, Webpage, BC5CDR, mit-movie, and mit-restaurant annotated by KB-matching, dictionary matching, ChatGPT, and UniNER. The paper additionally contributes five new LLM-annotated datasets and an analysis of noise transition matrices across annotation methods.

Significance. The paper makes several worthwhile contributions: it provides a clean separation of UEP and NEP that is well motivated by the noise transition matrices in Fig. 2; Theorem 1 is valid under its stated assumptions; the synthetic UEP experiments (Table 8) show large gains at high mask ratios; NPE is hyperparameter-free by design; and the authors release code and data. If the full NPE+UES method were consistently superior to state-of-the-art baselines, the paper would be a strong contribution to DS-NER. However, the empirical evidence as reported does not sustain the central claim: the full method is not the best row in several settings in the paper's own tables, and no significance tests are provided. The contribution is therefore real but the paper's headline claim needs substantial revision and additional evidence.

major comments (3)
  1. [Section 6.2, Tables 4 and 5] The sentence 'On all the datasets under various settings, our method achieves the best F1 performance among all baselines' is contradicted by the reported numbers. NPE+UES is not the best F1 in Webpage/KB (68.56 versus UES 70.01), BC5CDR/Dict (76.61 versus UES 77.21 and TopNeg 77.02), CoNLL03/UniNER (52.00 versus NPE 52.34), or mit-movie (56.07 versus DS 57.67, Neg. Sampling+ 57.80, TopNeg 57.72, and NPE 57.94). The abstract's claim of significant improvements on eight real-world distant supervision datasets is therefore not supported by the tables; at most four of the eight settings favor the full method. In addition, no paired significance tests or confidence intervals are reported, and the variances in Table 7 (0.3–1.9) are the same order as several reported gains (e.g., +0.60 on CoNLL03/KB), so the 'significant improvements' wording is not justified.
  2. [Theorem 1 and Section 6.4] Theorem 1 establishes Nce∩Nfal=∅ under the assumptions of pure UEP and non-overlapping true entities. These assumptions are violated in the real datasets on which the method is evaluated: ChatGPT and UniNER annotations exhibit substantial NEP (Fig. 2), and under NEP a noisy positive span can overlap a false-negative span and put that false negative into Nce. Section 6.4 acknowledges the resulting 'semi-labeled' cases and reports that they constitute only 0.1% of negatives in CoNLL03, but no analogous measurement is given for Webpage, BC5CDR, mit-movie, or mit-restaurant. Since the UES warm-up (Section 5.1.1, Eq. 17) relies on Nce being reliable, the theoretical guarantee does not currently cover the actual deployment conditions, and the empirical support for the assumption is limited to one dataset.
  3. [Section 2.1 and Section 6.1.3] The closely related SANTA method is discussed in Section 2.1 as a prior approach that also addresses inaccurate and incomplete annotation noise with separate strategies, which is conceptually the closest competitor to the proposed UEP/NEP decomposition. However, SANTA is absent from the baseline list in Section 6.1.3 and from Tables 4 and 5. Omitting this comparison makes the claim of superiority over current state-of-the-art methods incomplete.
minor comments (5)
  1. [Section 6.3.2, Eq. (18)] The equations use 'NER' and 'NEP' as abbreviations for recall and precision of noisy entities, which collides with the paper's established abbreviations for named entity recognition and noisy-entity problem. Please rename to avoid confusion.
  2. [Section 6, first sentence] 'Unlabled' should be 'Unlabeled'.
  3. [Table 6 caption] The caption says 'in two datasets' but the table appears to report only CoNLL03 classes; please clarify which datasets are included and whether the optimal-threshold analysis was run on other datasets.
  4. [Section 5.1.1] Calling Nce an 'unbiased negative set' is potentially misleading. Theorem 1 shows it is free of false negatives, but it is not a representative sample of all true negatives; Section 4.2.2 itself notes that training only on Nce hurts F1 because the sampled distribution is incomplete. A term like 'false-negative-free' would be more precise.
  5. [Section 6.1.5] The section says all hyper-parameters are tuned on the development set, but it is not stated whether the baselines were retuned under the same protocol; please specify this for a fair comparison.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the derivation is self-contained; thresholds are model-derived and Theorem 1 is a conditional set-theoretic claim.

full rationale

The derivation chain is not circular. The UES warm-up relies on Theorem 1, which states Nce ∩ Nfal = ∅ under span-based UEP and the premise that no two true entities overlap. This is a direct consequence of the definitions of Nce, Nbe, and Nfal (Eqs. 7, 9-11), not a restatement of the method's target outcome; the paper even acknowledges in Section 6.4 that relaxing the no-semi-unlabeled assumption introduces a small overlap (0.1% on CoNLL03), an honest limitation rather than a circular move. The NPE threshold t_l (Eq. 16) is computed from the model's own class-averaged confidence on the noisy training set and is not fitted to gold labels or test labels; the optimal-τ comparison in Section 6.3.2 is a diagnostic that uses gold labels to bound the achievable threshold, not a component of the method. The self-citations ([34], [42]) appear only as background for generative NER and for the general phenomenon of overfitting to noisy labels, so they are not load-bearing for the central claim. Finally, no fitted parameter is renamed as a prediction: the reported results are standard F1 evaluations against external baselines. Possible weaknesses—such as missing significance tests or the full model not being the best row on some datasets—are empirical/correctness concerns, not circularity.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The main extra assumptions are domain-level: non-overlapping entities and correct observed positives during warm-up. Free parameters are limited to the negative sampling ratio and warm-up length, both tuned on development data. No invented entities or new physical constructs are introduced.

free parameters (2)
  • Negative sampling ratio λ = 0.35
    Used in Eq. (2) and Eq. (17) to sample ⌈λn⌉ negative spans per sentence; tuned on development sets and held fixed across datasets.
  • Warm-up epoch count = 1
    Algorithm 1 trains with reliable negatives for the first epoch; no sensitivity analysis is reported for this choice.
assumptions (4)
  • domain assumption True named entity spans in the data do not overlap each other.
    Used in Theorem 1, premise 2, to show false negatives fall in the between-entity set Nbe. Standard for typical NER annotations but not guaranteed for all span-based noisy labels.
  • domain assumption The observed positive span set P is correct during reliable-negative construction.
    Theorem 1 is stated for the pure UEP setting where observed positives are true positives. When NEP corrupts P, the guarantee Nce ∩ Nfal = ∅ can fail.
  • domain assumption Model self-confidence in early epochs reflects label correctness.
    NPE and confident-negative selection rely on the model not having memorized noisy labels in the first epoch; the paper cites the early-learning phenomenon [40] as support.
  • domain assumption LLM outputs are mapped accurately back to original tokens via LCS.
    The ChatGPT annotation pipeline in Section 3.3 depends on longest-common-subsequence alignment; mapping errors inject noise not modeled as UEP or NEP.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Towards DS-NER: Unveiling and Addressing Latent Noise in Distant Annotations." pith.science (2026). https://pith.science/paper/IGNP5SMK

@misc{pith2026250512454,
  author       = {Pith},
  title        = {Pith review of: Towards DS-NER: Unveiling and Addressing Latent Noise in Distant Annotations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IGNP5SMK}},
  note         = {Machine review of arXiv:2505.12454}
}
read the original abstract

Distantly supervised named entity recognition (DS-NER) has emerged as a cheap and convenient alternative to traditional human annotation methods, enabling the automatic generation of training data by aligning text with external resources. Despite the many efforts in noise measurement methods, few works focus on the latent noise distribution between different distant annotation methods. In this work, we explore the effectiveness and robustness of DS-NER by two aspects: (1) distant annotation techniques, which encompasses both traditional rule-based methods and the innovative large language model supervision approach, and (2) noise assessment, for which we introduce a novel framework. This framework addresses the challenges by distinctly categorizing them into the unlabeled-entity problem (UEP) and the noisy-entity problem (NEP), subsequently providing specialized solutions for each. Our proposed method achieves significant improvements on eight real-world distant supervision datasets originating from three different data sources and involving four distinct annotation techniques, confirming its superiority over current state-of-the-art methods.

Figures

Figures reproduced from arXiv: 2505.12454 by the authors.

Figure 1
Figure 1. A noisy sample produced through distant supervision techniques, [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The standard label noise transition matrix on three datasets (CoNLL03, Webpage, and BC5CDR) annotated by four distantly supervised [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. The overall framework of our method. Left: An example of our span-based setting, illustrating the unlabeled entity problem and the noisy entity problem. Right: The process begins by warming up the model with entity spans and reliable negative spans. Subsequently, the model is used to filter out noisy entities and identify confident negatives for further training. 2d and 2h, the region indicated by P(˜y = O|y ∗ ∈ L) … view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: F NR, F NP of our method during training on synthetic datasets. TABLE 7 Standard variance of F1 Score of our methods, compared with competitive baselines. Methods CoNLL03 Webpage BC5CDR KB LLM KB LLM Dict BOND 0.9 0.6 0.9 1.0 0.9 DSCAU 1.2 2.6 1.2 1.3 0.9 Top Neg. 1.9 …
Figure 5
Figure 5. Figure 5: 1-shot prompt example for ChatGPT to annotate CoNLL03. [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

47 extracted references · 27 canonical work pages

  1. [1]

    A survey on recent advances in named entity recognition from deep learning models,

    V . Yadav and S. Bethard, “A survey on recent advances in named entity recognition from deep learning models,” arXiv preprint arXiv:1910.11470, 2019. IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING 13

    arXiv 1910

  2. [2]

    A survey on deep learning for named entity recognition,

    J. Li, A. Sun, J. Han, and C. Li, “A survey on deep learning for named entity recognition,” IEEE Transactions on Knowledge and Data Engineering, vol. 34, no. 1, pp. 50–70, 2020

    2020

  3. [3]

    Bidirectional lstm-crf models for sequence tagging,

    Z. Huang, W. Xu, and K. Yu, “Bidirectional lstm-crf models for sequence tagging,” arXiv preprint arXiv:1508.01991, 2015

    arXiv 2015

  4. [4]

    Fast and accurate entity recognition with iterated dilated convolutions,

    E. Strubell, P . Verga, D. Belanger, and A. McCallum, “Fast and accurate entity recognition with iterated dilated convolutions,” arXiv preprint arXiv:1702.02098, 2017

    arXiv 2017

  5. [5]

    Bert: Pre- training of deep bidirectional transformers for language under- standing,

    J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre- training of deep bidirectional transformers for language under- standing,” arXiv preprint arXiv:1810.04805, 2018

    arXiv 2018

  6. [6]

    Universalner: Targeted distillation from large language models for open named entity recognition,

    W. Zhou, S. Zhang, Y. Gu, M. Chen, and H. Poon, “Universalner: Targeted distillation from large language models for open named entity recognition,” Aug 2023

    work page 2023

  7. [7]

    Relation extraction using distant supervision: A survey,

    A. Smirnova and P . Cudr ´e-Mauroux, “Relation extraction using distant supervision: A survey,” ACM Computing Surveys (CSUR) , vol. 51, no. 5, pp. 1–35, 2018

    work page 2018

  8. [8]

    Bond: Bert-assisted open-domain named entity recognition with distant supervision,

    C. Liang, Y. Yu, H. Jiang, S. Er, R. Wang, T. Zhao, and C. Zhang, “Bond: Bert-assisted open-domain named entity recognition with distant supervision,” in Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining , 2020, pp. 1054–1064

    work page 2020

Show all 47 references
  1. [9]

    Is gpt-3 a good data annotator?

    B. Ding, C. Qin, L. Liu, L. Bing, S. Joty, and B. Li, “Is gpt-3 a good data annotator?” arXiv preprint arXiv:2212.10450, 2022

    2022 arXiv

  2. [10]

    Learn- ing named entity tagger using domain-specific dictionary,

    J. Shang, L. Liu, X. Ren, X. Gu, T. Ren, and J. Han, “Learn- ing named entity tagger using domain-specific dictionary,” arXiv preprint arXiv:1809.03599, 2018

    2018 arXiv

  3. [11]

    Dual t: Reducing estimation error for transition matrix in label- noise learning,

    Y. Yao, T. Liu, B. Han, M. Gong, J. Deng, G. Niu, and M. Sugiyama, “Dual t: Reducing estimation error for transition matrix in label- noise learning,” Advances in neural information processing systems , vol. 33, pp. 7260–7271, 2020

    2020

  4. [12]

    Distantly supervised named entity recognition using positive-unlabeled learning,

    M. Peng, X. Xing, Q. Zhang, J. Fu, and X. Huang, “Distantly supervised named entity recognition using positive-unlabeled learning,” arXiv preprint arXiv:1906.01378, 2019

    1906 arXiv

  5. [13]

    Distantly supervised named entity recognition via confidence-based multi-class positive and unla- beled learning,

    K. Zhou, Y. Li, and Q. Li, “Distantly supervised named entity recognition via confidence-based multi-class positive and unla- beled learning,” arXiv preprint arXiv:2204.09589, 2022

    2022 arXiv

  6. [14]

    Learning from positive and unlabeled exam- ples with different data distributions,

    X.-L. Li and B. Liu, “Learning from positive and unlabeled exam- ples with different data distributions,” in European conference on machine learning. Springer, 2005, pp. 218–229

    2005

  7. [15]

    Empirical analysis of unlabeled entity problem in named entity recognition,

    Y. Li, L. Liu, and S. Shi, “Empirical analysis of unlabeled entity problem in named entity recognition,” arXiv preprint arXiv:2012.05426, 2020

    2012 arXiv

  8. [16]

    Rethinking negative sampling for handling missing entity annotations,

    ——, “Rethinking negative sampling for handling missing entity annotations,” in Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , 2022, pp. 7188–7197

    2022

  9. [17]

    Better sampling of negatives for distantly supervised named entity recognition

    L. Xu, L. Bing, and W. Lu, “Better sampling of negatives for distantly supervised named entity recognition.”

  10. [18]

    Im- proving distantly-supervised named entity recognition with self- collaborative denoising learning,

    X. Zhang, B. Yu, T. Liu, Z. Zhang, J. Sheng, M. Xue, and H. Xu, “Im- proving distantly-supervised named entity recognition with self- collaborative denoising learning,” arXiv preprint arXiv:2110.04429, 2021

    2021 arXiv

  11. [19]

    An overview of distant supervision for relation extrac- tion with a focus on denoising and pre-training methods,

    W. Hogan, “An overview of distant supervision for relation extrac- tion with a focus on denoising and pre-training methods,” arXiv preprint arXiv:2207.08286, 2022

    2022 arXiv

  12. [20]

    Looking beyond label noise: Shifted label distribution matters in distantly supervised relation extraction,

    Q. Ye, L. Liu, M. Zhang, and X. Ren, “Looking beyond label noise: Shifted label distribution matters in distantly supervised relation extraction,” arXiv preprint arXiv:1904.09331, 2019

    1904 arXiv

  13. [21]

    Distantly supervised named entity recognition with spy-pu algorithm,

    H. Zheng, H. Yu, Y. Hao, Y. Wu, and S. Li, “Distantly supervised named entity recognition with spy-pu algorithm,” in 2021 IEEE 2nd International Conference on Pattern Recognition and Machine Learning (PRML). IEEE, 2021, pp. 56–63

    2021

  14. [22]

    Class-imbalanced- aware distantly supervised named entity recognition,

    Y. Mao, Y. Hao, W. Liu, X. Lin, and X. Cao, “Class-imbalanced- aware distantly supervised named entity recognition,”IEEE Trans- actions on Neural Networks and Learning Systems , 2023

    2023

  15. [23]

    Boundary smoothing for named entity recogni- tion,

    E. Zhu and J. Li, “Boundary smoothing for named entity recogni- tion,” Apr 2022

    2022

  16. [24]

    Scl-rai: Span-based contrastive learning with retrieval augmented inference for unlabeled entity problem in ner,

    S. Si, S. Zeng, J. Lin, and B. Chang, “Scl-rai: Span-based contrastive learning with retrieval augmented inference for unlabeled entity problem in ner,” arXiv preprint arXiv:2209.01646, 2022

    2022 arXiv

  17. [25]

    Distantly-supervised named entity recognition with noise-robust learning and language model augmented self-training,

    Y. Meng, Y. Zhang, J. Huang, X. Wang, Y. Zhang, H. Ji, and J. Han, “Distantly-supervised named entity recognition with noise-robust learning and language model augmented self-training,” arXiv preprint arXiv:2109.05003, 2021

    2021 arXiv

  18. [26]

    Distantly su- pervised ner with partial annotation learning and reinforcement learning,

    Y. Yang, W. Chen, Z. Li, Z. He, and M. Zhang, “Distantly su- pervised ner with partial annotation learning and reinforcement learning,” in Proceedings of the 27th International Conference on Computational Linguistics, 2018, pp. 2159–2169

    2018

  19. [27]

    De-biasing distantly supervised named entity recognition via causal intervention,

    W. Zhang, H. Lin, X. Han, and L. Sun, “De-biasing distantly supervised named entity recognition via causal intervention,” arXiv preprint arXiv:2106.09233, 2021

    2021 arXiv

  20. [28]

    Santa: Separate strate- gies for inaccurate and incomplete annotation noise in distantly- supervised named entity recognition

    S. Si, Z. Cai, S. Zeng, G. Feng, J. Lin, B. Chang, T. Ave, N. Seat- tle, W. Loc, L. Loc, and S. Washington, “Santa: Separate strate- gies for inaccurate and incomplete annotation noise in distantly- supervised named entity recognition.”

  21. [29]

    Is chatgpt a general-purpose natural language processing task solver?

    C. Qin, A. Zhang, Z. Zhang, J. Chen, M. Yasunaga, and D. Yang, “Is chatgpt a general-purpose natural language processing task solver?” arXiv preprint arXiv:2302.06476, 2023

    2023 arXiv

  22. [30]

    Large language model is not a good few-shot information extractor, but a good reranker for hard samples!

    Y. Ma, Y. Cao, Y. Hong, and A. Sun, “Large language model is not a good few-shot information extractor, but a good reranker for hard samples!” arXiv preprint arXiv:2303.08559, 2023

    2023 arXiv

  23. [31]

    Promptner: Prompting for named entity recognition,

    D. Ashok and Z. C. Lipton, “Promptner: Prompting for named entity recognition,” arXiv preprint arXiv:2305.15444, 2023

    2023 arXiv

  24. [32]

    Gpt-ner: Named entity recognition via large language models,

    S. Wang, X. Sun, X. Li, R. Ouyang, F. Wu, T. Zhang, J. Li, and G. Wang, “Gpt-ner: Named entity recognition via large language models,” arXiv preprint arXiv:2304.10428, 2023

    2023 arXiv

  25. [33]

    Instructuie: Multi-task instruction tun- ing for unified information extraction

    X. Wang, W. Zhou, C. Zu, H. Xia, T. Chen, Y. Zhang, R. Zheng, J. Ye, Q. Zhang, T. Gui, J. Kang, J. Yang, S. Li, C. Du, R. Al-Rfou, V . Kulkarni, B. Perozzi, S. Skiena, T. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P . Dhariwal, A. Neelakantan, P . Shyam, G. Sastry, A. As...

  26. [34]

    Rethinking negative instances for generative named entity recognition,

    Y. Ding, J. Li, P . Wang, Z. Tang, B. Yan, and M. Zhang, “Rethinking negative instances for generative named entity recognition,” arXiv preprint arXiv:2402.16602, 2024

    2024 arXiv

  27. [35]

    Introduction to the conll-2003 shared task: Language-independent named entity recognition,

    E. F. Sang and F. De Meulder, “Introduction to the conll-2003 shared task: Language-independent named entity recognition,” arXiv preprint cs/0306050, 2003

    2003 arXiv

  28. [36]

    Design challenges and misconceptions in named entity recognition,

    L. Ratinov and D. Roth, “Design challenges and misconceptions in named entity recognition,” in Proceedings of the thirteenth conference on computational natural language learning (CoNLL-2009) , 2009, pp. 147–155

    2009

  29. [37]

    Biocreative v cdr task corpus: a resource for chemical disease relation extraction,

    J. Li, Y. Sun, R. J. Johnson, D. Sciaky, C.-H. Wei, R. Leaman, A. P . Davis, C. J. Mattingly, T. C. Wiegers, and Z. Lu, “Biocreative v cdr task corpus: a resource for chemical disease relation extraction,” Database, vol. 2016, 2016

    2016

  30. [38]

    Training language models to follow instructions with human feedback,

    L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P . Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray et al., “Training language models to follow instructions with human feedback,” Advances in Neural Information Processing Systems , vol. 35, pp. 27 730–27 744, 2022

    2022

  31. [39]

    Llama: Open and efficient foundation language models,

    H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozi`ere, N. Goyal, E. Hambro, F. Azharet al., “Llama: Open and efficient foundation language models,” arXiv preprint arXiv:2302.13971, 2023

    2023 arXiv

  32. [40]

    Early-learning regularization prevents memorization of noisy labels,

    S. Liu, J. Niles-Weed, N. Razavian, and C. Fernandez-Granda, “Early-learning regularization prevents memorization of noisy labels,” Advances in neural information processing systems , vol. 33, pp. 20 331–20 342, 2020

    2020

  33. [41]

    How does disagreement help generalization against label corruption?

    X. Yu, B. Han, J. Yao, G. Niu, I. Tsang, and M. Sugiyama, “How does disagreement help generalization against label corruption?” in International Conference on Machine Learning . PMLR, 2019, pp. 7164–7173

    2019

  34. [42]

    Selfmix: Robust learning against textual label noise with self- mixup training,

    D. Qiao, C. Dai, Y. Ding, J. Li, Q. Chen, W. Chen, and M. Zhang, “Selfmix: Robust learning against textual label noise with self- mixup training,” arXiv preprint arXiv:2210.04525, 2022

    2022 arXiv

  35. [43]

    Confident learning: Esti- mating uncertainty in dataset labels,

    C. Northcutt, L. Jiang, and I. Chuang, “Confident learning: Esti- mating uncertainty in dataset labels,” Journal of Artificial Intelli- gence Research, vol. 70, pp. 1373–1411, 2021

    2021

  36. [44]

    Asgard: A portable architecture for multilingual dialogue systems,

    J. Liu, P . Pasupat, S. Cyphers, and J. Glass, “Asgard: A portable architecture for multilingual dialogue systems,” in 2013 IEEE International Conference on Acoustics, Speech and Signal Processing . IEEE, 2013, pp. 8386–8390

    2013

  37. [45]

    Roberta: A ro- bustly optimized bert pretraining approach,

    Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V . Stoyanov, “Roberta: A ro- bustly optimized bert pretraining approach,” arXiv preprint arXiv:1907.11692, 2019

    1907 arXiv

  38. [46]

    Decoupled weight decay regulariza- tion,

    I. Loshchilov and F. Hutter, “Decoupled weight decay regulariza- tion,” arXiv preprint arXiv:1711.05101, 2017. IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING 14 Yuyang Ding is now a PhD student at the nat- ural language processing laboratory, Soochow University, supervise...

    2017 arXiv

  39. [2011]

    His research interests include spatial/text/graph data management, query opti- mization and data mining

    He is currently a professor with the Soo- chow University. His research interests include spatial/text/graph data management, query opti- mization and data mining. Xiaofang Zhou (Fellow, IEEE) received the bachelor’s and master’s degrees in computer science from Nanjing Univer...

    1984

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.