Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

GuideX: Guided Synthetic Data Generation for Zero-Shot Information Extraction

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read GUIDEX shows that auto-generated annotation guidelines and examples let an 8B model beat prior zero-shot NER systems.

desk verdict GuideX is a genuinely reusable pipeline for schema/guideline induction, but the abstract overstates the synthetic-only result and the missing contamination check leaves the zero-shot claim under-supported. read the letter →

arxiv 2506.00649 v1 pith:GJFDDZVG submitted 2025-05-31 cs.CL

Neil De La Fuente , Oscar Sainz , Iker García-Ferrero , Eneko Agirre This is my paper
classification cs.CL
keywords informationextractionnamedentityrecognitionzero-shotlearningsyntheticdatagenerationannotationguidelinesschemainductionLLMfine-tuningdomainadaptation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

GUIDEX is a method for building information-extraction training data without human annotators: it takes raw documents from a target domain, asks a large language model to summarize them, organizes the content into JSON, generates an annotation schema with detailed guidelines expressed as Python dataclasses, and then extracts schema-compliant example annotations. The paper claims that fine-tuning an 8-billion-parameter Llama 3.1 model first on this synthetic dataset and then on gold data reaches 64.2 average F1 across seven zero-shot NER benchmarks, beating previous system families (GoLLIE, KnowCoder, UniNER, and gold-only fine-tuning). Training only on GUIDEX data lifts the zero-shot baseline from 30.1 to 37.1 F1, showing the synthetic data alone teaches the task. If the claim holds, automatic schema and guideline induction from plain text is a reusable way to adapt IE systems to new domains at low cost.

What carries the argument

The load-bearing mechanism is a four-stage pipeline run by an LLM on each source document: (1) document summarization into bulleted key points, (2) conversion of those points into a structured JSON representation, (3) generation of an annotation guideline expressed as a Python dataclass with a detailed docstring, and (4) extraction of concrete instances as a Python list. A consistency filter then executes the dataclass definitions and discards entries that do not conform, removing hallucinations and schema violations before training.

What would settle it

Run a near-duplicate search (such as MinHash or exact n-gram matching) between the 10,000 GUIDEX source documents and the sentences of CrossNER and MIT Movie/Restaurant; any substantial match would show the zero-shot gains could come from overlapping text rather than from learned generalization.

Watch

Extended reading notes

Core claim

The paper's central claim is that dynamically inferring annotation schemas and guidelines per document, rather than reusing fixed label definitions, produces synthetic training data that transfers well to unseen domains. Concretely, Llama 3.1-8B fine-tuned on GUIDEX data alone scores 37.1 F1 averaged over seven out-of-domain NER sets, and when combined with gold fine-tuning reaches 64.2 F1, outperforming GoLLIE (58.0), KnowCoder (60.1), and GoldFT (62.8). The gain is most visible on fine-grained domain-specific labels such as Scientist and PoliticalParty, where GUIDEX-trained models stop over-generalizing to Person and Organization.

Load-bearing premise

The 10,000 FineWeb-edu documents used to generate the synthetic data do not overlap, at the document or sentence level, with the texts in the CrossNER and MIT evaluation datasets; the paper checks label-name overlap but never tests text overlap.

Editorial extensions

If this is right

  • Zero-shot IE systems can be adapted to a new domain in hours: the four prompts and the consistency filter take any plain-text corpus and return a ready-to-train dataset.
  • Models trained with GUIDEX gain up to 7 F1 points over prior synthetic-data methods without any human labels, and nearly 2 F1 points when added on top of gold fine-tuning.
  • The benefit concentrates on fine-grained labels: Scientist, Politician, and PoliticalParty improve by up to 12.8 F1, while generic labels like Location and Country are unaffected.
  • Catch-all labels such as Other and Miscellaneous remain hard for guideline-driven models, since they lack consistent boundaries.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the same pipeline is task-agnostic, the document-to-schema procedure could be pointed at relation extraction or event extraction, not just NER; the paper lists document-level IE as future work.
  • The label-overlap analysis implies that about 40% of standard IE label names already appear verbatim in GUIDEX, so part of the benefit may come from the model learning canonical label semantics rather than from novel schemas; a controlled ablation that removes overlapping label names would test this.
  • A testable extension is to use the consistency filter's dataclass execution to measure per-label noise rates and then reweight or filter synthetic instances by noise, which the paper does not report.
  • The claimed zero-shot generalization would be more convincing if the 10,000 sampled FineWeb-edu document IDs were published, allowing independent verification that no evaluation text appears in the training corpus.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces GuideX, a four-step LLM-based pipeline that, given a plain-text document, generates a summary, a structured JSON representation, inferred annotation guidelines encoded as Python dataclasses, and final entity/attribute instances. The authors construct a synthetic training set from roughly 10,000 FineWeb-edu documents and fine-tune Llama-3.1-8B on this data, optionally followed by gold IE training data. The best system, GuideX FT + GoldFT, achieves an average F1 of 64.2 on seven zero-shot NER benchmarks (CrossNER and MIT), exceeding GoLLIE, KnowCoder, and GoldFT; GuideX FT alone reaches 37.1 from a 30.1 untuned baseline. The paper also provides label-level analyses of gains on fine-grained entity types. The main technical contribution is automatic schema and guideline induction from documents, with the dataset, code, and models released.

Significance. The method is potentially useful and well-motivated: it removes manual schema design from synthetic data generation, and the released open-source pipeline and dataset are valuable for reproducibility. The controlled comparison (GoldFT vs GuideX+GoldFT) suggests a consistent, though modest, benefit from GuideX data across most domains. However, the headline SOTA claim is weakened by two confounds: the best result requires gold human data and a newer/larger backbone than the compared models, and the paper does not check text-level overlap between its FineWeb-edu training sample and the evaluation sets, leaving open a memorization explanation for the gains. The label-overlap analysis (42.4% of gold labels appearing verbatim) further indicates that the 'automatic' schemas are not entirely novel, but the paper does not discuss this as a potential leakage path. Overall, the work is a solid empirical contribution pending a contamination check and a more careful framing of the SOTA claim.

major comments (4)
  1. [Section 4 (Document Collection) and Tables 2-3] There is no document- or sentence-level overlap check between the ~10,000 FineWeb-edu documents used to build GuideX and the CrossNER/MIT evaluation sets. Because GuideX generates annotations on the full documents and the training data is exactly those annotated texts, any verbatim or near-verbatim overlap could allow the fine-tuned model to memorize test spans; the reported gains (+7.06 F1 over the untuned base in Table 2, +1.4 over GoldFT in Table 3) would then reflect memorization rather than schema or guideline transfer. The paper's 'Entity–type overlap' analysis measures label names, not text duplication. Please report a deduplication analysis (exact or near-duplicate n-gram matching between the GuideX corpus and each evaluation split) and re-evaluate on a contamination-free subset.
  2. [Abstract and Section 1 (Introduction)] The state-of-the-art claim is supported only by the GuideX FT + GoldFT configuration, which includes human gold data and a Llama-3.1-8B backbone. GuideX FT alone (37.14 average in Table 2) is below every non-Vicuna baseline in Table 3 (e.g., USM 37.8, InstructUIE 47.2, GoLLIE 58.0, KnowCoder 60.1). Thus the abstract's statement that models trained with GuideX gain 'up to 7 F1 points over previous methods without human-labeled data' is not supported: the 7-point gain in Table 2 is relative to the untuned Llama baseline, not to previous methods. The introduction's 'average improvement of 10 F1 points' is also inconsistent with the 7.06 points reported in Table 2. Please revise these claims to focus on the controlled GoldFT comparison and the actual magnitude of the GuideX effect.
  3. [Section 6 and Table 3] The comparison against GoLLIE, KnowCoder, and GLiNER-L is confounded by the base model and training data. GoldFT (Llama-3.1-8B fine-tuned on gold data) already achieves 62.8 average F1, above all prior systems, and GuideX+GoldFT improves on GoldFT by only 1.38-1.4 points. The paper should explicitly state that the SOTA result is a combination of GuideX, a newer backbone, and gold data, and treat the GoldFT versus GuideX+GoldFT delta as the primary measure of GuideX's effectiveness. Without this framing, readers may attribute the SOTA result to GuideX when in fact most of the gain over prior work comes from the base model and the gold training set.
  4. [Section 4 (Entity–type overlap) and Section 5.1] The 42.4% verbatim label overlap with existing benchmark label spaces raises a secondary leakage path: Llama-3.1-70B, the generator, is likely to have seen those label definitions during pretraining, so the 'automatically inferred' schemas are not fully independent of the benchmarks. The paper presents this overlap as evidence of coverage, but it also means the zero-shot transfer is partly a test of the generator's memorized annotation conventions rather than of schema induction from documents. Please analyze whether the GuideX benefit changes when evaluation labels are restricted to types that do not appear in the generated label space, or at least condition the per-label results on label-name overlap.
minor comments (5)
  1. [Abstract and Title] The abstract uses 'GUIDEX' while the title and most of the body use 'GuideX'; please choose one convention and apply it consistently.
  2. [Section 1 (Introduction)] The statement of an 'average improvement of 10 F1 points' from GuideX-only fine-tuning does not match the 7.06-point improvement reported in Table 2; please harmonize the numbers.
  3. [Figure 1 caption] The caption contains typographical errors: 'GoldF T' should be 'GoldFT' and 'GuideXF T+ GoldF Tit' should be 'GuideXFT + GoldFT'.
  4. [Tables 2 and 3] There are formatting inconsistencies in the row and column headers, including 'GUIDEXFT+ GoldFT' with a stray space in Table 3 and the 'Movie Restaurant' column header lacking a separator in Table 2; these should be cleaned up.
  5. [Section 7 (Analysis)] The text says F1 gains of 'up to 12.8 points' for the Scientist/Politician cases, but Table 4 reports a gain of 12.78 for Scientist and 9.25 for Politician; please verify the summary statement against the table values.

Circularity Check

0 steps flagged · score 2.0 of 10

No load-bearing circularity: GuideX's gains are measured on external zero-shot benchmarks; self-citations to the authors' GoLLIE work are confined to format/gold-data reuse and do not force the result.

full rationale

The paper's derivation chain is empirical, not mathematical, and the central claim is externally tested. GuideX builds a synthetic dataset from ~10,000 FineWeb-edu documents through a four-step LLM pipeline (summarization, structured JSON, guideline generation, instance extraction), then fine-tunes Llama 3.1-8B and evaluates on seven unseen NER benchmarks (CrossNER, MIT Movie, MIT Restaurant). No parameter is fitted to the evaluation labels, and no evaluation annotation is used to construct the GuideX dataset; the gains (37.1 vs 30.1 for GuideXFT over base; 64.2 vs 62.8 for GuideXFT+GoldFT over GoldFT) are therefore not equal to the method's inputs by construction. The paper reports the 42.4% verbatim entity-type overlap with existing IE datasets as an observational coverage statistic, not as a training target. Self-citations to GoLLIE (Sainz et al., 2024) appear when adopting the code-style IE format and when reusing the same gold training data; these are implementation and controlled-baseline choices, not uniqueness claims or forced conclusions. The main validity threat is the untested document-level overlap between the randomly sampled FineWeb-edu corpus (Section 4, 'Document Collection') and the CrossNER/MIT evaluation sets; this could inflate F1 through memorization, but it is a contamination risk, not circularity, because the training corpus is not defined in terms of the test texts. The paper's stated limitations (sentence-level evaluation vs document-level training; catch-all 'Other'/'Miscellaneous' labels) similarly affect generalization, not circularity. Hence a low score reflecting only the presence of minor, non-load-bearing self-citations.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The method is empirical and does not derive any mathematical constants. Its trust rests on the generation model, the source corpus, the inherited annotation format, the correctness of weakly supervised labels, and the unseen evaluation premises listed above.

free parameters (3)
  • Synthetic generation sampling temperature = 0.7
    Chosen after evaluating alternatives; no ablation is reported for this choice, and it directly shapes the diversity and noise of the synthetic data.
  • Top-p for generation = 0.95
    Standard nucleus sampling setting; listed in Table 5 without sensitivity analysis.
  • Fine-tuning LoRA hyperparameters = rank 128, alpha 256, dropout 0.08/0.05, lr 3e-4
    Selected based on validation splits of the training datasets; no sweep is reported against the zero-shot evaluation domains.
assumptions (5)
  • domain assumption Llama 3.1-70B Instruct generates accurate summaries, structured JSON, guidelines, and entity instances from raw documents.
    The entire GuideX dataset is produced by this model; errors in any step propagate to the training signal. Invoked in Section 3 and Appendix A.1.
  • domain assumption FineWeb-edu documents are sufficiently representative of the evaluation domains and contain the entity types needed for zero-shot NER.
    The GuideX corpus is sampled from FineWeb-edu in Section 4, while evaluation uses CrossNER and MIT domains with different label spaces; the paper relies on this overlap being large enough to transfer.
  • domain assumption The code-style, dataclass-based annotation format inherited from GoLLIE remains effective when guidelines are generated automatically.
    Section 5.1 adopts the standard code-style IE format from Sainz et al. (2024); the benefit of the format was established for manually written guidelines, and the paper does not isolate whether gains come from the format or from GuideX's content.
  • domain assumption Generated annotations are correct and consistent enough to serve as weak supervision.
    The consistency filter in Section 3 removes parse errors and schema violations, but there is no human validation of label accuracy or boundary correctness on a sample of the synthetic data.
  • domain assumption The random FineWeb-edu sample does not include evaluation corpus texts at the document or sentence level.
    Not checked in the paper; only label-name overlap is reported in Section 4. This assumption is necessary for the zero-shot generalization claim.

how reviews work

0 comments
Cite this review

Pith. "Pith review of GuideX: Guided Synthetic Data Generation for Zero-Shot Information Extraction." pith.science (2026). https://pith.science/paper/GJFDDZVG

@misc{pith2026250600649,
  author       = {Pith},
  title        = {Pith review of: GuideX: Guided Synthetic Data Generation for Zero-Shot Information Extraction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GJFDDZVG}},
  note         = {Machine review of arXiv:2506.00649}
}
read the original abstract

Information Extraction (IE) systems are traditionally domain-specific, requiring costly adaptation that involves expert schema design, data annotation, and model training. While Large Language Models have shown promise in zero-shot IE, performance degrades significantly in unseen domains where label definitions differ. This paper introduces GUIDEX, a novel method that automatically defines domain-specific schemas, infers guidelines, and generates synthetically labeled instances, allowing for better out-of-domain generalization. Fine-tuning Llama 3.1 with GUIDEX sets a new state-of-the-art across seven zeroshot Named Entity Recognition benchmarks. Models trained with GUIDEX gain up to 7 F1 points over previous methods without humanlabeled data, and nearly 2 F1 points higher when combined with it. Models trained on GUIDEX demonstrate enhanced comprehension of complex, domain-specific annotation schemas. Code, models, and synthetic datasets are available at neilus03.github.io/guidex.com

Figures

Figures reproduced from arXiv: 2506.00649 by the authors.

Figure 1
Figure 1. Impact of GUIDEX for zero-shot NER on different domains. In GoldF T , the model is trained solely on gold training data, whereas in GuideXF T + GoldF T it is also trained on our synthetic dataset. Question Answering (Levy et al., 2017) or Natu￾ral Language Inference (Obamuyide and Vlachos, 2018; Sainz et al., 2021) paradigms, leveraging su￾pervised data from these auxiliary tasks. While showing initial promise, thes… view at source ↗
Figure 2
Figure 2. GUIDEX process overview. The approach transforms raw text into structured annotations by dynamically inferring schemas, generating executable guidelines, and resulting annotations. matches (Hoffmann et al., 2011; Lin et al., 2016; Han and Sun, 2016; Xiao et al., 2020), yet these pipelines often rely on rigid predefined schemas, making it challenging to adapt to novel entity or relation types. Beyond distant supervis… view at source ↗
Figure 3
Figure 3. GUIDEX follows a multi-step prompting pipeline which allows for the creation of the synthetic guidelines and annotations that conform GUIDEX and are used for GUIDEXFT [PITH_FULL_IMAGE:figures/full_fig_p014_3.png] view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Baryon and Meson Excited States

    hep-ph 2025-06 reject novelty 4.0 of 10

    The authors propose a universal logarithmic mass formula, M_n = α ln(n) + β, for all equal-quantum excited baryon and meson states.

Reference graph

Works this paper leans on

57 extracted references · 26 canonical work pages · cited by 1 Pith paper

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Dhananjay Ashok and Zachary C. Lipton. 2023. https://arxiv.org/abs/2305.15444 Promptner: Prompting for named entity recognition . Preprint, arXiv:2305.15444

    arXiv 2023

  4. [4]

    Fan Bai, Junmo Kang, Gabriel Stanovsky, Dayne Freitag, Mark Dredze, and Alan Ritter. 2024. https://doi.org/10.18653/v1/2024.findings-emnlp.600 Schema-driven information extraction from heterogeneous tables . In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 10252--10273, Miami, Florida, USA. Association for Computational Linguistics

    doi:10.18653/v1/2024.findings-emnlp.600 2024

  5. [5]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, and 12 others. 2020. https://proceedings.neurips.cc/paper_fil...

    2020

  6. [6]

    Yubo Chen, Shulin Liu, Xiang Zhang, Kang Liu, and Jun Zhao. 2017. https://doi.org/10.18653/v1/P17-1038 Automatically labeled data generation for large scale event extraction . In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 409--419, Vancouver, Canada. Association for Computational ...

    work page doi:10.18653/v1/p17-1038 2017

  7. [7]

    Gonzalez, Ion Stoica, and Eric P

    Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. 2023. https://lmsys.org/blog/2023-03-30-vicuna/ Vicuna: An open-source chatbot impressing gpt-4 with 90\

    2023

  8. [8]

    Leon Derczynski, Eric Nichols, Marieke van Erp, and Nut Limsopatham. 2017. https://doi.org/10.18653/v1/W17-4418 Results of the WNUT 2017 shared task on novel and emerging entity recognition . In Proceedings of the 3rd Workshop on Noisy User-generated Text, pages 140--147, Copenhagen, Denmark. Association for Computational Linguistics

    doi:10.18653/v1/w17-4418 2017

Show all 57 references
  1. [9]

    Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. Qlora: efficient finetuning of quantized llms. In Proceedings of the 37th International Conference on Neural Information Processing Systems, NIPS '23, Red Hook, NY, USA. Curran Associates Inc

    2023

  2. [10]

    Rezarta Islamaj Do g an, Robert Leaman, and Zhiyong Lu. 2014. Ncbi disease corpus: a resource for disease name recognition and concept normalization. Journal of biomedical informatics, 47:1--10

    2014

  3. [11]

    Kawin Ethayarajh, Yejin Choi, and Swabha Swayamdipta. 2022. https://proceedings.mlr.press/v162/ethayarajh22a.html Understanding dataset difficulty with V -usable information . In Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of...

    2022

  4. [12]

    Hermenegildo Fabregat, Juan Martinez-Romo, and Lourdes Araujo. 2018. Overview of the diann task: Disability annotation task. In IberEval@ SEPLN, pages 1--14

    2018

  5. [13]

    Steven Feng, Varun Gangal, Jason Wei, Sarath Chandar, Soroush Vosoughi, Teruko Mitamura, and Eduard Hovy. 2021. A survey of data augmentation approaches for nlp. Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021

    2021

  6. [14]

    Chufan Gao, Xulin Fan, Jimeng Sun, and Xuan Wang. 2024. https://doi.org/10.18653/v1/2024.knowllm-1.11 P rompt RE : Weakly-supervised document-level relation extraction via prompting-based data programming . In Proceedings of the 1st Workshop on Towards Knowledgeable Language M...

    2024 doi

  7. [15]

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Art...

    2024 arXiv

  8. [16]

    Ralph Grishman. 1997. https://api.semanticscholar.org/CorpusID:17479975 Information extraction: Techniques and challenges . In International Summer School on Information Extraction

    1997

  9. [17]

    Pan, Huajun Chen, and Ningyu Zhang

    Honghao Gui, Shuofei Qiao, Jintian Zhang, Hongbin Ye, Mengshu Sun, Lei Liang, Jeff Z. Pan, Huajun Chen, and Ningyu Zhang. 2025. Instructie: A bilingual instruction-based information extraction dataset. In The Semantic Web -- ISWC 2024, pages 59--79, Cham. Springer Nature Switzerland

    2025

  10. [18]

    Xianpei Han and Le Sun. 2016. Global distant supervision for relation extraction. In Proceedings of the AAAI conference on artificial intelligence, volume 30

    2016

  11. [19]

    Geoffrey Hinton. 2015. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531

    2015 arXiv

  12. [20]

    Raphael Hoffmann, Congle Zhang, Xiao Ling, Luke Zettlemoyer, and Daniel S. Weld. 2011. https://aclanthology.org/P11-1055/ Knowledge-based weak supervision for information extraction of overlapping relations . In Proceedings of the 49th Annual Meeting of the Association for Com...

    2011

  13. [21]

    Martin Josifoski, Marija Sakota, Maxime Peyrard, and Robert West. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.96 Exploiting asymmetry for synthetic training data generation: S ynth IE and the case of information extraction . In Proceedings of the 2023 Conference on Empir...

    2023 doi

  14. [22]

    Omer Levy, Minjoon Seo, Eunsol Choi, and Luke Zettlemoyer. 2017. https://doi.org/10.18653/v1/K17-1034 Zero-shot relation extraction via reading comprehension . In Proceedings of the 21st Conference on Computational Natural Language Learning ( C o NLL 2017) , pages 333--342, Va...

    2017 doi

  15. [23]

    Dongfang Li, Baotian Hu, and Qingcai Chen. 2022. https://aclanthology.org/2022.coling-1.164/ Prompt-based text entailment for low-resource named entity recognition . In Proceedings of the 29th International Conference on Computational Linguistics, pages 1896--1903, Gyeongju, R...

    2022

  16. [24]

    Jiao Li, Yueping Sun, Robin J Johnson, Daniela Sciaky, Chih-Hsuan Wei, Robert Leaman, Allan Peter Davis, Carolyn J Mattingly, Thomas C Wiegers, and Zhiyong Lu. 2016. Biocreative v cdr task corpus: a resource for chemical disease relation extraction. Database, 2016

    2016

  17. [25]

    Peng Li, Tianxiang Sun, Qiong Tang, Hang Yan, Yuanbin Wu, Xuanjing Huang, and Xipeng Qiu. 2023. Codeie: Large code generation models are better few-shot information extractors. arXiv preprint arXiv:2305.05711

    2023 arXiv

  18. [26]

    Zixuan Li, Yutao Zeng, Yuxin Zuo, Weicheng Ren, Wenxuan Liu, Miao Su, Yucan Guo, Yantao Liu, Xiang Li, Zhilei Hu, and 1 others. 2024. Knowcoder: Coding structured knowledge into llms for universal information extraction. arXiv preprint arXiv:2403.07969

    2024 arXiv

  19. [27]

    Xincheng Liao, Junwen Duan, Yixi Huang, and Jianxin Wang. 2025. https://aclanthology.org/2025.coling-main.645/ RUIE : Retrieval-based unified information extraction using large language model . In Proceedings of the 31st International Conference on Computational Linguistics, p...

    2025

  20. [28]

    Yankai Lin, Shiqi Shen, Zhiyuan Liu, Huanbo Luan, and Maosong Sun. 2016. https://doi.org/10.18653/v1/P16-1200 Neural relation extraction with selective attention over instances . In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume...

    2016 doi

  21. [29]

    Jingjing Liu, Panupong Pasupat, Scott Cyphers, and James R. Glass. 2013. https://doi.org/10.1109/ICASSP.2013.6639301 Asgard: A portable architecture for multilingual dialogue systems . In IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2013, Va...

    2013

  22. [30]

    Zihan Liu, Yan Xu, Tiezheng Yu, Wenliang Dai, Ziwei Ji, Samuel Cahyawijaya, Andrea Madotto, and Pascale Fung. 2020. https://arxiv.org/abs/2012.04373 Crossner: Evaluating cross-domain named entity recognition

    2020 arXiv

  23. [31]

    Jie Lou, Yaojie Lu, Dai Dai, Wei Jia, Hongyu Lin, Xianpei Han, Le Sun, and Hua Wu. 2023 a . Universal information extraction as unified semantic matching. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 13318--13326

    2023

  24. [32]

    Jie Lou, Yaojie Lu, Dai Dai, Wei Jia, Hongyu Lin, Xianpei Han, Le Sun, and Hua Wu. 2023 b . https://doi.org/10.1609/aaai.v37i11.26563 Universal information extraction as unified semantic matching . In Proceedings of the Thirty-Seventh AAAI Conference on Artificial Intelligence...

    2023 doi

  25. [33]

    Mike Mintz, Steven Bills, Rion Snow, and Dan Jurafsky. 2009. Distant supervision for relation extraction without labeled data. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing...

    2009

  26. [34]

    Ying Mo, Jiahao Liu, Jian Yang, Qifan Wang, Shun Zhang, Jingang Wang, and Zhoujun Li. 2024. https://doi.org/10.18653/v1/2024.findings-emnlp.590 C - ICL : Contrastive in-context learning for information extraction . In Findings of the Association for Computational Linguistics: ...

    2024 doi

  27. [35]

    Abiola Obamuyide and Andreas Vlachos. 2018. https://doi.org/10.18653/v1/W18-5511 Zero-shot relation classification as textual entailment . In Proceedings of the First Workshop on Fact Extraction and VER ification ( FEVER ) , pages 72--78, Brussels, Belgium. Association for Com...

    2018 doi

  28. [36]

    Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and...

    2022 arXiv

  29. [37]

    Chaoxu Pang, Yixuan Cao, Qiang Ding, and Ping Luo. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.950 Guideline learning for in-context information extraction . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 15372--15389, Si...

    2023 doi

  30. [38]

    Guilherme Penedo, Hynek Kydl \' c ek, Loubna Ben allal, Anton Lozhkov, Margaret Mitchell, Colin Raffel, Leandro Von Werra, and Thomas Wolf. 2024. https://openreview.net/forum?id=n6SCkn2QaG The fineweb datasets: Decanting the web for the finest text data at scale . In The Thirt...

    2024

  31. [39]

    Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Hwee Tou Ng, Anders Bj \"o rkelund, Olga Uryupina, Yuchen Zhang, and Zhi Zhong. 2013. Towards robust linguistic analysis using ontonotes. In Proceedings of the Seventeenth Conference on Computational Natural Language Learning,...

    2013

  32. [40]

    Yunjia Qi, Hao Peng, Xiaozhi Wang, Bin Xu, Lei Hou, and Juanzi Li. 2024. Adelie: Aligning large language models on information extraction. arXiv preprint arXiv:2405.05008

    2024 arXiv

  33. [41]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res., 21(1)

    2020

  34. [42]

    Oscar Sainz, Oier Lopez de Lacalle, Gorka Labaka, Ander Barrena, and Eneko Agirre. 2021. Label verbalization and entailment for effective zero-and few-shot relation extraction. arXiv preprint arXiv:2109.03659

    2021 arXiv

  35. [43]

    Oscar Sainz, Iker Garc \' a-Ferrero, Rodrigo Agerri, Oier Lopez de Lacalle, German Rigau, and Eneko Agirre. 2024. https://openreview.net/forum?id=Y3wpuxd7u9 Go LLIE : Annotation guidelines improve zero-shot information-extraction . In The Twelfth International Conference on Le...

    2024

  36. [44]

    Mihai Surdeanu, Julie Tibshirani, Ramesh Nallapati, and Christopher D Manning. 2012. Multi-instance multi-label learning for relation extraction. In Proceedings of the 2012 joint conference on empirical methods in natural language processing and computational natural language ...

    2012

  37. [45]

    Teknium. 2023. https://huggingface.co/datasets/teknium/OpenHermes-2.5 Openhermes 2.5: An open dataset of synthetic data for generalist llm assistants

    2023

  38. [46]

    Tjong Kim Sang and Fien De Meulder

    Erik F. Tjong Kim Sang and Fien De Meulder. 2003. https://aclanthology.org/W03-0419/ Introduction to the C o NLL -2003 shared task: Language-independent named entity recognition . In Proceedings of the Seventh Conference on Natural Language Learning at HLT - NAACL 2003 , pages...

    2003

  39. [47]

    Christopher Walker, Stephanie Strassel, Julie Medero, and Kazuaki Maeda. 2006. https://catalog.ldc.upenn.edu/LDC2006T06 ACE 2005 multilingual training corpus LDC2006T06

    2006

  40. [48]

    Chenguang Wang, Xiao Liu, Zui Chen, Haoyun Hong, Jie Tang, and Dawn Song. 2021. https://doi.org/10.18653/v1/2021.emnlp-main.94 Zero-shot information extraction as a unified text-to-triple translation . In Proceedings of the 2021 Conference on Empirical Methods in Natural Langu...

    2021 doi

  41. [49]

    Xiao Wang, Weikang Zhou, Can Zu, Han Xia, Tianze Chen, Yuansen Zhang, Rui Zheng, Junjie Ye, Qi Zhang, Tao Gui, Jihua Kang, Jingsheng Yang, Siyuan Li, and Chunsai Du. 2023. https://arxiv.org/abs/2304.08085 Instructuie: Multi-task instruction tuning for unified information extra...

    2023 arXiv

  42. [51]

    Xiang Wei, Xingyu Cui, Ning Cheng, Xiaobin Wang, Xin Zhang, Shen Huang, Pengjun Xie, Jinan Xu, Yufeng Chen, Meishan Zhang, and 1 others. 2023. Zero-shot information extraction via chatting with chatgpt. arXiv preprint arXiv:2302.10205

    2023 arXiv

  43. [52]

    Chaojun Xiao, Yuan Yao, Ruobing Xie, Xu Han, Zhiyuan Liu, Maosong Sun, Fen Lin, and Leyu Lin. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.300 Denoising relation extraction from document-level distant supervision . In Proceedings of the 2020 Conference on Empirical Method...

    2020 doi

  44. [53]

    Derong Xu, Wei Chen, Wenjun Peng, Chao Zhang, Tong Xu, Xiangyu Zhao, Xian Wu, Yefeng Zheng, Yang Wang, and Enhong Chen. 2024 a . Large language models for generative information extraction: A survey. Frontiers of Computer Science, 18(6):186357

    2024

  45. [54]

    Jun Xu, Mengshu Sun, Zhiqiang Zhang, and Jun Zhou. 2024 b . https://aclanthology.org/2024.lrec-main.279/ C hat UIE : Exploring chat-based unified information extraction using large language models . In Proceedings of the 2024 Joint International Conference on Computational Lin...

    2024

  46. [55]

    Zhangchen Xu, Fengqing Jiang, Luyao Niu, Yuntian Deng, Radha Poovendran, Yejin Choi, and Bill Yuchen Lin. 2025. https://openreview.net/forum?id=Pnk7vMbznK Magpie: Alignment data synthesis from scratch by prompting aligned LLM s with nothing . In The Thirteenth International Co...

    2025

  47. [56]

    Urchade Zaratiana, Nadi Tomeh, Pierre Holat, and Thierry Charnois. 2023. https://arxiv.org/abs/2311.08526 Gliner: Generalist model for named entity recognition using bidirectional transformer . Preprint, arXiv:2311.08526

    2023 arXiv

  48. [57]

    Yuhao Zhang, Victor Zhong, Danqi Chen, Gabor Angeli, and Christopher D Manning. 2017. Position-aware attention and supervised data improve slot filling. In Conference on empirical methods in natural language processing

    2017

  49. [58]

    Wenxuan Zhou, Sheng Zhang, Yu Gu, Muhao Chen, and Hoifung Poon. 2023. https://arxiv.org/abs/2308.03279 Universalner: Targeted distillation from large language models for open named entity recognition

    2023 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.