Generating Diverse Training Samples for Relation Extraction with Large Language Models

Hongliang Dai; Piji Li; Zexuan Li

REVIEW 3 major objections 5 minor 36 references

Generating Diverse Training Samples for Relation Extraction with Large Language Models

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Direct Preference Optimization makes LLMs generate relation-extraction training samples that are both more diverse and more correct, and those samples improve few-shot models.

desk verdict A useful, honestly-reported recipe for diverse LLM-generated RE samples, weakened by test-set selection of sample count and by transfer experiments that stay inside one dataset. read the letter →

arxiv 2505.23108 v1 pith:RW5HNGQ3 submitted 2025-05-29 cs.CL

Zexuan Li , Hongliang Dai , Piji Li This is my paper

classification cs.CL

keywords relationextractiondatagenerationdirectpreferenceoptimizationin-contextlearningdiversityfew-shotlargelanguagemodelstrainingsample

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that large language models, when directly prompted to create relation-extraction training data, tend to produce sentences that are structurally similar and use a narrow set of relation verbs. It proposes two remedies: an in-context learning prompt that asks for dissimilar samples, and a DPO fine-tuning step that teaches the model to prefer diverse, correct samples over automatically constructed undesirable ones. The authors show that training a small non-LLM relation-extraction model on these generated samples can outperform using the LLM directly for relation extraction, and that mixing generated data with a few human-labeled examples improves performance further. The central claim is that DPO, trained on relation categories separate from the ones used for generation, instills a transferable preference for diversity and correctness.

What carries the argument

The mechanism is DPO fine-tuning driven by automatically constructed preference pairs, combined with a one-by-one generation loop. The preference pair is an instruction prompt containing task description, a relation explanation, and a growing set of demonstration samples; the preferred response is a real human-labeled sample, and the dispreferred response is one of three automatically generated negatives: a relabeled other-relation sample, an entity-substituted context, or an exact copy of a demonstration. This teaches the model to avoid both incorrect annotations and repetitive imitation while preserving correctness. The one-by-one loop appends every generated sample to the demonstration set, so the model is always instructed to produce something different from what already exists.

What would settle it

Fine-tune the LLM with DPO on one domain's relation categories (for example, biomedical relations) and generate samples for a different domain's relations (such as news or Wikipedia). If the average pairwise cosine similarity of generated samples is not significantly lower than without DPO, and downstream few-shot F1 does not improve, then the claimed transferable diversity does not hold across domains.

Watch

Extended reading notes

Core claim

The paper establishes that prompting an LLM to generate relation-extraction training samples one by one, while adding each new sample back to the prompt, reduces structural similarity, and that fine-tuning the LLM with DPO using automatically constructed dispreferred samples substantially improves the quality of the generated data. Three types of dispreferred samples are used: a sample from another relation relabeled as the target relation, an entity-substituted and lightly edited copy of a demonstration, and an exact copy of a demonstration. The DPO preference data pairs a real human-labeled sample (preferred) with these constructed negatives (dispreferred), and the relation categories used for DPO fine-tuning are disjoint from those used for generation, preventing the model from simply memorizing the target relations. The resulting generated samples, when used to train KnowPrompt or RetrievalRE, yield higher Micro F1 than using manually labeled samples alone in several few-shot settings, and almost always beat directly prompting the LLM for relation extraction.

Load-bearing premise

The DPO fine-tuning is done on relation categories that are disjoint from the ones used for generation, so the method only works if the preference for diverse and correct phrasing learned on one set of relation types transfers to unseen relation types.

Editorial extensions

If this is right

LLM-generated training data can substitute for direct LLM prompting in few-shot relation extraction, cutting inference cost while improving accuracy.
Combining a small number of human-labeled samples with LLM-generated samples raises few-shot F1 beyond what either source achieves alone, especially when the human set is very small.
The automatically constructed dispreferred samples remove the need for human preference annotations in DPO for this data-generation task.
The diversity gains saturate near 32 generated samples, indicating a practical ceiling to how much variety an LLM can produce for a given relation without additional sources of novelty.
The disjoint-split evaluation shows that DPO fine-tuning transfers a notion of diversity and correctness to relation categories never seen during fine-tuning.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The three dispreferred sample types each target a distinct failure mode, so ablating them individually could reveal which mode matters most; the paper does not isolate their contributions.
The diversity ceiling near 32 samples suggests that the LLM's underlying knowledge, not the prompting strategy, limits novelty, so coupling generation with external corpora or retrieval could push the ceiling higher.
The same DPO recipe might transfer to other structured generation tasks, such as event extraction or slot filling, where repetitive phrasing also harms training-data quality.
Because the paper uses a single base LLM and two downstream RE models, the generality across model families and downstream architectures remains an open question the authors do not test.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: this is a solid, honestly written practical paper about making LLM-generated RE training data more diverse, with a real methodological soft spot (test-set selection of sample count) and an untested transfer claim (DPO generalization across domains). It deserves review, but it needs revision.

The new thing is using DPO to fine-tune an LLM to generate relation-extraction training samples that are both diverse and correct, with dispreferred samples constructed automatically from three sources: relabeled other-relation samples, entity-substituted variants, and duplicates of demonstrations. That is a reasonable application of an existing method, and the one-by-one ICL procedure—where each generated sample is appended to the prompt—is a simple extension that works in practice. The diversity metrics (cosine similarity, repetition rate) directly support the claim that DPO and the diversity instruction reduce inter-sample similarity, and the ablations mostly confirm the contribution of DPO and the diversity instruction.

The soft spots are real but fixable. First, the mixed-training results in Table 2, which are the strongest numbers, are obtained by choosing the number of generated samples (K in 8/16/32) that gives the best test performance. That is selection on the test set; it inflates the reported gains. They need to choose K on a validation split or report all K values without selecting. Second, there are no error bars or repeated-seed statistics anywhere, so we cannot tell if the differences are noise. Third, the DPO transfer is only tested within a dataset: half the relation categories are used for DPO fine-tuning and the other half for generation. The paper explicitly says DPO source should come from other datasets or domains in a data-scarce scenario, but no cross-domain experiment is run. That is the load-bearing assumption—that the learned preference for diversity/correctness transfers to new relations with different vocabulary and syntax. It may or may not hold; the current evidence does not establish it. Fourth, some ablations contradict the main story (for example, on TACRED at K=32, the AAO without DPO beats the DPO version by a large margin, and the OBO vs AAO comparison flips across datasets). The paper notes these exceptions but explains them ad hoc.

Who this is for: people doing data generation with LLMs for few-shot relation extraction, or more broadly, anyone interested in using DPO to shape LLM output diversity. The paper is not a major theoretical advance, but it is a useful recipe with code.

My recommendation: send it to peer review, but require a revision that fixes the K selection, adds error bars, and reports a cross-domain transfer experiment. Without those, the central claim is not established.

Referee Report

3 major / 5 minor

Summary. This paper addresses the problem of generating diverse and correct training samples for relation extraction (RE) using large language models (LLMs). It first shows that directly prompting an LLM (LLaMA2-7B-Chat) with in-context examples tends to produce structurally similar samples, then proposes two remedies: (i) an ICL protocol that generates samples one by one (OBO) while adding each new sample back into the prompt with an explicit diversity instruction, and (ii) Direct Preference Optimization (DPO) fine-tuning of the LLM using automatically constructed preference/dispreference pairs, where dispreferred samples are relabeled other-relation examples, entity-substituted variants of demonstrations, or exact copies of demonstrations. The authors evaluate the generated samples by measuring average pairwise cosine similarity and word repetition rate, and by training two non-LLM RE models (KnowPrompt and RetrievalRE) on the generated samples under 8-, 16-, and 32-shot settings, comparing against direct LLM-based RE and a prior data-generation baseline on TACRED, TACRED-Revisit, Re-TACRED, and SemEval. They report that DPO fine-tuning and OBO generation improve both diversity and downstream few-shot F1, and that training a small RE model on LLM-generated samples can outperform directly prompting the LLM for RE.

Significance. If the reported results hold, this paper makes a useful practical contribution: it offers a low-annotation pipeline that combines ICL prompting, DPO fine-tuning, and automatic dispreference construction to produce training data for few-shot RE, and it provides evidence that a non-LLM RE model trained on such data can beat direct LLM prompting. The paper is clearly written and includes several strengths: the code is released, the prompts and relation explanations are described in detail, comparisons are made against a data-generation baseline and against direct LLM RE, and diversity is measured with two complementary metrics (cosine similarity and word repetition rate). The internal diversity analysis in Figures 5 and 6 supports the claim that DPO reduces sample similarity. However, the downstream evaluation has load-bearing methodological weaknesses: the mixed-training results select the number of generated samples on the test set, no error bars or significance tests are reported, and the DPO transferability claim is tested only within the same dataset and domain, not across domains. These issues currently limit the strength of the central claims.

major comments (3)

[§5.1, Table 2] The mixed-training results, which are the strongest numbers in Table 2, are obtained by choosing, for each cell, the number of LLM-generated samples in [8, 16, 32] that gives the best performance for KnowPrompt and RetrievalRE. The paper states: 'we choose the value in [8, 16, 32] that gives the best performance.' This is test-set selection of a hyperparameter and makes these F1 scores an oracle estimate rather than a valid estimate of expected performance. The authors should either fix the generated-sample count a priori, select it on a validation split, or report performance for each generated-sample count without selecting the best per test set.
[§3.2, §4.3] The DPO fine-tuning scenario is motivated by data scarcity and the paper says in §3.2 that the source data for DPO 'should come from other already existing datasets that probably use different relation types or are of different domains.' However, the experiments in §4.3 split relation categories within each dataset and never evaluate cross-dataset or cross-domain transfer. All DPO and generation relations come from the same corpus and domain (e.g., TACRED news text), so the learned preference for diversity and correctness may not transfer to genuinely new domains with different entity types and surface forms. A cross-domain experiment (e.g., DPO on TACRED with generation and evaluation on SemEval, or on a different domain corpus) is needed to support the central claim that the method helps for data-scarce new domains. At minimum, the paper should limit the claim to within-domain unseen relation categories.
[§5, Tables 2 and 3] No standard deviations, confidence intervals, or significance tests are reported for any downstream F1 numbers. Many comparisons in Tables 2 and 3 differ by less than 1 F1 point (e.g., Ours (mix-OBO) 34.33 vs Ours (mix-AAO) 34.15 on TACRED-Revisit K=8; Ours (mix-OBO) 35.83 vs Ours (mix-AAO) 36.57 on TACRED K=16 for RetrievalRE), so without error bars the claimed improvements are not quantitatively secure. The authors should report means and standard deviations over multiple seeds (both for LLM generation and downstream training) and, where appropriate, use a pairwise significance test.

minor comments (5)

[Table 3] In the row 'Ours (AAO), w/o DPO', the first three values appear to be a formatting error: '18.9727.70 36.04' should likely read '18.97 27.70 36.04'.
[§1, §5.1] There are several typos: 'approch' in Section 1, 'identival' in Section 5.1, and inconsistent use of 'non-preference data' versus 'dispreferred data' in Section 3.2. These should be corrected.
[§5.1, Table 2] The text says 'For our methods, 64 samples are generated for training,' but Table 2 reports results for K=8, K=16, and K=32. It should be clarified whether K denotes the number of generated samples actually used for training and whether 64 is the maximum number generated before further selection.
[§5.3] The diversity metrics in Figures 5 and 6 are computed only for OBO generation, not for AAO generation, so the relative diversity of AAO versus OBO is not directly measured. Reporting the same metrics for AAO would make the comparison complete.
[§5.2, Table 3] The result on Re-TACRED K=16, where Ours (OBO) w/o DPO (41.47) outperforms Ours (OBO) with DPO (40.33), contradicts the general claim that DPO improves quality. The post-hoc explanation that the samples happen to be closer to the test data is plausible but speculative; this outlier should be discussed more carefully or addressed with repeated runs.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: DPO and generation relations are disjoint, evaluation uses held-out test data, and no load-bearing self-citations or fitted-then-predicted quantities were found.

full rationale

The paper's central derivation is self-contained against external benchmarks. DPO fine-tuning is performed on relation categories explicitly separated from the generation categories (Section 4.3: 'we separated the relation categories used for DPO fine-tuning from those to be generated'), so the target relations' test labels are not used to fit the generator. The downstream evaluation trains KnowPrompt and RetrievalRE on generated samples and measures Micro F1 on held-out test splits, which is an external benchmark rather than a quantity reconstructed from the model's own outputs. The diversity metric is related to the DPO training objective, but the paper's main claim—that such samples improve few-shot RE performance—is tested downstream and does not reduce to the construction of the preference data. There is no fitted parameter renamed as a prediction, and no self-citation is load-bearing; the references to KnowPrompt, RetrievalRE, and DPO are external prior work. Limitations about LLM strength and maximum useful sample count are acknowledged and do not indicate circularity.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on several choices that are not derived from first principles: the DPO objective is taken as given, the transfer of preference signals across relation categories is assumed, and the diversity metrics are not validated as the causal driver of downstream gains. The main free parameter is the number of generated samples, which is selected on the test set in mixed-training experiments.

free parameters (3)

Number of generated training samples (K) for mixed training = per cell, one of 8, 16, 32, chosen for best test F1
Section 5.1: 'we choose the value in [8, 16, 32] that gives the best performance for KnowPrompt and RetrievalRE.' This tunes a hyperparameter on the test set.
Sampling hyperparameters for LLaMA2-7B-Chat = temperature 0.4, top_p 0.9, top_k 20, repetition_penalty 1.15
Section 4.3; chosen by hand, no sensitivity analysis is provided.
LoRA and DPO training hyperparameters = lr 5e-5, batch size 4, epochs 20, truncation length 1024
Section 4.3; chosen by hand, and DPO beta and LoRA rank are not specified.

assumptions (4)

standard math The DPO objective in Eq. 1 is a valid way to optimize model behavior on preference data.
The paper relies on Rafailov et al. 2023 and applies Eq. 1 without modification.
domain assumption Manually labeled training samples from relation categories not used for generation are a suitable source of preference data.
Sections 3.2 and 4.3: DPO is trained on a disjoint half of relation categories, and the authors assume this transfers to the other half.
domain assumption Cosine similarity and word repetition rate capture the diversity that matters for downstream RE training.
Section 5.3 uses these metrics as evidence of improvement; no human evaluation or downstream ablation ties these metrics directly to F1.
domain assumption Generated samples can be treated as correctly labeled for training without filtering.
The pipeline does not validate labels of generated samples; the authors rely on DPO and prompt design to maintain correctness.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Generating Diverse Training Samples for Relation Extraction with Large Language Models." pith.science (2026). https://pith.science/paper/RW5HNGQ3

@misc{pith2026250523108,
  author       = {Pith},
  title        = {Pith review of: Generating Diverse Training Samples for Relation Extraction with Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RW5HNGQ3}},
  note         = {Machine review of arXiv:2505.23108}
}

read the original abstract

Using Large Language Models (LLMs) to generate training data can potentially be a preferable way to improve zero or few-shot NLP tasks. However, many problems remain to be investigated for this direction. For the task of Relation Extraction (RE), we find that samples generated by directly prompting LLMs may easily have high structural similarities with each other. They tend to use a limited variety of phrasing while expressing the relation between a pair of entities. Therefore, in this paper, we study how to effectively improve the diversity of the training samples generated with LLMs for RE, while also maintaining their correctness. We first try to make the LLMs produce dissimilar samples by directly giving instructions in In-Context Learning (ICL) prompts. Then, we propose an approach to fine-tune LLMs for diversity training sample generation through Direct Preference Optimization (DPO). Our experiments on commonly used RE datasets show that both attempts can improve the quality of the generated training data. We also find that comparing with directly performing RE with an LLM, training a non-LLM RE model with its generated samples may lead to better performance.

Figures

Figures reproduced from arXiv: 2505.23108 by the authors.

**Figure 2.** Construction of Prompt, which consists of three modules: Task Description Module, Relation Explanation [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** Construction of the DPO Fine-Tuning Training Dataset. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Figures from the paper (4 more)

**Figure 4.** Figure 4: Imitating one by one generation during Direct [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]

**Figure 5.** Figure 5: Average cosine similarity between generated training samples (K=32) for each relation category. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

**Figure 6.** Figure 6: Average repetition rate of words between generated training samples (K=32) for each relation category. [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]

**Figure 7.** Figure 7: Micro F1 (%) of KnowPrompt using different [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

36 extracted references · 2 canonical work pages

[1]

URL: " 'urlintro :=

ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...
[2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
[3]

Christoph Alt, Aleksandra Gabryszak, and Leonhard Hennig. 2020. https://doi.org/10.18653/V1/2020.ACL-MAIN.142 TACRED revisited: A thorough evaluation of the TACRED relation extraction task . In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020 , pages 1558--1569. Association for Comp...

doi:10.18653/v1/2020.acl-main.142 2020
[4]

Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert - Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litw...

2020
[5]

Xiang Chen, Lei Li, Ningyu Zhang, Chuanqi Tan, Fei Huang, Luo Si, and Huajun Chen. 2022 a . https://doi.org/10.1145/3477495.3531746 Relation extraction as open-book examination: Retrieval-enhanced prompt tuning . In SIGIR '22: The 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, Madrid, Spain, July 11 - 15, 202...

arXiv 2022
[6]

Xiang Chen, Ningyu Zhang, Xin Xie, Shumin Deng, Yunzhi Yao, Chuanqi Tan, Fei Huang, Luo Si, and Huajun Chen. 2022 b . https://doi.org/10.1145/3485447.3511998 Knowprompt: Knowledge-aware prompt-tuning with synergistic optimization for relation extraction . In WWW '22: The ACM Web Conference 2022, Virtual Event, Lyon, France, April 25 - 29, 2022 , pages 277...

arXiv 2022
[7]

Yew Ken Chia, Lidong Bing, Soujanya Poria, and Luo Si. 2022. https://doi.org/10.18653/V1/2022.FINDINGS-ACL.5 Relationprompt: Leveraging prompts to generate synthetic data for zero-shot relation triplet extraction . In Findings of the Association for Computational Linguistics: ACL 2022, Dublin, Ireland, May 22-27, 2022 , pages 45--57. Association for Compu...

doi:10.18653/v1/2022.findings-acl.5 2022
[8]

Jacob Devlin, Ming - Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. https://doi.org/10.18653/V1/N19-1423 BERT: pre-training of deep bidirectional transformers for language understanding . In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, ...

doi:10.18653/v1/n19-1423 2019

Show all 36 references

[9]

Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Zhiyong Wu, Baobao Chang, Xu Sun, Jingjing Xu, Lei Li, and Zhifang Sui. 2023. https://doi.org/10.48550/ARXIV.2301.00234 A survey for in-context learning . CoRR, abs/2301.00234

2023 doi arXiv
[10]

Jiahui Gao, Renjie Pi, Yong Lin, Hang Xu, Jiacheng Ye, Zhiyong Wu, Weizhong Zhang, Xiaodan Liang, Zhenguo Li, and Lingpeng Kong. 2023. https://openreview.net/forum?id=h5OpjGd\_lo6 Self-guided noise-free data generation for efficient zero-shot learning . In The Eleventh Interna...

2023
[11]

Zhijiang Guo, Guoshun Nan, Wei Lu, and Shay B. Cohen. 2020. https://doi.org/10.24963/IJCAI.2020/505 Learning latent forests for medical relation extraction . In Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI 2020 , pages 3651--...

2020 doi
[12]

Zhijiang Guo, Yan Zhang, and Wei Lu. 2019. https://doi.org/10.18653/V1/P19-1024 Attention guided graph convolutional networks for relation extraction . In Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- A...

2019 doi
[13]

Iris Hendrickx, Su Nam Kim, Zornitsa Kozareva, Preslav Nakov, Diarmuid \' O S \' e aghdha, Sebastian Pad \' o , Marco Pennacchiotti, Lorenza Romano, and Stan Szpakowicz. 2010. https://aclanthology.org/S10-1006/ Semeval-2010 task 8: Multi-way classification of semantic relation...

2010
[14]

Guozheng Li, Peng Wang, and Wenjun Ke. 2023. https://doi.org/10.18653/V1/2023.FINDINGS-EMNLP.459 Revisiting large language models as zero-shot relation extractors . In Findings of the Association for Computational Linguistics: EMNLP 2023, Singapore, December 6-10, 2023 , pages...

2023 doi
[15]

Yang Liu, Jinpeng Hu, Xiang Wan, and Tsung - Hui Chang. 2022. https://doi.org/10.18653/V1/2022.FINDINGS-NAACL.139 Learn from relation information: Towards prototype representation rectification for few-shot relation extraction . In Findings of the Association for Computational...

2022 doi
[16]

Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. http://arxiv.org/abs/1907.11692 Roberta: A robustly optimized BERT pretraining approach . CoRR, abs/1907.11692

2019 arXiv
[17]

Yu Meng, Jiaxin Huang, Yu Zhang, and Jiawei Han. 2022. http://papers.nips.cc/paper\_files/paper/2022/hash/0346c148ba1c21c6b4780a961ea141dc-Abstract-Conference.html Generating training data with language models: Towards zero-shot language understanding . In Advances in Neural I...

2022
[18]

OpenAI. 2022. https://openai.com/blog/ chatgpt/ Chatgpt: Optimizing language models for dialogue

2022
[19]

OpenAI. 2023. Gpt-4 technical report. arXiv preprint, abs/2303.08774

2023 arXiv
[20]

Giovanni Paolini, Ben Athiwaratkun, Jason Krone, Jie Ma, Alessandro Achille, Rishita Anubhai, C \' cero Nogueira dos Santos, Bing Xiang, and Stefano Soatto. 2021. https://openreview.net/forum?id=US-TP-xnXI Structured prediction as translation between augmented natural language...

2021
[21]

Meng Qu, Tianyu Gao, Louis - Pascal A. C. Xhonneux, and Jian Tang. 2020. http://proceedings.mlr.press/v119/qu20a.html Few-shot relation extraction via bayesian meta-learning on relation graphs . In Proceedings of the 37th International Conference on Machine Learning, ICML 2020...

2020
[22]

Manning, Stefano Ermon, and Chelsea Finn

Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning, Stefano Ermon, and Chelsea Finn. 2023. http://papers.nips.cc/paper\_files/paper/2023/hash/a85b405ed65c6477a4fe8302b5e06ce7-Abstract-Conference.html Direct preference optimization: Your language model is sec...

2023
[23]

Srihari and Wei Li

Rohini K. Srihari and Wei Li. 2000. https://doi.org/10.3115/974147.974170 A question answering system supported by information extraction . In 6th Applied Natural Language Processing Conference, ANLP 2000, Seattle, Washington, USA, April 29 - May 4, 2000 , pages 166--172. ACL

2000
[24]

George Stoica, Emmanouil Antonios Platanios, and Barnab \' a s P \' o czos. 2021. https://doi.org/10.1609/AAAI.V35I15.17631 Re-tacred: Addressing shortcomings of the TACRED dataset . In Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference...

2021 doi
[25]

Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton - Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu,...

2023 doi arXiv
[26]

Zhen Wan, Fei Cheng, Zhuoyuan Mao, Qianying Liu, Haiyue Song, Jiwei Li, and Sadao Kurohashi. 2023. https://doi.org/10.18653/V1/2023.EMNLP-MAIN.214 GPT-RE: in-context learning for relation extraction using large language models . In Proceedings of the 2023 Conference on Empiric...

2023 doi
[27]

Smith, Daniel Khashabi, and Hannaneh Hajishirzi

Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. 2023. https://doi.org/10.18653/V1/2023.ACL-LONG.754 Self-instruct: Aligning language models with self-generated instructions . In Proceedings of the 61st Annual Mee...

2023 doi
[28]

Xin Xu, Yuqi Zhu, Xiaohan Wang, and Ningyu Zhang. 2023. https://doi.org/10.18653/V1/2023.SUSTAINLP-1.13 How to unleash the power of large language models for few-shot relation extraction? In Proceedings of The Fourth Workshop on Simple and Efficient Natural Language Processing...

2023 doi
[29]

Jiacheng Ye, Jiahui Gao, Qintong Li, Hang Xu, Jiangtao Feng, Zhiyong Wu, Tao Yu, and Lingpeng Kong. 2022. https://doi.org/10.18653/V1/2022.EMNLP-MAIN.801 Zerogen: Efficient zero-shot learning via dataset generation . In Proceedings of the 2022 Conference on Empirical Methods i...

2022 doi
[30]

Daojian Zeng, Kang Liu, Yubo Chen, and Jun Zhao. 2015. https://doi.org/10.18653/V1/D15-1203 Distant supervision for relation extraction via piecewise convolutional neural networks . In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, EMNL...

2015 doi
[31]

Kai Zhang, Bernal Jimenez Gutierrez, and Yu Su. 2023. https://doi.org/10.18653/V1/2023.FINDINGS-ACL.50 Aligning instruction tasks unlocks large language models as zero-shot relation extractors . In Findings of the Association for Computational Linguistics: ACL 2023, Toronto, C...

2023 doi
[32]

Yuhao Zhang, Peng Qi, and Christopher D. Manning. 2018. https://doi.org/10.18653/V1/D18-1244 Graph convolution over pruned dependency trees improves relation extraction . In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgi...

2018 doi
[33]

Yuhao Zhang, Victor Zhong, Danqi Chen, Gabor Angeli, and Christopher D. Manning. 2017. https://doi.org/10.18653/V1/D17-1004 Position-aware attention and supervised data improve slot filling . In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Proces...

2017 doi
[34]

Lingfeng Zhong, Jia Wu, Qian Li, Hao Peng, and Xindong Wu. 2024. https://doi.org/10.1145/3618295 A comprehensive survey on automatic knowledge graph construction . ACM Comput. Surv. , 56(4):94:1--94:62

2024 doi
[35]

Peng Zhou, Wei Shi, Jun Tian, Zhenyu Qi, Bingchen Li, Hongwei Hao, and Bo Xu. 2016. https://doi.org/10.18653/V1/P16-2034 Attention-based bidirectional long short-term memory networks for relation classification . In Proceedings of the 54th Annual Meeting of the Association for...

2016 doi
[36]

Wenxuan Zhou and Muhao Chen. 2022. https://aclanthology.org/2022.aacl-short.21 An improved baseline for sentence-level relation extraction . In Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th Internati...

2022

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

URL: " 'urlintro :=

ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...

[2] [2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

[3] [3]

Christoph Alt, Aleksandra Gabryszak, and Leonhard Hennig. 2020. https://doi.org/10.18653/V1/2020.ACL-MAIN.142 TACRED revisited: A thorough evaluation of the TACRED relation extraction task . In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020 , pages 1558--1569. Association for Comp...

doi:10.18653/v1/2020.acl-main.142 2020

[4] [4]

Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert - Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litw...

2020

[5] [5]

Xiang Chen, Lei Li, Ningyu Zhang, Chuanqi Tan, Fei Huang, Luo Si, and Huajun Chen. 2022 a . https://doi.org/10.1145/3477495.3531746 Relation extraction as open-book examination: Retrieval-enhanced prompt tuning . In SIGIR '22: The 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, Madrid, Spain, July 11 - 15, 202...

arXiv 2022

[6] [6]

Xiang Chen, Ningyu Zhang, Xin Xie, Shumin Deng, Yunzhi Yao, Chuanqi Tan, Fei Huang, Luo Si, and Huajun Chen. 2022 b . https://doi.org/10.1145/3485447.3511998 Knowprompt: Knowledge-aware prompt-tuning with synergistic optimization for relation extraction . In WWW '22: The ACM Web Conference 2022, Virtual Event, Lyon, France, April 25 - 29, 2022 , pages 277...

arXiv 2022

[7] [7]

Yew Ken Chia, Lidong Bing, Soujanya Poria, and Luo Si. 2022. https://doi.org/10.18653/V1/2022.FINDINGS-ACL.5 Relationprompt: Leveraging prompts to generate synthetic data for zero-shot relation triplet extraction . In Findings of the Association for Computational Linguistics: ACL 2022, Dublin, Ireland, May 22-27, 2022 , pages 45--57. Association for Compu...

doi:10.18653/v1/2022.findings-acl.5 2022

[8] [8]

Jacob Devlin, Ming - Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. https://doi.org/10.18653/V1/N19-1423 BERT: pre-training of deep bidirectional transformers for language understanding . In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, ...

doi:10.18653/v1/n19-1423 2019