Pith. sign in

REVIEW 4 major objections 7 minor 38 references

CONSISTRE: A Unified Consistency-Aware Framework for Document-Level Relation Extraction with Large Language Models

T0 review · 4 major / 7 minor · reviewed 2026-07-31 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read LLM document relation extractors become reliable when forced to obey transitivity, symmetry, and uniqueness—either at inference or by training small models on that pressure.

desk verdict Useful dual-track DocRE systems paper with a real structured-vs-generic reflection result, but the headline Track A gain is under-explained and eval is subset-soft. read the letter →

arxiv 2607.24312 v1 pith:LO4HDFEQ submitted 2026-07-27 cs.CL

Mingxuan Sun This is my paper
classification cs.CL
keywords document-levelrelationextractionlargelanguagemodelsrelationalconsistencyself-reflectionknowledgedistillationreinforcementlearningGRPODocRE
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Large language models can pull relations out of long documents, but they usually score each entity pair alone, so the full set of triples often contradicts itself—missing transitive links, one-sided inverse pairs, or multiple answers where only one is allowed. CONSISTRE treats that global coherence as the missing objective. It defines three shared constraints (transitivity, symmetry/inverse, functional uniqueness) and applies them in two ways: at inference, black-box models are prompted with the rules, checked for violations, and asked to revise; at training, a strong teacher’s consistency-aware reasoning is distilled into 7–8B open models, then reinforced with a reward that mixes extraction F1 and a consistency score. On DocRED both paths beat their baselines, the inference path reaches competitive F1 without fine-tuning, and the training path brings compact local models close to proprietary API systems at far lower cost. The point for a reader is practical: consistency is not a post-hoc cleanup—it is a controllable signal that makes generative DocRE trustworthy under both API and on-prem deployment.

What carries the argument

The shared consistency score Cons(Ŷ)=1−V(Ŷ)/(T(Ŷ)+ε), built from counts of triggered vs violated transitivity, inverse, and functional-uniqueness instances. Track A uses it for verification and targeted self-reflection; Track B uses it inside the GRPO reward after gold-aligned teacher-trace SFT, so both tracks optimize the same structural target.

What would settle it

Rerun Track A’s best 5-shot CoT+self-reflection setup and Track B’s Qwen3-8B SFT+GRPO pipeline on the full DocRED dev set (or Re-DocRED) with the same Cons definition; if the F1/Cons gains over baselines and the ~0.04 F1 gap to the proprietary Track A peak disappear or reverse, the central transfer and reliability claims fail.

Watch

Extended reading notes

Core claim

The paper claims that framing document-level relation extraction as consistency-aware structured prediction—enforcing a unified set of transitivity, symmetry/inverse, and functional-uniqueness constraints—lets LLM systems cut relational contradictions and raise extraction quality, whether the constraints are applied only at inference on frozen black-box models or internalized in smaller open-source students through teacher-trace distillation plus GRPO with a composite F1-plus-consistency reward.

Load-bearing premise

That scores on a chosen 200-document DocRED slice (with only a small ranking check on 50 docs) are a faithful stand-in for full-benchmark behavior and for the claimed gap-closing between small open models and proprietary APIs.

Editorial extensions

If this is right

  • Black-box DocRE APIs can raise coherence without any fine-tuning by adding constraint cues, violation feedback, and bounded reflection.
  • 7–8B open models can approach proprietary DocRE F1 and consistency after reasoning-trace distillation plus a consistency-weighted GRPO reward, at a fraction of inference cost.
  • Generic free-form self-reflection alone can hurt DocRE F1; structured violation feedback is what turns reflection into a gain.
  • SFT mainly installs local inverse symmetry; GRPO is what repairs multi-hop transitivity violations—so the two stages target different error types.
  • The same Cons formulation can be swapped into either verification or reward design without changing the rest of the stack.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because gold DocRED itself has low transitivity Cons on the evaluation slice, part of the remaining gap may be annotation incompleteness; testing on completed labels would separate model error from label noise.
  • The same composite reward could be tried on other multi-triple IE tasks (event argument sets, knowledge-graph completion) where local generation routinely breaks global constraints.
  • If teacher reasoning text stays unfaithful even after gold entity/relation swap, process-level rewards on intermediate consistency checks may matter more than outcome Cons alone.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper proposes CONSISTRE, a framework for document-level relation extraction (DocRE) with LLMs built around a shared formulation of relational consistency — transitivity, symmetry/inverse, and functional uniqueness — with a scalar Cons score. Track A applies this at inference time to black-box LLMs (GPT-5.2, Gemini-2.5 Pro) via constraint-aware prompting, constraint-based verification, and iterative self-reflection; Track B distills consistency-aware teacher traces into 7–8B students (Qwen2.5-7B, Qwen3-8B) via SFT with gold-truth alignment, then GRPO with a composite F1+Cons+format reward. On a 200-document DocRED dev subset, Track A reaches 0.543 macro-F1 (5-shot +CoT+Self-Reflection, Gemini) and Track B reaches 0.492 (Qwen3-8B SFT+GRPO), with ablations attributing gains to the constraint machinery: disabling transitivity drops Track A F1 from 0.518 to 0.405, and generic reflection baselines (Reflexion, Self-Refine, CoVe) all reduce F1 relative to base while CONSISTRE improves it. A per-type analysis claims SFT primarily fixes inverse-consistency while GRPO primarily fixes transitivity.

Significance. If the results hold, the paper makes a useful contribution to LLM-based DocRE: a single constraint formulation (transitivity, inverse, functional uniqueness) instantiated in both inference-time verification and an RL reward, with evidence that the two training stages of Track B repair different structural error types (SFT→inverse symmetry, GRPO→transitivity, with a concrete 26.5% transitivity-violation reduction at near-constant triple counts — a good control against the 'consistency by suppression' confound). The work is candid about several of its own weaknesses: it acknowledges DocRED's annotation incompleteness (gold transitivity Cons = 0.265), reports multi-seed variance for Track B in the supplement, and includes a reward-weight sensitivity scan showing F1 stability. The empirical claims are falsifiable and the Track B gains over near-zero baselines are large. However, the headline numbers rest on a 200-document subset, several load-bearing tables contain internally inconsistent values for the same configuration, and promised precision/recall metrics never appear, so the strength of the central claims cannot yet be fully assessed.

major comments (4)
  1. [Table III, §IV-B (Constraint Ablation)] The key ablation conflates two mechanisms: each variant 'simultaneously removes the corresponding constraint cue from CoT/Reflection prompts and disables the corresponding post-processing enforcement.' The narrative then concludes the gains 'stem from consistency modeling rather than from CoT and reflection in isolation,' but the design cannot separate the effect of mechanical transitive-closure propagation from prompt wording. This matters because the same mechanism attribution motivates Track B's Cons reward term. Moreover, the paper itself reports (§V, supp. B) that DocRED gold annotations have transitivity Cons of only 0.265 on the eval subset, so closing over predicted chains should inject mostly gold-mismatched triples and hurt precision; a net +0.113 F1 from this operation is surprising and is never decomposed. §IV-A promises Precision, Recall, and macro-F1 as primary metrics, yet
  2. [Tables I–III and VI] Nominally the same configuration — GPT-5.2, 5-shot, +CoT+Self-Reflection, 200-document subset — is reported as F1 = 0.5305 in Table I, 0.518 in Table III ('Full'), and 0.530 in Table VI; the Table II value of 0.468 is presumably a single-reflection-round restriction, but this is never stated for the CONSISTRE row itself (the text only says baselines were run with one round 'matching the inference structure of CONSISTRE Track A'). These discrepancies are unexplained, and the cross-track gap-closing claim ('narrowing the gap to approximately 0.04') depends on which Track A number is used. Please reconcile these numbers, state the exact configuration behind each table, and report run-to-run variance for Track A (API nondeterminism and model drift), since no error bars appear anywhere for Track A.
  3. [§IV-A (Dataset) and Table VII, §IV-F] All headline numbers are computed on a 200-document subset whose selection procedure is never specified (random? stratified? hand-chosen?), while Table VII compares these subset numbers against prior systems evaluated on the full DocRED dev set (DREEAM 0.674, LMRC 0.611, LoRA FT 0.393) and even mixes dev and test numbers (0-shot GPT-4, 0.156 test). The claim that CONSISTRE 'establishes a new state of the art' in the low-resource paradigm is therefore not supported by a like-for-like comparison. At minimum: state the subset selection protocol, report bootstrap confidence intervals on the 200-document subset, and ideally validate the best Track A and Track B configurations on the full dev set — the 50-document ranking check is suggestive but insufficient given that the relevant gaps (e.g., Track B vs. LoRA FT LLaMA2-13B, 0.492 vs. 0.393) are within the range where subset choice could matte
  4. [§III-C (Consistency Formulation)] The entire framework rests on the sets R_trans, R_inv, and R_func, yet the paper never enumerates which of DocRED's 96 relations are declared transitive, which inverse pairs are used, and which relations are functional, nor how these assignments were derived (annotation guidelines? author judgment? Wikidata properties?). Different choices would change both the Cons metric and the enforcement behavior, so the main results are not reproducible as written. Please list the constraint relation inventory (in an appendix if necessary) and discuss sensitivity to borderline assignments (e.g., 'located in' is transitive only under containment semantics that DocRED may not consistently follow).
minor comments (7)
  1. [Table IV, footnote ‡] The main table reports the single-seed result (F1 = 0.492) while the supplementary mean is 0.483 ± 0.009; the main table should report the multi-seed mean, with the footnote reversed.
  2. [§III-D2/D3] Track A hyperparameters — the verification penalty λ, the acceptance threshold, and the maximum number of reflection iterations — are never given in the main text, although they appear in every Track A configuration.
  3. [§IV-C and §IV-E] The claim that Qwen3-8B has 'approximately two orders of magnitude fewer parameters than the proprietary backbones' presumes knowledge of GPT-5.2 and Gemini-2.5 Pro parameter counts that is not public; please soften or attribute this claim.
  4. [§IV-B (Human Validation of the Cons Metric)] The human validation uses a single human rater (the lead author) plus two LLM judges; describing LLMs as 'independent evaluators' and characterizing Claude–Gemini r = 0.608 as 'consistently high' overstates the evidence, and it is odd that the human correlates more strongly with each LLM (0.828/0.893) than the LLMs do with each other. Please temper the wording and report the rating protocol (Likert anchors, blinding).
  5. [Table II] The near-zero baseline inverse-consistency values for Reflexion/Self-Refine/CoVe (0.014–0.033) deserve one sentence of explanation — presumably these methods rarely produce both directions of an inverse pair — otherwise the contrast with CONSISTRE's 1.000 looks like a metric artifact.
  6. [Various] Typos and presentation: 'The pipeline is consist in three parts' (§III-E); malformed bullet spacing in the contributions list; supplementary Sections B, C, D, and H are cited but the supplementary material should be confirmed as available to reviewers; Figure 1 caption refers to colors (blue/green) that may not survive grayscale printing.
  7. [General] Reproducibility: no code, prompt templates, or the exact 200-document subset are stated as released. Given that the central claims are empirical and Track A depends on proprietary APIs, releasing prompts, the subset document IDs, and the Track B training code would substantially strengthen the paper.

Circularity Check

1 steps flagged · score 2.0 of 10

Mostly non-circular empirical ML paper; only mild by-construction Cons after Track A post-processing enforcement.

  1. self definitional [§III-C Cons definition; §III-D2–D3 verification/refinement; Table III Track A constraint ablation]
    "Cons(Ŷ)=1−V(Ŷ)/(T(Ŷ)+ϵ) ... Each variant simultaneously removes the corresponding constraint cue from CoT/Reflection prompts and disables the corresponding post-processing enforcement: transitive closure propagation for Transitivity, symmetric/inverse triple completion for Inverse, and deduplication of (head, rfunc,·) tuples for Functional Uniqueness. ... Full 0.518 ... Trans Cons 0.961 Inv Cons 1.000 Func Cons 1.000 Macro Cons 0.983"

    Under Full, Track A applies mechanical post-processors that complete transitive/inverse triples and dedupe functional tails, which directly drive violation count V toward zero for those constraint families. Reporting near-saturated Trans/Inv/Func Cons for Full is therefore largely the definitional consequence of running those enforcers, not an independent measurement that unconstrained generations satisfied the constraints. The F1 column is still empirical; only the Cons-as-evidence-of-mitigated-contradictions step is partly by construction.

full rationale

CONSISTRE’s load-bearing claims are empirical comparisons on DocRED (macro-F1 vs gold; ablations vs prompting/reflection baselines; Track B SFT/GRPO vs base). F1 is scored against external gold labels and is not defined in terms of the method’s consistency machinery. The Cons formula is author-defined, but that alone is ordinary metric design, not circular derivation. The one mild circularity is that Track A’s reported near-perfect per-type Cons under the Full configuration is largely forced by the same post-processing enforcers (transitive closure propagation, inverse completion, functional dedup) that the ablation disables—so high Cons after Full is partly by construction of the verifier/post-processor, not an independent discovery that free-form LLM outputs became coherent. Track B’s inclusion of Cons in the GRPO reward and subsequent Cons reporting is standard optimize-and-measure practice and does not make the F1 gains tautological. No self-citation uniqueness chain, no fitted parameter renamed as a first-principles prediction, and no renaming of a known closed-form result. Central gap-closing and F1 claims remain externally falsifiable; score 2 for the Cons-after-enforcement step only.

Assumptions & free parameters 4 free parameters · 5 assumptions · 3 invented entities

The work is empirical NLP systems research. Load-bearing premises are standard DocRE task assumptions, the three hand-chosen constraint families, the Cons aggregate formula, reward weights, and the claim that a 200-doc subset plus teacher traces faithfully represent the problem. No new physical entities; invented pieces are methodological constructs (Cons score, dual tracks, composite reward).

free parameters (4)
  • Reward weights α, β, γ in R = α·F1 + β·Cons + γ·Fmt = α=0.7, β=0.2, γ=0.1
    Set empirically to 0.7, 0.2, 0.1; ablations claim robustness but central GRPO behavior depends on this hand-chosen mix.
  • Verification penalty λ and acceptance threshold in Track A S(Ŷ)=S_base−λV(Ŷ)
    Controls when reflection triggers; values not fully specified as a unique principled choice in the main text.
  • GRPO KL coefficient β_KL and group size G = β_KL=0.15, G=4
    Training stability knobs (β_KL=0.15, G=4) chosen for practice; affect how far the student moves from SFT.
  • SFT/GRPO data split sizes (1200 / 500) and 200-doc eval subset = 1200 SFT, 500 GRPO, 200 eval
    Sampling choices driven by cost; rankings claimed stable vs 50-doc check but subset is still a free experimental parameter.
assumptions (5)
  • domain assumption DocRE predictions should obey predefined schema-level constraints: transitivity for R_trans, inverse pairs R_inv, and functional uniqueness for R_func.
    Section III-C; standard relational lore, but coverage and which DocRED relations belong to each set are design choices not derived from data.
  • ad hoc to paper Cons(Ŷ)=1−V(Ŷ)/(T(Ŷ)+ε) is an adequate scalar of global relational coherence for both verification and reward.
    Author aggregate used uniformly in Tracks A and B; human study partially supports correlation on transitivity.
  • ad hoc to paper Teacher reasoning and consistency_check fields remain useful supervision after entity/relation fields are replaced by DocRED gold (gold-truth alignment).
    §III-E1; ablation Table V supports F1 gains but notes SFT Cons drop from narrative–label mismatch.
  • domain assumption Macro-F1 on a 200-document subset is a sufficient primary comparison signal for Track A API methods and Track B checkpoints.
    §IV-A cost justification; load-bearing for all reported leaderboard-style claims.
  • standard math Standard token-level SFT NLL and GRPO group-relative advantages are valid optimizers for transferring consistency-aware DocRE behavior.
    §III-E2–E3; conventional ML training assumptions.
invented entities (3)
  • CONSISTRE dual-track framework (Track A inference-time consistency + Track B training-time consistency)
    purpose: Unify API and local deployment under one constraint formulation for DocRE.
    Organizational contribution of the paper; not an external natural kind.
  • Composite consistency-aware GRPO reward R(y,x)
    purpose: Jointly push extraction F1, Cons, and schema format in student alignment.
    Method-specific reward; evidence is internal ablations only.
  • Constraint-aware teacher trace schema (reasoning, consistency_check, entity, relation)
    purpose: Distill process-level consistency reasoning into students.
    Prompt/trace format invented for the pipeline.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CONSISTRE: A Unified Consistency-Aware Framework for Document-Level Relation Extraction with Large Language Models." pith.science (2026). https://pith.science/paper/LO4HDFEQ

@misc{pith2026260724312,
  author       = {Pith},
  title        = {Pith review of: CONSISTRE: A Unified Consistency-Aware Framework for Document-Level Relation Extraction with Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LO4HDFEQ}},
  note         = {Machine review of arXiv:2607.24312}
}
read the original abstract

Document-level relation extraction (DocRE) aims to extract relations among multiple entities across extended contexts while maintaining consistency across predicted triples. Although large language models (LLMs) show remarkable reasoning capabilities in information extraction, their predictions are typically generated independently for each candidate triple and may violate fundamental relational constraints such as transitivity, symmetry, and functional uniqueness, leading to contradictory and unreliable outputs. We propose CONSISTRE, a unified consistency-aware framework for DocRE that addresses this limitation through two complementary tracks. The first operates at inference time for black-box LLMs, combining constraint-aware prompting, constraint-based verification, and iterative self-reflection to refine predictions without task-specific fine-tuning. The second injects consistency knowledge into smaller open-source models via a knowledge distillation and reinforcement learning pipeline: reasoning traces from a powerful teacher are distilled into a student via supervised fine-tuning, followed by GRPO alignment using a composite reward that jointly optimizes extraction performance and relational consistency. Together, the two tracks cover both API-accessible and locally deployable scenarios under a unified consistency formulation. Experiments on DocRED show that both tracks outperform their baselines, with the inference-time track achieving competitive F1 using off-the-shelf black-box LLMs and the training-time track substantially narrowing the gap between 7--8B open-source models and state-of-the-art proprietary LLMs at a fraction of their inference cost. Ablation studies confirm that explicit consistency modeling mitigates relational contradictions and enhances the reliability of LLM-based DocRE across both deployment paradigms.

Figures

Figures reproduced from arXiv: 2607.24312 by the authors.

Figure 1
Figure 1. Overall architecture of CONSISTRE. Track A (left, blue) refines extracted triples for black-box LLMs via constraint-aware prompting, verification, [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Per-type Cons across the three stages of Track B on the 200- [PITH_FULL_IMAGE:figures/full_fig_p011_2.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

38 extracted references · 11 linked inside Pith

  1. [1]

    Docred: A large-scale document-level relation extraction dataset,

    Y . Yao, D. Ye, P. Li, X. Han, Y . Lin, Z. Liu, J. Zhou, and M. Sun, “Docred: A large-scale document-level relation extraction dataset,” inProceedings of ACL, 2019

    2019

  2. [2]

    Double graph based reasoning for document- level relation extraction,

    S. Zeng, R. Xu, B. Zhou, Z. Zhao, Z. Liu, L. Li, and J. Zhao, “Double graph based reasoning for document- level relation extraction,” inProceedings of EMNLP, 2020

    2020

  3. [3]

    Augmenting neural networks with first-order logic,

    T. Li and V . Srikumar, “Augmenting neural networks with first-order logic,” inProceedings of the 57th Annual Meeting of the Association for Computational Linguis- tics, 2019, pp. 292–302

    2019

  4. [4]

    Document-level relation extraction with adaptive thresh- olding and localized context pooling,

    W. Zhou, Y . Xu, T. Gui, Q. Tan, X. Zhang, and X. Huang, “Document-level relation extraction with adaptive thresh- olding and localized context pooling,” inProceedings of AAAI, 2021

    2021

  5. [5]

    Learning logic rules for document- level relation extraction,

    D. Ru, C. Sun, J. Feng, L. Qiu, H. Zhou, W. Zhang, Y . Yu, and L. Li, “Learning logic rules for document- level relation extraction,” inProceedings of the 2021 conference on empirical methods in natural language processing, 2021, pp. 1239–1250

    2021

  6. [6]

    Boosting document-level relation extraction by mining and injecting logical rules,

    S. Fan, S. Mo, and J. Niu, “Boosting document-level relation extraction by mining and injecting logical rules,” inProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 2022, pp. 10 311–10 323

    2022

  7. [7]

    Towards better document-level relation extrac- tion via iterative inference,

    L. Zhang, J. Su, Y . Chen, Z. Miao, M. Zijun, Q. Hu, and X. Shi, “Towards better document-level relation extrac- tion via iterative inference,” inProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 2022, pp. 8306–8317

    2022

  8. [8]

    Revisiting relation extraction in the era of large language models,

    S. Wadhwa, S. Amir, and B. C. Wallace, “Revisiting relation extraction in the era of large language models,” inProceedings of the 61st Annual Meeting of the Asso- ciation for Computational Linguistics (Volume 1: Long Papers), 2023, pp. 15 566–15 589

    2023

Show all 38 references
  1. [9]

    Large language models for generative information extraction: A survey,

    D. Xu, W. Chen, W. Peng, C. Zhang, T. Xu, X. Zhao, X. Wu, Y . Zheng, Y . Wang, and E. Chen, “Large language models for generative information extraction: A survey,” Frontiers of Computer Science, vol. 18, no. 6, p. 186357, 2024

    2024

  2. [10]

    Reflexion: Language agents with verbal reinforcement learning,

    N. Shinn, F. Cassano, A. Gopinath, K. Narasimhan, and S. Yao, “Reflexion: Language agents with verbal reinforcement learning,”Advances in neural information processing systems, vol. 36, pp. 8634–8652, 2023

    2023

  3. [11]

    Minillm: Knowledge distillation of large language models,

    Y . Gu, L. Dong, F. Wei, and M. Huang, “Minillm: Knowledge distillation of large language models,” inThe twelfth international conference on learning representa- tions, 2024

    2024

  4. [12]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models,

    Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, X. Bi, H. Zhang, M. Zhang, Y . K. Li, Y . Wu, and D. Guo, “Deepseekmath: Pushing the limits of mathematical reasoning in open language models,” 2024. [Online]. Available: https://arxiv.org/abs/2402.03300

    2024 arXiv

  5. [13]

    En- tity structure within and throughout: Modeling mention dependencies for document-level relation extraction,

    B. Xu, Q. Wang, Y . Lyu, Y . Zhu, and Z. Mao, “En- tity structure within and throughout: Modeling mention dependencies for document-level relation extraction,” in Proceedings of the AAAI conference on artificial intelli- gence, vol. 35, no. 16, 2021, pp. 14 149–14 157

    2021

  6. [14]

    Document-level rela- tion extraction as semantic segmentation,

    N. Zhang, X. Chen, X. Xie, S. Deng, C. Tan, M. Chen, F. Huang, L. Si, and H. Chen, “Document-level rela- tion extraction as semantic segmentation,”arXiv preprint arXiv:2106.03618, 2021

    2021 arXiv

  7. [15]

    Dreeam: Guiding attention with evidence for improving document-level relation extraction,

    Y . Ma, A. Wang, and N. Okazaki, “Dreeam: Guiding attention with evidence for improving document-level relation extraction,” inProceedings of the 17th Confer- ence of the European Chapter of the Association for Computational Linguistics, 2023, pp. 1971–1983

    2023

  8. [16]

    End-to-end learning of logical rules for enhancing document-level relation extraction,

    K. Qi, J. Du, and H. Wan, “End-to-end learning of logical rules for enhancing document-level relation extraction,” inProceedings of the 62nd Annual Meeting of the Asso- ciation for Computational Linguistics (Volume 1: Long IEEE/ACM TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE P...

    2024

  9. [17]

    Rapl: A relation-aware prototype learning ap- proach for few-shot document-level relation extraction,

    S. Meng, X. Hu, A. Liu, S. Li, F. Ma, Y . Yang, and L. Wen, “Rapl: A relation-aware prototype learning ap- proach for few-shot document-level relation extraction,” inProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 2023, pp. 5208–5226

    2023

  10. [18]

    Ptr: Prompt tuning with rules for text classification,

    X. Han, W. Zhao, N. Ding, Z. Liu, and M. Sun, “Ptr: Prompt tuning with rules for text classification,”AI Open, vol. 3, pp. 182–192, 2022

    2022

  11. [19]

    Autore: Document-level relation extraction with large language models,

    L. Xue, D. Zhang, Y . Dong, and J. Tang, “Autore: Document-level relation extraction with large language models,” 2024. [Online]. Available: https://arxiv.org/abs/ 2403.14888

    2024 arXiv

  12. [20]

    Consistency guided knowledge retrieval and denoising in llms for zero-shot document-level relation triplet extraction,

    Q. Sun, K. Huang, X. Yang, R. Tong, K. Zhang, and S. Poria, “Consistency guided knowledge retrieval and denoising in llms for zero-shot document-level relation triplet extraction,” inProceedings of the ACM Web Con- ference 2024, 2024, pp. 4407–4416

    2024

  13. [21]

    Semi-automatic data en- hancement for document-level relation extraction with distant supervision from large language models,

    J. Li, Z. Jia, and Z. Zheng, “Semi-automatic data en- hancement for document-level relation extraction with distant supervision from large language models,” inPro- ceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 2023, pp. 5495–5505

    2023

  14. [22]

    Gpt-re: In-context learning for relation extraction using large language models,

    Z. Wan, F. Cheng, Z. Mao, Q. Liu, H. Song, J. Li, and S. Kurohashi, “Gpt-re: In-context learning for relation extraction using large language models,” inProceedings of the 2023 conference on empirical methods in natural language processing, 2023, pp. 3534–3547

    2023

  15. [23]

    Prompt- free and efficient few-shot learning with language mod- els,

    R. K. Mahabadi, L. Zettlemoyer, J. Henderson, L. Math- ias, M. Saeidi, V . Stoyanov, and M. Yazdani, “Prompt- free and efficient few-shot learning with language mod- els,” inProceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long...

    2022

  16. [24]

    Self-refine: Iterative refinement with self- feedback,

    A. Madaan, N. Tandon, P. Gupta, S. Hallinan, L. Gao, S. Wiegreffe, U. Alon, N. Dziri, S. Prabhumoye, Y . Yanget al., “Self-refine: Iterative refinement with self- feedback,”Advances in neural information processing systems, vol. 36, pp. 46 534–46 594, 2023

    2023

  17. [25]

    Chain-of-verification reduces hallucination in large language models,

    S. Dhuliawala, M. Komeili, J. Xu, R. Raileanu, X. Li, A. Celikyilmaz, and J. Weston, “Chain-of-verification reduces hallucination in large language models,” inFind- ings of the association for computational linguistics: ACL 2024, 2024, pp. 3563–3578

    2024

  18. [26]

    Document-level relationship extraction by bidirectional constraints of beta rules,

    Y . Liu, Z. Zhu, X. Zhang, Z. Feng, D. Chen, and Y . Li, “Document-level relationship extraction by bidirectional constraints of beta rules,” inProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 2023, pp. 2256–2266

    2023

  19. [27]

    Distilling the knowledge in a neural network,

    G. Hinton, O. Vinyals, and J. Dean, “Distilling the knowledge in a neural network,” 2015. [Online]. Available: https://arxiv.org/abs/1503.02531

    2015 arXiv

  20. [28]

    Distilling step-by-step! outperforming larger language models with less training data and smaller model sizes,

    C.-Y . Hsieh, C.-L. Li, C.-K. Yeh, H. Nakhost, Y . Fujii, A. Ratner, R. Krishna, C.-Y . Lee, and T. Pfister, “Distilling step-by-step! outperforming larger language models with less training data and smaller model sizes,” 2023. [Online]. Available: https://arxiv.org/abs/ 2305.02301

    2023 arXiv

  21. [29]

    Constitutional ai: Harmlessness from ai feedback,

    Y . Bai, S. Kadavath, S. Kundu, A. Askell, J. Kernion, A. Jones, A. Chen, A. Goldie, A. Mirhoseini, C. McK- innonet al., “Constitutional ai: Harmlessness from ai feedback,”arXiv preprint arXiv:2212.08073, 2022

    2022 arXiv

  22. [30]

    Proximal policy optimization algorithms,

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  23. [31]

    Training language models to follow instructions with human feedback,

    L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. L. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, J. Schulman, J. Hilton, F. Kelton, L. Miller, M. Simens, A. Askell, P. Welinder, P. Christiano, J. Leike, and R. Lowe, “Training language models to follow instructions...

    2022 arXiv

  24. [32]

    Direct preference optimization: Your language model is secretly a reward model,

    R. Rafailov, A. Sharma, E. Mitchell, S. Ermon, C. D. Manning, and C. Finn, “Direct preference optimization: Your language model is secretly a reward model,” 2024. [Online]. Available: https://arxiv.org/abs/2305.18290

    2024 arXiv

  25. [33]

    Let’s verify step by step,

    H. Lightman, V . Kosaraju, Y . Burda, H. Edwards, B. Baker, T. Lee, J. Leike, J. Schulman, I. Sutskever, and K. Cobbe, “Let’s verify step by step,” inThe twelfth in- ternational conference on learning representations, 2023

    2023

  26. [34]

    Lora: Low-rank adaptation of large language models,

    E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, and W. Chen, “Lora: Low-rank adaptation of large language models,” 2021. [Online]. Available: https://arxiv.org/abs/2106.09685

    2021 arXiv

  27. [35]

    Large language models cannot self-correct reasoning yet,

    J. Huang, X. Chen, S. Mishra, H. S. Zheng, A. Yu, X. Song, and D. Zhou, “Large language models cannot self-correct reasoning yet,” inInternational conference on learning representations, vol. 2024, 2024, pp. 32 808– 32 824

    2024

  28. [36]

    Refining chatgpt for document-level relation extraction: a multi- dimensional prompting approach,

    W. Zhu, X. Wang, X. Chen, and X. Luo, “Refining chatgpt for document-level relation extraction: a multi- dimensional prompting approach,” inInternational Con- ference on Intelligent Computing. Springer, 2024, pp. 190–201

    2024

  29. [37]

    Llm with relation classifier for document-level relation extraction,

    X. Li, K. Chen, Y . Long, and M. Zhang, “Llm with relation classifier for document-level relation extraction,” 2024. [Online]. Available: https://arxiv.org/ abs/2408.13889

    2024 arXiv

  30. [38]

    Revisiting docred-addressing the false negative prob- lem in relation extraction,

    Q. Tan, L. Xu, L. Bing, H. T. Ng, and S. M. Aljunied, “Revisiting docred-addressing the false negative prob- lem in relation extraction,” inProceedings of the 2022 conference on empirical methods in natural language processing, 2022, pp. 8472–8487

    2022

Pith tools

Reviewed July 31, 2026 · model on record in the stance chip above.