Pith. sign in

REVIEW 5 major objections 9 minor 195 references

Structured IE self-correction works better when the model names its error type before it revises.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-30 22:47 UTC pith:TWKS7LPY

load-bearing objection Real same-backbone IE gains, but the headline ~20 OOD F1 is mostly second-pass correction—not the label-aware story the abstract sells. the 5 major comments →

arxiv 2607.23420 v1 pith:TWKS7LPY submitted 2026-07-26 cs.CL

LA-RL: Label-Aware Self-Reflection for Reinforcement Learning in Information Extraction

Xiao You , Tianwei Yan , Zixu Shan , Longyu Du , Shan Zhao This is my paper
classification cs.CL
keywords information extractionself-reflectionreinforcement learningdiagnostic error labelsrelation extractionevent extractionGRPOstructured generation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Large language models still make systematic mistakes when extracting entities, relations, and events: missing spans, wrong labels, bad boundaries, invalid types, reversed arguments. Free-form reflection often notices that something is wrong without saying which failure mode it is, so the repair step stays vague. This paper argues that IE self-correction should be organized as label-guided diagnosis then revision. One backbone first extracts, then emits task-specific error labels from a fixed taxonomy, then revises conditioned on that diagnosis. Training starts from annotator-labeled diagnostic examples for cold-start fine-tuning, then uses two stages of outcome-only group-relative policy optimization that reward final extraction quality, format validity, and eventually first-pass correctness—without a separate process reward model. On the same 7B backbone, the method improves relation and event extraction over supervised fine-tuning alone, with especially large gains under domain shift, and ablations show that tighter reflection structure helps relation extraction more than named entity recognition.

Core claim

For structured information extraction, inserting a discrete, task-grounded diagnostic error-label state between first-pass extraction and final revision, then training that diagnosis-to-revision path with cold-start SFT plus two-stage outcome-supervised GRPO, yields consistent same-backbone gains over SFT and free-form reflection—most clearly on relation extraction and event extraction, and more strongly out of distribution than in distribution.

What carries the argument

LA-RL’s diagnostic state: a multi-label set of task-specific error names (for example Missing relation, Wrong entity boundary, Reversed entities) produced after the first pass and used to condition the revised answer, optimized with two GRPO stages that score final F1 and format, then also first-pass quality.

Load-bearing premise

The method assumes that a larger annotation model’s multi-label diagnoses over a hand-written error taxonomy are accurate enough that cold-start training on them teaches real diagnosis-to-revision behavior rather than annotator quirks.

What would settle it

Retrain with deliberately scrambled or empty diagnostic labels (or with a weak annotator) while keeping the same two-stage outcome rewards: if final F1 gains collapse relative to true labels, the label-aware path is doing the work; if gains stay, outcome RL alone explains the result.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • IE reflection should be designed as a constrained diagnosis-to-revision path, not as unconstrained natural-language thinking.
  • Two-stage outcome RL—first reward revised answers, then also reward first-pass answers—can raise final quality without discarding the correction pathway.
  • Relation extraction and event extraction, with richer multi-error schemas, benefit more from stronger label constraints than NER under domain shift.
  • The same backbone can learn extract → diagnose → revise without training a separate process reward model.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Hand-defined error taxonomies may become the main transfer bottleneck; automatic discovery of error labels would be the natural next pressure test.
  • If diagnosis quality is the load-bearing step, smaller models may need distilled or verified labels rather than raw self-labels.
  • The larger OOD gains suggest label-aware reflection acts partly as a schema-alignment regularizer, not only as in-domain error repair.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 9 minor

Summary. The paper proposes LA-RL, a reflective RL framework for information extraction in which a single Qwen2.5-7B-Instruct backbone produces a first-pass extraction, emits a set of task-grounded diagnostic error labels (e.g., Missing relation, Wrong entity boundary, Reversed entities), and then revises the extraction conditioned on that diagnosis. Training uses cold-start SFT on diagnostic data labeled by DeepSeek-V3 (comparing base-model predictions against gold), followed by two GRPO stages: Stage I rewards only final-answer F1 plus format validity, and Stage II adds a first-pass quality reward. Experiments on SciER (NER/RE), an OOD RE setting, and DuEE1.0 (EE) show consistent same-backbone gains over SFT: +6.68 Rel F1 on SciER, +19.87 Rel F1 on OOD, and +14.80/+17.50 trigger/argument F1 on DuEE1.0. Ablations compare Generic Reflection (free-form second pass, no labels), Label-only Correction, and full LA-RL, finding that RE benefits from stronger reflection structure while OOD NER does not.

Significance. If the results hold, the paper makes a useful contribution to a timely question: how to structure self-reflection for structured-prediction tasks where free-form rationales are poorly aligned with the output schema. Strengths include: (i) controlled same-backbone comparisons against SFT, GRPO, and a reproduced R2GRPO baseline across three IE tasks; (ii) a genuinely informative ablation design — the inclusion of a Generic Reflection control that isolates the label mechanism is more honest than typical ablations in this space, and the authors explicitly decline to overclaim uniformity ("we avoid claiming that more structure is uniformly better"); (iii) reproducibility detail — full SFT/RL/inference hyperparameters in Tables 7–9, deterministic decoding, system prompts in Figures 4–5, and label definitions in Table 12 and Figure 6; (iv) the rewards are external outcome metrics (micro F1, format checks) computed from gold annotations, so there is no circularity in the training signal. The main weakness is that the paper's own ablation substantially narrows what the evidence supports relative to the headline framing, and no experiment verifies that the diagnostic labels are accurate or c

major comments (5)
  1. [Abstract / §4.3, Table 2] Table 2 undercuts the mechanism attribution carried by the abstract and §1. On OOD Rel, SFT is 36.81, Generic Reflection (a free-form second pass with NO diagnostic labels) is 56.20, and full LA-RL is 56.68 — i.e., +19.39 F1 comes from having any second-pass correction step and only +0.48 from the label-aware diagnose-and-revise structure. Label-only Correction (55.78) is also within ~1 F1 of LA-RL and exceeds it on OOD NER (76.21 vs 75.34). Thus the '~+20 F1 on OOD RE' figure featured in the abstract is almost entirely explained by second-pass inference, not by the diagnostic labels. §4.3's ablation discussion is appropriately cautious, but the abstract and contributions still attribute the headline gains to 'task-grounded diagnostic labels.' The framing must be revised to match the evidence: the defensible claim is that second-pass reflection drives the OOD gain and that label structur
  2. [§3.2 / §4.3] The central mechanism — that predicted diagnostic labels guide revision — is never verified. Stage I rewards are 'evaluated exclusively at the final answer' (§3.2), Stage II adds only first-pass RE F1, and no reward term, ablation, or audit measures the trained model's diagnostic precision/recall against gold-derived error categories. Table 3 gives the gold error distribution and Table 11 the annotator's distribution on training data, but nowhere is the trained policy's label accuracy measured. Under outcome-only GRPO the policy can retain the label-emitting format (which costs nothing) while the revision step improves independently, which would make the 'label-aware' mechanism partly decorative — consistent with the Table 2 pattern noted above. A small audit is feasible with the existing pipeline: run the trained model on held-out data, derive true error labels by comparing first-pass p
  3. [§4.2, Tables 1, 2, 4] No variance estimates are reported anywhere, and several load-bearing differences are small. All RL runs use 1,000 subsampled training instances (§4.2) with a single seed, yet key ablation gaps are 0.5–1.0 F1 (LA-RL vs Label-only Correction: 56.68 vs 55.78 OOD Rel; LA-RL vs Generic Reflection: +0.48). Single-run LoRA RL at this scale commonly fluctuates by more than that. The task-sensitivity conclusion ('RE improves monotonically with stronger constraints, NER does not') rests on these small differences and on the single OOD NER reversal. At minimum, report mean ± std over 3 seeds for Tables 1, 2, and 4 (or for Table 2 alone if compute is constrained), and soften ordinal claims that do not survive the noise floor.
  4. [§3.1, Algorithm 1, Table 11] The cold-start data pipeline depends entirely on DeepSeek-V3's labeling fidelity over a hand-defined taxonomy (Algorithm 1), yet no inter-annotator agreement, human spot-check, or error rate for the annotation model is reported. If the annotator systematically mislabels boundaries, misses multi-error cases, or over-assigns 'Missing relation' (50.83% of assignments per Table 11), cold-start SFT may teach annotator artifacts. A 100–200 example human audit of Manno labels, with agreement statistics, would substantially de-risk the foundation on which both SFT and the reward-free diagnostic format rest.
  5. [§4.2, §3.2 Eqs. (3), (8)] Experimental-control asymmetries need clarification or justification. (a) For NER/RE, SFT and LA-RL are initialized from 'the reproduced R2GRPO-trained model,' while the GRPO baseline is trained from the base model — this is a fair ablation of cold-start, but the text (§4.2) is confusing about whether the SFT baseline and LA-RL see identical data and identical R2GRPO initialization; please state this explicitly. (b) For EE, LA-RL is trained directly from Qwen2.5-7B-Instruct rather than from the R2GRPO checkpoint — the rationale for this asymmetry is not given, and it means the DuEE results do not test the identical recipe claimed in §4.3. (c) Eq. (3) adds a first-pass reward only for RE F1 in the joint NER/RE setting, and Eq. (8) only for argument F1 in EE; the choice of which first-pass signal to reward is a free design decision that could itself drive the Stage-II gains and deserves ju
minor comments (9)
  1. [Abstract vs §4.3] Numerical inconsistency: the abstract reports '+6.83 average F1' on SciER RE, while §4.3 reports +6.68 Rel F1 (66.26 − 59.58). Please reconcile which quantity 6.83 averages over.
  2. [Tables 1–2, Figure 1] Formatting errors in tables: Table 1 has '64.5375.34' (missing column break in the LA-RL_II row), Table 2 has '76.2155.78' and '64.5375.34' similarly. Figure 1's rendered example text is garbled (stray ellipses and broken lines).
  3. [§4.2, Evaluation Metrics] The Rel+ metric definition ('It further requires the entity type is correct in the triples') is confusingly worded given that Rel already requires both entity spans and types to match (§4.2); please clarify the actual distinction between Rel and Rel+ as used in the R2GRPO protocol.
  4. [§3.1] Please report the size of D_diag after the Algorithm 1 filtering (how many training instances had ŷ ≠ y), per task; this determines the cold-start data budget and is currently absent.
  5. [§4.1 / Table 6] DuEE1.0 evaluation uses the validation split as test because the official test set is not released (Table 6 caption) — this is understandable, but should be stated in the main text (§4.1), not only in the appendix, since test-set numbers in the EE literature are not directly comparable.
  6. [§3.2, Appendix B] The reward weights (λ_ner, λ_re) = (1.0, 2.0), λ_fmt = 0.5, λ_first = 1.0 (Eqs. 1–3, 6–8) are stated without sensitivity analysis or motivation beyond 'argument prediction is more difficult.' A brief discussion of robustness to these choices would strengthen the design section.
  7. [Appendix C, Table 10] The zero-shot transfer evaluation (Table 10) shows gains of only ~1.7 F1 on CoNLL04 and ~1.9 on NYT10 over the base model; given no variance estimates, the claim that 'the learned reflection behavior can transfer' should be tempered further.
  8. [§5] Related work omits recent RL-for-IE and structured self-correction work beyond R2GRPO (e.g., process-reward and critique-based extraction methods); a short paragraph positioning LA-RL against self-critique prompting baselines at inference time (not just trained baselines) would clarify the contribution.
  9. [Appendix B] Whether code, prompts-as-data, or the trained LoRA adapters will be released is not stated; given the reproducibility detail provided elsewhere, an explicit statement (and ideally a release) is warranted.

Circularity Check

0 steps flagged

No circular derivation: gains are measured by external gold F1, and diagnostic labels are intermediate representations, not the scored target.

full rationale

LA-RL is an empirical training recipe (cold-start SFT on annotator-labeled errors, then two-stage outcome GRPO). The load-bearing claims are same-backbone F1 improvements on SciER, OOD RE, and DuEE1.0. Those metrics are computed from gold annotations and format checks (Eqs. 1–3, 6–8), not from quantities defined by the policy’s own diagnostic labels. Stage I rewards only final extraction quality; Stage II adds first-pass F1 still scored against gold—standard outcome RL, not a fit renamed as prediction. Diagnostic labels are constructed by an external annotator (DeepSeek-V3) comparing base predictions to gold (Algorithm 1) and used as intermediate text in the generation trajectory; they are never the evaluation target. There is no self-definitional loop, no parameter fit presented as an independent forecast, no uniqueness theorem imported from overlapping authors, and no renaming of a known closed-form result. Mechanism-attribution weaknesses (e.g., Generic Reflection capturing most of the OOD gain in Table 2, or lack of diagnosis accuracy audits) are empirical/causal concerns, not circularity. The derivation chain is self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

6 free parameters · 5 axioms · 2 invented entities

Empirical ML methods paper. Load-bearing premises are modeling and data-construction choices rather than mathematical axioms: hand-built error taxonomies, annotator-model labels, GRPO outcome rewards with fixed λ weights, and the assumption that discrete diagnostic states improve revision more than free-form reflection for schema-constrained IE.

free parameters (6)
  • λ_ner, λ_re (task reward weights) = (1.0, 2.0)
    Fixed to (1.0, 2.0) in Eq. (1) without reported sweep; directly shapes Stage-I/II optimization emphasis toward RE.
  • λ_fmt (format reward weight) = 0.5
    Set to 0.5 in Eq. (2); hand-chosen tradeoff between structure validity and extraction F1.
  • λ_first (first-pass reward weight) = 1.0
    Set to 1.0 in Eq. (3)/EE analogue; controls how hard Stage II pushes early correctness vs reflection.
  • λ_trig, λ_arg for EE = (1.0, 2.0)
    EE task reward weights fixed at (1.0, 2.0), privileging argument F1 by design.
  • RL subsample size = 1000
    Both RL stages use 1,000 selected training instances following R2GRPO setup; selection rule impact not analyzed.
  • LoRA ranks / LR / epochs for SFT and RL = see Tables 8–9
    SFT LoRA r=16 α=32 lr=1e-4; RL LoRA r=64 α=128 lr=5e-6; 2 epochs each—standard but claim-sensitive hyperparameters.
axioms (5)
  • domain assumption IE errors are systematic and adequately covered by a small hand-defined multi-label taxonomy per task.
    Motivates discrete diagnostic state (§1, §3); label sets in Figs. 4–6 and Table 12 are predefined, not learned.
  • domain assumption A larger annotator LLM (DeepSeek-V3) can reliably map (x, gold, prediction) to those error labels for cold-start data.
    Algorithm 1 Stage 1 depends on this; no human agreement study is reported.
  • domain assumption Outcome rewards from micro-F1 and format validity, without a process reward model, suffice to instill useful diagnose-then-revise behavior under GRPO.
    Stated design choice in abstract and §3.2; intermediate labels are not directly rewarded in Stage I.
  • domain assumption Group Relative Policy Optimization with KL to the previous-stage reference is a stable enough optimizer for this reflective policy.
    Optimization objective Eqs. (4)–(5); standard in recent LLM RL but still an unproved transfer assumption here.
  • standard math Structured IE can be cast as constrained text generation with JSON-like schemas shared across NER/RE/EE.
    Task formulation §2; common generative IE framing, not ad hoc.
invented entities (2)
  • LA-RL diagnostic reflective policy (extract → task error-label set → revise) no independent evidence
    purpose: Structure IE self-correction as a discrete diagnostic state inside one sequence-generation policy.
    Core proposed object; evaluated only via downstream F1, not via independent process metrics of diagnosis accuracy at scale.
  • Two-stage reflective vs integrated GRPO schedule (Stage I final-only, Stage II final+first-pass) no independent evidence
    purpose: Separate learning to correct from learning to solve early while keeping the correction path.
    Training paradigm contribution; evidence is ablation-style stage comparisons on the same benchmarks.

pith-pipeline@v1.2.0-grok45-kimik3 · 21338 in / 3865 out tokens · 73448 ms · 2026-07-30T22:47:01.227167+00:00 · methodology

0 comments
read the original abstract

Large language models show strong promise for information extraction (IE), but existing reflection-based correction methods are often misaligned with structured extraction outputs. Free-form self-reflection can flag an error, yet it rarely identifies whether the failure is a missing span, wrong label, boundary mismatch, invalid relation type, or reversed argument order. We introduce LA-RL (Label-Aware Reflective Reinforcement Learning), an outcome-supervised framework that guides IE self-correction with task-grounded diagnostic labels. A single backbone first predicts an extraction, diagnoses task-specific error labels, and then revises its output conditioned on the diagnosis. Training starts from diagnostic data labeled by an annotation model for cold-start supervised fine-tuning and proceeds through two GRPO stages that reward final extraction quality, format validity, and first-pass correctness, without a process reward model. Experiments on named entity recognition, relation extraction, and event extraction show consistent same-backbone gains over SFT, including 6.83 average F1 on SciER relation extraction, about 20 F1 on out-of-distribution relation extraction, and 14.80 trigger F1 plus 17.50 argument F1 on DuEE1.0. Ablations show that reflection structure is task-sensitive: stronger constraints benefit relation extraction, whereas named entity recognition needs less restrictive correction under domain shift.

Figures

Figures reproduced from arXiv: 2607.23420 by Longyu Du, Shan Zhao, Tianwei Yan, Xiao You, Zixu Shan.

Figure 1
Figure 1. Figure 1: Relation extraction example illustrating why [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of LA-RL. The first-pass extraction is followed by a task-grounded diagnostic state, which [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Group Relative Policy Optimization (GRPO) process for optimizing the diagnostic reflective policy with [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: System prompt used in LA-RL training for joint named entity recognition and relation extraction with [PITH_FULL_IMAGE:figures/full_fig_p013_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: System prompt used in LA-RL training for event extraction with task-specific EE error labels. [PITH_FULL_IMAGE:figures/full_fig_p014_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Relation extraction error label set used for multi-label diagnosis. The RE setting uses a six-label set. [PITH_FULL_IMAGE:figures/full_fig_p014_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: A representative case in which label-aware reflection corrects missing relations and entity-boundary errors. [PITH_FULL_IMAGE:figures/full_fig_p015_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: An example of correct direct inference in the first stage. [PITH_FULL_IMAGE:figures/full_fig_p016_8.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

195 extracted references · 7 canonical work pages · 2 internal anchors

  1. [1]

    arXiv preprint arXiv:2410.21155 , year=

    SciER: An Entity and Relation Extraction Dataset for Datasets, Methods, and Tasks in Scientific Documents , author=. arXiv preprint arXiv:2410.21155 , year=

    Pith/arXiv arXiv

  2. [2]

    CCF International Conference on Natural Language Processing and Chinese Computing , pages=

    DuEE1.0: a large-scale dataset for Chinese event extraction in real-world scenarios , author=. CCF International Conference on Natural Language Processing and Chinese Computing , pages=. 2020 , organization=

    2020

  3. [3]

    arXiv preprint arXiv:2402.11254 , year=

    C-ICL: contrastive in-context learning for information extraction , author=. arXiv preprint arXiv:2402.11254 , year=

    Pith/arXiv arXiv

  4. [4]

    arXiv preprint arXiv:2303.17651 , year=

    Self-Refine: Iterative Refinement with Self-Feedback , author=. arXiv preprint arXiv:2303.17651 , year=

    Pith/arXiv arXiv

  5. [5]

    arXiv preprint arXiv:2303.11366 , year=

    Reflexion: Language Agents with Verbal Reinforcement Learning , author=. arXiv preprint arXiv:2303.11366 , year=

    Pith/arXiv arXiv

  6. [6]

    arXiv preprint arXiv:2305.20050 , year=

    Let's Verify Step by Step , author=. arXiv preprint arXiv:2305.20050 , year=

    Pith/arXiv arXiv

  7. [7]

    HCL: A Hierarchical Contrastive Learning Framework for Zero-Shot Relation Extraction , year=

    Yan, Tianwei and Zhao, Shan and Hu, Minghao and Wang, Mengzhu and Zhang, Xiang and Luo, Zhigang and Wang, Meng , journal=. HCL: A Hierarchical Contrastive Learning Framework for Zero-Shot Relation Extraction , year=

  8. [8]

    Hierarchical Label-Enhanced Contrastive Learning for Chinese NER , year=

    Wang, Chengyu and Zhao, Shan and Yan, Tianwei and Song, Shezheng and Ma, Wentao and Liu, Kuien and Wang, Meng , journal=. Hierarchical Label-Enhanced Contrastive Learning for Chinese NER , year=

  9. [9]

    Dynamic Modeling Cross-Modal Interactions in Two-Phase Prediction for Entity-Relation Extraction , year=

    Zhao, Shan and Hu, Minghao and Cai, Zhiping and Liu, Fang , journal=. Dynamic Modeling Cross-Modal Interactions in Two-Phase Prediction for Entity-Relation Extraction , year=

  10. [10]

    arXiv preprint arXiv:1603.01360 , year=

    Neural architectures for named entity recognition , author=. arXiv preprint arXiv:1603.01360 , year=

    Pith/arXiv arXiv

  11. [11]

    Wireless Communications and Mobile Computing , volume=

    Recognizing Continuous and Discontinuous Adverse Drug Reaction Mentions from Social Media Using LSTM-CRF , author=. Wireless Communications and Mobile Computing , volume=. 2018 , publisher=

    2018

  12. [12]

    IEEE Transactions on Neural Networks and Learning Systems , volume=

    Dynamic modeling cross-modal interactions in two-phase prediction for entity-relation extraction , author=. IEEE Transactions on Neural Networks and Learning Systems , volume=. 2021 , publisher=

    2021

  13. [13]

    arXiv preprint arXiv:2210.07022 , year=

    CROP: zero-shot cross-lingual named entity recognition with multilingual labeled sequence translation , author=. arXiv preprint arXiv:2210.07022 , year=

    Pith/arXiv arXiv

  14. [15]

    Advances in neural information processing systems , volume=

    Training language models to follow instructions with human feedback , author=. Advances in neural information processing systems , volume=

  15. [16]

    arXiv preprint arXiv:2411.08923 , year=

    Aligning visual contrastive learning models via preference optimization , author=. arXiv preprint arXiv:2411.08923 , year=

    Pith/arXiv arXiv

  16. [17]

    Preprint , year=

    Rlaif: Scaling reinforcement learning from human feedback with ai feedback , author=. Preprint , year=

  17. [18]

    Advances in neural information processing systems , volume=

    Direct preference optimization: Your language model is secretly a reward model , author=. Advances in neural information processing systems , volume=

  18. [19]

    arXiv preprint arXiv:2506.04746 , year=

    Multi-Layer GRPO: Enhancing Reasoning and Self-Correction in Large Language Models , author=. arXiv preprint arXiv:2506.04746 , year=

    Pith/arXiv arXiv

  19. [20]

    arXiv preprint arXiv:2501.12948 , year=

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning , author=. arXiv preprint arXiv:2501.12948 , year=

    Pith/arXiv arXiv

  20. [21]

    Advances in neural information processing systems , volume=

    Distributed representations of words and phrases and their compositionality , author=. Advances in neural information processing systems , volume=

  21. [22]

    Bert: Pre-training of deep bidirectional transformers for language understanding , author=. Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers) , pages=

    2019

  22. [23]

    arXiv preprint arXiv:2410.22071 , year=

    Distinguishing ignorance from error in llm hallucinations , author=. arXiv preprint arXiv:2410.22071 , year=

    Pith/arXiv arXiv

  23. [24]

    Preprints , year=

    Error analysis prompting enables human-like translation evaluation in large language models: A case study on chatgpt , author=. Preprints , year=

  24. [25]

    IEEE Transactions on Neural Networks and Learning Systems , year=

    Hierarchical label-enhanced contrastive learning for Chinese NER , author=. IEEE Transactions on Neural Networks and Learning Systems , year=

  25. [26]

    IEEE Transactions on Neural Networks and Learning Systems , volume=

    Hcl: A hierarchical contrastive learning framework for zero-shot relation extraction , author=. IEEE Transactions on Neural Networks and Learning Systems , volume=. 2024 , publisher=

    2024

  26. [27]

    CoRR , volume =

    Lingfeng Zhong and Jia Wu and Qian Li and Hao Peng and Xindong Wu , title =. CoRR , volume =. 2023 , url =. doi:10.48550/ARXIV.2302.05019 , eprinttype =. 2302.05019 , timestamp =

    Pith/arXiv arXiv doi:10.48550/arxiv.2302.05019 2023

  27. [28]

    Named Entity Recognition for Question Answering , booktitle =

    Diego Moll. Named Entity Recognition for Question Answering , booktitle =. 2006 , url =

    2006

  28. [29]

    Tom B. Brown and Benjamin Mann and Nick Ryder and Melanie Subbiah and Jared Kaplan and Prafulla Dhariwal and Arvind Neelakantan and Pranav Shyam and Girish Sastry and Amanda Askell and Sandhini Agarwal and Ariel Herbert. Language Models are Few-Shot Learners , booktitle =. 2020 , url =

    2020

  29. [30]

    IJCAI 2022 , pages =

    Jian Yang and Yuwei Yin and Shuming Ma and Dongdong Zhang and Zhoujun Li and Furu Wei , title =. IJCAI 2022 , pages =

    2022

  30. [31]

    IJCAI 2022 , pages =

    Jian Yang and Yuwei Yin and Shuming Ma and Dongdong Zhang and Shuangzhi Wu and Hongcheng Guo and Zhoujun Li and Furu Wei , editor =. IJCAI 2022 , pages =

    2022

  31. [32]

    WMT 2021 , pages =

    Jian Yang and Shuming Ma and Haoyang Huang and Dongdong Zhang and Li Dong and Shaohan Huang and Alexandre Muzio and Saksham Singhal and Hany Hassan and Xia Song and Furu Wei , title =. WMT 2021 , pages =

    2021

  32. [33]

    Findings of the Association for Computational Linguistics:

    Jian Yang and Shaohan Huang and Shuming Ma and Yuwei Yin and Li Dong and Dongdong Zhang and Hongcheng Guo and Zhoujun Li and Furu Wei , editor =. Findings of the Association for Computational Linguistics:. 2022 , url =. doi:10.18653/V1/2022.FINDINGS-EMNLP.34 , timestamp =

    work page doi:10.18653/v1/2022.findings-emnlp.34 2022

  33. [34]

    arXiv preprint arXiv:2012.15547 , year=

    Xlm-t: Scaling up multilingual machine translation with pretrained cross-lingual transformer encoders , author=. arXiv preprint arXiv:2012.15547 , year=

    Pith/arXiv arXiv 2012

  34. [35]

    Naman Goyal and Cynthia Gao and Vishrav Chaudhary and Peng. The. CoRR , volume =

  35. [36]

    arXiv preprint arXiv:2310.10462 , year=

    Adaptive Neural Ranking Framework: Toward Maximized Business Goal for Cascade Ranking Systems , author=. arXiv preprint arXiv:2310.10462 , year=

    Pith/arXiv arXiv

  36. [37]

    Harnessing pre-trained neural networks with rules for formality style transfer , author=. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) , pages=

    2019

  37. [38]

    Proceedings of the 28th International Conference on Computational Linguistics , pages=

    Formality style transfer with shared latent space , author=. Proceedings of the 28th International Conference on Computational Linguistics , pages=

  38. [39]

    Natural Language Processing and Chinese Computing: 7th CCF International Conference, NLPCC 2018, Hohhot, China, August 26--30, 2018, Proceedings, Part I 7 , pages=

    Response selection of multi-turn conversation with deep neural networks , author=. Natural Language Processing and Chinese Computing: 7th CCF International Conference, NLPCC 2018, Hohhot, China, August 26--30, 2018, Proceedings, Part I 7 , pages=. 2018 , organization=

    2018

  39. [40]

    arXiv preprint arXiv:2309.16609 , year=

    Qwen technical report , author=. arXiv preprint arXiv:2309.16609 , year=

    Pith/arXiv arXiv

  40. [41]

    Rethinking the Role of Demonstrations: What Makes In-Context Learning Work? , booktitle =

    Sewon Min and Xinxi Lyu and Ari Holtzman and Mikel Artetxe and Mike Lewis and Hannaneh Hajishirzi and Luke Zettlemoyer , editor =. Rethinking the Role of Demonstrations: What Makes In-Context Learning Work? , booktitle =. 2022 , url =. doi:10.18653/V1/2022.EMNLP-MAIN.759 , timestamp =

    doi:10.18653/v1/2022.emnlp-main.759 2022

  41. [42]

    Long Ouyang and Jeffrey Wu and Xu Jiang and Diogo Almeida and Carroll L. Wainwright and Pamela Mishkin and Chong Zhang and Sandhini Agarwal and Katarina Slama and Alex Ray and John Schulman and Jacob Hilton and Fraser Kelton and Luke Miller and Maddie Simens and Amanda Askell and Peter Welinder and Paul F. Christiano and Jan Leike and Ryan Lowe , editor =...

    2022

  42. [43]

    arXiv preprint arXiv:2307.09288 , year=

    Llama 2: Open foundation and fine-tuned chat models , author=. arXiv preprint arXiv:2307.09288 , year=

    Pith/arXiv arXiv

  43. [44]

    arXiv preprint arXiv:2303.08774 , year=

    Gpt-4 technical report , author=. arXiv preprint arXiv:2303.08774 , year=

    Pith/arXiv arXiv

  44. [45]

    CoRR , volume =

    Bo Li and Gexiang Fang and Yang Yang and Quansen Wang and Wei Ye and Wen Zhao and Shikun Zhang , title =. CoRR , volume =. 2023 , url =. doi:10.48550/ARXIV.2304.11633 , eprinttype =. 2304.11633 , timestamp =

    Pith/arXiv arXiv doi:10.48550/arxiv.2304.11633 2023

  45. [46]

    CoRR , volume =

    Derong Xu and Wei Chen and Wenjun Peng and Chao Zhang and Tong Xu and Xiangyu Zhao and Xian Wu and Yefeng Zheng and Enhong Chen , title =. CoRR , volume =. 2023 , url =. doi:10.48550/ARXIV.2312.17617 , eprinttype =. 2312.17617 , timestamp =

    Pith/arXiv arXiv doi:10.48550/arxiv.2312.17617 2023

  46. [47]

    Structured Prediction as Translation between Augmented Natural Languages , booktitle =

    Giovanni Paolini and Ben Athiwaratkun and Jason Krone and Jie Ma and Alessandro Achille and Rishita Anubhai and C. Structured Prediction as Translation between Augmented Natural Languages , booktitle =. 2021 , url =

    2021

  47. [48]

    Unified Structure Generation for Universal Information Extraction , booktitle =

    Yaojie Lu and Qing Liu and Dai Dai and Xinyan Xiao and Hongyu Lin and Xianpei Han and Le Sun and Hua Wu , editor =. Unified Structure Generation for Universal Information Extraction , booktitle =. 2022 , url =. doi:10.18653/V1/2022.ACL-LONG.395 , timestamp =

    doi:10.18653/v1/2022.acl-long.395 2022

  48. [49]

    arXiv preprint arXiv:2302.10205 , year=

    Zero-shot information extraction via chatting with chatgpt , author=. arXiv preprint arXiv:2302.10205 , year=

    Pith/arXiv arXiv

  49. [50]

    arXiv preprint arXiv:2304.08085 , year=

    InstructUIE: Multi-task Instruction Tuning for Unified Information Extraction , author=. arXiv preprint arXiv:2304.08085 , year=

    Pith/arXiv arXiv

  50. [51]

    Learning In-context Learning for Named Entity Recognition , booktitle =

    Jiawei Chen and Yaojie Lu and Hongyu Lin and Jie Lou and Wei Jia and Dai Dai and Hua Wu and Boxi Cao and Xianpei Han and Le Sun , editor =. Learning In-context Learning for Named Entity Recognition , booktitle =. 2023 , url =. doi:10.18653/V1/2023.ACL-LONG.764 , timestamp =

    doi:10.18653/v1/2023.acl-long.764 2023

  51. [52]

    Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers),

    Xinxi Lyu and Sewon Min and Iz Beltagy and Luke Zettlemoyer and Hannaneh Hajishirzi , editor =. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers),. 2023 , url =. doi:10.18653/V1/2023.ACL-LONG.129 , timestamp =

    doi:10.18653/v1/2023.acl-long.129 2023

  52. [53]

    CodeIE: Large Code Generation Models are Better Few-Shot Information Extractors , booktitle =

    Peng Li and Tianxiang Sun and Qiong Tang and Hang Yan and Yuanbin Wu and Xuanjing Huang and Xipeng Qiu , editor =. CodeIE: Large Code Generation Models are Better Few-Shot Information Extractors , booktitle =. 2023 , url =. doi:10.18653/V1/2023.ACL-LONG.855 , timestamp =

    doi:10.18653/v1/2023.acl-long.855 2023

  53. [54]

    Chain of Thought with Explicit Evidence Reasoning for Few-shot Relation Extraction , booktitle =

    Xilai Ma and Jing Li and Min Zhang , editor =. Chain of Thought with Explicit Evidence Reasoning for Few-shot Relation Extraction , booktitle =. 2023 , url =

    2023

  54. [55]

    Wallace , editor =

    Somin Wadhwa and Silvio Amir and Byron C. Wallace , editor =. Revisiting Relation Extraction in the era of Large Language Models , booktitle =. 2023 , url =. doi:10.18653/V1/2023.ACL-LONG.868 , timestamp =

    doi:10.18653/v1/2023.acl-long.868 2023

  55. [56]

    GoLLIE: Annotation Guidelines improve Zero-Shot Information-Extraction , journal =

    Oscar Sainz and Iker Garc. GoLLIE: Annotation Guidelines improve Zero-Shot Information-Extraction , journal =. 2023 , url =. doi:10.48550/ARXIV.2310.03668 , eprinttype =. 2310.03668 , timestamp =

    Pith/arXiv arXiv doi:10.48550/arxiv.2310.03668 2023

  56. [57]

    Instruct and Extract: Instruction Tuning for On-Demand Information Extraction , booktitle =

    Yizhu Jiao and Ming Zhong and Sha Li and Ruining Zhao and Siru Ouyang and Heng Ji and Jiawei Han , editor =. Instruct and Extract: Instruction Tuning for On-Demand Information Extraction , booktitle =. 2023 , url =

    2023

  57. [58]

    Universal Information Extraction as Unified Semantic Matching , booktitle =

    Jie Lou and Yaojie Lu and Dai Dai and Wei Jia and Hongyu Lin and Xianpei Han and Le Sun and Hua Wu , editor =. Universal Information Extraction as Unified Semantic Matching , booktitle =. 2023 , url =. doi:10.1609/AAAI.V37I11.26563 , timestamp =

    work page doi:10.1609/aaai.v37i11.26563 2023

  58. [59]

    Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,

    Zhen Wan and Fei Cheng and Zhuoyuan Mao and Qianying Liu and Haiyue Song and Jiwei Li and Sadao Kurohashi , editor =. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,. 2023 , url =

    2023

  59. [60]

    CodeKGC: Code Language Model for Generative Knowledge Graph Construction

    Zhen Bi and Jing Chen and Yinuo Jiang and Feiyu Xiong and Wei Guo and Huajun Chen and Ningyu Zhang , title =. CoRR , volume =. 2023 , url =. doi:10.48550/ARXIV.2304.09048 , eprinttype =. 2304.09048 , timestamp =

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2304.09048 2023

  60. [61]

    CoRR , volume =

    Yucan Guo and Zixuan Li and Xiaolong Jin and Yantao Liu and Yutao Zeng and Wenxuan Liu and Xiang Li and Pan Yang and Long Bai and Jiafeng Guo and Xueqi Cheng , title =. CoRR , volume =. 2023 , url =. doi:10.48550/ARXIV.2311.02962 , eprinttype =. 2311.02962 , timestamp =

    Pith/arXiv arXiv doi:10.48550/arxiv.2311.02962 2023

  61. [62]

    Code4Struct: Code Generation for Few-Shot Event Structure Prediction , booktitle =

    Xingyao Wang and Sha Li and Heng Ji , editor =. Code4Struct: Code Generation for Few-Shot Event Structure Prediction , booktitle =. 2023 , url =. doi:10.18653/V1/2023.ACL-LONG.202 , timestamp =

    doi:10.18653/v1/2023.acl-long.202 2023

  62. [63]

    What Makes Good In-Context Examples for GPT-3? , booktitle =

    Jiachang Liu and Dinghan Shen and Yizhe Zhang and Bill Dolan and Lawrence Carin and Weizhu Chen , editor =. What Makes Good In-Context Examples for GPT-3? , booktitle =. 2022 , url =. doi:10.18653/V1/2022.DEELIO-1.10 , timestamp =

    doi:10.18653/v1/2022.deelio-1.10 2022

  63. [64]

    Thinking about

    Bernal Jimenez Gutierrez and Nikolas McNeal and Clayton Washington and You Chen and Lang Li and Huan Sun and Yu Su , editor =. Thinking about. Findings of the Association for Computational Linguistics:. 2022 , url =. doi:10.18653/V1/2022.FINDINGS-EMNLP.329 , timestamp =

    doi:10.18653/v1/2022.findings-emnlp.329 2022

  64. [65]

    Le and Ed H

    Xuezhi Wang and Jason Wei and Dale Schuurmans and Quoc V. Le and Ed H. Chi and Sharan Narang and Aakanksha Chowdhery and Denny Zhou , title =. The Eleventh International Conference on Learning Representations,. 2023 , url =

    2023

  65. [66]

    Empirical Study of Zero-Shot

    Tingyu Xie and Qi Li and Jian Zhang and Yan Zhang and Zuozhu Liu and Hongwei Wang , editor =. Empirical Study of Zero-Shot. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,. 2023 , url =

    2023

  66. [67]

    A Unified

    Xiaoya Li and Jingrong Feng and Yuxian Meng and Qinghong Han and Fei Wu and Jiwei Li , editor =. A Unified. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics,. 2020 , url =. doi:10.18653/V1/2020.ACL-MAIN.519 , timestamp =

    doi:10.18653/v1/2020.acl-main.519 2020

  67. [68]

    2023 , url =

    Ying Mo and Hongyin Tang and Jiahao Liu and Qifan Wang and Zenglin Xu and Jingang Wang and Wei Wu and Zhoujun Li , title =. 2023 , url =. doi:10.1109/ICASSP49357.2023.10094905 , timestamp =

    arXiv 2023

  68. [69]

    CoRR , volume =

    Wenhao Zhu and Hongyi Liu and Qingxiu Dong and Jingjing Xu and Lingpeng Kong and Jiajun Chen and Lei Li and Shujian Huang , title =. CoRR , volume =. 2023 , url =. doi:10.48550/ARXIV.2304.04675 , eprinttype =. 2304.04675 , timestamp =

    Pith/arXiv arXiv doi:10.48550/arxiv.2304.04675 2023

  69. [70]

    CoRR , volume =

    Xiang Wei and Xingyu Cui and Ning Cheng and Xiaobin Wang and Xin Zhang and Shen Huang and Pengjun Xie and Jinan Xu and Yufeng Chen and Meishan Zhang and Yong Jiang and Wenjuan Han , title =. CoRR , volume =. 2023 , url =. doi:10.48550/ARXIV.2302.10205 , eprinttype =. 2302.10205 , timestamp =

    Pith/arXiv arXiv doi:10.48550/arxiv.2302.10205 2023

  70. [71]

    2023 , url =

    Linzheng Chai and Dongling Xiao and Zhao Yan and Jian Yang and Liqun Yang and Qian. 2023 , url =. doi:10.1007/978-981-99-7022-3\_24 , timestamp =

    work page doi:10.1007/978-981-99-7022-3 2023

  71. [72]

    Learning to Select Relevant Knowledge for Neural Machine Translation , booktitle =

    Jian Yang and Juncheng Wan and Shuming Ma and Haoyang Huang and Dongdong Zhang and Yong Yu and Zhoujun Li and Furu Wei , editor =. Learning to Select Relevant Knowledge for Neural Machine Translation , booktitle =. 2021 , url =. doi:10.1007/978-3-030-88480-2\_7 , timestamp =

    doi:10.1007/978-3-030-88480-2 2021

  72. [73]

    arXiv preprint arXiv:2309.09298 , year=

    Owl: A large language model for it operations , author=. arXiv preprint arXiv:2309.09298 , year=

    Pith/arXiv arXiv

  73. [74]

    arXiv preprint arXiv:2401.07037 , year=

    xCoT: Cross-lingual Instruction Tuning for Cross-lingual Chain-of-Thought Reasoning , author=. arXiv preprint arXiv:2401.07037 , year=

    Pith/arXiv arXiv

  74. [75]

    GanLM: Encoder-Decoder Pre-training with an Auxiliary Discriminator , booktitle =

    Jian Yang and Shuming Ma and Li Dong and Shaohan Huang and Haoyang Huang and Yuwei Yin and Dongdong Zhang and Liqun Yang and Furu Wei and Zhoujun Li , editor =. GanLM: Encoder-Decoder Pre-training with an Auxiliary Discriminator , booktitle =. 2023 , url =. doi:10.18653/V1/2023.ACL-LONG.522 , timestamp =

    work page doi:10.18653/v1/2023.acl-long.522 2023

  75. [76]

    CoRR , volume =

    Ying Mo and Jian Yang and Jiahao Liu and Qifan Wang and Ruoyu Chen and Jingang Wang and Zhoujun Li , title =. CoRR , volume =. 2023 , url =. doi:10.48550/ARXIV.2308.09073 , eprinttype =. 2308.09073 , timestamp =

    Pith/arXiv arXiv doi:10.48550/arxiv.2308.09073 2023

  76. [77]

    Tjong Kim Sang and Fien De Meulder , editor =

    Erik F. Tjong Kim Sang and Fien De Meulder , editor =. Introduction to the CoNLL-2003 Shared Task: Language-Independent Named Entity Recognition , booktitle =. 2003 , url =

    2003

  77. [78]

    Doddington and Alexis Mitchell and Mark A

    George R. Doddington and Alexis Mitchell and Mark A. Przybocki and Lance A. Ramshaw and Stephanie M. Strassel and Ralph M. Weischedel , title =. Proceedings of the Fourth International Conference on Language Resources and Evaluation,. 2004 , url =

    2004

  78. [79]

    ACE 2005 Multilingual Training Corpus , author=

    2005

  79. [80]

    A Linear Programming Formulation for Global Inference in Natural Language Tasks , booktitle =

    Dan Roth and Wen. A Linear Programming Formulation for Global Inference in Natural Language Tasks , booktitle =. 2004 , url =

    2004

  80. [81]

    Multi-Task Identification of Entities, Relations, and Coreference for Scientific Knowledge Graph Construction , booktitle =

    Yi Luan and Luheng He and Mari Ostendorf and Hannaneh Hajishirzi , editor =. Multi-Task Identification of Entities, Relations, and Coreference for Scientific Knowledge Graph Construction , booktitle =. 2018 , url =. doi:10.18653/V1/D18-1360 , timestamp =

    doi:10.18653/v1/d18-1360 2018

Showing first 80 references.