{"id":"1fd38daa-1dfd-4102-b0f9-cbcadad2996f","arxiv_id":"2501.02196","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"CPTuning adds contrastive learning and label smoothing to generative relation extraction, letting a T5 model extract zero, one, or multiple relations per entity pair with Trie-constrained decoding.","lead":"This paper trains a T5 model to generate relation phrases between two entities and accepts every generated relation whose estimated likelihood passes a threshold, so one entity pair can yield several relations. The method reports small F1 gains on four relation-extraction benchmarks, but the written inference rule contradicts the loss and no code is released.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. 7 and Algorithm 1 invert the meaning of the score f defined in Eq. 3, so the stated objective rewards low-probability gold relations and the stated inference rule keeps low-probability candidates; the central threshold mechanism is not reproducible from the text.","rationale":"The reader's weakest assumption focused on negative sampling in distantly supervised NYT, which is a real concern but would only weaken one dataset. The threshold sign contradiction is more fundamental: it affects every experiment and the entire conceptual claim of learning a probability-mass threshold. The manuscript contains no code, so the ambiguity is not merely stylistic—it is impossible to determine from the text whether the method is even the one described. The ablations are internally consistent with the reported trends, which is credit to the authors, but that consistency cannot repair a formal inversion in the core objective and inference rule. I therefore recommend moving from CONDITIONAL to REJECT: as written, the method cannot work, and the empirical results are disconnected from the formal specification.","tokens_in":14774,"tokens_out":6313,"duration_ms":63236,"concrete_test":"Run the training objective exactly as written (Eq. 3, 7, 8 with λ=1.0, ζ=1.2, β=0.2, µ=0.1) on a TACRED subset for one epoch and compare the average gold-relation token probability before/after: if the equations are literal, gold probabilities should drop. Then run the full pipeline with Algorithm 1's 'keep if f>λ' rule; if the final F1 does not match the reported 76.7, the published algorithm is not the one that produced the results.","verdict_should_be":"REJECT","load_bearing_attack":"The formal core of the method is internally inconsistent. Eq. 3 defines f(r_i) as the length-normalized negative log-likelihood, so smaller f means higher token probability. Eq. 7's contrastive term for gold relations is max(ζ−f(r_i),0), which is minimized only when f(r_i) ≥ ζ, i.e., when the gold relation has *lower* probability; the fake-relation term max(f(r_j)−ζ,0) is minimized when f(r_j) ≤ ζ, i.e., when fake relations have *higher* probability. Algorithm 1 then keeps produced candidates whenever f(r_i) > λ, again selecting for lower probability. Taken literally, training would suppress exactly the relations the method claims to extract, so the reported F1 gains (e.g., 91.4 on NYT) cannot follow from the written algorithm. Since the CTL ablation in Table 3 shows contrastive learning *helps*, the implemented objective must use a different sign convention (e.g., f as raw log-likelihood, or flipped inequalities), but no code is provided to disambiguate. This makes the central claim—that thresholded candidate likelihoods solve entity-pair overlap—unverifiable from the text. The negative-sampling concern is secondary: even if negative sampling were clean, the threshold direction would still be wrong as written.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CPTuning, a contrastive prompt-tuning method for generative relation extraction. It reformulates RE as a Seq2Seq text-infilling task with sentinel tokens, uses Trie-constrained beam search to generate candidate verbalized relations, and combines layer-based label smoothing (LBLS) with a contrastive loss (CTL) to let the model assign probability mass above or below a threshold. Experiments on TACRED, TACREV, Re-TACRED, and NYT report state-of-the-art micro-F1 scores for T5-large, with ablations showing that both LBLS and CTL contribute to the gains.","tokens_in":15098,"tokens_out":5275,"duration_ms":48734,"significance":"If the method works as described, it addresses a real limitation of generative prompt-tuning RE—entity-pair overlap—and the reported gains over GenPT and other baselines are substantial. The paper's ablations (Table 3) support the contribution of both proposed components, and the H-index analysis (Table 4) is a useful diagnostic of semantic behavior. However, the central scoring and threshold mechanism is written inconsistently, so the claimed results cannot be verified from the text; no code is provided.","major_comments":[{"comment":"Eq. (3) defines f(ri) = −Σ log P(ri(j)) / |ri|^α, so larger f means lower token probability. Eq. (7) then minimizes max(ζ − f(ri), 0) for gold relations, which is zero only when f(ri) ≥ ζ (i.e., low probability), and max(f(rj) − ζ, 0) for fake relations, which is zero only when f(rj) ≤ ζ (i.e., high probability). Algorithm 1 line 14 keeps candidates with f(ri) > λ, again selecting low-probability candidates. Taken literally, the training objective and inference rule suppress gold relations and promote fake ones, so the reported F1 gains in Table 2 cannot follow from the written algorithm. Please correct the sign of f or flip the inequalities, and clarify which convention was actually implemented.","section":"§3.2, §3.3, Eq. (3), Eq. (7), Algorithm 1"},{"comment":"Results in Table 2 are reported as single micro-F1 numbers without error bars, significance tests, or the number of seeds, despite the abstract's claim of \"significantly outperforms.\" The authors should either report multiple runs with variance and a statistical test or soften the significance claim.","section":"§4.4, Table 2"},{"comment":"Figure 5 shows performance of CPTuning-s1 and -s2 against μ on TACRED and NYT, and μ = 0.1 selected from these curves is then fixed for all experiments, including TACREV and Re-TACRED. If these curves are computed on the test splits, this is tuning on the test set; please specify the split used and report sensitivity to λ and ζ, which are otherwise fixed without analysis.","section":"§5, Figure 5"},{"comment":"Equation (7) samples fake relations R− uniformly from the verbalized relation vocabulary and treats them as definitely absent. In a distantly supervised multi-relation dataset such as NYT, an unannotated relation can be true for a given entity pair, so random negative sampling can suppress correct relations and undermine the EPO handling the method targets. Please discuss this issue and consider filtering negatives by entity-type constraints or evaluating alternative negative-sampling strategies.","section":"§3.3, Eq. (7)"}],"minor_comments":[{"comment":"The manuscript contains template artifacts: Section 5 includes \"2 F. Author et al.\" and Springer template captions (e.g., \"Fig. 1: Caption\") immediately before Figure 5; these should be removed.","section":"§5, Figure 5"},{"comment":"There are typos: \"ralation\" in the Section 4.4 headings, \"citiy\" in Figure 4, and \"T able\" in table captions.","section":"§4.4 and Figure 4"},{"comment":"Algorithm 1 initializes r1..rK as the prefix z, but candidate relations should arguably be continuations after the prefix; please clarify how full target texts are represented.","section":"§3.2, Algorithm 1"},{"comment":"No code or reproducibility statement is provided; given the sign ambiguity in Eq. (3)/(7), releasing code would materially help verification.","section":"General"},{"comment":"Equation (6) uses r*(i) both as the predicted token and as the summation index; please clean up the notation to avoid confusion.","section":"§3.3, Eq. (6)"},{"comment":"The text says the NYT training and test sets contain 9782 and 987 EPO entity pairs, but Table 1 only lists \"Yes\" in the EPO column; please make the table consistent with the text.","section":"§4.1, Table 1"}],"recommendation":"major_revision","confidential_remarks":"The sign inconsistency in Eqs. (3)/(7) and Algorithm 1 is the main obstacle. My reading is that the implemented system likely used a different convention than the one written, because the CTL ablation helps; nevertheless, the text as it stands makes the core mechanism impossible to reproduce. I would invite a revision with corrected equations, a clear statement of the implemented scoring convention, and preferably code. The paper otherwise fits the journal's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know this before citing: the central threshold mechanism is written with the sign inverted. Eq. 3 defines f as length-normalized negative log-likelihood, so smaller f means higher probability. Yet the contrastive loss in Eq. 7 penalizes gold relations with f < ζ and rewards fake relations with f > ζ, and Algorithm 1 keeps candidates with f > λ. Literally, training suppresses the relations the method claims to extract and inference keeps low-probability candidates. The reported F1 gains cannot follow from the written algorithm. Since the CTL ablation shows contrastive learning helps, the implementation must use a different convention (e.g., raw log-probability or flipped inequalities), but no code is provided to disambiguate.\n\nWhat the paper does well: it addresses entity pair overlap, a real gap in generative RE; it combines known ingredients (GenPT-style text-infilling, Trie-constrained decoding from SURE, BRIO-style contrastive loss, label smoothing) into a single pipeline; and the ablations show both LBLS and CTL contribute. The empirical results are consistent across four datasets and beat strong baselines by a few F1 points. If the sign is corrected, the method is plausible.\n\nOther soft spots: no error bars or significance tests; hyperparameters λ, ζ, β, α, µ are hand-set, and µ is chosen from Figure 5 without indicating whether the curves are dev or test. Negative sampling of fake relations on distantly supervised NYT is a real risk—random negatives may be true but unannotated—though that is minor relative to the sign error.\n\nBottom line: the paper deserves a serious referee because the idea is useful and the experiments are substantial, but the formal core needs correction and the artifacts need to be released. I would not cite it in its current form.","headline":"Solid multi-relation RE results, but the core threshold objective and inference rule are written backwards.","tokens_in":15645,"tokens_out":2935,"would_cite":false,"duration_ms":28175,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"CPTuning claims that contrastive prompt tuning with layer-based label smoothing lets a sequence-to-sequence model assign each candidate relation a probability above or below a threshold, so thresholding generated candidates extracts all…","keywords":["relation extraction","entity pair overlap","contrastive learning","prompt tuning","sequence-to-sequence generation","text infilling","label smoothing","Trie-constrained decoding"],"falsifier":"Manually inspect a sample of the fake negative relations used in Eq. 7 on the NYT training set: if a meaningful fraction of them are actually expressed in the sentence, then the contrastive loss is suppressing true relations, and replacing random negatives with verified or entity-type-incompatible negatives should raise multi-relation F1 above the reported 91.2-91.4.","tokens_in":14517,"feed_emoji":"🔗","tokens_out":13726,"duration_ms":117097,"temperature":0.7,"pith_summary":"Relation extraction traditionally assigns exactly one relation to each entity pair, but real sentences often state several relations at once; the paper calls this entity pair overlap. CPTuning is a training and decoding scheme that lets a sequence-to-sequence model output all valid relations instead of one. The central claim is that contrastive training with layer-based label smoothing makes the model associate each candidate relation with probability mass above or below a threshold, and that thresholding the generated candidates recovers every true relation. The authors report that T5-large fine-tuned with CPTuning outperforms previous generative and classificational methods on four datasets, covering both single- and multi-relation extraction.","feed_headline":"One T5 model now extracts multiple relations per entity pair","feed_subtitle":"Contrastive training plus a threshold turns a text-to-text generator into a multi-relation extractor that beats prior methods.","key_machinery":"The central object is a thresholded probability mass over verbalized relation phrases. The training objective is $L(\\theta) = L_{\\mathrm{ctl}}(\\theta) + \\mu L_{\\mathrm{lbls}}(\\theta)$, where $L_{\\mathrm{lbls}}$ is the layer-based label smoothing loss that distributes target probability $1-\\beta$ to the gold token and $\\beta/(|T_i|-1)$ to the other tokens allowed at layer $i$ of the Trie, and $L_{\\mathrm{ctl}}$ is a contrastive loss that penalizes gold-relation scores $f(r_i)$ below $\\zeta$ and fake-relation scores above $\\zeta$. The score $f$ is a length-normalized negative log-likelihood. Decoding uses a Trie, a prefix tree over the verbalized relation vocabulary, to restrict beam search to valid relation phrases, and the final threshold $\\lambda$ converts the ranked candidate list into the extracted relation set.","core_discovery":"The paper claims that entity pair overlap can be handled by a generative model that learns a graded, threshold-able likelihood over relation phrases rather than a one-hot gold label. Concretely, CPTuning combines layer-based label smoothing, which gives every valid token at each decoding layer a small nonzero target probability, with a contrastive loss that pushes gold relations above a threshold $\\zeta$ and randomly sampled fake relations below it. At inference, Trie-constrained prefix-given beam search generates a set of candidate relations and a length-normalized log-likelihood score $f(r_i)$ is compared with a threshold $\\lambda$ to decide which relations exist. The paper reports that T5-large trained this way reaches the best micro F1 among compared methods on TACRED, TACREV and NYT, with Re-TACRED used in ablations and low-resource experiments, outperforming both classificational and generative baselines in single- and multi-relation settings.","pith_inferences":["The paper leaves implicit that the same contrastive thresholding recipe could apply to other structured generation problems with variable numbers of valid outputs per input, such as event extraction or open information extraction, since the machinery only requires a verbalized candidate set and a decision threshold.","Because the negative relations in Eq. 7 are sampled uniformly from the relation vocabulary, the method's practical ceiling on distantly supervised data may be set by annotation noise; an obvious test is to replace uniform negatives with entity-type-constrained or confidence-weighted negatives and check whether multi-relation F1 rises.","The H-index analysis in the paper suggests likelihood and semantic similarity align under CPTuning; one concrete extension is to use semantic similarity rankings as an additional training signal, which could reduce the number of beam candidates needed at inference."],"forward_implications":["On NYT, a dataset built for entity pair overlap, CPTuning with T5-large reaches micro F1 of 91.4 while the strongest prior generative prompt-tuning baseline reaches 87.27, so thresholded generative decoding works for multi-relation extraction.","On single-relation datasets, the same training recipe beats the previous best generative method by 1.8 F1 points on TACRED and 0.9 on TACREV, so supporting multiple relations does not degrade single-relation accuracy.","Ablation removes either the contrastive loss or the layer-based label smoothing and NYT F1 drops from 91.2 to about 87.7-87.9, indicating both components are needed for the reported multi-relation gain.","With only 8-32 labeled instances per relation, CPTuning is competitive with or better than prior prompt-tuning baselines on most settings, suggesting the thresholded likelihood signal remains useful in low-resource conditions."],"supporting_citations":[{"why":"Provides the generative prompt-tuning baseline and the Seq2Seq text-infilling formulation that CPTuning builds on.","marker":"[11]"},{"why":"The pre-trained T5 model used as the backbone; its text-infilling objective motivates the sentinel-mask transformation.","marker":"[38]"},{"why":"Supplies the label-smoothing technique that CPTuning adapts into layer-based label smoothing.","marker":"[31]"},{"why":"Introduces the Trie-constrained decoding and the summarization-style transformation that CPTuning uses to generate only valid relations.","marker":"[28]"},{"why":"Provides the contrastive loss structure, length normalization, and balance factor used in scoring and training.","marker":"[26]"},{"why":"The NYT dataset, the multi-relation benchmark with entity pair overlap that the method is designed to handle.","marker":"[39]"},{"why":"The TACRED dataset, the primary single-relation benchmark that supports the single-relation performance claims.","marker":"[56]"},{"why":"The diverse beam search idea behind prefix-given constrained decoding for sampling multiple candidate relations.","marker":"[44]"},{"why":"The main classificational prompt-tuning baseline used in the comparison.","marker":"[53]"}],"fun_headline_variants":["CPTuning turns T5 into a multi-relation extractor","One T5, many relations: contrastive tuning nails overlapping pairs","Thresholded contrastive generation extracts multiple relations","Single T5 handles overlapping relations via contrastive prompt tuning","Contrastive tuning beats dedicated multi-relation extractors"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a relation sampled at random from the relation vocabulary and not labelled for a given entity pair is genuinely absent; on automatically annotated data such as NYT this can fail because a sampled relation may be true but simply unannotated.","fun_headline_variants_meta":{"raw":{"variants":["CPTuning turns T5 into a multi-relation extractor","One T5, many relations: contrastive tuning nails overlapping pairs","Thresholded contrastive generation extracts multiple relations","Single T5 handles overlapping relations via contrastive prompt tuning","Contrastive tuning beats dedicated multi-relation extractors"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000575,"raw_usage":{"total_tokens":2710,"prompt_tokens":936,"completion_tokens":1774,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":552,"completion_tokens_details":{"reasoning_tokens":1692}},"tokens_in":552,"tokens_out":1774,"duration_ms":11398,"temperature":1.0,"reasoning_tokens":1692,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T22:13:54.458680+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Manually inspect a sample of the fake negative relations used in Eq. 7 on the NYT training set: if a meaningful fraction of them are actually expressed in the sentence, then the contrastive loss is suppressing true relations, and replacing random negatives with verified or entity-type-incompatible negatives should raise multi-relation F1 above the reported 91.2-91.4.","supporting_citations":[{"cited_title":"In: Findings of the Association for Computational Linguistics: EMNLP 2022","cited_arxiv_id":null,"evidence_quote":"Provides the generative prompt-tuning baseline and the Seq2Seq text-infilling formulation that CPTuning builds on."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The pre-trained T5 model used as the backbone; its text-infilling objective motivates the sentinel-mask transformation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the label-smoothing technique that CPTuning adapts into layer-based label smoothing."},{"cited_title":"In: Findings of the Association for Computational Linguistics: EMNLP 2022","cited_arxiv_id":null,"evidence_quote":"Introduces the Trie-constrained decoding and the summarization-style transformation that CPTuning uses to generate only valid relations."},{"cited_title":"In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2022","cited_arxiv_id":null,"evidence_quote":"Provides the contrastive loss structure, length normalization, and balance factor used in scoring and training."},{"cited_title":"In: Balc´ azar, J.L., Bonchi, F., Gionis, A., Sebag, M","cited_arxiv_id":null,"evidence_quote":"The NYT dataset, the multi-relation benchmark with entity pair overlap that the method is designed to handle."},{"cited_title":"In: EMNLP 2017","cited_arxiv_id":null,"evidence_quote":"The TACRED dataset, the primary single-relation benchmark that supports the single-relation performance claims."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The main classificational prompt-tuning baseline used in the comparison."}],"review_version":1}