{"id":"de911c92-0476-4984-bdeb-0ab6cd625b95","arxiv_id":"2505.06605","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"A BERT-based model that injects WordNet lexical-relation signals into attention and adaptively fuses them claims consistent accuracy improvements on 10 semantic matching datasets.","lead":"This paper adds WordNet relation signals into BERT's attention mechanism, using gates to fuse semantic and knowledge-based representations. It reports accuracy gains over BERT and other knowledge-enhanced models on ten sentence-pair datasets.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The Kprior construction never links WordNet relations to self-attention: Section 3.2 defines only an m×n co-attention matrix and then assumes an L×L form, so Eq. (5)'s injection is unimplementable and the claimed gains cannot be attributed to external knowledge.","rationale":"The reader's weakest assumption identifies the same load-bearing concern: Kprior is never concretely defined for self-attention, so Eq. (5) cannot be instantiated. My stress-test confirms this and sharpens it by noting that the only proposed Kprior is itself derived from the model's augmented attention weights, making it circular rather than a fixed external-knowledge prior. This matters because the entire empirical claim rests on the ability to attribute the reported gains to external knowledge infusion; if Kprior cannot be constructed, or if it merely re-encodes the PLM's own semantic similarity, the method has no identifiable mechanism. The manuscript's own bracketed notes and 'Self-correction' passages are in-scope evidence that the authors replaced equations during drafting, which further undermines reproducibility. There is no code, no formal verification, and no significance testing, so the tables cannot independently corroborate the method. Therefore the REJECT verdict stands unchanged; a resubmission with a fully specified Kprior, a concrete mapping to self-attention positions, and reproducible experiments could warrant reconsideration.","tokens_in":11310,"tokens_out":5981,"duration_ms":59795,"concrete_test":"Specify an explicit indexing map φ: {1..m}×{1..n} → {1..L}×{1..L} for the concatenated input [CLS] A [SEP] B [SEP], defining Kprior[i,j] = γI(relation) for cross-sentence word pairs and 0 for within-sentence, [CLS], [SEP], and padding positions, and reimplement Eq. (5) with this matrix. Then run MRPC and QQP with BERT-base and compare three variants: (a) the exact proposed model, (b) the same model with Kprior ≡ 0, and (c) the same model with Kprior replaced by random attention weights. If (b) or (c) reproduces the reported gains, the external-knowledge attribution fails. If no such φ can be defined, the method is currently unimplementable.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that a WordNet-derived prior injected through Eq. (5) improves PLM semantic matching. The load-bearing link is Kprior, and it is not actually specified. Section 3.2 defines Kprior only as an m×n cross-sentence co-attention matrix, then states \"we assume Kprior is appropriately formatted, potentially focusing on self-attention within a concatenated sequence or adapted based on the specific PLM layer\" and \"Let's assume Kprior ∈ R^{L×L}\". No mapping is given from the m×n word-pair space to the L×L positions of the concatenated BERT input, and no treatment of [CLS], [SEP], or padding tokens is provided. Without that mapping, Eq. (5)'s elementwise multiplication QK^T ⊙ Kprior is undefined in any concrete implementation.\n\nThe problem is worse than an unspecified formatting detail. The only concrete construction offered is Kprior[i,j] = (ωA_ij + ωB_ij)/2, where ωA and ωB are softmax attention weights computed from augmented scores s_ij = h_i^T h_j + γI(k_ij). Thus Kprior is a function of the PLM's own contextualized hidden states and of the same knowledge signal it is supposed to inject; it is not a fixed external-knowledge prior. If no relevant WordNet relation exists, s_ij reduces to ordinary dot-product similarity, so the 'knowledge prior' degenerates to a re-encoding of semantic similarity. The manuscript's inserted notes ('we assume', 'Self-correction' before Eq. 6) are explicit admissions that this central step was not worked out. Consequently the 1.66% and 2.31% average gains in Tables 1 and 2 have no clear mechanism and cannot be assigned to external knowledge.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes augmenting pre-trained language models (BERT-base and BERT-large) for semantic matching with an external lexical knowledge prior derived from WordNet. The method introduces a knowledge-aware co-attention mechanism, injects a prior matrix into self-attention via elementwise multiplication, and applies an adaptive gated fusion module to combine semantic and knowledge-modulated signals. Experiments are reported on six GLUE tasks and four additional datasets (SNLI, SciTail, SICK, TwitterURL), plus robustness evaluations using TextFlint, with claimed average accuracy gains of 1.66% over BERT-base and 2.31% over BERT-large, together with improvements over prior knowledge-enhanced baselines such as SemBERT, UERBERT, and SyntaxBERT.","tokens_in":11647,"tokens_out":4362,"duration_ms":42278,"significance":"If the proposed mechanism were precisely specified and reproducible, the idea of calibrating attention with explicit lexical relations (synonymy, antonymy, hypernymy) could be a useful contribution to semantic matching and model interpretability. The experimental scope is broad, including 10 datasets and robustness transformations, which is commendable. However, the manuscript's central construction is not concretely defined: the knowledge prior is never mapped to the self-attention space, and the 'external' prior is actually derived from the PLM's own attention weights. The paper also contains explicit 'Self-correction' and 'Please ensure' notes indicating unresolved equations and references. Consequently, the reported gains cannot currently be attributed to the proposed method, and the paper is not in a publishable state.","major_comments":[{"comment":"The knowledge prior K_prior is defined only as an m×n co-attention matrix via K_prior[i,j] = (ωA_ij + ωB_ij)/2, where ωA and ωB are computed from the augmented scores s_ij. The text then states 'we assume Kprior is appropriately formatted' and 'Let's assume Kprior ∈ R^{L×L}', but no mapping is given from the (m,n) word-pair indices to the L positions of the concatenated BERT input. There is no treatment of [CLS], [SEP], or padding tokens, and no description of how the m×n matrix is expanded or aligned to the L×L self-attention score matrix. As a result, Eq. (5)'s elementwise multiplication QK^T ⊙ K_prior is undefined in any concrete implementation; this is the load-bearing step of the proposed method.","section":"§3.2, Eqs. (3)–(5)"},{"comment":"The 'external' knowledge prior is not an external fixed matrix. K_prior is the average of the model's own attention weights ωA_ij and ωB_ij, which are computed from s_ij = (h_i^A)^T h_j^B + γ I(k_ij). Thus K_prior depends on the contextualized hidden states h_i^A, h_j^B and on the same knowledge signal G(k_ij) that it is supposed to inject. The injection is therefore circular: the calibration target and the injected prior both derive from the identical augmented scores. Moreover, when no WordNet relation is found, I(k_ij)=0 and K_prior degenerates to the model's ordinary dot-product soft-alignment weights, so the 'knowledge' component does not act as an independent prior at all.","section":"§3.2, Eqs. (1)–(5)"},{"comment":"The adaptive fusion module is not fully specified. In Eq. (6), the ⊕ operator is described as 'concatenation then linear layer', but the dimensions of the operand WOknw O^T_knw ⊕ (W_osem o_sem_i + b_osem) are not defined, and the text inserts 'Ensure weight dimensions match' as an instruction rather than a specification. Further, Eq. (8) defines y_i solely as a gated re-scaling of u_i (y_i = g_filter ⊙ tanh(W_y u_i)), with no direct dependence on o_sem_i except through the gate; yet §5.3 interprets low filter-gate values as the model 'relying more on the original semantic representation (o_sem_i, implicitly weighted higher when g_filter is low)'. This interpretation is inconsistent with the equation as written.","section":"§3.4, Eqs. (6)–(8)"},{"comment":"The manuscript contains explicit non-final notes that undermine its status as a completed submission: §3.4 includes a 'Self-correction' passage saying the original equations were 'overly complex/potentially incorrect' and that the replacements should be assumed, while §5.1 and §5.2 contain 'Please ensure this reference points to the correct table' and 'details assumed to be in Table 2 - ensure this ref is correct'. These passages indicate unresolved equations and references, making it impossible to verify the experimental claims or reproduce the method from the text.","section":"§5.1, §5.2, §3.4"}],"minor_comments":[{"comment":"Several entries are missing separators between numbers, e.g., '87.5085.48', '50.2083.98', and '60.89/61.3783.23/83.19', making the table difficult to read and some values ambiguous.","section":"Table 3"},{"comment":"The table formatting is broken: fields such as 'label:1 label:0filter gate:0.93label:0' are concatenated without spaces or separators, and the label semantics (what 0 and 1 denote) are not stated in the text.","section":"Table 4"},{"comment":"The title contains a grammatical error ('Using External knowledge to Enhanced PLM'), and the abstract repeats informal phrasing such as 'state-ofthe-art'; the paper would benefit from a careful language edit.","section":"Abstract and title"},{"comment":"Reference [10] is cited in the text as 'UERBERT' but the reference title is 'Using prior knowledge to guide bert's attention in semantic textual matching tasks'; the naming should be made consistent.","section":"References"}],"recommendation":"reject","confidential_remarks":"This manuscript appears to be an unfinished draft: it contains explicit 'Self-correction' notes, 'Please ensure' table-reference notes, and a central method step (mapping the m×n knowledge prior to the L×L self-attention space) that is simply assumed rather than defined. The 'external' prior is also circular, being derived from the model's own attention weights. These issues are load-bearing and would require re-specification and re-running of the method to address, not a routine revision. I recommend rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper attacks a real problem—injecting lexical knowledge like WordNet relations into PLMs for semantic matching—and the high-level design is reasonable: a knowledge-aware co-attention that builds a prior, injection into self-attention, then adaptive gating to fuse the semantic and knowledge streams. The reported gains are consistent across the GLUE subset, SNLI, SciTail, SICK, and TwitterURL, and the robustness experiments on TextFlint transformations are a nice addition. The case study in Table 4, with filtration gate values, is a genuine attempt at interpretability.\n\nThe problem is that the central mechanism is not actually specified. K_prior is defined in Section 3.2 only as an m×n co-attention matrix, then 'assumed' to be L×L for self-attention. No mapping is given from word pairs to the concatenated BERT token positions; no treatment of [CLS], [SEP], or padding. Eq. (5) multiplies QK^T by K_prior, but the dimensions only work if K_prior is L×L, and the paper never says how to get there. The stress-test note is correct: as written, the injection is unimplementable.\n\nEven if the formatting were fixed, the prior is not 'external' in any meaningful sense. It is defined as the average of the model's own co-attention weights, which already contain the knowledge signal via the indicator function. Injecting that back into self-attention is more like a self-consistency loop than a knowledge prior. The manuscript's own insertions—'we assume,' 'Let's assume,' 'Self-correction' before Eq. (6), 'Please ensure' before Table 1—are explicit admissions that the work is not finished.\n\nThe comparisons also lack error bars and significance tests; several reference entries are mismatched (e.g., [30] points to a POS tagging paper, not UERBERT); and no code is released. These are not cosmetic issues. A reader cannot determine what the method is, let alone reproduce it.\n\nThe direction is plausible, and a cleaned-up version with a fully specified K_prior, concrete alignment rules, and released code could be a credible incremental contribution. But this submission is not that version. I'd desk reject it. If a revised version addresses the mapping and the self-derivation issue, it's worth another look.\n\nRecommendation: don't send to referees in this state.","headline":"Incomplete draft: the central knowledge-prior mapping into self-attention is never specified, so the reported gains cannot be evaluated.","tokens_in":12242,"tokens_out":2559,"would_cite":false,"duration_ms":22866,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that calibrating BERT's self-attention with WordNet-derived lexical relation priors yields consistent accuracy gains in semantic matching, with average improvements of 1.66 points over BERT-base and 2.31 points over…","keywords":["semantic relevance modeling","external knowledge","attention calibration","adaptive fusion","WordNet relations","pre-trained language models","natural language inference","paraphrase identification"],"falsifier":"Replace the WordNet-based prior matrix with a randomly permuted matrix of the same shape, keeping the architecture and training unchanged; if the accuracy gains over BERT persist, the reported improvements come from the added capacity of the fusion network, not from lexical knowledge. A direct implementation also requires a concrete rule mapping the $m\\times n$ relation matrix to the $L\\times L$ self-attention positions, and without that rule Equation (5) cannot be executed.","tokens_in":11045,"feed_emoji":"🧠","tokens_out":8353,"duration_ms":73507,"temperature":0.7,"pith_summary":"The paper argues that pretrained language models cannot reliably learn all the lexical knowledge needed for semantic matching from co-occurrence data alone, and that injecting structured lexical relations from WordNet into BERT's attention improves performance. It proposes a knowledge-aware co-attention that builds a prior matrix from synonym, antonym, hypernym, and hyponym links, injects that prior into self-attention, and uses an adaptive gated fusion to combine semantic and knowledge signals. On ten public datasets, the authors report average accuracy gains of 1.66% over BERT-base and 2.31% over BERT-large on six GLUE tasks, with additional gains on SNLI, SciTail, SICK, and TwitterURL. If the claim holds, this offers a modest but consistent way to make PLMs more sensitive to fine-grained semantic differences without changing pretraining, including better resistance to antonym swaps and similar lexical perturbations.","feed_headline":"WordNet priors add 2.31% to BERT-large's GLUE accuracy","feed_subtitle":"A gated attention module uses synonym and antonym links to beat BERT on 10 NLP benchmarks.","key_machinery":"The central object is the knowledge prior matrix $K_{\\mathrm{prior}}\\in\\mathbb{R}^{L\\times L}$, built from WordNet relation indicators that boost the co-attention scores between words in the two sentences. This matrix is multiplied element-wise into the PLM's scaled dot-product attention scores, producing a knowledge-modulated attention output parallel to the standard semantic output. The second key machinery is the adaptive fusion module: a cross-attention step where each signal gathers context from the other, a fusion gate $g_{\\mathrm{fuse}}$ that balances the two contributions, and a filtration gate $g_{\\mathrm{filter}}$ that scales the combined signal before it replaces the self-attention output. Together these carry the argument that explicit lexical relations can sharpen attention alignment and then be selectively trusted.","core_discovery":"The central claim is that calibrating BERT's self-attention with an external lexical-knowledge prior improves semantic relevance modeling beyond what BERT can learn from text alone. The authors construct a co-attention matrix over the two input sentences, augment each pairwise similarity score with an indicator of whether the word pair is linked in WordNet by synonymy, antonymy, hypernymy, or hyponymy, and average the resulting attention weights to form a prior matrix $K_{\\mathrm{prior}}$ that is multiplied element-wise into the scaled dot-product attention scores in each head. A two-stage adaptive fusion then lets the semantic and knowledge-modulated attention outputs adjust each other before a gating network decides how much of the fused signal to keep, yielding a final self-attention output that can fall back on the original semantic representation when the external signal is noisy. The paper reports consistent accuracy improvements over BERT on all evaluated tasks, with the largest relative gains on contradiction-heavy and lexical-variation cases such as swapped antonyms.","pith_inferences":["Inference: because the injection is formulated as an additive prior on attention scores, the same mechanism could absorb other discrete lexical resources, such as domain ontologies or medical terminologies, without retraining the PLM; the paper only demonstrates WordNet relations.","Inference: the reported gate values in the case studies suggest the model is learnably deciding when to trust external knowledge; one could test this directly by measuring whether gate values correlate with WordNet coverage of the word pairs in each input.","Inference: if the $m\\times n$ to $L\\times L$ reshaping is resolved, the method could be applied to single-sentence tasks by treating the second input as the same sentence, though the paper does not explore this."],"forward_implications":["On the six GLUE sentence-pair tasks, the WordNet prior raises average accuracy by 1.66 percentage points over BERT-base and 2.31 points over BERT-large, with the largest single-task gain on QQP.","Across ten datasets, the method averages higher accuracy than SemBERT, UERBERT, and SyntaxBERT, indicating lexical-relation priors can complement syntax- and semantic-role-based knowledge infusion.","In robustness tests, the model gains roughly 10 points over the best baseline on swapped-antonym pairs in QQP, showing the prior sharpens contradiction detection.","The case studies show the filtration gate sometimes heavily weights the fused knowledge signal and sometimes nearly ignores it, so the adaptive gate is what makes noisy external knowledge safe to use."],"supporting_citations":[{"why":"Supplies the WordNet relation types — synonymy, antonymy, hypernymy, hyponymy — from which the knowledge prior is constructed.","marker":"[2]"},{"why":"The BERT backbone whose self-attention is modified; its base and large variants are the direct baselines for the reported gains.","marker":"[8]"},{"why":"The prior-work baseline that injects synonym knowledge to guide BERT attention; the paper compares against it as UERBERT.","marker":"[10]"},{"why":"A knowledge-enhanced BERT baseline using semantic role labels; Tables 1-3 report comparisons and gains over it.","marker":"[9]"},{"why":"SyntaxBERT is the strongest previous knowledge-enhanced baseline in the evaluation; the paper reports average gains of 0.91% over it with BERT-large.","marker":"[4]"},{"why":"GLUE supplies six of the ten datasets (MRPC, QQP, STS-B, MNLI, RTE, QNLI) used to compute the headline average-accuracy improvements.","marker":"[25]"},{"why":"ESIM is a representative non-pretrained inference baseline used in both main and robustness comparisons.","marker":"[7]"},{"why":"TextFlint provides the transformations (SwapAnt, NumWord, etc.) used in the robustness experiments.","marker":"[33]"}],"fun_headline_variants":["WordNet boosts BERT semantic matching on 10 benchmarks","External lexical knowledge sharpens BERT relevance scoring","Synonym links enhance BERT semantic relevance by 2.31%","WordNet knowledge yields consistent BERT gains on 10 datasets"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the matrix of WordNet relations computed between words of the two sentences can be rearranged into the square attention matrix inside the pretrained model; the paper assumes this formatting rather than defining it.","fun_headline_variants_meta":{"raw":{"variants":["WordNet boosts BERT semantic matching on 10 benchmarks","External lexical knowledge sharpens BERT relevance scoring","Synonym links enhance BERT semantic relevance by 2.31%","WordNet knowledge yields consistent BERT gains on 10 datasets"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000207,"raw_usage":{"total_tokens":1375,"prompt_tokens":895,"completion_tokens":480,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":511,"completion_tokens_details":{"reasoning_tokens":411}},"tokens_in":511,"tokens_out":480,"duration_ms":4378,"temperature":1.0,"reasoning_tokens":411,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T22:38:06.516062+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Replace the WordNet-based prior matrix with a randomly permuted matrix of the same shape, keeping the architecture and training unchanged; if the accuracy gains over BERT persist, the reported improvements come from the added capacity of the fusion network, not from lexical knowledge. A direct implementation also requires a concrete rule mapping the $m\\times n$ relation matrix to the $L\\times L$ self-attention positions, and without that rule Equation (5) cannot be executed.","supporting_citations":[{"cited_title":"Bert: Pre-training of deep bidirectional transformers for language understanding,","cited_arxiv_id":null,"evidence_quote":"The BERT backbone whose self-attention is modified; its base and large variants are the direct baselines for the reported gains."},{"cited_title":"Using prior knowledge to guide bert’s attention in semantic textual matching tasks,","cited_arxiv_id":null,"evidence_quote":"The prior-work baseline that injects synonym knowledge to guide BERT attention; the paper compares against it as UERBERT."},{"cited_title":"Semantics-aware bert for language understanding,","cited_arxiv_id":null,"evidence_quote":"A knowledge-enhanced BERT baseline using semantic role labels; Tables 1-3 report comparisons and gains over it."},{"cited_title":"Glue: A multi-task benchmark and analysis platform for natural language understanding,","cited_arxiv_id":null,"evidence_quote":"GLUE supplies six of the ten datasets (MRPC, QQP, STS-B, MNLI, RTE, QNLI) used to compute the headline average-accuracy improvements."}],"review_version":1}