{"id":"2ca3e428-a79b-45a6-930c-36c53fb92058","arxiv_id":"1909.02560","paper_version":5,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Paraphrase identification models suffer dramatic accuracy drops on examples modified by replacing or adding shared words, and adversarial training partially restores robustness.","lead":"The paper shows that paraphrase identification models are easily fooled when words shared by both sentences are swapped or introduced. This matters because it reveals a new failure mode and a possible defense via adversarial training.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported accuracy drops may be inflated: modified-example labels are only spot-checked on a biased BERT-failure sample, with 12–21% human disagreement, and no validation for BiMPM/DIIN.","rationale":"Good-faith reading: the paper proposes a new attack formulation and shows large accuracy drops on modified examples, with a manual check that most (but not all) modified examples are label-correct. The central claim requires that the gold labels of modified examples are largely preserved; otherwise the drop measures label corruption, not model fragility. Sec. 4.4 provides the only direct evidence, and it has three limitations: (1) it samples only from examples where BERT is wrong, so the reported 79%/88% human agreement does not estimate validity on the full modified test set; (2) 12-21% disagreement is non-trivial and no error bars are given; (3) it covers only BERT, while Table 1 includes BiMPM and DIIN. Under a conservative correction, the BERT-on-QQP drop could shrink from 94.3 to 24.1 to roughly 94.3 to 40, which still indicates a substantial issue but not as dramatic as claimed. The adversarial-training result is also scored against the same possibly-noisy labels, so the mitigation claim is similarly uncertain. The missing '??' citation in Sec. 3.2 is a separate editorial defect. These concerns do not invalidate the paper, but they justify the reader's CONDITIONAL verdict: the work is promising and likely correct in direction, but the headline numbers need re-validation on random samples covering all models. The proposed annotation test would settle the matter.","tokens_in":9167,"tokens_out":14254,"duration_ms":155955,"concrete_test":"Re-run human annotation on a random sample of 150 modified examples per model (BiMPM, DIIN, BERT) per dataset, drawn uniformly from the full modified test sets used for Table 1 rather than from the model-failure subset. Use three annotators per example and ask them to label the pair and rate grammaticality (1–3). Then recompute Table 1 accuracies on the subset of examples where at least two annotators confirm the original gold label. If the recomputed drops remain large (e.g., below 50% accuracy), the robustness claim stands; if they rise to near original accuracy, the reported drops are largely an artifact of label corruption. Report annotator agreement (e.g., Fleiss' kappa) to bound label noise.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—that paraphrase models are highly fragile under shared-word modification—depends on the modified examples actually preserving the original label. The only evidence is the manual evaluation in Sec. 4.4, which samples 100 modified examples per dataset, for BERT only, and only from the subset where BERT makes wrong predictions. On that biased sample, human agreement with the original label is 79% (QQP) and 88% (MRPC). Because the sample excludes examples where BERT is correct, the paper cannot estimate the overall mislabeling rate in the 1,000-example test set used for Table 1. If, say, 16% of all modified examples are mislabeled, the reported BERT-on-QQP accuracy of 24.1% would correspond to a true accuracy of roughly 40%—still a drop from 94.3%, but materially smaller. The same label noise contaminates the adversarial-training evaluation, which scores against the original labels. No human validation is reported for BiMPM or DIIN, where the drops are even larger (11.1% and 1.7% on QQP/MRPC). The phrase 'some recent works (??)' in Sec. 3.2 also indicates an unresolved missing reference, though this is secondary. Thus the magnitude of the claimed robustness issue, and the reported mitigation, are not yet established with sufficient label validity.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies robustness of paraphrase identification models under a new type of modification: changing words that are shared by the two sentences (for positive examples) or introducing new shared words (for negative examples). The proposed algorithm samples original examples, restricts replaceable positions by heuristic rules (no stopwords; shared words for positives; same POS for negatives), generates candidate substitutions with a BERT masked language model, and uses beam search to find modifications that increase the target model's loss. The authors evaluate BiMPM, DIIN, and BERT on QQP and MRPC, reporting large accuracy drops on the modified examples (e.g., BERT on QQP from 94.3% to 24.1%) and improved accuracy after adversarial training with modified examples. A manual evaluation on 100 BERT-failure examples per dataset reports human label agreement of 79% (QQP) and 88% (MRPC) on modified examples.","tokens_in":9419,"tokens_out":3257,"duration_ms":38694,"significance":"If label preservation in the modified examples is adequately established, the paper makes a useful contribution. The shared-word modification perspective is genuinely different from PAWS-style word scrambling, and the finding that several architectures, including BERT, can be moved from confident correct predictions to confident wrong predictions by coordinated word replacement is practically important. The adversarial-training result, showing partial recovery without large original-data degradation, is a constructive step. The method is simple, reproducible in principle, and the paper includes several concrete examples. The main weakness is that the validity of the measured accuracy drops rests almost entirely on a small, biased manual evaluation; the quantitative claims, as currently supported, are provisional.","major_comments":[{"comment":"The manual evaluation is the only evidence that the modified examples preserve the original labels, but it samples only 100 examples per dataset, exclusively from the subset where BERT makes wrong predictions, and only for BERT as the target model. Human agreement with the original label is 79% on QQP and 88% on MRPC, so 21% and 12% of the sampled modified examples are not confirmed to preserve the label. Because the sample is conditioned on adversarial failure, it cannot estimate the label-preservation rate in the 1,000-example sets used for Table 1, and no validation is reported for BiMPM or DIIN. Since all accuracies in Table 1 and the adversarial-training rows are scored against the original labels, a mislabel rate of this magnitude could materially inflate the reported drops; for example, BERT's QQP accuracy of 24.1% from Table 1 would be substantially higher if a nontrivial fraction of the modified examples are actually incorrectly labeled. The paper should validate labels on a random sample of the full modified test sets (or per model), or report accuracies corrected for label noise.","section":"Section 4.4, Table 2"},{"comment":"The experiments do not include a random-substitution control. Table 1 compares model accuracy on original examples with accuracy on adversarially modified examples, but it never compares against modifications using the same replacement positions and the same BERT-MLM candidate words without optimizing the target-model loss. Without such a control, the large accuracy drops cannot be attributed specifically to the introduction or replacement of shared words; they could in part reflect any lexical substitution or the grammaticality degradation reported in Section 4.4. Adding a control where substitutions are random (or chosen to minimize loss) would directly support the paper's central claim about shared words as the source of fragility.","section":"Section 4.2, Table 1"},{"comment":"All results are single point estimates on 1,000 sampled examples, with no standard errors, confidence intervals, or multiple random seeds. Given the 12-21% label disagreement in the manual evaluation and the fact that some reported differences are small (e.g., QQP adversarial-training gains for BiMPM: 11.1% to 21.4%), the statistical stability of the comparisons is unclear. The paper should report bootstrap intervals over the sampled examples or variance across training/generation seeds for the main accuracy columns.","section":"Table 1"}],"minor_comments":[{"comment":"The phrase \"some recent works ( ??)\" is an unresolved placeholder and should be replaced with the actual citations or removed before publication.","section":"Section 3.2"},{"comment":"The hyperparameters S, K, and B are set to 5, 25, and 25 on QQP and doubled on MRPC, but no sensitivity analysis or rationale is given; a sentence explaining why these values were chosen and whether results are stable to them would strengthen the paper.","section":"Appendix A"},{"comment":"The manual evaluation reports only accuracy and grammaticality averages; reporting annotator-level agreement (e.g., Fleiss' kappa) would help quantify how difficult the label judgment is, especially given the low agreement on QQP modified examples.","section":"Section 4.4"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is within scope for the journal and the core idea is worth publishing if the label-validity concern is adequately addressed. The missing citation placeholder in Section 3.2 should be fixed. I would also encourage the authors to release code or a data appendix with the modified examples for reproducibility."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know about this paper: it studies a genuinely new failure mode for paraphrase identification. Instead of semantic-preserving edits, the attack replaces or introduces shared content words between the two sentences, which is different from PAWS word scrambling and covers positive examples too. The BERT-masked-LM joint substitution candidate generation is a neat, concise trick, and the reported accuracy drops are dramatic (BERT on QQP from 94.3% to 24.1%). The paper also shows adversarial training recovers a good chunk of the drop, which is a useful practical result.\n\nWhat is actually good: the method is simple and reproducible, tested on three architecture families (BiMPM, DIIN, BERT), and the authors do report manual evaluation, which is more than many attack papers bother with. The qualitative finding—that shared-word modifications expose real fragility—is credible and worth knowing about.\n\nNow the soft spots. The label-validity evidence is the weakest link. The manual check covers only 100 modified examples per dataset, only for BERT as target, and only from the subset where BERT made mistakes. On that biased sample, human agreement with the original label is 79% on QQP and 88% on MRPC, versus 92%/95% on original examples. So a fifth of the QQP examples used to demonstrate the drop may actually be mislabeled, and the paper cannot estimate the mislabeling rate on the full 1,000-example test set. That doesn't erase the finding, but it means the headline numbers are likely overstated. Also missing: error bars, a random-substitution control, and human validation for the BiMPM/DIIN results. The unresolved citation placeholder '(??)' in Sec. 3.2 is a minor but real blemish.\n\nWho is this for? People working on adversarial robustness, sentence-pair models, or evaluation methodology. It deserves a serious referee, and I would send it back for a round of revisions: report variance, add a random-control baseline, and expand human evaluation to all models and to examples where the model is correct. The central claim that paraphrase models are fragile under shared-word modification holds up in broad strokes; the precise magnitudes are not yet established.\n\nRecommendation: engage with it. Cite it with care, and don't repeat the 24.1% without noting the label-noise caveat.","headline":"A genuinely new attack for paraphrase models with a real fragility signal, but the drop magnitudes are not fully nailed down because label validity is only spot-checked.","tokens_in":9924,"tokens_out":2492,"would_cite":true,"duration_ms":27330,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Paraphrase-identification models are fragile when words shared by both sentences change, and adversarial training repairs much of the damage.","keywords":["paraphrase identification","adversarial examples","shared words","robustness","BERT","adversarial training","beam search","question pairs"],"falsifier":"Collect a large sample of modified examples (thousands, not hundreds) on which the target model flips, and have expert annotators judge the original label under a strict protocol; if the disagreement rate climbs well above the reported 12-21%, the measured accuracy drop partly reflects label-changing edits rather than model fragility, while disagreement near that level confirms the vulnerability.","tokens_in":8947,"feed_emoji":"💬","tokens_out":8028,"duration_ms":80107,"temperature":0.7,"pith_summary":"Paraphrase identification is the task of deciding whether two sentences mean the same thing. The paper claims that modern models, including BERT, are fragile when words shared by both sentences are changed, even when the paraphrase relation is preserved. On modified examples built from Quora Question Pairs, BERT's accuracy falls from 94.3% to 24.1%, and on MRPC from 97.0% to 23.8%. The same construction also makes negative pairs look positive by injecting shared content words into unrelated templates. Training on a small share of such modified examples restores much of the lost accuracy, suggesting the failure is correctable rather than intrinsic.","feed_headline":"Shared-word edits crash paraphrase models from 94% to 24%","feed_subtitle":"Meaning-preserving word swaps flip BERT's verdicts; training on the edits restores most of the lost accuracy.","key_machinery":"The load-bearing object is the shared-word modification: a pair of positions, one in each sentence, is replaced by a word that appears in both sentences. For positive examples, a shared word in both sentences is replaced by one new shared word, preserving the paraphrase; for negative examples, new shared words are introduced into two unrelated sentences, which still leaves them non-paraphrases. A constrained beam search chooses which positions and which substitution words to use, and BERT's masked language model both supplies candidates and scores their compatibility with both masked contexts. The mechanism isolates the model's sensitivity to lexical overlap from its ability to judge semantic equivalence.","core_discovery":"The central claim is that paraphrase-identification models rely on shared vocabulary more than on meaning. For positive pairs, replacing a word that appears in both sentences with the same new word keeps the pair a paraphrase (e.g., switching \"purpose of life\" to \"measure of value\") but makes the target model predict negative. For negative pairs, replacing a noun, verb, or adjective in each sentence so that the two sentences acquire new shared content words flips the prediction to positive. The authors generate such examples with a two-stage beam search scored by the target model's loss, using BERT's masked language model to propose substitutions that fit both contexts. Human annotation of a 100-example sample from each dataset indicates the generated pairs mostly keep their original labels (79-88% agreement), so the accuracy collapse is not simply the result of changing the label.","pith_inferences":["The same generation recipe could be applied to other sentence-pair tasks (duplicate detection, retrieval, natural language inference) to measure how much of their performance is lexical-overlap bias; the paper does not test those tasks.","The recovery pattern in the tables suggests adversarial training is more effective on negative examples (BERT QQP negative accuracy rises from 14.8% to 79.0%) than on positive examples (33.4% to 53.0%), so future defenses may need to spend extra effort on positive shared-word replacements.","If the apparent fragility is real, then paraphrase benchmarks with naturally varying word overlap may overstate model quality; an explicit diagnostic controlling overlap would make this bias measurable.","The small human-validation sample leaves room for a stricter test; an annotator disagreement rate much larger than the reported 12-21% would require revising the size of the claimed accuracy drop."],"forward_implications":["On adversarially modified examples, all three tested architectures—BiMPM, DIIN, and BERT—drop to near or below chance on at least one dataset (e.g., BiMPM on MRPC falls to 1.7% overall).","The failure is bidirectional: positive examples become false negatives and negative examples become false positives, so both paraphrase and non-paraphrase decisions are affected.","Adversarial training with about 10% modified examples per batch improves accuracy on modified examples substantially (BERT on QQP from 24.1% to 66.0%; on MRPC from 23.8% to 87.0%) while leaving accuracy on the original test sets nearly unchanged.","Because the perturbations are not semantic-preserving, they reveal a distinct vulnerability from PAWS word-order adversaries, which only tested negative examples with identical bags of words."],"supporting_citations":[{"why":"Supplies the Quora Question Pairs dataset, the main source of original examples used for modification and evaluation.","marker":"Iyer et al., 2017"},{"why":"Supplies the MRPC corpus, the second benchmark on which the same modifications and adversarial training are tested.","marker":"Dolan and Brockett, 2005"},{"why":"Provides BERT as a target model and the pre-trained masked language model used to generate context-compatible substitution words.","marker":"Devlin et al., 2018"},{"why":"Provides BiMPM, one of the three target models, and the QQP data partition followed in the experiments.","marker":"Wang et al., 2017"},{"why":"Provides DIIN, the second target architecture whose performance collapses on modified examples.","marker":"Gong et al., 2017"},{"why":"Introduces PAWS, the word-order paraphrase adversary this work contrasts with and extends to shared-word modifications.","marker":"Zhang et al., 2019b"},{"why":"Supplies the two-stage position-selection and word-selection framework that the beam search adapts.","marker":"Yang et al., 2018"}],"fun_headline_variants":["Shared word swaps fool paraphrase detectors","Paraphrase models fooled by shared-word edits","Word sharing tricks paraphrase AI","Paraphrase ID fails on clever word swaps","Shared-word tweaks flip paraphrase predictions"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole accuracy-drop measurement rests on the assumption that the generated sentence pairs really do keep their original paraphrase labels; the paper only spot-checks this on 100 examples per dataset, where human raters agree with the original label only 79% to 88% of the time.","fun_headline_variants_meta":{"raw":{"variants":["Shared word swaps fool paraphrase detectors","Paraphrase models fooled by shared-word edits","Word sharing tricks paraphrase AI","Paraphrase ID fails on clever word swaps","Shared-word tweaks flip paraphrase predictions"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000288,"raw_usage":{"total_tokens":1649,"prompt_tokens":861,"completion_tokens":788,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":477,"completion_tokens_details":{"reasoning_tokens":723}},"tokens_in":477,"tokens_out":788,"duration_ms":7910,"temperature":1.0,"reasoning_tokens":723,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T04:45:39.772929+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Collect a large sample of modified examples (thousands, not hundreds) on which the target model flips, and have expert annotators judge the original label under a strict protocol; if the disagreement rate climbs well above the reported 12-21%, the measured accuracy drop partly reflects label-changing edits rather than model fragility, while disagreement near that level confirms the vulnerability.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the Quora Question Pairs dataset, the main source of original examples used for modification and evaluation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides BiMPM, one of the three target models, and the QQP data partition followed in the experiments."}],"review_version":1}