{"id":"9cd8d0d5-78ca-4c9f-9697-5435221c1939","arxiv_id":"1908.01165","paper_version":3,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A soft-attention based adversarial attack keeps English-German and English-French translations unchanged while replacing many words in the source sentence.","lead":"This paper introduces a gradient-based attack that replaces many words in a source sentence while a neural machine translation model still outputs the same translation. The study shows that widely used BLSTM and Transformer NMT models can be fooled this way, and it proposes a new metric for measuring how nonsensical the altered sentence is.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Semantic-failure claim rests on unvalidated BLEU proxy: low cross-model BLEU may reflect out-of-distribution brittleness, not changed meaning; human validation is required.","rationale":"I read the paper as an empirical study claiming that a proposed attack can make NMT systems output the same translation for source sentences whose meaning has been changed. The attack optimization and success-rate numbers are plausible, and the comparison with HotFlip is clearly reported. The load-bearing step is the inference from 'same output under attack' to 'the model fails to capture semantics.' That inference depends on the attacked source sentences actually differing in meaning from the originals, which Section IV-B attempts to establish through a BLEU-based proxy involving other NMT models. The proxy is the weakest link: it is not validated against human semantic judgments, and low BLEU between translations of a nonsensical input by other models is exactly what one would expect if those models are also brittle on out-of-distribution text. This does not make the attack uninteresting, but it means the headline semantic conclusion is overreached. The reader's CONDITIONAL verdict already captures this; my stress-test does not identify a different or more severe flaw. I therefore recommend no change to the verdict.","tokens_in":10933,"tokens_out":4806,"duration_ms":47831,"concrete_test":"Sample 100 successful attack pairs (roughly 25 per language-pair/model combination for BLSTM and Transformer) and have multiple fluent annotators independently rate whether the adversarial source expresses a different meaning than the original (or rate semantic similarity on a scale). Compute the correlation (e.g., Spearman) between these human ratings and the BLEU scores of the 'other NMT models' used in Eq. (3), and also compute e(M). If the cross-model BLEU does not track human semantic-difference judgments--or if known meaning-preserving paraphrase pairs also yield low cross-model BLEU--then the BLEU-based metric does not establish the semantic-failure conclusion, and the central claim must be weakened.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The experimental finding that Min-Grad + Soft-Att often makes an NMT model reproduce the original translation is credible, but the paper's headline conclusion--'state-of-the-art NMT systems are unable to capture the semantics of the source language'--does not follow from the reported metrics. Success rate (Section IV-A) counts cases where the model's output is unchanged; since the attack explicitly optimizes for that outcome, it cannot by itself show that the changed source has different meaning. The paper tries to close this gap in Section IV-B with a BLEU-based composite e(M) (Eq. 3), using low BLEU between translations of original and adversarial sentences by other NMT models as evidence that the source meanings differ. This proxy is never validated against human judgments, and it has a specific failure mode: nonsensical, out-of-distribution inputs can make any NMT model produce low-BLEU translations even when the semantic content (insofar as it exists) is similar or when the difference is purely distributional. Low cross-model BLEU is therefore consistent with the alternative explanation that the attack merely finds inputs on which the target model is invariant while other models are also brittle, not inputs whose meaning a fluent speaker would judge as changed. The title and conclusion themselves call the inputs 'nonsensical'; for nonsensical strings, 'semantics of the source language' is ill-defined, so the strongest semantic claim is unsupported without a validated semantic-similarity check.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies whether neural machine translation (NMT) systems can be made to output the same translation after multiple words in the source sentence are replaced. It proposes a white-box attack, Min-Grad + Soft-Att, in which Min-Grad selects replacement positions by minimizing the gradient norm of the loss with respect to input embeddings, and Soft-Att chooses replacement words by soft-attention over a pruned vocabulary. Experiments on English-German and English-French with BLSTM-based attention and Transformer models report that the proposed attack preserves the original translation for 61%–71% of the tested 500 sentences, outperforming HotFlip and random baselines. The paper also introduces a BLEU-based composite metric, e(M), intended to measure the semantic change induced by the attack, and concludes that state-of-the-art NMT systems are unable to capture the semantics of the source language.","tokens_in":11245,"tokens_out":6003,"duration_ms":61432,"significance":"If the conclusions were fully supported, the paper would provide a striking demonstration that large, semantically destructive edits to source sentences can leave NMT translations unchanged, with a proposed attack that clearly dominates HotFlip in success rate and replacement count. The algorithmic specification is detailed and reproducible in principle, the evaluation covers two architectures and two language pairs, and the paper is commendably clear about the target-language ambiguity issue in Section IV-B and attempts to address it through cross-model BLEU scores. However, the headline semantic claim goes beyond what the reported evidence establishes: the BLEU-based proxy used to infer meaning change is never validated against human judgments, and for inputs the paper itself calls 'nonsensical,' the notion of source-language semantics is ill-defined. The paper is better viewed as demonstrating distributional brittleness and invariance of NMT models to out-of-distribution inputs, which is a valuable finding but not the same as showing an inability to capture semantics.","major_comments":[{"comment":"The abstract and Section V conclude that NMT systems 'are unable to capture the semantics of the source language.' This conclusion is not supported by the reported metrics. The attack explicitly optimizes to keep the model's loss low while replacing words, so a high success rate in Table IV is expected for a successful optimizer and does not by itself demonstrate a change in meaning. The only evidence offered for meaning change is the low BLEU between the original and adversarial source sentences and between their translations by other NMT models (Tables V and VI), aggregated in Eq. (3). This BLEU-based proxy is never validated against human judgments, and it has a clear failure mode: nonsensical, out-of-distribution inputs can make any NMT model produce low-BLEU translations even when the semantic content (insofar as it exists) is similar or when the difference is purely distributional. The paper's own acknowledgment in Section IV-B that same translations may arise from target-language properties rather than model deficiency is not resolved by the cross-model BLEU check, because low cross-model BLEU is also consistent with model brittleness. I recommend adding a human evaluation of meaning change on a sample of successful attacks, or rephrasing the central claim as one about invariance/brittleness rather than semantic failure.","section":"Section IV-B, Tables V-VII, Eq. (3)"},{"comment":"The paper states that Min-Grad gives a 'significant improvement' over random traversal and that the proposed method outperforms HotFlip by a 'significant margin,' but Table IV reports only point estimates on 500 test sentences, with no confidence intervals, standard errors, or statistical tests. This matters for several comparisons that are not obviously robust to sampling variation: for Transformer en-fr, random+Soft-Att achieves 59.0% success and Min-Grad+Soft-Att 64.8%, a difference that is borderline under a two-proportion test; also, Min-Grad+HotFlip versus random+HotFlip for Transformer en-fr is 44.0% versus 40.6%, which is well within noise. Adding 95% confidence intervals (e.g., Wilson intervals or bootstrap) or a paired significance test over the 500 sentences would substantiate the claimed advantage of the Min-Grad traversal, which is a stated contribution in Section I-B, item 2. Without this, the 'significant margin' language in the abstract is not fully supported.","section":"Section IV-A, Table IV"}],"minor_comments":[{"comment":"The displayed formula for e(M) is ambiguous as typeset; the denominator n+1 must apply to the entire sum, not only to the final term. Please add parentheses or a clear fraction so the definition is unambiguous.","section":"Eq. (3)"},{"comment":"In Algorithm 3, the variable 'loss' returned by Soft-Att and the variable 'l' denoting the loss of the current sentence are easy to confuse; consider renaming them to 'candidate_loss' and 'current_loss' for clarity.","section":"Algorithm 3"},{"comment":"The phrase 'V is the set of proper words in the source language' is unclear; it would be clearer to say that V is the set of source-language words that appear in the shared vocabulary and in the source training corpus.","section":"Section II-A"},{"comment":"The caption of Table IV does not state the number of test sentences used for the reported percentages; adding 'n = 500' and confidence intervals would improve interpretability, as noted in Major Comment 2.","section":"Table IV"}],"recommendation":"major_revision","confidential_remarks":"The core experimental finding—that the proposed attack often preserves the translation under substantial word replacements, and that it outperforms HotFlip—appears credible and reproducible from the description. The main risk is overinterpretation of the BLEU-based metric as evidence of semantic change; if the authors can add human validation or carefully soften the semantic claim, the paper would be a solid empirical contribution. The borderline statistical significance on some Min-Grad versus random comparisons should also be addressed with confidence intervals or tests. This is fixable within the manuscript's scope, hence major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a useful, mostly solid empirical paper about a real phenomenon, but it oversells the conclusion. The core result — that you can replace multiple words (around 40–60% of the source sentence) and get many NMT models to output the same translation, with Min-Grad + Soft-Att beating HotFlip — looks credible and is worth having in the literature. The method description is clear, the baselines are fair, and the tables are internally consistent.\n\nWhat's new: the invariance-based attack framing, Min-Grad position selection (minimum norm of gradient wrt input embedding), soft-attention word replacement, and a BLEU-based composite metric. These are useful building blocks. The comparison against HotFlip and random baselines is reasonable, and the observation that Transformer is more robust than BLSTM (at least to this attack) is plausible.\n\nWhere I have problems: the headline claim that the results show NMT systems 'are unable to capture the semantics of the source language' does not follow from the metrics. Success rate only measures whether the model's output is unchanged; since the attack optimizes for exactly that, it is expected. The paper attempts to close the gap with cross-model BLEU scores — the idea being that if other models translate the original and adversarial sentences differently, the meanings must have changed. That proxy is never validated against human judgments. Low cross-model BLEU is exactly what you would expect from out-of-distribution nonsensical inputs even if the 'meaning' (insofar as one exists) is similar. So the semantic claim is overreached. The authors themselves call the inputs nonsensical; for such strings, the phrase 'semantics of the source language' is ill-defined without human annotation.\n\nSmaller issues: no confidence intervals or significance tests on the success rates, and no code release despite the promise in the text. The formula for the composite metric in Eq. (3) is also under-specified (missing parentheses) — minor but worth fixing.\n\nWho this is for: people working on adversarial robustness of NLP/MT. It deserves a serious referee — the attack method and the empirical comparison are worth engaging with, and the semantic question is important even if the current evidence doesn't settle it. I'd recommend sending it to review, with the expectation that the authors either soften the semantic conclusion or add a proper human study.","headline":"A genuinely useful adversarial-attack method for NMT, with a headline semantic claim that the experiments do not actually support.","tokens_in":11706,"tokens_out":2292,"would_cite":true,"duration_ms":21785,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper shows that modern machine translation systems often output the same translation after many source words are replaced with nonsense, and argues this reveals a failure to capture semantics.","keywords":["adversarial attack","neural machine translation","invariance-based attack","semantic understanding","Transformer","BLSTM","BLEU-based metric","robustness"],"falsifier":"Ask native speakers of English to judge a random sample of the paper's original/adversarial sentence pairs for meaning difference; if many pairs are judged synonymous, or the identical translations turn out to be fully explained by missing gender or tense distinctions in the target language, the claim that NMT systems ignore source semantics would be falsified.","tokens_in":10770,"feed_emoji":"🔁","tokens_out":9716,"duration_ms":87512,"temperature":0.7,"pith_summary":"Neural machine translation is supposed to change its output when the source sentence stops meaning what it originally did. This paper asks whether modern systems actually do that, and reports that they often do not: a purpose-built attack that replaces several words per sentence keeps the translation unchanged for 67.8% and 70.8% of English-German and English-French test sentences with a BLSTM encoder-decoder, and for 61.6% and 64.8% with a Transformer. The proposed recipe, Min-Grad + Soft-Att, selects the least sensitive word positions by gradient norm and then picks replacement words by gradient descent over a soft attention distribution on the pruned vocabulary. Because the rewritten source sentences are visibly nonsensical, the identical translations are read as evidence that the models rely on surface patterns rather than meaning. The paper also proposes a BLEU-based composite score, e(M), to evaluate invariance attacks in a way that success rate alone cannot.","feed_headline":"Machine translators keep the same output after nonsense word swaps","feed_subtitle":"Rewriting several source words still yields the same translation in about two-thirds of test sentences.","key_machinery":"The carrying mechanism is the two-part replacement loop. Min-Grad chooses the position to attack: it computes the negative log-likelihood loss of the model on the current input and picks the position whose input-embedding gradient has the smallest ℓ2 norm, on the reasoning that the model is least sensitive there. Soft-Att then chooses the replacement word: it initializes a uniform probability distribution over the pruned source vocabulary, runs gradient descent on that distribution to keep the translation loss low, and picks the word with maximum probability once the distribution concentrates. The two steps are iterated over sweeps with a loss threshold that caps each replacement at the original loss. Around this loop, the paper wraps a BLEU-based composite score e(M) that rewards high BLEU between the attacked model's two translations and low BLEU between the translations produced by other models, as a check that the source meaning truly changed.","core_discovery":"On the paper's own terms, the discovery is that word-level invariance in NMT is broad and easy to produce. The success rate of the proposed Min-Grad + Soft-Att attack — the percentage of adversarial sentences assigned exactly the same translation as the original — reaches 67.8% for en-de and 70.8% for en-fr on the BLSTM model, and 61.6% for en-de and 64.8% for en-fr on the Transformer, with normalized replacement counts around 0.4–0.6 of sentence length. The authors interpret this as showing that state-of-the-art NMT systems cannot capture the semantics of the source language, because the models stay perfectly calm while the input's meaning is destroyed. They further argue, through the proposed e(M) metric and cross-model BLEU checks, that these unchanged translations are not explained by target-language properties such as missing gender or tense distinctions, and that the adversarial pairs do not transfer across models.","pith_inferences":["An extension the paper leaves implicit: Min-Grad's position ranking can be reused as an interpretability probe to map where a model's semantic sensitivity is lowest, not just as an attack.","A testable follow-up the paper does not run: feed the surviving adversarial pairs back into training with a loss that penalizes unchanged translations, and measure whether robustness to this attack improves without hurting BLEU.","The e(M) score is defined for translation, but the same logic applies to any conditional generator; one could evaluate invariance attacks on summarization or dialogue with the same composite of source-change and output-stability measures."],"forward_implications":["If the results hold, high BLEU scores on clean test sets say little about whether an NMT system understood the input, since wholesale meaning destruction can leave the output untouched.","The attack produces model-specific adversarial pairs: sentences that fool one model do not generally fool the other architecture, so defenses cannot be shared across models without retesting.","Min-Grad's improvement over random position selection indicates that gradient norms with respect to input embeddings are a usable, cheap signal for where an NMT model is semantically indifferent.","Soft-Att's large margin over HotFlip suggests that first-order directional word flips are weaker than continuous soft-attention search over the vocabulary for producing invariance.","The e(M) metric gives future work a way to compare invariance attacks while controlling for target-language coincidences, rather than relying on success rate alone."],"supporting_citations":[{"why":"Supplies the Transformer architecture that the paper attacks.","marker":"[1]"},{"why":"Supplies the attention mechanism of the BLSTM encoder-decoder model under attack.","marker":"[10]"},{"why":"Introduces the HotFlip word/character replacement baseline that Soft-Att is compared against.","marker":"[4]"},{"why":"Extends HotFlip to neural machine translation and provides the prior adversarial method used as a baseline.","marker":"[6]"},{"why":"Earlier robustness work expecting similar translations for semantically similar inputs, the contrast that frames the paper's invariance question.","marker":"[8]"},{"why":"Companion robustness study whose expectation of translation invariance for similar inputs the paper inverts.","marker":"[9]"},{"why":"Provides the English-German and English-French parallel corpora used for training and testing.","marker":"[13]"},{"why":"Defines the byte-pair encoding that creates the subword vocabulary on which replacements operate.","marker":"[15]"},{"why":"Defines BLEU, used both for model quality and inside the paper's proposed e(M) attack metric.","marker":"[16]"}],"fun_headline_variants":["NMTs keep translations identical after nonsense word swaps","Soft-attack exposes NMTs ignore source semantics","Word chaos: NMTs output same translation regardless","Invariance attack: NMTs blind to meaning-destroying edits","NMT systems fail semantics test under soft-attack"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central conclusion rests on the assumption that the generated original/adversarial pairs really do differ in meaning; the paper checks this by eye and with a BLEU-based proxy whose reliability is not independently established.","fun_headline_variants_meta":{"raw":{"variants":["NMTs keep translations identical after nonsense word swaps","Soft-attack exposes NMTs ignore source semantics","Word chaos: NMTs output same translation regardless","Invariance attack: NMTs blind to meaning-destroying edits","NMT systems fail semantics test under soft-attack"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000204,"raw_usage":{"total_tokens":1390,"prompt_tokens":947,"completion_tokens":443,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":563,"completion_tokens_details":{"reasoning_tokens":363}},"tokens_in":563,"tokens_out":443,"duration_ms":5280,"temperature":1.0,"reasoning_tokens":363,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T15:21:07.490250+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Ask native speakers of English to judge a random sample of the paper's original/adversarial sentence pairs for meaning difference; if many pairs are judged synonymous, or the identical translations turn out to be fully explained by missing gender or tense distinctions in the target language, the claim that NMT systems ignore source semantics would be falsified.","supporting_citations":[{"cited_title":"Attention is all you need,","cited_arxiv_id":null,"evidence_quote":"Supplies the Transformer architecture that the paper attacks."},{"cited_title":"Effective approac hes to attention-based neural machine translation,","cited_arxiv_id":null,"evidence_quote":"Supplies the attention mechanism of the BLSTM encoder-decoder model under attack."},{"cited_title":"Hotflip: White-b ox adversarial examples for text classification,","cited_arxiv_id":null,"evidence_quote":"Introduces the HotFlip word/character replacement baseline that Soft-Att is compared against."},{"cited_title":"On adversarial example s for character-level neural machine translation,","cited_arxiv_id":null,"evidence_quote":"Extends HotFlip to neural machine translation and provides the prior adversarial method used as a baseline."},{"cited_title":"Towards robu st neural machine translation,","cited_arxiv_id":null,"evidence_quote":"Earlier robustness work expecting similar translations for semantically similar inputs, the contrast that frames the paper's invariance question."},{"cited_title":"Robust neural machi ne transla- tion with doubly adversarial inputs,","cited_arxiv_id":null,"evidence_quote":"Companion robustness study whose expectation of translation invariance for similar inputs the paper inverts."},{"cited_title":"When and why are pre-trained word embeddings useful for neural ma chine translation?","cited_arxiv_id":null,"evidence_quote":"Provides the English-German and English-French parallel corpora used for training and testing."},{"cited_title":"Neural machine tr anslation of rare words with subword units,","cited_arxiv_id":null,"evidence_quote":"Defines the byte-pair encoding that creates the subword vocabulary on which replacements operate."},{"cited_title":"Bleu: a m ethod for automatic evaluation of machine translation,","cited_arxiv_id":null,"evidence_quote":"Defines BLEU, used both for model quality and inside the paper's proposed e(M) attack metric."}],"review_version":1}