Pith. sign in

REVIEW 2 major objections 4 minor 1 cited by

Exploring the Robustness of NMT Systems to Nonsensical Inputs

T0 review · 2 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper shows that modern machine translation systems often output the same translation after many source words are replaced with nonsense, and argues this reveals a failure to capture semantics.

desk verdict A genuinely useful adversarial-attack method for NMT, with a headline semantic claim that the experiments do not actually support. read the letter →

arxiv 1908.01165 v3 pith:ZXNKIN6P submitted 2019-08-03 cs.LG cs.CLcs.CRstat.ML

Akshay Chaturvedi , Abijith KP , Utpal Garain This is my paper · ORCID
classification cs.LGcs.CLcs.CRstat.ML
keywords adversarialattackneuralmachinetranslationinvariance-basedsemanticunderstandingTransformerBLSTMBLEU-basedmetricrobustness
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Neural machine translation is supposed to change its output when the source sentence stops meaning what it originally did. This paper asks whether modern systems actually do that, and reports that they often do not: a purpose-built attack that replaces several words per sentence keeps the translation unchanged for 67.8% and 70.8% of English-German and English-French test sentences with a BLSTM encoder-decoder, and for 61.6% and 64.8% with a Transformer. The proposed recipe, Min-Grad + Soft-Att, selects the least sensitive word positions by gradient norm and then picks replacement words by gradient descent over a soft attention distribution on the pruned vocabulary. Because the rewritten source sentences are visibly nonsensical, the identical translations are read as evidence that the models rely on surface patterns rather than meaning. The paper also proposes a BLEU-based composite score, e(M), to evaluate invariance attacks in a way that success rate alone cannot.

What carries the argument

The carrying mechanism is the two-part replacement loop. Min-Grad chooses the position to attack: it computes the negative log-likelihood loss of the model on the current input and picks the position whose input-embedding gradient has the smallest ℓ2 norm, on the reasoning that the model is least sensitive there. Soft-Att then chooses the replacement word: it initializes a uniform probability distribution over the pruned source vocabulary, runs gradient descent on that distribution to keep the translation loss low, and picks the word with maximum probability once the distribution concentrates. The two steps are iterated over sweeps with a loss threshold that caps each replacement at the original loss. Around this loop, the paper wraps a BLEU-based composite score e(M) that rewards high BLEU between the attacked model's two translations and low BLEU between the translations produced by other models, as a check that the source meaning truly changed.

What would settle it

Ask native speakers of English to judge a random sample of the paper's original/adversarial sentence pairs for meaning difference; if many pairs are judged synonymous, or the identical translations turn out to be fully explained by missing gender or tense distinctions in the target language, the claim that NMT systems ignore source semantics would be falsified.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that word-level invariance in NMT is broad and easy to produce. The success rate of the proposed Min-Grad + Soft-Att attack — the percentage of adversarial sentences assigned exactly the same translation as the original — reaches 67.8% for en-de and 70.8% for en-fr on the BLSTM model, and 61.6% for en-de and 64.8% for en-fr on the Transformer, with normalized replacement counts around 0.4–0.6 of sentence length. The authors interpret this as showing that state-of-the-art NMT systems cannot capture the semantics of the source language, because the models stay perfectly calm while the input's meaning is destroyed. They further argue, through the proposed e(M) metric and cross-model BLEU checks, that these unchanged translations are not explained by target-language properties such as missing gender or tense distinctions, and that the adversarial pairs do not transfer across models.

Load-bearing premise

The central conclusion rests on the assumption that the generated original/adversarial pairs really do differ in meaning; the paper checks this by eye and with a BLEU-based proxy whose reliability is not independently established.

Editorial extensions

If this is right

  • If the results hold, high BLEU scores on clean test sets say little about whether an NMT system understood the input, since wholesale meaning destruction can leave the output untouched.
  • The attack produces model-specific adversarial pairs: sentences that fool one model do not generally fool the other architecture, so defenses cannot be shared across models without retesting.
  • Min-Grad's improvement over random position selection indicates that gradient norms with respect to input embeddings are a usable, cheap signal for where an NMT model is semantically indifferent.
  • Soft-Att's large margin over HotFlip suggests that first-order directional word flips are weaker than continuous soft-attention search over the vocabulary for producing invariance.
  • The e(M) metric gives future work a way to compare invariance attacks while controlling for target-language coincidences, rather than relying on success rate alone.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An extension the paper leaves implicit: Min-Grad's position ranking can be reused as an interpretability probe to map where a model's semantic sensitivity is lowest, not just as an attack.
  • A testable follow-up the paper does not run: feed the surviving adversarial pairs back into training with a loss that penalizes unchanged translations, and measure whether robustness to this attack improves without hurting BLEU.
  • The e(M) score is defined for translation, but the same logic applies to any conditional generator; one could evaluate invariance attacks on summarization or dialogue with the same composite of source-change and output-stability measures.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper studies whether neural machine translation (NMT) systems can be made to output the same translation after multiple words in the source sentence are replaced. It proposes a white-box attack, Min-Grad + Soft-Att, in which Min-Grad selects replacement positions by minimizing the gradient norm of the loss with respect to input embeddings, and Soft-Att chooses replacement words by soft-attention over a pruned vocabulary. Experiments on English-German and English-French with BLSTM-based attention and Transformer models report that the proposed attack preserves the original translation for 61%–71% of the tested 500 sentences, outperforming HotFlip and random baselines. The paper also introduces a BLEU-based composite metric, e(M), intended to measure the semantic change induced by the attack, and concludes that state-of-the-art NMT systems are unable to capture the semantics of the source language.

Significance. If the conclusions were fully supported, the paper would provide a striking demonstration that large, semantically destructive edits to source sentences can leave NMT translations unchanged, with a proposed attack that clearly dominates HotFlip in success rate and replacement count. The algorithmic specification is detailed and reproducible in principle, the evaluation covers two architectures and two language pairs, and the paper is commendably clear about the target-language ambiguity issue in Section IV-B and attempts to address it through cross-model BLEU scores. However, the headline semantic claim goes beyond what the reported evidence establishes: the BLEU-based proxy used to infer meaning change is never validated against human judgments, and for inputs the paper itself calls 'nonsensical,' the notion of source-language semantics is ill-defined. The paper is better viewed as demonstrating distributional brittleness and invariance of NMT models to out-of-distribution inputs, which is a valuable finding but not the same as showing an inability to capture semantics.

major comments (2)
  1. [Section IV-B, Tables V-VII, Eq. (3)] The abstract and Section V conclude that NMT systems 'are unable to capture the semantics of the source language.' This conclusion is not supported by the reported metrics. The attack explicitly optimizes to keep the model's loss low while replacing words, so a high success rate in Table IV is expected for a successful optimizer and does not by itself demonstrate a change in meaning. The only evidence offered for meaning change is the low BLEU between the original and adversarial source sentences and between their translations by other NMT models (Tables V and VI), aggregated in Eq. (3). This BLEU-based proxy is never validated against human judgments, and it has a clear failure mode: nonsensical, out-of-distribution inputs can make any NMT model produce low-BLEU translations even when the semantic content (insofar as it exists) is similar or when the difference is purely distributional. The paper's own acknowledgment in Section IV-B that same translations may arise from target-language properties rather than model deficiency is not resolved by the cross-model BLEU check, because low cross-model BLEU is also consistent with model brittleness. I recommend adding a human evaluation of meaning change on a sample of successful attacks, or rephrasing the central claim as one about invariance/brittleness rather than semantic failure.
  2. [Section IV-A, Table IV] The paper states that Min-Grad gives a 'significant improvement' over random traversal and that the proposed method outperforms HotFlip by a 'significant margin,' but Table IV reports only point estimates on 500 test sentences, with no confidence intervals, standard errors, or statistical tests. This matters for several comparisons that are not obviously robust to sampling variation: for Transformer en-fr, random+Soft-Att achieves 59.0% success and Min-Grad+Soft-Att 64.8%, a difference that is borderline under a two-proportion test; also, Min-Grad+HotFlip versus random+HotFlip for Transformer en-fr is 44.0% versus 40.6%, which is well within noise. Adding 95% confidence intervals (e.g., Wilson intervals or bootstrap) or a paired significance test over the 500 sentences would substantiate the claimed advantage of the Min-Grad traversal, which is a stated contribution in Section I-B, item 2. Without this, the 'significant margin' language in the abstract is not fully supported.
minor comments (4)
  1. [Eq. (3)] The displayed formula for e(M) is ambiguous as typeset; the denominator n+1 must apply to the entire sum, not only to the final term. Please add parentheses or a clear fraction so the definition is unambiguous.
  2. [Algorithm 3] In Algorithm 3, the variable 'loss' returned by Soft-Att and the variable 'l' denoting the loss of the current sentence are easy to confuse; consider renaming them to 'candidate_loss' and 'current_loss' for clarity.
  3. [Section II-A] The phrase 'V is the set of proper words in the source language' is unclear; it would be clearer to say that V is the set of source-language words that appear in the shared vocabulary and in the source training corpus.
  4. [Table IV] The caption of Table IV does not state the number of test sentences used for the reported percentages; adding 'n = 500' and confidence intervals would improve interpretability, as noted in Major Comment 2.

Circularity Check

1 steps flagged · score 3.0 of 10

Semantic-failure conclusion restates the self-defined BLEU metric; the attack comparison itself is not circular.

  1. self definitional [Section IV-B, Eq. (3) and the paragraph after Table VII]
    "To summarize, an effective invariance based attack is expected to give pair of original/adversarial source sentences whose corresponding translations by the model under attack have high BLEU scores and whose corresponding translations by the other NMT models have low BLEU scores. ... The e(M ) values demonstrate that the state-of-the-art NMT systems are unable to capture the semantics of the adversarial examples generated by the the proposed method, Min-Grad+Soft-Att."

    Eq. (3) defines e(M) as a linear combination in which low values require exactly high BLEU for the model under attack (small 100-b_l1) and low BLEU for all other models (small b_li). The paper first labels this condition "an effective invariance based attack" and then treats low e(M) as demonstrating that the systems cannot capture semantics. The conclusion thus restates the metric's definition rather than testing it: the semantic-failure claim holds by construction only if low cross-model BLEU is accepted as equivalent to changed meaning, an identification the paper never validates against human judgments or any external ground truth. The attack comparison itself remains independent, so this is a partial, interpretation-level circularity.

full rationale

The paper's substantive contribution is an adversarial search procedure: Min-Grad selects positions with minimal gradient norm, and Soft-Att chooses replacement words by optimizing an attention distribution over the pruned vocabulary to minimize NLL against the original translation. This is not a derivation from a theory, and the reported success rates are the outcome of a search, not a fitted prediction. The comparison with HotFlip and random baselines is external and provides independent evidence that the proposed traversal and replacement choices are more effective at preserving the target translation. The one place where the argument becomes circular is the semantic interpretation in Section IV-B: the composite e(M) is defined so that low values require high BLEU for the attacked model and low BLEU for all other models, and this exact condition is then called an effective invariance-based attack and read as demonstrating that NMT systems are unable to capture semantics. Without external validation that low cross-model BLEU equals changed meaning (e.g., human judgments), the semantic-failure conclusion is a restatement of the metric's definition. This does not undermine the attack comparison, but it does mean the abstract's strongest claim is only as strong as the unvalidated operationalization. No self-citation or uniqueness-import patterns are present.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The ledger captures the hand-set attack hyperparameters and the domain assumptions needed to interpret the experimental results. There are no invented entities; the paper builds on standard NMT training and BLEU.

free parameters (4)
  • nsweep = 5
    Number of sweeps over the source sentence; chosen by hand, no sensitivity analysis reported.
  • maxiter = 1000
    Maximum gradient descent iterations for Soft-Att word selection.
  • maxprob = 0.9
    Probability threshold that triggers early stopping in Soft-Att.
  • niter = 10
    Number of consecutive iterations above maxprob required to stop word search.
assumptions (4)
  • domain assumption The trained BLSTM and Transformer models are representative of state-of-the-art NMT systems.
    Claims about NMT systems generalize from two architectures and two language pairs (Section III).
  • domain assumption BLEU score is a valid proxy for semantic similarity between translations.
    The proposed e(M) metric assumes high BLEU among translations indicates unchanged meaning, and low BLEU indicates changed meaning (Section IV-B).
  • ad hoc to paper Same translation output for different source sentences implies a semantic deficiency in the model.
    The paper concludes NMT systems cannot capture source semantics from same-translation examples; this inference is not proven (Section V).
  • domain assumption Subword-level tokenization with BPE is an appropriate unit for word replacement.
    Replacements operate at subword level within words, producing strings like 'undsubmitted covered'; conclusion assumes this still constitutes meaningful language change (Section II-A).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Exploring the Robustness of NMT Systems to Nonsensical Inputs." pith.science (2026). https://pith.science/paper/ZXNKIN6P

@misc{pith2026190801165,
  author       = {Pith},
  title        = {Pith review of: Exploring the Robustness of NMT Systems to Nonsensical Inputs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZXNKIN6P}},
  note         = {Machine review of arXiv:1908.01165}
}
read the original abstract

Neural machine translation (NMT) systems have been shown to give undesirable translation when a small change is made in the source sentence. In this paper, we study the behaviour of NMT systems when multiple changes are made to the source sentence. In particular, we ask the following question "Is it possible for an NMT system to predict same translation even when multiple words in the source sentence have been replaced?". To this end, we propose a soft-attention based technique to make the aforementioned word replacements. The experiments are conducted on two language pairs: English-German (en-de) and English-French (en-fr) and two state-of-the-art NMT systems: BLSTM-based encoder-decoder with attention and Transformer. The proposed soft-attention based technique achieves high success rate and outperforms existing methods like HotFlip by a significant margin for all the conducted experiments. The results demonstrate that state-of-the-art NMT systems are unable to capture the semantics of the source language. The proposed soft-attention based technique is an invariance-based adversarial attack on NMT systems. To better evaluate such attacks, we propose an alternate metric and argue its benefits in comparison with success rate.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Extend Adversarial Policy Against Neural Machine Translation via Unknown Token

    cs.CL 2025-01 conditional novelty 5.0 of 10

    DexChar adds UNK-mediated character perturbations and noisy discriminator augmentation to produce semantic-preserving adversarial examples for subword NMT.

Reference graph

Works this paper leans on

17 extracted references · 16 canonical work pages · cited by 1 Pith paper

  1. [1]

    Attention is all you need,

    A. V aswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones , A. N. Gomez, L. u. Kaiser, and I. Polosukhin, “Attention is all you need,” in Advances in Neural Information Processing Systems 30 , I. Guyon, U. V . Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Gar nett, Eds. Curran Associates, Inc., 2017, pp. 5998–6008

    work page 2017

  2. [2]

    BERT: P re- training of deep bidirectional transformers for language u nderstanding,

    J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: P re- training of deep bidirectional transformers for language u nderstanding,” in Proceedings of the 2019 Conference of the North American Cha pter of the Association for Computational Linguistics: Human La nguage Technologies, V olume 1 (Long and Short Papers) . Minneapolis, Minnesota: Association...

    2019

  3. [3]

    Pathologies of neural models make interpretation s difficult,

    S. Feng, E. Wallace, A. Grissom II, M. Iyyer, P . Rodriguez , and J. Boyd- Graber, “Pathologies of neural models make interpretation s difficult,” in Proceedings of the 2018 Conference on Empirical Methods in N atural Language Processing. Association for Computational Linguistics, 2018, pp. 3719–3728

    work page 2018

  4. [4]

    Hotflip: White-b ox adversarial examples for text classification,

    J. Ebrahimi, A. Rao, D. Lowd, and D. Dou, “Hotflip: White-b ox adversarial examples for text classification,” in Proceedings of the 56th Annual Meeting of the Association for Computational Li nguistics (V olume 2: Short Papers). Association for Computational Linguistics, 2018, pp. 31–36

    work page 2018

  5. [5]

    Synthetic and natural noise bot h break neural machine translation,

    Y . Belinkov and Y . Bisk, “Synthetic and natural noise bot h break neural machine translation,” in International Conference on Learning Representations, 2018

    work page 2018

  6. [6]

    On adversarial example s for character-level neural machine translation,

    J. Ebrahimi, D. Lowd, and D. Dou, “On adversarial example s for character-level neural machine translation,” in Proceedings of the 27th International Conference on Computational Linguistics . Association for Computational Linguistics, 2018, pp. 653–663

    work page 2018

  7. [7]

    Character-ba sed neural machine translation,

    M. R. Costa-juss` a and J. A. R. Fonollosa, “Character-ba sed neural machine translation,” in Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (V olume 2: S hort Papers). Berlin, Germany: Association for Computational Linguisti cs, Aug. 2016, pp. 357–361

    work page 2016

  8. [8]

    Towards robu st neural machine translation,

    Y . Cheng, Z. Tu, F. Meng, J. Zhai, and Y . Liu, “Towards robu st neural machine translation,” in Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (V olume 1: Long Papers). 8 Melbourne, Australia: Association for Computational Ling uistics, Jul. 2018, pp. 1756–1766

    work page 2018

Show all 17 references
  1. [9]

    Robust neural machi ne transla- tion with doubly adversarial inputs,

    Y . Cheng, L. Jiang, and W. Macherey, “Robust neural machi ne transla- tion with doubly adversarial inputs,” in Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics . Florence, Italy: Association for Computational Linguistics, Jul. 20 19, pp....

  2. [10]

    Effective approac hes to attention-based neural machine translation,

    T. Luong, H. Pham, and C. D. Manning, “Effective approac hes to attention-based neural machine translation,” in Proceedings of the 2015 Conference on Empirical Methods in Natural Language Proces sing. Association for Computational Linguistics, 2015, pp. 1412 –1421

    2015

  3. [11]

    Detecting egregious responses in ne ural sequence- to-sequence models,

    T. He and J. Glass, “Detecting egregious responses in ne ural sequence- to-sequence models,” in International Conference on Learning Repre- sentations, 2019

    2019

  4. [12]

    Robust neura l machine translation with joint textual and phonetic embedding,

    H. Liu, M. Ma, L. Huang, H. Xiong, and Z. He, “Robust neura l machine translation with joint textual and phonetic embedding,” in Proceedings of the 57th Annual Meeting of the Association for Computatio nal Linguistics. Florence, Italy: Association for Computational Linguist ic...

    2019

  5. [13]

    When and why are pre-trained word embeddings useful for neural ma chine translation?

    Y . Qi, D. Sachan, M. Felix, S. Padmanabhan, and G. Neubig , “When and why are pre-trained word embeddings useful for neural ma chine translation?” in Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Ling uistics: Human Lang...

    2018

  6. [14]

    Parameter sharing methods for multilin- gual self-attentional translation models,

    D. Sachan and G. Neubig, “Parameter sharing methods for multilin- gual self-attentional translation models,” in Proceedings of the Third Conference on Machine Translation . Association for Computational Linguistics, 2018

    2018

  7. [15]

    Neural machine tr anslation of rare words with subword units,

    R. Sennrich, B. Haddow, and A. Birch, “Neural machine tr anslation of rare words with subword units,” in Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics ( V olume 1: Long Papers) . Berlin, Germany: Association for Computational Linguisti...

    2016

  8. [16]

    Bleu: a m ethod for automatic evaluation of machine translation,

    K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu, “Bleu: a m ethod for automatic evaluation of machine translation,” in Proceedings of the 40th Annual Meeting of the Association for Computational Lingui stics, 2002

    2002

  9. [17]

    Towards deep learning models resistant to adversarial attacks,

    A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vlad u, “Towards deep learning models resistant to adversarial attacks,” in 6th Interna- tional Conference on Learning Representations, ICLR 2018, V ancouver , BC, Canada, April 30 - May 3, 2018, Conference Track Proceedi ngs, 2018

    2018

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.