Hard but Robust, Easy but Sensitive: How Encoder and Decoder Perform in Neural Machine Translation

Tao Qin; Tianyu He; Xu Tan

REVIEW 3 major objections 7 minor 1 cited by

Hard but Robust, Easy but Sensitive: How Encoder and Decoder Perform in Neural Machine Translation

T0 review · 3 major / 7 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper argues that in Transformer-based neural machine translation the decoder performs the easier task yet is the more fragile half, because it leans on the immediately preceding target words as strong conditional cues.

desk verdict Decent empirical study: the decoder's noise sensitivity is well supported, but the 'easier task' claim is undercut by a confounded convergence-speed experiment; worth refereeing with revisions. read the letter →

arxiv 1908.06259 v1 pith:2FNUCXHI submitted 2019-08-17 cs.CL

Tianyu He , Xu Tan , Tao Qin This is my paper · ORCID

classification cs.CL

keywords neuralmachinetranslationencoder-decoderframeworkTransformerdecodersensitivityinputnoiseconditionallanguagemodelingnon-autoregressivedecodingteacherforcing

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper studies the two halves of a translation model—the encoder that reads the source sentence and the decoder that writes the target sentence—to see which one works harder and which one breaks more easily. Using the Transformer, a self-attention-based architecture, as the test bed, it argues that the decoder has the easier job: training curves and layer-scaling experiments suggest the encoder needs more capacity and more training time. Yet the decoder is the fragile half: corrupting the decoder's input during translation hurts quality more than corrupting the encoder's input. The explanation offered is that the decoder acts as a conditional language model, leaning heavily on the words it has just produced; that strong nearby context makes prediction easy, but it also makes the decoder depend on exactly those tokens and therefore sensitive to their corruption. Understanding this asymmetry matters for designing better translation architectures and for knowing where robustness work should focus.

What carries the argument

The carrying mechanism is the decoder's autoregressive conditioning: each target word is predicted from the previously generated words as well as the encoded source. The paper isolates this mechanism with three probes: perturbing encoder input and decoder input separately using random dropping, noising, and swapping; dropping a token at distance n before the predicted token and measuring the change in translation quality; and comparing autoregressive decoding with a non-autoregressive decoder that generates all target words in parallel. A second probe is the convergence-speed comparison, where one well-trained half is fixed and the other is randomly initialized to see which half learns faster. The sharp decay in accuracy when nearby tokens are dropped, together with the absence of that decay in the non-autoregressive model, is the evidence that preceding tokens are the load-bearing conditional information.

What would settle it

On a held-out test set, measure translation quality when the token immediately before the predicted word is removed and compare it with removal of a token several positions back, averaging over many sentence positions; if distant-token removal degrades translation as much as or more than nearby-token removal, the paper's explanation that strong nearby conditional information drives the decoder's ease and sensitivity is falsified.

Watch

Extended reading notes

Core claim

The paper's central discovery is that the encoder and decoder in NMT are not equally hard-working or equally robust: the encoder performs the harder task and is the more robust half, while the decoder performs an easier task and is more sensitive to input noise. Three experiments carry the claim. Increasing encoder depth produces larger BLEU gains than increasing decoder depth, and a randomly initialized encoder trained against a fixed well-trained decoder converges more slowly than the reverse, indicating the encoder's task is more difficult. When tokens are randomly dropped, noised, or swapped, perturbing the decoder input degrades translation more than perturbing the encoder input under teacher forcing, which feeds the correct previous words rather than the model's own guesses. Finally, dropping a token immediately before the predicted word hurts far more than dropping a distant token, and this nearby-token dependence disappears in non-autoregressive decoding, supporting the explanation that the decoder's strong conditioning on preceding tokens is what makes its task easy and its output sensitive.

Load-bearing premise

The conclusion that the decoder handles an easier task assumes that how fast a randomly initialized encoder learns with a fixed well-trained decoder, compared with the reverse arrangement, measures the intrinsic difficulty of each half's task; but the fixed half may already be doing much of the work, so the comparison may measure adaptation difficulty rather than intrinsic task difficulty.

Editorial extensions

If this is right

Adding layers to the encoder is likely to yield larger translation-quality gains than adding equal layers to the decoder, because the encoder's harder task needs more representation capacity.
Robustness work for NMT should protect the decoder's input as well as the encoder's, since token-level corruption of the decoder input degrades output more.
The immediate-prefix dependence found by the dropping experiments gives a concrete target for smoothing or regularizing the decoder's conditioning.
Non-autoregressive translation shifts sensitivity to the encoder side, so robustness conclusions about autoregressive decoders do not automatically transfer to parallel decoding.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Going beyond the paper: the same perturb-one-component diagnostic could be applied to other encoder-decoder tasks such as summarization or speech recognition, where the balance of difficulty and robustness may shift.
Going beyond the paper: the convergence-speed measure is a proxy for task difficulty; a fairer test would hold the fixed half's contribution constant by comparing against a randomly initialized frozen baseline, since a well-trained frozen decoder already supplies much of the target-side structure.
Going beyond the paper: the nearby-token result suggests a testable architectural recipe—throttling or making position-adaptive the decoder's attention to the immediate prefix may improve robustness to input noise with little loss in average quality.
Going beyond the paper: the noise operators used here are token-level; perturbing continuous encoder representations or attention patterns could reveal whether the encoder's robustness comes from representational redundancy or from the source sentence's structure.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

The headline is this: the paper gives a clean empirical answer to one question — the Transformer decoder is more sensitive to input noise than the encoder — and a weaker, partly confounded answer to the other, that the decoder's task is easier. The title wants both, but the evidence only fully supports the first.

What is new and good: the comparison of encoder vs decoder under teacher-forced noise is a sensible protocol. Because they feed ground-truth targets during inference, they separate sensitivity from error propagation, which is a real confound in earlier robustness work. The result is consistent across three noise types and four language-pair directions, so I trust it. The non-autoregressive control is a clever piece of evidence: it flips the sensitivity pattern, which supports the explanation that autoregressive decoding leans on previous tokens. The token-dropping analysis showing nearby tokens matter more is also clean and matches what we now know about language models.

Where the paper slips: Section 3.2's comparison of convergence speed freezes a well-trained decoder and retrains the encoder from scratch, and vice versa. That does not measure intrinsic task difficulty. A randomly initialized encoder must learn source representations that map into the frozen decoder's cross-attention subspace; a randomly initialized decoder can lean on the frozen self-attention language-model prior. The convergence gap is at least partly a compatibility cost, not a difficulty gap. The layer-scaling results in Section 3.1 have a similar ambiguity: needing more encoder depth could mean source encoding is harder, or it could mean encoders need more capacity to align with the decoder. So the 'easy but sensitive' framing is asymmetric: the sensitivity finding is solid, the ease finding is a hypothesis that needs a better test. Also, there are no error bars or multi-seed runs for any BLEU comparison. That is a real omission, though common in 2019 NMT work.

The citation pattern is fine; they engage with prior analysis work on attention, representations, and robustness. They don't overclaim much in the abstract, and the supplementary details are adequate.

Bottom line: this paper is for people working on robust translation or decoder architecture. It deserves a serious referee, but the referee should demand either a cleaned-up convergence experiment or a softening of the 'easier task' claim. If the authors can't fix the confound, the sensitivity result alone is worth publishing; the difficulty claim should be flagged as speculative.

Recommendation: accept for peer review with a request for major revision focused on Section 3.2.

Referee Report

3 major / 7 minor

Summary. This paper reports an empirical study of the encoder and decoder in Transformer-based neural machine translation. On IWSLT14 De↔En and Ro↔En and WMT17 Zh↔En, the authors manipulate (i) the number of encoder/decoder layers, (ii) the convergence speed when one module is frozen and the other is re-initialized, (iii) input perturbation during inference (dropping, noising, swapping), and (iv) dropping of nearby target tokens and comparisons with non-autoregressive NMT. They conclude that the decoder handles an easier task than the encoder, that the decoder is more sensitive to input noise, and that strong conditional information from preceding target tokens explains both observations.

Significance. The sensitivity finding in Section 4 is the strongest part of the paper: the decoder's larger BLEU drop under all three perturbation types is consistent across four language-pair directions and is partially controlled for error propagation by teacher forcing. The comparison with non-autoregressive NMT is a commendable attempt to test the proposed mechanism, and Section 5's nearby-token ablation gives direct evidence of the decoder's reliance on immediate context. However, the convergence-speed experiment in Section 3.2 is confounded by frozen-module alignment, so the 'easy' pillar of the central claim is not supported as stated. If the authors can repair or substantially soften the 'easier task' claim, the paper would be a useful empirical contribution; as it stands, the abstract and title overreach relative to the evidence.

major comments (3)

[Section 3.2, Figure 1c; Appendix A.1.1] The convergence-speed experiment does not cleanly measure intrinsic task difficulty because the protocol freezes a well-trained module while randomly initializing the other. When a randomly initialized encoder is paired with a frozen well-trained decoder, the encoder must not only learn source encoding but also learn to produce representations inside the subspace that the frozen decoder's cross-attention, feed-forward, and output projections expect; when the encoder is frozen and the decoder is re-initialized, the random decoder can exploit the frozen target-side self-attention and target embeddings. The measured convergence time therefore conflates task difficulty with cross-module compatibility. Since this experiment is the second operationalization of 'the decoder handles an easier task' (Abstract finding 1), the 'easy' pillar is not established by the current data. A cleaner test would, for example, freeze a randomly initialized counterpart rather than a well-trained one, compare both modules trained from matched random initializations under a fixed budget, or use a difficulty measure that does not depend on joint training history.
[Section 3.1, Figures 1a and 1b] The layer-count evidence is also not decisive for intrinsic task difficulty. A decoder layer contains an additional encoder-decoder attention sublayer, so increasing the layer count changes the parameter count and the optimization landscape differently for the encoder and decoder. Gains from depth may reflect where additional capacity is most useful for optimization rather than which task is intrinsically harder. In addition, no BLEU values or confidence intervals are reported for the layer variants, so the differences in Figures 1a and 1b cannot be assessed for statistical reliability. At minimum, the conclusion should be rephrased as 'under this architecture and protocol, adding layers to the encoder helped more than adding layers to the decoder,' not as a direct statement about task difficulty.
[Section 5, Figures 3a-3c and Table 1] The proposed explanation is explicitly introduced with 'We guess' and is not independently quantified. The nearby-token dropping experiment shows that the decoder's predictions degrade more when nearby target tokens are dropped, but it does not by itself show that the decoder's task is easy. The non-autoregressive comparison shows an interesting reversal, but the NAT model is a different architecture with much lower BLEU (De→En 24.06 vs 32.98 in Table 1), and the term 'decoder input' in NAT refers to a different quantity than in autoregressive decoding. The reversal may therefore reflect model quality or the different role of decoder input in NAT rather than the claimed mechanism. The paper should provide a matched-capacity NAT baseline or present the explanation as a hypothesis with clearly stated limitations.

minor comments (7)

[Abstract vs Section 1 and Section 5] The Abstract states that preceding tokens 'account for the two observations,' while the Introduction says they 'partially explain' the observations and Section 5 says 'We guess.' Align the wording across these locations to avoid overclaiming.
[Section 3.2, Figure 1c] Please report the x-axis units, the number of training steps or updates, and a quantitative convergence criterion (for example, time to reach a fixed BLEU threshold) instead of relying on visual inspection of the curves.
[Figures 1 and 2] No error bars or multiple-seed results accompany the BLEU differences; differences of 1-2 BLEU points can be within run-to-run variance for the datasets used. Reporting means with standard deviations or bootstrap confidence intervals would strengthen the empirical claims.
[Section 4, random noising] Please specify how the random noise is generated (sampling distribution, scale, and whether it is normalized relative to the embedding norm); without this information, the comparison between encoder and decoder noising is difficult to calibrate.
[Section 5, Figure 3a] Please clarify what 'drop the same number of tokens for each n' means concretely (per-sentence count vs corpus-wide count) and whether the dropped positions are chosen uniformly among all positions at distance n.
[Section 3.1, Figure 1b] For the fixed-total-12 condition, list the exact encoder-decoder layer splits (for example, 2-10, 4-8, 6-6, 8-4, 10-2) in the main text, since the text currently refers only to a figure.
[Appendix A.1.1 and A.1.2] There are minor typos: 'workpieces' should be 'wordpieces' or 'subword units,' and 'transaltion tasks' should be 'translation tasks.'

Circularity Check

0 steps flagged · score 0.0 of 10

Empirical benchmark study with no fitted-parameter derivation; no circular steps identified.

full rationale

This paper is an empirical comparison of encoder and decoder behavior in Transformer NMT, not a derivation whose conclusions are equivalent to its inputs. The difficulty claim in Section 3 is an operational interpretation of two independent measurements: layer-count gains and convergence speed. The convergence-speed protocol may be confounded by frozen-module alignment, but a confounded proxy is a validity threat, not circularity, and the paper reports the raw BLEU trajectories rather than renaming a fitted parameter as a prediction. The sensitivity claim in Section 4 is measured directly by perturbation experiments under teacher forcing. The Section 5 explanation that preceding tokens provide strong conditional information is tested by token-dropping and by contrasting autoregressive with non-autoregressive NMT; the non-autoregressive comparison shows the opposite sensitivity pattern, so the explanation is not simply a restatement of the autoregressive architecture. The paper cites several works co-authored by its own authors (e.g., Wu et al. 2018, Guo et al. 2018, He et al. 2018), but none is used as a load-bearing uniqueness theorem or as the sole justification for a central premise; they support experimental choices that are independently standard or externally checkable. No equation is equal to another by construction, and no measured quantity is relabeled as a prediction. I therefore find no significant circularity and assign score 0.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The central conclusions rest on several domain assumptions: BLEU measures quality, teacher forcing removes error propagation effects, and convergence speed with a frozen counterpart reflects task difficulty. The last is load-bearing and potentially confounded.

assumptions (4)

domain assumption BLEU score is an adequate measure of translation quality for comparing models.
All comparisons rely on BLEU differences without significance tests or confidence intervals.
domain assumption Teacher forcing during inference eliminates error propagation and isolates the model's sensitivity to input noise.
Section 4 applies noise to ground-truth decoder inputs rather than generated outputs, assuming the resulting accuracy drop reflects intrinsic sensitivity.
domain assumption The convergence speed of a randomly initialized encoder with a fixed pretrained decoder, and vice versa, reflects the intrinsic difficulty of the trained component's task.
Section 3.2 uses this protocol to conclude the decoder's task is easier; however, the frozen component may already perform much of the task, confounding the measure.
domain assumption The same perturbation rates applied to source tokens and target tokens make the encoder and decoder noise conditions comparable.
Section 4 perturbs encoder input and decoder input at the same rates, though the token distributions and lengths differ.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Hard but Robust, Easy but Sensitive: How Encoder and Decoder Perform in Neural Machine Translation." pith.science (2026). https://pith.science/paper/2FNUCXHI

@misc{pith2026190806259,
  author       = {Pith},
  title        = {Pith review of: Hard but Robust, Easy but Sensitive: How Encoder and Decoder Perform in Neural Machine Translation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2FNUCXHI}},
  note         = {Machine review of arXiv:1908.06259}
}

read the original abstract

Neural machine translation (NMT) typically adopts the encoder-decoder framework. A good understanding of the characteristics and functionalities of the encoder and decoder can help to explain the pros and cons of the framework, and design better models for NMT. In this work, we conduct an empirical study on the encoder and the decoder in NMT, taking Transformer as an example. We find that 1) the decoder handles an easier task than the encoder in NMT, 2) the decoder is more sensitive to the input noise than the encoder, and 3) the preceding words/tokens in the decoder provide strong conditional information, which accounts for the two observations above. We hope those observations can shed light on the characteristics of the encoder and decoder and inspire future research on NMT.

Figures

Figures reproduced from arXiv: 1908.06259 by the authors.

**Figure 1.** Experimental results for analyzing the training effort for the encoder and decoder. X-axes in Figure 1a and 1b indicate the number of layers for encoder and decoder respectively (Encoder-Decoder). Y-axes in Figure 1a and 1b indicate the delta BLEU to the 2-2 layer model (Figure 1a) and the 6-6 layer model (Figure 1b). (a) Random dropping (b) Random noising (c) Random swapping [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗

**Figure 2.** The accuracy drop (compared to the model with no input noise) when adding noise to the input tokens of the encoder and decoder. X-axis indicates the perturbation rate (%), i.e., the ratio of tokens being perturbed by noise. Discussion From the analyses in the last two subsections, it can be seen that increasing the layer of encoder brings more gains than increasing the layer of the decoder, and encoder needs more tr… view at source ↗

**Figure 3.** Further analyses on characteristics of the encoder and decoder. X-axis in Figure 3a indicates the distance between the dropped token and the current predicted token. X-axes in Figure 3b and 3c indicate the perturbation rate (%). See text for the detailed description. on certain input tokens for prediction as the decoder does, demonstrating that the encoder is more robust than the decoder. 5 Further Analysis Given t… view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: The sensitivity of different kinds of POS [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

SimLabel: Consistency-Guided OOD Detection with Pretrained Vision-Language Models
cs.CV 2025-01 conditional novelty 6.0 of 10

SimLabel improves zero-shot OOD detection by scoring images based on their consistency with a set of semantically similar class labels.

Reference graph

Works this paper leans on

36 extracted references · 14 canonical work pages · cited by 1 Pith paper

[1]

URL: " 'urlintro :=

ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...
[2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
[3]

Mikel Artetxe, Gorka Labaka, Eneko Agirre, and Kyunghyun Cho. 2018. Unsupervised neural machine translation. In International Conference on Learning Representations

work page 2018
[4]

Dzmitry Bahdanau, Philemon Brakel, Kelvin Xu, Anirudh Goyal, Ryan Lowe, Joelle Pineau, Aaron Courville, and Yoshua Bengio. 2017. An actor-critic algorithm for sequence prediction. ICLR

work page 2017
[5]

Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In International Conference on Learning Representations

2015
[6]

Yonatan Belinkov and Yonatan Bisk. 2018. Synthetic and natural noise both break neural machine translation. In International Conference on Learning Representations

work page 2018
[7]

Yonatan Belinkov, Llu \' s M \`a rquez, Hassan Sajjad, Nadir Durrani, Fahim Dalvi, and James Glass. 2017. Evaluating layers of representation in neural machine translation on part-of-speech and semantic tagging tasks. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers), volume 1, pages 1--10

work page 2017
[8]

Samy Bengio, Oriol Vinyals, Navdeep Jaitly, and Noam Shazeer. 2015. Scheduled sampling for sequence prediction with recurrent neural networks. In Advances in Neural Information Processing Systems, pages 1171--1179

2015

Show all 36 references

[9]

Denny Britz, Anna Goldie, Minh-Thang Luong, and Quoc Le. 2017. Massive exploration of neural machine translation architectures. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1442--1451

2017
[10]

Mostafa Dehghani, Stephan Gouws, Oriol Vinyals, Jakob Uszkoreit, and Lukasz Kaiser. 2019. Universal transformers. In International Conference on Learning Representations

2019
[11]

Yanzhuo Ding, Yang Liu, Huanbo Luan, and Maosong Sun. 2017. Visualizing and understanding neural machine translation. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 1150--1159

2017
[12]

Tobias Domhan. 2018. How much attention do you need? a granular analysis of neural machine translation architectures. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 1799--1808

2018
[13]

Sergey Edunov, Myle Ott, Michael Auli, David Grangier, et al. 2018. Classical structured prediction losses for sequence to sequence learning. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Te...

2018
[14]

Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, and Yann N Dauphin. 2017. Convolutional sequence to sequence learning. In Proceedings of the 34th International Conference on Machine Learning-Volume 70, pages 1243--1252. JMLR. org

2017
[15]

Hamidreza Ghader and Christof Monz. 2017. What does attention in neural machine translation pay attention to? In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers), volume 1, pages 30--39

2017
[16]

Jiatao Gu, James Bradbury, Caiming Xiong, Victor OK Li, and Richard Socher. 2017. Non-autoregressive neural machine translation. arXiv preprint arXiv:1711.02281

2017 arXiv
[17]

Li, and Richard Socher

Jiatao Gu, James Bradbury, Caiming Xiong, Victor O.K. Li, and Richard Socher. 2018. Non-autoregressive neural machine translation. In International Conference on Learning Representations

2018
[18]

Junliang Guo, Xu Tan, Di He, Tao Qin, Linli Xu, and Tie-Yan Liu. 2018. Non-autoregressive neural machine translation with enhanced decoder input. arXiv preprint arXiv:1812.09664

2018 arXiv
[19]

Hany Hassan, Anthony Aue, Chang Chen, Vishal Chowdhary, Jonathan Clark, Christian Federmann, Xuedong Huang, Marcin Junczys-Dowmunt, William Lewis, Mu Li, et al. 2018. Achieving human parity on automatic chinese to english news translation. arXiv preprint arXiv:1803.05567

2018 arXiv
[20]

Tianyu He, Xu Tan, Yingce Xia, Di He, Tao Qin, Zhibo Chen, and Tie-Yan Liu. 2018. Layer-wise coordination between encoder and decoder for neural machine translation. In Advances in Neural Information Processing Systems, pages 7944--7954

2018
[21]

Gomez, and Francois Chollet

Lukasz Kaiser, Aidan N. Gomez, and Francois Chollet. 2018. Depthwise separable convolutions for neural machine translation. In International Conference on Learning Representations

2018
[22]

Urvashi Khandelwal, He He, Peng Qi, and Dan Jurafsky. 2018. Sharp nearby, fuzzy far away: How neural language models use context. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 284--294

2018
[23]

Guillaume Lample, Alexis Conneau, Ludovic Denoyer, and Marc'Aurelio Ranzato. 2018. Unsupervised machine translation using monolingual corpora only. In International Conference on Learning Representations

2018
[24]

Thang Luong, Hieu Pham, and Christopher D Manning. 2015. Effective approaches to attention-based neural machine translation. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1412--1421

2015
[25]

Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting on association for computational linguistics, pages 311--318. Association for Computational Linguistics

2002
[26]

Marc'Aurelio Ranzato, Sumit Chopra, Michael Auli, and Wojciech Zaremba. 2015. Sequence level training with recurrent neural networks. arXiv preprint arXiv:1511.06732

2015 arXiv
[27]

Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 1715--1725

2016
[28]

Yanyao Shen, Xu Tan, Di He, Tao Qin, and Tie-Yan Liu. 2018. Dense information flow for neural machine translation. arXiv preprint arXiv:1806.00722

2018 arXiv
[29]

Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, and Tie-Yan Liu. 2019. Mass: Masked sequence to sequence pre-training for language generation. arXiv preprint arXiv:1905.02450

2019 arXiv
[30]

Kaitao Song, Tan Xu, Furong Peng, and Jianfeng Lu. 2018. Hybrid self-attention network for machine translation. arXiv preprint arXiv:1811.00253

2018 arXiv
[31]

Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. In Advances in neural information processing systems, pages 3104--3112

2014
[32]

Gongbo Tang, Rico Sennrich, and Joakim Nivre. 2018. An analysis of attention mechanisms: The case of word sense disambiguation in neural machine translation. In Proceedings of the Third Conference on Machine Translation, pages 26--35

2018
[33]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems, pages 5998--6008

2017
[34]

Ronald J Williams and David Zipser. 1989. A learning algorithm for continually running fully recurrent neural networks. Neural computation, 1(2):270--280

1989
[35]

Lijun Wu, Xu Tan, Di He, Fei Tian, Tao Qin, Jianhuang Lai, and Tie-Yan Liu. 2018. Beyond error propagation in neural machine translation: Characteristics of language also matter. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3...

2018
[36]

Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, et al. 2016. Google's neural machine translation system: Bridging the gap between human and machine translation. arXiv preprint arXiv:1609.08144

2016 arXiv

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

URL: " 'urlintro :=

ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...

[2] [2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

[3] [3]

Mikel Artetxe, Gorka Labaka, Eneko Agirre, and Kyunghyun Cho. 2018. Unsupervised neural machine translation. In International Conference on Learning Representations

work page 2018

[4] [4]

Dzmitry Bahdanau, Philemon Brakel, Kelvin Xu, Anirudh Goyal, Ryan Lowe, Joelle Pineau, Aaron Courville, and Yoshua Bengio. 2017. An actor-critic algorithm for sequence prediction. ICLR

work page 2017

[5] [5]

Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In International Conference on Learning Representations

2015

[6] [6]

Yonatan Belinkov and Yonatan Bisk. 2018. Synthetic and natural noise both break neural machine translation. In International Conference on Learning Representations

work page 2018

[7] [7]

Yonatan Belinkov, Llu \' s M \`a rquez, Hassan Sajjad, Nadir Durrani, Fahim Dalvi, and James Glass. 2017. Evaluating layers of representation in neural machine translation on part-of-speech and semantic tagging tasks. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers), volume 1, pages 1--10

work page 2017

[8] [8]

Samy Bengio, Oriol Vinyals, Navdeep Jaitly, and Noam Shazeer. 2015. Scheduled sampling for sequence prediction with recurrent neural networks. In Advances in Neural Information Processing Systems, pages 1171--1179

2015