Regularized Context Gates on Transformer for Machine Translation

Guoping Huang; Lemao Liu; Max Meng; Rui Wang; Xintong Li

REVIEW 2 major objections 5 minor 31 references

Regularized Context Gates on Transformer for Machine Translation

T0 review · 2 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Adding regularized source-target context gates to the Transformer improves machine translation by about 1.0 BLEU on average across four datasets.

desk verdict Credible +1.0 BLEU gain from regularized context gates for Transformer, but the mechanistic claim is undercut by using the same PMI-derived labels in both training and the error analysis. read the letter →

arxiv 1908.11020 v2 pith:JONTENI2 submitted 2019-08-29 cs.CL

Xintong Li , Lemao Liu , Rui Wang , Guoping Huang , Max Meng This is my paper · ORCID

classification cs.CL

keywords contextgatesTransformerneuralmachinetranslationpointwisemutualinformationregularizationsourcetargetBLEU

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to prove that the Transformer's translation quality is limited by an imbalance between source context and target context, and that a regularized gate can correct that imbalance. It defines a target-context stream and a source-context stream inside each decoder layer, combines them with a learned gate vector, and then regularizes the gates using automatically generated word-co-occurrence labels. On four translation benchmarks the regularized model beats a strong Transformer baseline by about 1.0 BLEU on average. The paper also reports that roughly one third of forced-decoding translation errors are context-selection errors, and that the regularized gates reduce these errors while moving the average gate value from 0.38 to 0.51, closer to balanced use of source and target context.

What carries the argument

The load-bearing object is the context gate vector $z_i^l$, a sigmoid value per hidden unit that interpolates between the target-context stream and the source-context stream in every decoder layer; it turns the Transformer's additive residual connection into a highway-style mixture. The companion piece is the automatic label generator: for each target token $y_i$, the paper computes the strongest pointwise mutual information with any source word and with any preceding target word, and sets the training target $z_i^*$ to whichever side is stronger. This label drives the regularizer in Eq. (5), which penalizes gates whose value lies on the opposite side of 0.5 from $z_i^*$. The same labels are reused in Section 3.3 to count context-selection errors, so the machinery both trains the model and provides the paper's explanation of where translation errors come from.

What would settle it

Take a held-out sample with human word-alignment or golden source-target contribution labels, compare them with the $z_i^*$ labels from Eq. (7), and also compare BLEU when training with PMI labels versus with the human labels. A large disagreement rate, or a big BLEU gap between the two training signals, would show that PMI supervision, not the gate mechanism, is responsible for the reported gain and that the assumption that PMI tracks true context origin is wrong.

Watch

Extended reading notes

Core claim

The central claim is that adding a dimension-wise context gate and supervising it with a corpus-derived binary label improves both translation quality and interpretability of the Transformer. The model replaces the decoder's residual addition with a weighted combination: $c_i^l = \mathrm{rn}\circ\mathrm{ln}\circ\mathrm{ff}\big((1-z_i^l)\otimes t_i^l + z_i^l\otimes s_i^l\big)$, where $t_i^l$ is the target-context stream built from preceding target words and $s_i^l$ is the source-context stream built from the encoder output. The gate $z_i^l=\sigma(\mathrm{ff}(t_i^l\|s_i^l))$ is a sigmoid vector, and the supervision label $z_i^*$ is 1 if the maximum PMI between $y_i$ and any source word exceeds the maximum PMI between $y_i$ and any earlier target word, else 0. The paper's empirical finding is that this regularized gating yields an average gain of 1.0 BLEU over a strong Transformer baseline on four datasets and measurably reduces context-selection errors, whereas the unregularized gate alone gives only a modest gain.

Load-bearing premise

The load-bearing premise is that the binary label $z_i^*$ computed by comparing word-co-occurrence strengths correctly identifies whether a target word is translated from the source or generated from the target context; if corpus co-occurrence statistics mislabel this origin for a sizable fraction of words, the regularizer and the error analysis are internally consistent but not anchored to truth.

Editorial extensions

If this is right

If the 1.0 BLEU average gain reproduces in other language pairs, Transformer systems can pick up a consistent quality improvement by replacing the residual connection with a gated mixture and adding the PMI regularizer at training time only.
Because the gate's average value moves from 0.38 to 0.51, the regularizer is best understood as a bias-correction term; other architectures with stronger target-context bias would need the same correction to stay faithful.
The layer ablations indicate that regularizing every decoder layer beats regularizing any single layer, so future Transformer variants should apply the gate uniformly rather than at a chosen depth.
The error analysis implies an upper bound on what context gating alone can fix: roughly one third of forced-decoding errors are context-selection errors, so the remaining two thirds require different mechanisms.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A natural test is to replace the PMI-derived labels with human word alignments on a small sample; if gate behavior and BLEU change substantially, the supervision signal, not the gate structure, is doing the work.
The same regularizer could be applied at inference time as a decoding constraint by re-ranking hypotheses whose gates more closely match the PMI labels; the paper only uses regularization during training.
Because the context-selection errors are defined under forced decoding, an untested extension is whether the gate's online distribution at test time could be calibrated to the label distribution, which might recover gains the current BLEU numbers do not show.
The mean gate value near 0.5 suggests the method is implicitly learning a per-dimension attention over source versus target; probing those dimensions could reveal interpretable linguistic categories, a question the paper does not address.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Worth a read for anyone who works on controlling source versus target context in Transformer NMT. The headline result, a 1.0 BLEU average gain over a solid Transformer baseline across four datasets, looks real. The paper does what it says: it defines separate source and target contexts in the decoder, adds a gate, and regularizes that gate with automatically generated PMI labels. The decomposition in Eq. (2) is a natural but non-obvious adaptation, and the PMI supervision is a reasonable heuristic that buys a consistent improvement.

The main weakness is the double role of z*. The same PMI-based labels supervise the regularizer in Eq. (5) and serve as ground truth in the error analysis in Section 3.3. Claiming that one third of translation errors are context-selection errors and that the regularizer reduces them is then partly self-consistency: the model is trained to agree with z*, then evaluated on agreement with z*. That does not invalidate the BLEU gain, which is external and independent, but it leaves the causal story unestablished. I also want significance tests or variance in Table 1; the FER drop from 40.5 to 40.0 is small, so the word "significantly" appears twice in Section 3.3 without statistical support. Lambda is tuned once on DE-EN and fixed elsewhere; that is minor given the small robustness table. No code is released, which hurts reproducibility for an architecture paper.

What is genuinely good: the BLEU improvement is consistent across ZH-EN, EN-DE, DE-EN, and FR-EN, and the baseline looks strong. Tu et al. (2017) is properly credited as the source of context gates, so the novelty claim is careful. Table 4, showing the gate means move from 0.38 to 0.51 with regularization, is a nice sanity check that the mechanism does what the authors intend.

This paper is for NMT practitioners who want an adoptable, incremental improvement to Transformer, and for researchers interested in interpreting what gates learn. It is not a paradigm shift. It deserves a serious referee: the BLEU result should be published after revision. Ask for paired significance tests, a small manual validation of the PMI labels, and code or detailed hyperparameters. The central claim is likely to replicate; the mechanism story needs more support before it is sold as an explanation.

Referee Report

2 major / 5 minor

Summary. The paper presents Regularized Context Gates for the Transformer architecture. It first proposes a decomposition of the decoder context into source and target components (Eq. 2), introducing a context gate to rebalance them (Eqs. 3-4). Because freely learned gates show a target-side bias, the authors add a regularizer (Eq. 5) supervised by binary labels z*_i (Eq. 7) derived from pointwise mutual information in bilingual and monolingual corpora. Experiments on four translation tasks show an average improvement of about 1.0 BLEU over a strong Transformer baseline, and an error analysis (Table 3) claims that roughly one third of translation errors are context-selection errors that the regularization reduces.

Significance. The paper addresses a real phenomenon in NMT, the imbalance between source and target context, and proposes a simple, architecture-specific solution. Its strengths include the use of a strong Transformer baseline, consistent gains across six test sets (NIST MT05/06/08, WMT14, IWSLT14, IWSLT17), and a transparent modeling of supervision via PMI. If the BLEU gains are statistically reliable, the method is a useful contribution. However, the paper's explanatory claim about context-selection errors is currently supported only by an analysis that reuses the same labels that define the training supervision, so the mechanism remains unvalidated.

major comments (2)

[Section 3.3, Eq. (7)] The context-selection error rate in Table 3 is computed using z*_i, the same PMI-based labels that serve as the regularizer's training target in Eq. (5). This makes the observed reduction in CER a measure of agreement with the training heuristic, not an independent test of whether translation errors are caused by misusing source versus target context. To support the causal claim, the authors should validate z*_i against human judgments or an independent word-alignment tool on a sample, and then recompute the error analysis on that validated subset.
[Tables 1 and 3] The paper reports BLEU scores and error rates as point estimates without significance tests, confidence intervals, or multiple-run variance. The 1.0 BLEU average gain and the 0.5-point FER drop in Table 3 are therefore not demonstrated to be statistically reliable; the word "significantly" in Sections 3.3 and 4 is not justified. At minimum, report paired bootstrap resampling or multiple-seed variance for the main comparisons.

minor comments (5)

[Section 3.1, Table 2] The regularization coefficient lambda is tuned on DE-EN only and fixed for all other tasks; while Table 2 shows robustness on that single task, a sensitivity analysis on a second language pair would make the transferability claim more convincing.
[Section 2.3, Eq. (7)] The indicator notation 1_{...} is used in Eq. (7) without definition; please define it explicitly.
[Figure 1] The word "purposed" should be "proposed," and the phonetic transcription "t¯i q´ıu" is unclear and should be replaced with a standard romanization or gloss.
[Section 3.4] The mean and variance of context gates in Table 4 are computed over all dimensions of the gate vectors and test tokens, but this aggregation is not stated in the text; please clarify the exact computation.
[Section 3.6] The claim that Regularized Context Gates "significantly improves" the translation of long and short sentences is not supported by significance tests; please add error bars or soften the wording.

Circularity Check

1 steps flagged · score 4.0 of 10

Explanatory error analysis reuses the PMI supervision labels, making the 'context selection error' reduction a self-consistency check rather than an independent mechanism.

self definitional [Section 2.2 Eq. (5), Section 2.3 Eq. (7), Section 3.3 Table 3]
"the training objective is defined as follows: 𝓁 = − logP (y| x)+λ ∑_{l,i}( z∗_i max(0.5−z_l_i, 0) + (1−z∗_i ) max(z_l_i− 0.5, 0) ) ... The context selection error is defined as z∗_i (y_i) ≠ z∗_i (ŷ_i), where z∗_i is defined in equation 7."

The same z∗_i from Eq. (7) is both the supervision target in the Eq. (5) regularizer and the ground-truth label in the Section 3.3 error analysis. The regularizer is explicitly trained to push each gate toward z∗_i, so the reported reduction in 'context selection error' and the CE/FE ratio measure agreement with the very PMI heuristic used to train the gates, not an independent cause of translation quality. The BLEU gain in Table 1 is an external result, but the mechanistic explanation that improvements are caused by avoiding context-selection errors is self-consistent rather than independently validated.

full rationale

The paper's headline result—an averaged 1.0 BLEU gain over the Transformer baseline on four held-out datasets—is an external, non-circular claim. The model is compared against standard benchmarks and the BLEU metric is not derived from the proposed z∗ labels. The circularity is confined to the explanatory analysis. Equation (5) adds a hinge-style regularizer that supervises each context gate with z∗_i, defined in Eq. (7) from PMI co-occurrence counts. Section 3.3 then defines 'context selection error' using the same z∗_i and, in Table 3, credits the regularized gates with reducing that error. Thus the CER/CE-FE statistics and the 'approximately one third translation errors' statement do not independently validate the mechanism; they show the model has learned to match the PMI heuristic that was used as its training signal. This is partial circularity in the interpretation, not in the core BLEU result. The self-citation to Li et al. (2019) is not load-bearing here, and no other circular step is present. The lack of significance testing is a separate statistical concern, not a circularity issue.

Assumptions & free parameters 2 free parameters · 2 assumptions · 0 invented entities

The central BLEU claim rests on a standard training and evaluation setup. The load-bearing external assumptions are the decoder context decomposition and the PMI heuristic for source/target attribution. The main manually tuned free parameter is the regularization coefficient lambda.

free parameters (2)

regularization coefficient lambda = 1
Tuned on DE-EN (Table 2) with values 0.1, 0.5, 1, 2, 10; lambda=1 gives the best BLEU and is used for all tasks.
gate threshold for regularizer = 0.5
The regularizer in Eq. (5) uses max(0.5 - z_i, 0) and max(z_i - 0.5, 0); this threshold is hand-chosen and not tuned.

assumptions (2)

domain assumption The standard Transformer decoder layer can be split into a target self-attention context and a source encoder-decoder attention context.
Section 2.1, Eq. (2). If this decomposition does not isolate the true source and target contributions, the gate mechanism is mis-specified.
domain assumption PMI co-occurrences between words in the training data are a valid proxy for lexical translation equivalence.
Section 2.3 uses PMI to binarize source vs target contribution. There is no independent validation that this heuristic matches human judgment of context contribution.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Regularized Context Gates on Transformer for Machine Translation." pith.science (2026). https://pith.science/paper/JONTENI2

@misc{pith2026190811020,
  author       = {Pith},
  title        = {Pith review of: Regularized Context Gates on Transformer for Machine Translation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JONTENI2}},
  note         = {Machine review of arXiv:1908.11020}
}

read the original abstract

Context gates are effective to control the contributions from the source and target contexts in the recurrent neural network (RNN) based neural machine translation (NMT). However, it is challenging to extend them into the advanced Transformer architecture, which is more complicated than RNN. This paper first provides a method to identify source and target contexts and then introduce a gate mechanism to control the source and target contributions in Transformer. In addition, to further reduce the bias problem in the gate mechanism, this paper proposes a regularization method to guide the learning of the gates with supervision automatically generated using pointwise mutual information. Extensive experiments on 4 translation datasets demonstrate that the proposed model obtains an averaged gain of 1.0 BLEU score over a strong Transformer baseline.

Figures

Figures reproduced from arXiv: 1908.11020 by the authors.

**Figure 2.** Translation performance on MT08 test set [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

31 extracted references · 17 canonical work pages

[1]

URL: " 'urlintro :=

ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...
[2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
[3]

Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. 2016. https://arxiv.org/pdf/1607.06450.pdf Layer normalization . arXiv preprint arXiv:1607.06450

arXiv 2016
[4]

Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. https://arxiv.org/pdf/1409.0473.pdf Neural machine translation by jointly learning to align and translate . arXiv preprint arXiv:1409.0473

arXiv 2014
[5]

Gerlof Bouma. 2009. https://svn.spraakdata.gu.se/repos/gerlof/pub/www/Docs/npmi-pfd.pdf Normalized (pointwise) mutual information in collocation extraction . Proceedings of GSCL, pages 31--40

work page 2009
[6]

Kehai Chen, Rui Wang, Masao Utiyama, Eiichiro Sumita, and Tiejun Zhao. 2018. https://www.aaai.org/ocs/index.php/AAAI/AAAI18/paper/view/16060/16008 Syntax-directed attention for neural machine translation . In Thirty-Second AAAI Conference on Artificial Intelligence

work page 2018
[7]

Yong Cheng, Zhaopeng Tu, Fandong Meng, Junjie Zhai, and Yang Liu. 2018. https://arxiv.org/pdf/1805.06130.pdf Towards robust neural machine translation . arXiv preprint arXiv:1805.06130

arXiv 2018
[8]

David Chiang. 2005. https://www3.nd.edu/ dchiang/papers/chiang-acl05.pdf A hierarchical phrase-based model for statistical machine translation . In Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics, pages 263--270. Association for Computational Linguistics

work page 2005

Show all 31 references

[9]

Maha Elbayad, Laurent Besacier, and Jakob Verbeek. 2018. https://aclweb.org/anthology/K18-1010 Pervasive attention: 2d convolutional neural networks for sequence-to-sequence prediction . arXiv preprint arXiv:1808.03867

2018 arXiv
[10]

Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, and Yann N Dauphin. 2017. https://arxiv.org/pdf/1705.03122.pdf Convolutional sequence to sequence learning . In Proceedings of the 34th International Conference on Machine Learning-Volume 70, pages 1243--1252. JMLR. org

2017 arXiv
[11]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. http://openaccess.thecvf.com/content_cvpr_2016/papers/He_Deep_Residual_Learning_CVPR_2016_paper.pdf Deep residual learning for image recognition . In Proceedings of the IEEE conference on computer vision and pattern ...

2016
[12]

Tianyu He, Xu Tan, Yingce Xia, Di He, Tao Qin, Zhibo Chen, and Tie-Yan Liu. 2018. http://papers.nips.cc/paper/8019-layer-wise-coordination-between-encoder-and-decoder-for-neural-machine-translation.pdf Layer-wise coordination between encoder and decoder for neural machine tran...

2018
[13]

Philipp Koehn. 2009. Statistical machine translation. Cambridge University Press

2009
[14]

Philipp Koehn, Franz Josef Och, and Daniel Marcu. 2003. http://www.aclweb.org/anthology/N03-1017 Statistical phrase-based translation . In Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Techno...

2003
[15]

Xintong Li, Guanlin Li, Lemao Liu, Max Meng, and Shuming Shi. 2019. https://www.aclweb.org/anthology/P19-1124 On the word alignment from neural machine translation . In Proceedings of the 57th Conference of the Association for Computational Linguistics, pages 1293--1303

2019
[16]

Xintong Li, Lemao Liu, Zhaopeng Tu, Shuming Shi, and Max Meng. 2018. https://www.aclweb.org/anthology/N18-1125 Target foresight based attention for neural machine translation . In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computati...

2018
[17]

Lemao Liu, Masao Utiyama, Andrew Finch, and Eiichiro Sumita. 2016. https://arxiv.org/pdf/1609.04186.pdf Neural machine translation with supervised attention . arXiv preprint arXiv:1609.04186

2016 arXiv
[18]

Zhanyi Liu, Haifeng Wang, Hua Wu, and Sheng Li. 2009. http://aclweb.org/anthology/D09-1051 Collocation extraction using monolingual word alignment method . In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing: Volume 2-Volume 2, pages 487--...

2009
[19]

Minh-Thang Luong, Hieu Pham, and Christopher D Manning. 2015. https://arxiv.org/pdf/1508.04025.pdf Effective approaches to attention-based neural machine translation . arXiv preprint arXiv:1508.04025

2015 arXiv
[20]

Shuming Ma, Xu Sun, Yizhong Wang, and Junyang Lin. 2018. https://arxiv.org/pdf/1805.04871.pdf Bag-of-words as target for neural machine translation . arXiv preprint arXiv:1805.04871

2018 arXiv
[21]

Haitao Mi, Zhiguo Wang, and Abe Ittycheriah. 2016. https://arxiv.org/pdf/1608.00112.pdf Supervised attentions for neural machine translation . arXiv preprint arXiv:1608.00112

2016 arXiv
[22]

Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. https://www.aclweb.org/anthology/P02-1040.pdf Bleu: a method for automatic evaluation of machine translation . In Proceedings of the 40th annual meeting on association for computational linguistics, pages 311--...

2002
[23]

Rico Sennrich, Barry Haddow, and Alexandra Birch. 2015. https://arxiv.org/pdf/1508.07909.pdf Neural machine translation of rare words with subword units . arXiv preprint arXiv:1508.07909

2015 arXiv
[24]

Rupesh Kumar Srivastava, Klaus Greff, and J \"u rgen Schmidhuber. 2015. https://arxiv.org/pdf/1505.00387.pdf Highway networks . arXiv preprint arXiv:1505.00387

2015 arXiv
[25]

Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. https://papers.nips.cc/paper/5346-sequence-to-sequence-learning-with-neural-networks.pdf Sequence to sequence learning with neural networks . In Advances in neural information processing systems, pages 3104--3112

2014
[26]

Zhaopeng Tu, Yang Liu, Zhengdong Lu, Xiaohua Liu, and Hang Li. 2017. http://www.aclweb.org/anthology/Q17-1007 Context gates for neural machine translation . Transactions of the Association for Computational Linguistics, 5:87--99

2017
[27]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. 2017. https://papers.nips.cc/paper/7181-attention-is-all-you-need.pdf Attention is all you need . In Advances in Neural Information Processing Systems, p...

2017
[28]

Mingming Yang, Min Zhang, Kehai Chen, Rui Wang, and Tiejun Zhao. 2020. https://search.ieice.org/bin/summary.php?id=e103-d_3_684 Neural machine translation with target-attention model . IEICE TRANSACTIONS on Information and Systems, 103(3):684--694

2020
[29]

Jiacheng Zhang, Yanzhuo Ding, Shiqi Shen, Yong Cheng, Maosong Sun, Huanbo Luan, and Yang Liu. 2017. https://arxiv.org/pdf/1706.06415.pdf Thumt: an open source toolkit for neural machine translation . arXiv preprint arXiv:1706.06415

2017 arXiv
[30]

Jiacheng Zhang, Huanbo Luan, Maosong Sun, Feifei Zhai, Jingfang Xu, Min Zhang, and Yang Liu. 2018. https://arxiv.org/abs/1810.03581 Improving the transformer translation model with document-level context . arXiv preprint arXiv:1810.03581

2018 arXiv
[31]

Yang Zhao, Jiajun Zhang, Zhongjun He, Chengqing Zong, and Hua Wu. 2018. http://www.aclweb.org/anthology/D18-1036 Addressing troublesome words in neural machine translation . In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 391--400

2018

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

URL: " 'urlintro :=

ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...

[2] [2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

[3] [3]

Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. 2016. https://arxiv.org/pdf/1607.06450.pdf Layer normalization . arXiv preprint arXiv:1607.06450

arXiv 2016

[4] [4]

Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. https://arxiv.org/pdf/1409.0473.pdf Neural machine translation by jointly learning to align and translate . arXiv preprint arXiv:1409.0473

arXiv 2014

[5] [5]

Gerlof Bouma. 2009. https://svn.spraakdata.gu.se/repos/gerlof/pub/www/Docs/npmi-pfd.pdf Normalized (pointwise) mutual information in collocation extraction . Proceedings of GSCL, pages 31--40

work page 2009

[6] [6]

Kehai Chen, Rui Wang, Masao Utiyama, Eiichiro Sumita, and Tiejun Zhao. 2018. https://www.aaai.org/ocs/index.php/AAAI/AAAI18/paper/view/16060/16008 Syntax-directed attention for neural machine translation . In Thirty-Second AAAI Conference on Artificial Intelligence

work page 2018

[7] [7]

Yong Cheng, Zhaopeng Tu, Fandong Meng, Junjie Zhai, and Yang Liu. 2018. https://arxiv.org/pdf/1805.06130.pdf Towards robust neural machine translation . arXiv preprint arXiv:1805.06130

arXiv 2018

[8] [8]

David Chiang. 2005. https://www3.nd.edu/ dchiang/papers/chiang-acl05.pdf A hierarchical phrase-based model for statistical machine translation . In Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics, pages 263--270. Association for Computational Linguistics

work page 2005