Pith. sign in

REVIEW 2 major objections 4 minor 55 references

Improving Back-Translation with Uncertainty-based Confidence Estimation

T0 review · 2 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Uncertainty-based confidence scoring lifts back-translation BLEU by up to 0.82.

desk verdict Solid back-translation paper; the uncertainty-weighting works empirically, but the paper never isolates per-sentence confidence from a global downweighting of synthetic data. read the letter →

arxiv 1909.00157 v1 pith:X36CDEQU submitted 2019-08-31 cs.CL

Shuo Wang , Yang Liu , Chao Wang , Huanbo Luan , Maosong Sun This is my paper · ORCID
classification cs.CL
keywords back-translationneuralmachinetranslationmodeluncertaintyMonteCarlodropoutconfidenceestimationsyntheticbilingualdataattentionreweightinglow-resourceNMT
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that the noisy synthetic sentences produced by back-translation can be judged by the NMT model itself, using model uncertainty rather than translation probability. After a target-to-source model generates a synthetic source sentence, the authors run the same model $K$ times with dropout active, turning word- and sentence-level translation probabilities into random variables. Confidence is defined from the expectation and variance of those samples, with the CEV measure $(1-\mathrm{Var}/E)^\beta$; high variance marks a prediction the model is not sure about. Sentence-level confidence down-weights unreliable synthetic pairs in the training loss, and word-level confidence re-weights attention so the trained model leans less on probably erroneous source words. On Chinese-English and English-German benchmarks, this produces BLEU gains up to +0.82 over standard beam-search back-translation and statistically significant improvements on the combined test sets.

What carries the argument

The load-bearing object is the Monte Carlo dropout estimate of the distribution of translation probabilities over the model's parameter posterior. Given a fixed prediction $\hat{x}$ from a target sentence $y$, the model is run $K$ times with different dropout masks, producing samples $P(\hat{x}\mid y,\hat{\theta}^{(k)})$. Their mean approximates the expected translation probability and their variance approximates model uncertainty (Eqs. 7-8). The CEV confidence $(1-\mathrm{Var}/E)^\beta$ combines expectation and variance, and the same computation is done for each word. These scores are not used to re-decode; they are inserted into training: sentence-level confidence reweights the likelihood of each synthetic pair (Eq. 13), and word-level confidence is broadcast-multiplied into the attention function (Eq. 15).

What would settle it

Take a synthetic back-translated corpus, align each generated source word to its actual error status (by comparison with a human translation or a round-trip reference), and measure whether word-level dropout variance separates correct from incorrect words. If the area under the ROC curve is near 0.5, or if inverse-confidence weighting (up-weighting low-confidence pairs) also improves BLEU, then the CEV signal is not the mechanism behind the reported gains.

Watch

Extended reading notes

Core claim

The central claim is that translation probability is not a good confidence signal, but the variance of translation probability under dropout is. Beam-search back-translation assigns probabilities to its synthetic sentences, and the paper shows that using those probabilities directly (PTP) or their expectation (EXP) as training weights hurts BLEU (45.41 and 45.22 vs. 46.23 on NIST06). Variance-based measures help: VAR gives +0.54, and CEV, which divides variance by expectation before scaling, gives +0.82 (47.05). The same logic extends to words: a word-level confidence vector multiplies attention weights in Eq. (15), so the model pays less attention to words the generator was unsure about. On the combined NIST test sets, uncertainty-aware back-translation reaches 45.72 BLEU versus 44.76 for beam-search back-translation, and on WMT14 English-German it reaches 28.28 versus 27.04, both statistically significant; it also beats a neural quality-estimation baseline that needs labeled post-edited data.

Load-bearing premise

The method's usefulness rests on the variance of translation probabilities under dropout actually marking erroneous translations: if wrong words are often produced with confidently low variance, the confidence signal adds nothing beyond the probability baselines, which the paper shows fail.

Editorial extensions

If this is right

  • Back-translation can be made noise-aware without any external quality-estimation model or labeled post-edited data; the generator's own dropout statistics provide the signal.
  • The method complements sampling-based back-translation and difficult-word data selection, so it stacks with existing ways of improving synthetic data.
  • Confidence-weighted training keeps all synthetic pairs instead of filtering them, so low-resource systems can keep using large monolingual corpora without discarding uncertain translations.
  • When the synthetic corpus grows, unweighted back-translation plateaus or falls while confidence-weighted training keeps improving, which is evidence that weighting is what lets extra data help.
  • Because it only needs dropout and attention, the approach is architecture- and language-pair-agnostic and should transfer to other sequence-generation tasks trained on model-generated data.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct calibration check would measure whether word-level dropout variance separates correct from incorrect synthetic source words; if it does, CEV could serve as a cheap pseudo-quality estimator for any model-generated text.
  • If variance is the operative signal, the exact sampling method may be replaceable by deep ensembles or other approximate-inference techniques, and the confidence scores could be computed once and reused across training runs.
  • The soft attention re-weighting suggests a testable variant: hard-masking low-confidence source words, or annealing $\beta$ upward during training, might outperform the fixed $\beta=2$ used here.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. Back-translation generates synthetic source sentences from monolingual target data, but the synthetic data are noisy. This paper uses Monte Carlo Dropout to estimate the expectation and variance of word- and sentence-level translation probabilities of the target-to-source model, and turns them into confidence measures: predicted probability (PTP), expected probability (EXP), variance-based (VAR), and combined expectation-variance (CEV). Sentence-level confidence is used as a weight on synthetic sentence log-likelihoods in the final training objective (Eq. 13), and word-level confidence is used to modulate attention weights (Eq. 15). Experiments on Chinese-English (NIST) and English-German (WMT14) report that CEV improves over plain back-translation (e.g., +0.82 BLEU on NIST06; aggregate All 45.72 vs 44.76 for SEARCH on zh-en; All 28.28 vs 27.04 for en-de), and that uncertainty-based confidence outperforms a neural QE baseline on en-de. The authors also report scaling and data-selection experiments. The implementation is released.

Significance. If the improvement is caused by the per-sentence uncertainty signal, the result would be valuable: it is a label-free way to make back-translation more robust to noise, applicable to any language pair and model, and it outperforms a supervised QE baseline. The paper is generally well executed: two language pairs, standard data and BLEU evaluation, paired bootstrap significance tests, comparisons with neural QE, and released code. The main substantive gap is that the experiments do not disentangle per-sentence reweighting from an overall downweighting of the synthetic corpus.

major comments (2)
  1. [Section 3.3, Eq. (13)] Because every synthetic sentence is multiplied by C_CEV in [0,1] while authentic pairs keep weight 1, the objective simultaneously changes the relative weight among synthetic sentences and the total weight of the synthetic corpus relative to the authentic corpus. The PTP and EXP baselines in Eqs. (9)-(10) also apply nonuniform weights, but their total weight is not matched to CEV's, so they do not control for this confound. Neither a uniform downweighting run (all synthetic sentences weighted by the corpus-average C_CEV) nor a random permutation of the same weights is reported. Without these controls, the observed BLEU gains over SEARCH and SAMPLE do not establish that the per-sentence uncertainty signal is the active ingredient. Please add at least one such control, or otherwise argue quantitatively that total-weight effects cannot explain the gains.
  2. [Section 3.3, Eq. (15) and Figure 3] The modified attention weights are computed as softmax(...) ⊙ c without renormalizing each row to sum to one, so the total attention mass varies with the confidence vector; moreover, the notation Q,K,V ∈ R^{I×D} does not make clear whether the source-side confidence vector c of length I is broadcast over rows or columns when encoder and decoder lengths differ. The paper should clarify the exact tensor layout and state whether renormalization was tried; this matters because the word-level gains in Table 2 are attributed to this attention modification.
minor comments (4)
  1. [Section 4.1] Alpha and beta are set to 2 based on the development set, but no sensitivity analysis or error bars over independent training runs are reported; please state whether the reported BLEU scores are from a single run and, if feasible, add repeat-run statistics for the main comparisons.
  2. [Section 3.1] K=20 is fixed, but there is no ablation over the number of Monte Carlo Dropout samples; a sentence or table showing sensitivity to K would help the reader judge the method's robustness.
  3. [Section 4.4, Table 3] For SAMPLE+U, the gains are not significant on MT02 and MT08 individually, and MT02 is lower than SAMPLE, yet the aggregate All is significant; this should be stated in the text for transparency.
  4. [Section 6] The phrase 'qualifying model uncertainty' should read 'quantifying model uncertainty'.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: confidence weights are computed from the back-translation model's own MC-dropout statistics and evaluated on held-out external test sets, so no prediction reduces to its input by construction.

full rationale

The claimed improvement is empirical: Eq. (13) weights each synthetic sentence by C_CEV, but the outcome is BLEU on NIST and WMT test sets, not the confidence values themselves. The confidence measures in Eqs. (9)-(12) are derived from MC-dropout samples of translation probabilities (Gal and Ghahramani, 2016, an external citation), and neither α nor β is fit to test BLEU; the paper states they work best when set to 2, and the comparisons among PTP, EXP, VAR, and CEV are made on development data. The main reported gains are on held-out test sets, so the derivation is not self-definitional. The skeptical concern that CEV changes the aggregate weight of the synthetic corpus relative to the authentic corpus is a missing-control/attribution issue about whether per-sentence uncertainty is the active ingredient, not a definitional equivalence or a self-citation chain. Self-citations to THUMT and prior author papers are implementation or related-work references, not load-bearing justifications of the central claim. No circular step meeting the required evidentiary standard is present.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The only tuned quantities are the confidence-scoring hyperparameters (alpha, beta) and the MC sample count K; these are standard and are set on development data. The method introduces no new theoretical entities or physical objects.

free parameters (3)
  • alpha = 2
    Hyperparameter controlling the gap in VAR confidence measure, tuned on the development set (Section 4.1).
  • beta = 2
    Hyperparameter controlling the gap in CEV confidence measure, tuned on the development set (Section 4.1).
  • K (MC Dropout samples) = 20
    Number of stochastic forward passes used to estimate expectation and variance, chosen in experiments (Section 4.1).
assumptions (3)
  • domain assumption Monte Carlo Dropout provides a valid approximation to the Bayesian posterior over model parameters.
    Relied on in Section 3.1 to interpret dropout samples as samples from the posterior and to compute model uncertainty.
  • domain assumption The back-translation pipeline as formalized by Sennrich et al. (2016a) is a valid baseline and the three-step procedure is followed.
    Used in Section 2 to define the training objective and the synthetic corpus generation.
  • domain assumption BLEU is a reliable evaluation metric for translation quality.
    Used throughout Section 4 as the primary measure of performance, with bootstrap significance testing.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Improving Back-Translation with Uncertainty-based Confidence Estimation." pith.science (2026). https://pith.science/paper/X36CDEQU

@misc{pith2026190900157,
  author       = {Pith},
  title        = {Pith review of: Improving Back-Translation with Uncertainty-based Confidence Estimation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/X36CDEQU}},
  note         = {Machine review of arXiv:1909.00157}
}
read the original abstract

While back-translation is simple and effective in exploiting abundant monolingual corpora to improve low-resource neural machine translation (NMT), the synthetic bilingual corpora generated by NMT models trained on limited authentic bilingual data are inevitably noisy. In this work, we propose to quantify the confidence of NMT model predictions based on model uncertainty. With word- and sentence-level confidence measures based on uncertainty, it is possible for back-translation to better cope with noise in synthetic bilingual corpora. Experiments on Chinese-English and English-German translation tasks show that uncertainty-based confidence estimation significantly improves the performance of back-translation.

Figures

Figures reproduced from arXiv: 1909.00157 by the authors.

Figure 1
Figure 1. Confidence estimation for back-translation. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Illustration of uncertainty calculation. Given a target sentence [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Using word-level confidence in confidence-aware training. The basic idea is to use confidence to modify [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Effect of training corpus size. 4.6 Effect of Data Selection Instead of randomly selecting monolingual sen￾tences to generate synthetic data, we also used the method proposed by (Fadaee and Monz, 2018) to select monolingual data by targeting difficult words. In this se…
Figure 5
Figure 5. Figure 5: Example of confidence measures. (2018) find that iterative back-translation can fur￾ther improve the performance of NMT. Fadaee and Monz (2018) show that words with high pre￾dicted loss during training benefit most. Our work differs from existing methods in that we pro…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

55 extracted references · 44 canonical work pages

  1. [1]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Nguyen Bach, Fei Huang, and Yaser Al-Onaizan. 2011. https://www.aclweb.org/anthology/P11-1022 G oodness: A method for measuring machine translation confidence . In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 211--219, Portland, Oregon, USA. Association for Computational Linguistics

    work page 2011

  4. [4]

    Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In Proceedings of ICLR 2015

    work page 2015

  5. [5]

    John Blatz, Erin Fitzgerald, George Foster, Simona Gandrabur, Cyril Goutte, Alex Kulesza, Alberto Sanchis, and Nicola Ueffing. 2004. https://www.aclweb.org/anthology/C04-1046 Confidence estimation for machine translation . In Coling 2004: Proceedings of the 20th international conference on computational linguistics, pages 315--321

    work page 2004

  6. [6]

    Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, and Daan Wierstra. 2015. Weight uncertainty in neural networks. In Proceedings of ICML 2015

    work page 2015

  7. [7]

    Wray Buntine and Andreas S. Weigend. 1991. Bayesian back-propagation. Complex Systems

    work page 1991

  8. [8]

    Yun Chen, Yang Liu, Yong Cheng, and Victor O.K. Li. 2017. https://doi.org/10.18653/v1/P17-1176 A teacher-student framework for zero-resource neural machine translation . In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1925--1935, Vancouver, Canada. Association for Computational Linguistics

    doi:10.18653/v1/p17-1176 2017

Show all 55 references
  1. [9]

    Yong Cheng, Wei Xu, Zhongjun He, Wei He, Hua Wu, Maosong Sun, and Yang Liu. 2016. https://www.aclweb.org/anthology/P16-1185 Semi-supervised learning for neural machine translation . In Proceedings of ACL 2016

    2016

  2. [10]

    Ryan Cotterell and Julia Kreutzer. 2018. Explaining and generalizing back-translation through wake-sleep. CoRR, abs/1806.04402

    2018 arXiv

  3. [11]

    Anna Currey, Antonio Valerio Miceli Barone, and Kenneth Heafield. 2017. https://www.aclweb.org/anthology/W17-4715 Copied monolingual data improves low-resource neural machine translation . In Proceedings of the Second Conference on Machine Translation, pages 148--156. Associat...

    2017

  4. [12]

    Li Dong, Chris Quirk, and Mirella Lapata. 2018. https://doi.org/10.18653/v1/P18-1069 Confidence modeling for neural semantic parsing . In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 743--753, Melbourne,...

    2018 doi

  5. [13]

    Sergey Edunov, Myle Ott, Michael Auli, and David Grangier. 2018. https://aclweb.org/anthology/D18-1045 Understanding back-translation at scale . In Proceedings of EMNLP 2018

    2018

  6. [14]

    Marzieh Fadaee, Arianna Bisazza, and Christof Monz. 2017. https://doi.org/10.18653/v1/P17-2090 Data augmentation for low-resource neural machine translation . In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), p...

    2017 doi

  7. [15]

    Marzieh Fadaee and Christof Monz. 2018. https://www.aclweb.org/anthology/D18-1040 Back-translation sampling by targeting difficult words in neural machine translation . In Proceedings of EMNLP 2018

    2018

  8. [16]

    Yarin Gal and Zoubin Ghahramani. 2016. Dropout as a bayesian approaximation: Representing model uncertainty in deep learning. In Proceedings of ICML 2016

    2016

  9. [17]

    Yonatan Geifman, Guy Uziel, and Ran El-Yaniv. 2019. https://openreview.net/forum?id=SJfb5jCqKm Bias-reduced uncertainty estimation for deep neural classifiers . In International Conference on Learning Representations

    2019

  10. [18]

    Alex Graves. 2011. Practical variational inference for neural networks. In Proceedings of NeurIPS

    2011

  11. [19]

    Hany Hassan, Anthony Aue, Chang Chen, Vishal Chowdhary, Jonathan Clark, Christian Federmann, Xuedong Huang, Marcin Junczys-Dowmunt, William Lewis, Mu Li, Shujie Liu, Tie-Yan Liu, Renqian Luo, Arul Menezes, Tao Qin, Frank Seide, Xu Tan, Fei Tian, Lijun Wu, Shuangzhi Wu, Yingce ...

    2018 arXiv

  12. [20]

    Cong Duy Vu Hoang, Philipp Keohn, Gholamreza Haffari, and Trevor Cohn. 2018. https://www.aclweb.org/anthology/W18-2703 Iterative back-translation for neural machine translation . In Proceedings of the 2nd Workshop on Neural Machine Translation and Generation

    2018

  13. [21]

    Kenji Imamura, Atsushi Fujita, and Eiichiro Sumita. 2018. https://www.aclweb.org/anthology/W18-2707 Enhancement of encoder and attention using target monolingual corpora in neural machine translation . In Proceedings of the 2nd Workshop on Neural Machine Translation and Genera...

    2018

  14. [22]

    Julia Ive, Fr \'e d \'e ric Blain, and Lucia Specia. 2018. https://www.aclweb.org/anthology/C18-1266 deep Q uest: A framework for neural-based quality estimation . In Proceedings of the 27th International Conference on Computational Linguistics, pages 3146--3157, Santa Fe, New...

    2018

  15. [23]

    Alex Kendall, Vijay Badrinarayanan, and Roberto Cipolla. 2015. Bayesian segnet: Model uncertainty in deep convolutional encoder-decoder architectures for scene understanding. arXiv:1511.02680

    2015 arXiv

  16. [24]

    Alex Kendall and Yarin Gal. 2017. What uncertainties do we need in bayesian deep learning for computer vision? In Advances in neural information processing systems, pages 5574--5584

    2017

  17. [25]

    Fabio Kepler, Jonay Tr \'e nous, Marcos Treviso, Miguel Vera, and Andr \'e F. T. Martins. 2019. https://www.aclweb.org/anthology/P19-3020 O pen K iwi: An open source framework for quality estimation . In Proceedings of the 57th Annual Meeting of the Association for Computation...

    2019

  18. [26]

    Hyun Kim, Jong-Hyeok Lee, and Seung-Hoon Na. 2017. https://doi.org/10.18653/v1/W17-4763 Predictor-estimator using multilevel task learning with stack propagation for neural quality estimation . In Proceedings of the Second Conference on Machine Translation, pages 562--568, Cop...

    2017 doi

  19. [27]

    Kingma and Jimmy Ba

    Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. CoRR, abs/1412.6980

    2015 arXiv

  20. [28]

    Philipp Koehn. 2004. https://www.aclweb.org/anthology/W04-3250 Statistical significance tests for machine translation evaluation . In Proceedings of the 2004 conference on empirical methods in natural language processing, pages 388--395

    2004

  21. [29]

    Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondrej Bojar, Alexandra Constantin, and Evan Herbst. 2007. https://www.aclweb.org/anthology/P07-2045 Moses:...

    2007

  22. [30]

    Och, and Daniel Marcu

    Philipp Koehn, Franz J. Och, and Daniel Marcu. 2003. https://www.aclweb.org/anthology/N03-1017 Statistical phrase-based translation . In Proceedings of NAACL 2003

    2003

  23. [31]

    Guillaume Lample, Myle Ott, Alexis Conneau, Ludovic Denoyer, and Marc'Aurelio Ranzato. 2018. https://aclweb.org/anthology/D18-1549 Phrase-based & neural unsupervised machine translation . In Proceedings of EMNLP 2018

    2018

  24. [32]

    Srinivasa

    Gilwoo Lee, Brian Hou, Aditya Mandalika, Jeongseok Lee, and Siddhartha S. Srinivasa. 2019. https://openreview.net/forum?id=SJGvns0qK7 Bayesian policy optimization for model uncertainty . In International Conference on Learning Representations

    2019

  25. [33]

    Ngoc-Quang Luong, Laurent Besacier, and Benjamin Lecouteux. 2017. Find the errors, get the better: Enhancing machine translation via word confidence estimation. Natural Language Engineering, 23:617--639

    2017

  26. [34]

    Gallagher, Kevin P

    Seong Joon Oh, Andrew C. Gallagher, Kevin P. Murphy, Florian Schroff, Jiyan Pan, and Joseph Roth. 2019. https://openreview.net/forum?id=r1xQQhAqKX Modeling uncertainty with hedged instance embeddings . In International Conference on Learning Representations

    2019

  27. [35]

    Myle Ott, Michael Auli, David Grangier, and Marc'Aurelio Ranzato. 2018. Analyzing uncertainty in neural machine translation. In Proceedings of ICML 2018

    2018

  28. [36]

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2001. https://www.aclweb.org/anthology/P02-1040 Bleu: a method for automatic evaluation of machine translation . In Proceedings of ACL 2001

    2001

  29. [37]

    Gabriel Pereyra, George Tucker, Jan Chorowski, Lukasz Kaiser, and Geoffrey E. Hinton. 2017. Regularizing neural networks by penalizing confident output distributions. CoRR, abs/1701.06548

    2017 arXiv

  30. [38]

    M \= a rcis Pinnis, Rihards Kri s lauks, Daiga Deksne, and Toms Miks. 2017. Neural machine translation for morphologically rich languages with improved sub-word units and synthetic data. In Text, Speech, and Dialogue, pages 237--245, Cham. Springer International Publishing

    2017

  31. [39]

    Alberto Poncelas, Dimitar Shterionov, Andy Way, Gideon Maillette de Buy Wenniger, and Peyman Passban. 2018. Investigating backtranslation in neural machine translation. CoRR, abs/1804.06189

    2018 arXiv

  32. [40]

    Shuo Ren, Wenhu Chen, Shujie Liu, Mu Li, Ming Zhou, and Shuai Ma. 2018. https://doi.org/10.18653/v1/P18-1006 Triangular architecture for rare language translation . In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Paper...

    2018 doi

  33. [41]

    Matiss Rikters and Mark Fishel. 2017. Confidence through attention. CoRR, abs/1710.03743

    2017 arXiv

  34. [42]

    Marzieh Salehi, Shahram Khadivi, and Nooshin Riahi. 2014. Confidence estimation for machine translation using context vectors. 7'th International Symposium on Telecommunications (IST'2014), pages 524--528

    2014

  35. [43]

    Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016 a . https://www.aclweb.org/anthology/P16-1009 Improving neural machine translation models with monolingual data . In Proceedings of ACL 2016

    2016

  36. [44]

    Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016 b . https://www.aclweb.org/anthology/P16-1162 Neural machine translation of rare words with subword units . In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)...

    2016

  37. [45]

    Lucia Specia, Marco Turchi, Zhuoran Wang, John Shawe-Taylor, and Craig Saunders. 2009. Improving the confidence of machine translation quality estimates. In Twelfth Machine Translation Summit

    2009

  38. [46]

    Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to sequence learning with neural networks. In Proceedings of NeurIPS 2014

    2014

  39. [47]

    Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jonathon Shlens, and Zbigniew Wojna. 2016. Rethinking the inception architecture for computer vision. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2818--2826

    2016

  40. [48]

    Nicola Ueffing and Hermann Ney. 2007. https://www.aclweb.org/anthology/J07-1003 Word-level confidence estimation for machine translation . Computational Linguistics

    2007

  41. [49]

    Gomez, Lukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of NeurIPS 2017

    2017

  42. [50]

    Jiayi Wang, Kai Fan, Bo Li, Fengming Zhou, Boxing Chen, Yangbin Shi, and Luo Si. 2018. https://doi.org/10.18653/v1/W18-6465 A libaba submission for WMT 18 quality estimation task . In Proceedings of the Third Conference on Machine Translation: Shared Task Papers, pages 809--81...

    2018 doi

  43. [51]

    Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V. Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, Jeff Klingner, Apurva Shah, Melvin Johnson, Xiaobing Liu, Lukasz Kaiser, Stephan Gouws, Yoshikiyo Kato, Taku Kudo, Hideto Kazawa, Keith St...

    2016 arXiv

  44. [52]

    Yijun Xiao and William Yang Wang. 2019. Quantifying uncertainties in natural language processing tasks. In Proceedings of AAAI 2019

    2019

  45. [53]

    Jiacheng Zhang, Yanzhuo Ding, Shiqi Shen, Yong Cheng, Maosong Sun, Huan - Bo Luan, and Yang Liu. 2017. http://arxiv.org/abs/1706.06415 THUMT: an open source toolkit for neural machine translation . CoRR, abs/1706.06415

    2017 arXiv

  46. [54]

    Lingxue Zhu and Nikolay Laptev. 2017. Deep and confident prediction for time series at uber. In Proceedings of ICDM 2017

    2017

  47. [55]

    Barret Zoph, Deniz Yuret, Jonathan May, and Kevin Knight. 2016. https://aclweb.org/anthology/D16-1163 Transfer learning for low-resource neural machine translation . In Proceedings of EMNLP 2016

    2016

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.