Pith. sign in

REVIEW 4 major objections 5 minor 37 references

Controllable Dual Skew Divergence Loss for Neural Machine Translation

T0 review · 4 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A dual skew divergence loss with a proportionally controlled balance weight lifts BLEU on WMT14 translation tasks when applied after maximum-likelihood convergence.

desk verdict Useful drop-in training loss with modest, consistent gains, but the 'general and convenient' claim rests on unvalidated controller constants and no code release; worth refereeing to force the missing sensitivity analysis. read the letter →

arxiv 1908.08399 v2 pith:4OIBITMF submitted 2019-08-22 cs.CL

Zuchao Li , Hai Zhao , Yingting Wu , Fengshun Xiao , Shu Jiang This is my paper · ORCID
classification cs.CL
keywords dualskewdivergencecontrollableoptimizationneuralmachinetranslationlossfunctionKullback-LeiblerPIcontrollersequence-to-sequenceBLEU
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes a new training loss for neural machine translation, the dual skew divergence (DSD), which combines two skew divergences: one that drives the model toward the true data distribution and one that trusts the model's own predicted distribution, blended by a weight β. The authors find that a fixed β works for shallow RNN models but destabilizes deep CNN and Transformer models, so they introduce a controllable version (cDSD) that adapts β at each training step using a proportional-integral controller, keeping the forward skew divergence close to a preset target value. Switching from cross-entropy to DSD/cDSD after maximum-likelihood convergence is claimed to act like simulated annealing, letting the model escape local optima. On WMT 2014 English-German and English-French tasks, the loss improves BLEU over strong baselines across all three architecture families, with reported gains of roughly 0.2 to 1.1 BLEU. The paper aims to establish that this loss is a general and convenient drop-in replacement for the final phase of NMT training.

What carries the argument

The machinery is the dual skew divergence loss D_DS = β s_α(Q,P) + (1-β) s_α(P,Q), where each skew divergence s_α(P,Q)=D_KL(P||αP+(1-α)Q) is a KL divergence with the argument smoothed by α. The forward term s_α(Q,P) approximates the standard ML objective, while the reverse term s_α(P,Q) weights loss by the model's own distribution; the balanced weight β interpolates between them. In the controllable variant, β(t) is updated by a nonlinear PI controller, β(t)=min(β_max, K_p/(1+exp(e(t))) - K_i Σ_{j=0}^t e(j) + β_min), with error e(t)=u(t)-u*, so the divergence term stays near the set point. This mechanism gives the loss its claimed stability on deep models.

What would settle it

Run a controlled sweep over the set point u* (for example, values 25, 30, 35, 40) and over Kp and Ki (for example, Kp from 0.001 to 0.1 and Ki from 1e-5 to 1e-3) on the WMT14 English-German Transformer-big task; if the BLEU improvement over the cross-entropy baseline is not stable across a reasonable range of these values, or if the best performance requires per-task tuning, the claim that cDSD is a general and convenient loss is falsified.

Watch

Extended reading notes

Core claim

The central claim is that the standard cross-entropy objective for NMT, equivalent to minimizing KL(Q||P) where Q is the data distribution and P the model distribution, is one-sided: it forces the model to cover all training data, including noise, and ignores its own prediction errors. The paper argues that adding the reverse direction KL(P||Q) through skew divergences produces a more balanced objective. The DSD loss is defined as D_DS = β s_α(Q,P) + (1-β) s_α(P,Q), with s_α(P,Q)=D_KL(P||αP+(1-α)Q) and α=0.01. For deep models, a fixed β is unstable, so the authors propose cDSD, in which β(t) is regulated by a PI controller to keep s_α(Q,P) near a set point u* defined by the divergence reached under ML training with label smoothing. The empirical claim is that after ML convergence, switching to DSD or cDSD consistently raises BLEU on WMT14 En-De and En-Fr across RNN, CNN, and Transformer models, with the improvement largest at beam width 1 and diminishing but present with larger beams.

Load-bearing premise

The cDSD controller assumes that the set point u*, taken from the divergence value of ML training with label smoothing, is the right target for the forward skew divergence, and that the controller constants Kp=0.01, Ki=0.0001 and α=0.01 work across tasks without sensitivity tuning; if that assumption fails, the stable gains on deep models disappear and the general claim reduces to a tuned special case.

Editorial extensions

If this is right

  • If the claim holds, any sequence-to-sequence model trained with cross entropy can, after convergence, be fine-tuned with DSD/cDSD without changing the model architecture, inference procedure, or data pipeline, yielding free BLEU gains.
  • The effect is strongest for greedy search (beam width 1) and shrinks as beam size increases, implying the loss improves top-1 prediction quality rather than sequence-level reranking.
  • The gains persist on top of back-translation data augmentation, indicating the loss improves the model's generalization rather than compensating for data quantity.
  • An optimizer-switch baseline (Adam to SGD with unchanged cross-entropy loss) produces only marginal improvement, so the reported gains are attributed to the loss itself, not the training schedule.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension would be to test whether the same two-directional divergence principle applies to other sequence prediction tasks, such as speech recognition or abstractive summarization, where overgeneralization from noisy targets is also an issue; the paper does not report such experiments.
  • The set point u* is defined as the divergence value of ML with label smoothing; if this value is task-dependent and must be tuned per dataset, the 'convenient' nature of the loss is weaker than claimed. A systematic sensitivity test over u* across language pairs would clarify this.
  • The PI controller constants Kp=0.01 and Ki=0.0001 are inherited from a different domain; the paper provides no ablation showing they are near-optimal for NMT, so a reader might reasonably expect that tuned constants could yield larger gains or that untuned constants could fail on other architectures.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a dual skew divergence (DSD) loss for NMT, formed by interpolating two skew divergences sα(Q,P) and sα(P,Q) with a balanced weight β, and then a controllable version (cDSD) in which β(t) is adjusted online by a PI controller to keep sα(Q,P) near a set point u*. Training is hybrid: standard cross-entropy training first, then switching to DSD or cDSD. Experiments on WMT14 English-German and English-French with RNN, CNN, and Transformer models report BLEU gains over the corresponding cross-entropy baselines, with sign-test significance markers and additional ablation studies including back-translation and optimizer-switch finetuning.

Significance. If the reported gains are robust, the work offers a simple, mostly drop-in loss modification that improves strong NMT baselines, which is practically useful. The paper is transparent about its hybrid training protocol and includes several control experiments (back-translation, OSF) that help attribute the gains to the loss rather than to optimizer switching. The algebraic derivation from the skew divergence to the implementable loss is straightforward. The main limitation is that the cDSD controller is not validated: the set point u*, PI constants Kp and Ki, and β bounds are fixed without sensitivity analysis, and no measurement of the controlled divergence u(t) is reported. Since fixed DSD is worse than baseline on CNN and Transformer models, the cDSD-specific gains are the load-bearing evidence for the 'general and convenient' claim, and that evidence is currently incomplete.

major comments (4)
  1. [§4.2, Tables 2–5] The cDSD controller's set point u* is defined as the divergence value that ML with label smoothing reaches, but no measurement of this value is reported; the numbers 35 and 33 for the two language pairs are introduced without derivation. The PI constants Kp=0.01 and Ki=0.0001 are taken from ControlVAE without verifying their scale for NMT loss units. Because Table 2 shows fixed DSD below the baseline on CNN and Transformer models (22.60 vs 22.64 and 24.55 vs 24.76), the claimed cDSD improvements depend entirely on the controller keeping sα(Q,P) near u*. Please report the measured u(t) trajectory, verify that u* lies between the stated upper and lower bounds, and provide a sensitivity analysis over u*, Kp, Ki, βmin, and βmax. Without this, the 0.2–0.3 BLEU deep-model gains cannot be distinguished from a carefully tuned special case.
  2. [§3.5–§3.6] The paper's 'adversarial-like' interpretation of the two DSD terms is not supported by the loss geometry. For one-hot Q, both sα(Q,P) and sα(P,Q) are minimized when P concentrates its mass on the correct token; the second term penalizes probability on incorrect tokens rather than pulling against the first term. Please demonstrate concretely where the conflict between the two terms arises, e.g., with label-smoothed Q or with non-one-hot targets, or revise the motivation accordingly. This matters because the balanced weight and the PI controller are motivated by an assumed conflict between the two terms.
  3. [§4.6, Tables 3–6] The headline BLEU numbers appear to come from a single training run per configuration. The sign test is a per-sentence test and does not account for seed or initialization variance. With reported gains as small as 0.2–0.3 BLEU on the Transformer and CNN models, the paper should report multiple seeds (at least three) or otherwise bound run-to-run variance to support the 'stable performance improvements' claim.
  4. [§4.3, Table 2 vs. Table 1] The paper does not reconcile the discrepancy between Table 1 (RNN dev baseline 20.51; DSD β=0 21.72; cDSD 21.96) and Table 2 (RNN dev baseline 20.11; DSD 21.32; cDSD 21.44) for what appear to be the same task, model family, and greedy search. If these are different model configurations or different training setups, please state this explicitly; as presented, the reader cannot tell which configuration produces the final test-set numbers in Tables 3 and 4.
minor comments (5)
  1. [§3.3] In the paragraph after the DKL formulas, the phrase 'minimizing the DKL(Q||P) divergence' appears twice; the second occurrence should refer to DKL(P||Q). Also, the summation index in the displayed formula for DKL(P||Q) is written as 'N∑ i=n' but should be 'n∑ i=1'.
  2. [§4.6] The text says 'sigh test results' where 'sign test results' is meant.
  3. [Table 2] Table 2 reports no significance markers, while Tables 3 and 4 use +/++ markers. Adding the same markers to Table 2 would make the dev-set comparisons easier to interpret.
  4. [§4.2] The notation for the initial value of β(t) is written as 'beta(0)' in prose rather than 'β(0)', and the controller formula uses βmax/βmin while the text sometimes uses beta_max/beta_min; please standardize the notation.
  5. [References] Reference [26] has a garbled author list ('Koehn, Philipp, Hoang, Hieu, Alexandra, CallisonBurch, Chris, Federico, and Marcello') that should be corrected.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the cDSD loss is evaluated against external BLEU benchmarks, and the set-point and controller constants are hyperparameter choices rather than fitted quantities presented as predictions.

full rationale

The central claim is an empirical claim about BLEU on WMT14 after switching from cross-entropy training to DSD/cDSD training, tested against external baselines and held-out test sets. The DSD loss is derived from standard KL and skew-divergence definitions (Eqs. 1 and 2) without using BLEU or any baseline output as an input to the loss formula. The cDSD controller regulates the sampled skew divergence toward a set point taken from ML with label smoothing; this is a hyperparameter choice described in Section 4.2, not a fitted quantity later presented as a prediction, and no reported BLEU score is used to fit the controller. The only self-citation, reference [20], is invoked contrastively to distinguish DSD from D2GPo and is not load-bearing for the main claim. The ControlVAE constants and the set-point values are borrowed or chosen empirically; the absence of a sensitivity analysis is a robustness concern, not circularity. Under the stated standard, there is no step where an input quantity is renamed as a derived result, so no circular step is flagged.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central contribution rests on the choice of alpha, beta, the PI controller constants, and the task-specific set point, all set empirically or borrowed from prior work. The mathematical derivation from KL divergence is standard, but the stabilization mechanism is asserted rather than derived.

free parameters (6)
  • skew divergence constant alpha = 0.01
    Hand-set in Section 3.4 with no ablation; it defines the skew approximation and is used in all DSD and cDSD experiments.
  • balanced weight beta = 0, 0.5, 1 tested; best RNN result uses beta=0
    Selected on the En-De development set in Table 1; the final experiments use DSD for RNN and cDSD for deeper models.
  • set point u* = 35 En-De, 33 En-Fr
    Target divergence for cDSD, computed from the ML with label smoothing loss; task-specific and not otherwise justified in Section 4.2.
  • PI controller constants Kp, Ki = 0.01, 0.0001
    Borrowed unchanged from ControlVAE [21] with no sensitivity analysis.
  • beta_min and beta_max = 0.85 and 0.95
    Application-specific bounds for the PI controller, stated in Section 4.2 without tuning details.
  • loss switching point = around 245K-313K steps for RNN; final epoch unclear
    The paper shows switching around ML convergence is best on dev (Figure 2), then uses this hybrid strategy without a precise rule.
assumptions (5)
  • standard math Minimizing cross entropy is equivalent to minimizing KL(Q||P) because the entropy of the training data is constant.
    Used in Section 3.2 to motivate DSD.
  • ad hoc to paper ML training with cross entropy overgeneralizes and falls into local optima; combining the two KL directions yields better generalization.
    Stated in the Introduction and Section 3.5 without proof; the simulated-annealing analogy is weakened by the footnote that switching back does not improve.
  • ad hoc to paper The skew divergence s_alpha(P,Q) with alpha=0.01 is a safe approximation to KL(P||Q) for NMT training.
    The paper sets alpha=0.01 in Section 3.4 and never ablates it.
  • ad hoc to paper The target divergence value of ML with label smoothing is an appropriate set point for stabilizing cDSD training.
    Section 4.2 states the set point is taken from ML with label smoothing; no independent justification or sensitivity study is given.
  • domain assumption The sign test over sentences is a valid significance test for BLEU differences.
    Used for the plus marks in Tables 3 and 4; assumes sentence-level BLEU differences are exchangeable.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Controllable Dual Skew Divergence Loss for Neural Machine Translation." pith.science (2026). https://pith.science/paper/4OIBITMF

@misc{pith2026190808399,
  author       = {Pith},
  title        = {Pith review of: Controllable Dual Skew Divergence Loss for Neural Machine Translation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4OIBITMF}},
  note         = {Machine review of arXiv:1908.08399}
}
read the original abstract

In sequence prediction tasks like neural machine translation, training with cross-entropy loss often leads to models that overgeneralize and plunge into local optima. In this paper, we propose an extended loss function called \emph{dual skew divergence} (DSD) that integrates two symmetric terms on KL divergences with a balanced weight. We empirically discovered that such a balanced weight plays a crucial role in applying the proposed DSD loss into deep models. Thus we eventually develop a controllable DSD loss for general-purpose scenarios. Our experiments indicate that switching to the DSD loss after the convergence of ML training helps models escape local optima and stimulates stable performance improvements. Our evaluations on the WMT 2014 English-German and English-French translation tasks demonstrate that the proposed loss as a general and convenient mean for NMT training indeed brings performance improvement in comparison to strong baselines.

Figures

Figures reproduced from arXiv: 1908.08399 by the authors.

Figure 1
Figure 1. (a) The derivation of our proposed DSD and cDSD loss function. (b) PI controller in cDSD loss. [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. BLEU scores and training loss on English-German dev set with [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. BLEU scores with different beam sizes on English-German dev [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

37 extracted references · 35 canonical work pages

  1. [1]

    Recurrent continuous translation models,

    N. Kalchbrenner and P . Blunsom, “Recurrent continuous translation models,” in Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, Seattle, Washington, USA, October 2013, pp. 1700–1709

    work page 2013

  2. [2]

    Sequence to sequence learning with neural networks,

    I. Sutskever, O. Vinyals, and Q. V . Le, “Sequence to sequence learning with neural networks,” in Advances in Neural Information Processing Systems, 2014, pp. 3104–3112

    work page 2014

  3. [3]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” in Advances in Neural Information Processing Systems , 2017, pp. 6000– 6010

    work page 2017

  4. [4]

    Statistical phrase-based translation,

    P . Koehn, F. J. Och, and D. Marcu, “Statistical phrase-based translation,” in Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology - Volume 1, Stroudsburg, PA, USA, 2003, pp. 48–54

    work page 2003

  5. [5]

    Neural machine translation by jointly learning to align and translate,

    D. Bahdanau, K. Cho, and Y. Bengio, “Neural machine translation by jointly learning to align and translate,” in Proceedings of 3rd International Conference on Learning Representations, 2015

    work page 2015

  6. [6]

    Effective approaches to attention-based neural machine translation,

    T. Luong, H. Pham, and C. D. Manning, “Effective approaches to attention-based neural machine translation,” in Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing , Lisbon, Portugal, September 2015, pp. 1412–1421

    work page 2015

  7. [7]

    Sequence level training with recurrent neural networks,

    M. Ranzato, S. Chopra, M. Auli, and W. Zaremba, “Sequence level training with recurrent neural networks,” in Proceedings of 4rd International Conference on Learning Representations, 2016

    work page 2016

  8. [8]

    Bleu: a method for automatic evaluation of machine translation,

    K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu, “Bleu: a method for automatic evaluation of machine translation,” in Proceedings of 40th Annual Meeting of the Association for Computational Linguistics , Philadelphia, Pennsylvania, USA, July 2002, pp. 311–318

    work page 2002

Show all 37 references
  1. [9]

    Scheduled sampling for sequence prediction with recurrent neural networks,

    S. Bengio, O. Vinyals, N. Jaitly, and N. Shazeer, “Scheduled sampling for sequence prediction with recurrent neural networks,” in Advances in Neural Information Processing Systems , 2015, pp. 1171– 1179

    2015

  2. [10]

    Sequence-to-sequence learning as beam-search optimization,

    S. Wiseman and A. M. Rush, “Sequence-to-sequence learning as beam-search optimization,” in Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing , Austin, Texas, November 2016, pp. 1296–1306. [Online]. Available: https://aclweb.org/anthology/D16-1137

    2016

  3. [11]

    Minimum risk training for neural machine translation,

    S. Shen, Y. Cheng, Z. He, W. He, H. Wu, M. Sun, and Y. Liu, “Minimum risk training for neural machine translation,” in Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics , Berlin, Germany, August 2016, pp. 1683–1692. [Online]. Available: ht...

    2016

  4. [12]

    An actor-critic algorithm for sequence prediction,

    D. Bahdanau, P . Brakel, K. Xu, A. Goyal, R. Lowe, J. Pineau, A. C. Courville, and Y. Bengio, “An actor-critic algorithm for sequence prediction,” in Proceedings of 5rd International Conference on Learning Representations, 2017

    2017

  5. [13]

    On information and sufficiency,

    S. Kullback and R. A. Leibler, “On information and sufficiency,” Ann. Math. Statist., vol. 22, no. 1, pp. 79–86, 03 1951

    1951

  6. [14]

    How (not) to train your generative model: Scheduled sampling, likelihood, adversary?

    F. Husz´ar, “How (not) to train your generative model: Scheduled sampling, likelihood, adversary?” arXiv e-print arXiv:1511.05101 , 2015

    2015 arXiv

  7. [15]

    Convolutional sequence to sequence learning,

    J. Gehring, M. Auli, D. Grangier, D. Yarats, and Y. N. Dauphin, “Convolutional sequence to sequence learning,” arXiv preprint arXiv:1705.03122, 2017. ARXIV , APRIL 2021 10

    2017 arXiv

  8. [16]

    Learning phrase representations using rnn encoder–decoder for statistical machine translation,

    K. Cho, B. van Merrienboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, and Y. Bengio, “Learning phrase representations using rnn encoder–decoder for statistical machine translation,” in Proceedings of the 2014 Conference on Empirical Methods in Natural Language Process...

    2014

  9. [17]

    Long short-term memory,

    S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural Comput., vol. 9, pp. 1735–1780, 1997

    1997

  10. [18]

    Measures of distributional similarity,

    L. Lee, “Measures of distributional similarity,” in Proceedings of the 37th Annual Meeting of the Association for Computational Linguistics , College Park, Maryland, USA, June 1999, pp. 25–32

    1999

  11. [19]

    An empirical study of smoothing techniques for language modeling,

    S. F. Chen and J. Goodman, “An empirical study of smoothing techniques for language modeling,” in 34th Annual Meeting of the Association for Computational Linguistics . Santa Cruz, California, USA: Association for Computational Linguistics, Jun. 1996, pp. 310–318. [Online]. Av...

    1996

  12. [20]

    Data-dependent gaussian prior objective for language generation,

    Z. Li, R. Wang, K. Chen, M. Utiyama, E. Sumita, Z. Zhang, and H. Zhao, “Data-dependent gaussian prior objective for language generation,” in International Conference on Learning Representations, 2020. [Online]. Available: https://openreview.net/ forum?id=S1efxTVYDr

    2020

  13. [21]

    Controlvae: Controllable variational autoencoder,

    H. Shao, S. Yao, D. Sun, A. Zhang, S. Liu, D. Liu, J. Wang, and T. Abdelzaher, “Controlvae: Controllable variational autoencoder,” in International Conference on Machine Learning. PMLR, 2020, pp. 8655–8664

    2020

  14. [22]

    K. J. ˚Astr¨om, T. H ¨agglund, and K. J. Astrom, Advanced PID control. ISA-The Instrumentation, Systems, and Automation Society Research Triangle Park, 2006, vol. 461

    2006

  15. [23]

    Generative adversarial nets,

    I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” Advances in Neural Information Processing Systems , vol. 27, pp. 2672– 2680, 2014

    2014

  16. [24]

    Clause restructuring for statistical machine translation,

    M. Collins, P . Koehn, and I. Kuˇcerov´a, “Clause restructuring for statistical machine translation,” in Proceedings of the 43rd annual meeting on association for computational linguistics . Association for Computational Linguistics, 2005, pp. 531–540

    2005

  17. [25]

    Neural machine translation of rare words with subword units,

    R. Sennrich, B. Haddow, and A. Birch, “Neural machine translation of rare words with subword units,” in Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, Berlin, Germany, August 2016, pp. 1715–1725

    2016

  18. [26]

    Moses: open source toolkit for statistical machine translation,

    Koehn, Philipp, Hoang, Hieu, Alexandra, CallisonBurch, Chris, Federico, and Marcello, “Moses: open source toolkit for statistical machine translation,” in Proceedings of the 45th Annual Meeting of the ACL on Interactive Poster and Demonstration Sessions , 2007, pp. 177–180

    2007

  19. [27]

    Minimum error rate training in statistical machine trans- lation,

    F. J. Och, “Minimum error rate training in statistical machine trans- lation,” in Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics, Sapporo, Japan, July 2003, pp. 160–167

    2003

  20. [28]

    Srilm — an extensible language modeling toolkit,

    A. Stolcke, “Srilm — an extensible language modeling toolkit,” in International Conference on Spoken Language Processing, 2002, pp. 901–904

    2002

  21. [29]

    On using very large target vocabulary for neural machine translation,

    S. Jean, K. Cho, R. Memisevic, and Y. Bengio, “On using very large target vocabulary for neural machine translation,” in Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Proce...

    2015

  22. [30]

    Six challenges for neural machine translation,

    P . Koehn and R. Knowles, “Six challenges for neural machine translation,” in Proceedings of the First Workshop on Neural Machine Translation . Vancouver: Association for Computational Linguistics, Aug. 2017, pp. 28–39. [Online]. Available: https://www.aclweb.org/anthology/W17-3204

    2017

  23. [31]

    Analyzing uncer- tainty in neural machine translation,

    M. Ott, M. Auli, D. Grangier, and M. Ranzato, “Analyzing uncer- tainty in neural machine translation,” in International Conference on Machine Learning. PMLR, 2018, pp. 3956–3965

    2018

  24. [32]

    Empirical analysis of beam search perfor- mance degradation in neural sequence models,

    E. Cohen and C. Beck, “Empirical analysis of beam search perfor- mance degradation in neural sequence models,” in International Conference on Machine Learning. PMLR, 2019, pp. 1290–1299

    2019

  25. [33]

    Classical structured prediction losses for sequence to sequence learning,

    S. Edunov, M. Ott, M. Auli, D. Grangier, and M. Ranzato, “Classical structured prediction losses for sequence to sequence learning,” in Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologie...

    2018

  26. [34]

    SMART: Robust and efficient fine-tuning for pre-trained natural language models through principled regularized optimization,

    H. Jiang, P . He, W. Chen, X. Liu, J. Gao, and T. Zhao, “SMART: Robust and efficient fine-tuning for pre-trained natural language models through principled regularized optimization,” in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Onli...

    2020

  27. [35]

    Better fine-tuning by reducing representational collapse,

    A. Aghajanyan, A. Shrivastava, A. Gupta, N. Goyal, L. Zettlemoyer, and S. Gupta, “Better fine-tuning by reducing representational collapse,” in International Conference on Learning Representations ,

  28. [36]

    Regularizing neural machine translation by target-bidirectional agreement,

    Z. Zhang, S. Wu, S. Liu, M. Li, M. Zhou, and T. Xu, “Regularizing neural machine translation by target-bidirectional agreement,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 33, no. 01, 2019, pp. 443–450

    2019

  29. [2021]

    Available: https://openreview.net/forum?id= OQ08SN70M1V

    [Online]. Available: https://openreview.net/forum?id= OQ08SN70M1V

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.