Pith. sign in

REVIEW 4 major objections 6 minor 43 references

Adversarial Bootstrapping for Dialogue Model Training

T0 review · 4 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Adversarial bootstrapping trains a dialogue generator on its own sampled replies, weighting the loss by an adversarial critic's score, to produce more relevant and diverse responses than teacher forcing alone.

desk verdict A plausible new training recipe for open-domain dialogue whose key new ingredient—discriminator bootstrapping—is under-supported and possibly unstable; the state-of-the-art claim outruns the evidence. read the letter →

arxiv 1909.00925 v2 pith:VUSZRSBK submitted 2019-09-03 cs.CL cs.LG

Oluwatobi Olabiyi , Erik T. Mueller , Christopher Larson , Tarek Lahlou This is my paper · ORCID
classification cs.CLcs.LG
keywords adversarialbootstrappingdialogueresponsegenerationexposurebiasmulti-turnpolicygradienttrainingdiscriminatorcalibrationdiversityHRED
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Neural dialogue models trained by maximum likelihood with teacher forcing tend to produce short, generic, and repetitive responses, because they are never trained on their own outputs and the data is full of high-frequency dull replies. This paper tries to fix both problems with adversarial bootstrapping: the generator is trained in autoregressive mode as well as teacher-forcing mode, and its maximum-likelihood loss on its own outputs is weighted by the score of an adversarially trained discriminator. The discriminator is in turn bootstrapped by matching the intermediate representations of the generated response to those of the ground truth, a cosine-similarity target meant to calibrate its scores. The authors report that the resulting models outperform existing multi-turn dialogue systems on the Movie and Ubuntu corpora on both automatic metrics (BLEU, ROUGE, distinct n-grams) and human preference judgments. If this holds, the method offers a way to train open-domain chatbots that are more informative without a separate diversity-promoting objective.

What carries the argument

The central object is the adversarial bootstrapping objective, a weighted maximum-likelihood target in which the generator's loss on its own autoregressive outputs is scaled by a discriminator score. For the generator, the target $t_G(y_i)$ equals $\beta$ for the ground-truth response, $0$ for the argmax output, and $\alpha Q_{\theta_D}(y_i, x_i)$ for other sampled responses. For the discriminator, the target is $t_D(y_i) = S(h_D(y_i), h_D(x_{i+1}))$, the cosine similarity between the discriminator's intermediate representations of a sampled response and the ground truth. A shared hierarchical recurrent encoder gives both models multi-turn context, and top-k categorical or uniform sampling combined with REINFORCE or a deterministic policy gradient provides the exploration strategy. This combination is what the paper claims carries the improvement: exposure bias is reduced by training on autoregressive outputs, and the calibrated discriminator suppresses bland responses without collapsing diversity.

What would settle it

Keep the entire training setup identical but replace the discriminator's bootstrapping target $t_D(y_i) = S(h_D(y_i), h_D(x_{i+1}))$ with the cosine similarity between the generated response and a randomly chosen other response from the same batch; if the reported gains over the non-bootstrapped discriminator persist, the similarity-to-ground-truth term is not doing the claimed work. A second check: if discriminator scores under the bootstrapped target do not track human relevance ratings on held-out responses, the calibration story is unsupported.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that a dialogue generator can be trained to produce better responses by bootstrapping its maximum-likelihood objective with an adversarial critic. The generator is trained on three kinds of targets simultaneously: ground-truth responses (teacher forcing), its own most-likely output, and sampled outputs whose loss weight comes from the discriminator. The discriminator is trained to distinguish ground truth from teacher-forcing outputs and from distractors sampled from the training set, which pushes it to penalize generic high-frequency responses. The new ingredient is second-level bootstrapping of the discriminator: its target for generated samples is the cosine similarity between an intermediate-layer representation of the generated response and the corresponding representation of the ground truth, rather than a hard 0/1 label. The authors argue this calibration signal is what makes the discriminator's feedback reliable enough to improve relevance without sacrificing diversity, and they present state-of-the-art results across Movie and Ubuntu with word-level discrimination and top-k categorical sampling.

Load-bearing premise

The argument depends on the assumption that the cosine similarity between the discriminator's internal representation of a generated response and of the ground truth is a trustworthy calibration signal, one that improves the discriminator's scoring rather than being gameable by the generator.

Editorial extensions

If this is right

  • Training a dialogue generator on its own sampled outputs with a discriminator-weighted loss should reduce exposure bias, because the model learns to recover from its own decoding errors during training.
  • Including distractors sampled from the training set as negative examples should teach the discriminator to down-weight high-frequency generic responses, making the generator less likely to reproduce them.
  • The best configuration reported, word-level discrimination with top-k categorical sampling, should give the largest gains in both relevance and diversity among the variants tested.
  • Discriminator bootstrapping via feature similarity should improve the calibration of the critic's score, and removing it should degrade performance across all variants, as the ablation in Table 3 indicates.
  • On the Movie and Ubuntu corpora, the method should outperform the (V)HRED, hredGAN, DAIM, and Transformer baselines on both automatic metrics and human preference.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The cosine-similarity calibration target is not tied to dialogue, so the same double-bootstrap recipe could be tested on other autoregressive sequence generators, such as machine translation or summarization models, where exposure bias also matters.
  • The top-k categorical sampling strategy could be read as a general variance-reduction technique for policy-gradient text generation; one could test it in isolation on a non-dialogue language modeling task.
  • The paper's positional-entropy observation suggests a testable extension: a curriculum that upsamples mid-sequence tokens or reweights loss by positional entropy might yield similar diversity gains without an adversarial component.
  • If the discriminator's intermediate-feature similarity is the active ingredient, one would predict that deeper-layer similarities correlate with human relevance judgments; that correlation is not reported and could be measured directly.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. This paper proposes adversarial bootstrapping (aBoots) for training multi-turn dialogue response generators. The generator is trained with a maximum-likelihood objective in teacher-forcing mode and an additional maximum-likelihood term on autoregressive outputs, weighted by a discriminator score Q (Eqs. 1-2). The discriminator is trained with ground-truth responses, the generator's argmax teacher-forcing outputs, and dataset distractors as negative examples (Eqs. 3-4); it is further bootstrapped by replacing the target for generator samples with the cosine similarity between intermediate discriminator representations of the generated response and the ground truth (Eq. 5). Several variants are explored: word- vs utterance-level discrimination and uniform, categorical, or Gaussian-noise sampling strategies. Experiments on the Movie and Ubuntu datasets report BLEU, ROUGE, distinct n-gram, and normalized average sequence length, plus pairwise human preference judgments over 200 samples, and the paper claims state-of-the-art results, with the categorical top-k word-level variant (aBoots w cat) performing best.

Significance. The empirical results, if reproducible, are impressive in magnitude: aBoots w cat improves BLEU from 0.0613 to 0.0940 and DIST-1 from 0.0179 to 0.0613 over hredGAN w on Movie, with large ROUGE and diversity gains on Ubuntu as well. The paper also carefully compares several design choices (word vs utterance discrimination, sampling strategies) and provides an ablation of the discriminator bootstrapping term. However, the significance is curtailed by missing error bars and significance tests on all automatic metrics, an under-powered human evaluation in which the key comparison against hredGAN w is not statistically significant, and an ablation for the paper's main novel component that is mixed and unreplicated. The self-referential nature of Eq. (5) also needs a theoretical or empirical justification before the component can be credited.

major comments (4)
  1. [Discriminator Bootstrapping, Eq. (5)] The target tD(yi) for a generated sample is the cosine similarity between the discriminator's own intermediate representations hD(yi) and hD(xi+1). Because hD is exactly the function optimized by Eq. (4), this target is not a fixed label: unless a stop-gradient/detachment is applied (which is not stated), the discriminator can reduce its cross-entropy loss by altering hD so that generated and ground-truth representations become similar, independently of true response relevance. The paper provides no gradient analysis, no fixed-point or equilibrium argument, and no layer-wise analysis of why this similarity is meaningful. Since Eq. (5) is the main new ingredient separating the method from prior GAN dialogue work, this is a load-bearing gap. The authors should either detach the target, analyze the resulting gradient, or supply controlled experiments showing the bootstrapped discriminator is better calibrated.
  2. [Table 3 vs Table 1] The ablation supporting discriminator bootstrapping is not statistically convincing. For the best variant aBoots w cat on Movie, adding Eq. (5) changes BLEU from 0.0928 to 0.0940, leaves DIST-1 unchanged at 0.0613, and decreases ROUGE from 0.4029 to 0.3973; on Ubuntu it improves BLEU from 0.0202 to 0.0233 but decreases ROUGE from 0.2343 to 0.2292. No error bars, significance tests, or multiple runs are reported. These mixed, small deltas do not establish that discriminator bootstrapping helps to better calibrate the discriminator, and without this component the novelty-dependent part of the state-of-the-art claim is weakened.
  3. [Table 2 and Table 1] The human evaluation uses only 200 samples and the authors themselves note that the comparison with hredGAN w is not significant (0.550 vs 0.450 on Movie, 0.556 vs 0.444 on Ubuntu). Combined with the absence of error bars and significance tests for all automatic metrics in Table 1, the paper's statement of state-of-the-art results with respect to human evaluations overstates the evidence. At minimum, the automatic metric comparisons should be accompanied by variance estimates or significance tests.
  4. [Sampling Strategy, Eq. (6)] The REINFORCE estimator in Eq. (6) is written as an expectation under pθG, but the categorical and uniform top-k strategies described in the same section sample from truncated distributions over the top k outputs. Unless an importance-weighting correction pθG/q is included, Eq. (6) is not an unbiased estimator of the gradient of the loss in Eq. (1) under the actual sampling distribution. The paper should specify the proposal distribution and the exact estimator used for aBoots cat and aBoots uni, or state that the gradient is taken with respect to the truncated policy.
minor comments (6)
  1. [Abstract] The abstract spells the metrics as BLUE and ROGUE; these should be BLEU and ROUGE.
  2. [Training section] The sentence about top k is unclear and appears self-contradictory: the authors say they used a single top k value during training but also say they searched for the optimum top k on the validation set using BLEU. Please clarify how the validation search was performed and whether models were retrained for each top k.
  3. [Eq. (13)] The notation uses sigma for both the logit projection and the sigmoid activation, which is confusing; please use distinct symbols for the projection and the sigmoid function.
  4. [Figure 1] Figure 1 is referenced in the text but the caption is minimal; please explain what is plotted and what conclusion the reader should draw from the positional entropy plot.
  5. [References] Some references (e.g., Graves et al. 2016; Graves, Wayne, and Danihelka 2014; Zhang et al. 2018a) do not appear to be cited in the body; please add citations or remove them.
  6. [Author affiliation] The affiliation line contains a typo: Vienna, V A should be Vienna, VA.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the training objective and the evaluation metrics are distinct, and the claimed gains rest on held-out test data.

full rationale

The paper's central claim is that adversarial bootstrapping improves response relevance and diversity, and this is tested on held-out test splits of Movie and Ubuntu with automatic metrics (BLEU, ROUGE, distinct n-gram) and human pairwise preferences that are not part of any training objective. The only data-dependent hyperparameter, top k, is selected on the validation set using BLEU, which is standard practice and does not force the test-set result. The discriminator-bootstrapping target in Eq. (5), tD(yi) = S(hD(yi), hD(xi+1)), is self-referential in the sense that the label depends on the discriminator's own features, so a poorly regularized discriminator could trivially lower its cross-entropy by collapsing representations; however, the paper does not derive the claimed calibration benefit from Eq. (5) by definition, but instead offers an empirical ablation (Table 3). That ablation is unreplicated and shows mixed deltas, which is a legitimate robustness or correctness concern but not circularity. Self-citations to Olabiyi et al. (2018, 2019) are used as baselines and background, not as load-bearing justification for the new method; the comparison against hredGAN is an external falsifiable benchmark. No equation or fitted parameter is renamed as a prediction, and no uniqueness theorem is imported. Accordingly, no circular step can be exhibited with the required quote-and-reduction evidence.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The method relies on several design assumptions that are introduced explicitly in this paper, most notably the cosine-similarity bootstrapping of the discriminator and the zero-target for argmax outputs. The only data-dependent free parameter is top k, while α, β, and τ are hand-set without sensitivity analysis. No new physical or conceptual entities are posited.

free parameters (4)
  • top k = 10 (selected from 1-20 on validation via BLEU score)
    Number of generator outputs considered in stochastic sampling; tuned on validation, so it is data-dependent.
  • alpha (α) = 1
    Weight on the discriminator score for non-truth, non-argmax generator outputs in Eq. (2); set by hand with no sensitivity analysis.
  • beta (β) = 1
    Weight on the ground-truth target in Eqs. (2) and (3); set by hand with no sensitivity analysis.
  • temperature tau (τ) = 1
    Softmax temperature in Eq. (11); set by hand with no sensitivity analysis.
assumptions (5)
  • standard math REINFORCE gradient estimator (Eq. 6) is unbiased given the policy.
    Invoked in the Sampling Strategy section to justify the stochastic policy gradient update.
  • domain assumption The Movie and Ubuntu datasets are representative of open-domain dialogue, and BLEU, ROUGE, and distinct n-gram scores reflect response quality as judged by humans.
    Used in the Experiments section to interpret the automatic evaluation results.
  • ad hoc to paper Cosine similarity between the discriminator's intermediate representations of a generated response and the ground truth is a valid calibration signal for the discriminator.
    Introduced in Eq. (5) and in the Discriminator Bootstrapping paragraph; this is the core new assumption of the method.
  • domain assumption Adversarial bootstrapping converges under simultaneous generator and discriminator updates with the stated hyperparameters.
    Assumed in the Training section; no convergence analysis is provided.
  • ad hoc to paper Assigning a target of zero to the generator's argmax output and to dataset distractors in the discriminator is beneficial.
    Defined in Eqs. (2) and (3); this design choice is not independently justified beyond the reported experiments.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Adversarial Bootstrapping for Dialogue Model Training." pith.science (2026). https://pith.science/paper/VUSZRSBK

@misc{pith2026190900925,
  author       = {Pith},
  title        = {Pith review of: Adversarial Bootstrapping for Dialogue Model Training},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VUSZRSBK}},
  note         = {Machine review of arXiv:1909.00925}
}
read the original abstract

Open domain neural dialogue models, despite their successes, are known to produce responses that lack relevance, diversity, and in many cases coherence. These shortcomings stem from the limited ability of common training objectives to directly express these properties as well as their interplay with training datasets and model architectures. Toward addressing these problems, this paper proposes bootstrapping a dialogue response generator with an adversarially trained discriminator. The method involves training a neural generator in both autoregressive and traditional teacher-forcing modes, with the maximum likelihood loss of the auto-regressive outputs weighted by the score from a metric-based discriminator model. The discriminator input is a mixture of ground truth labels, the teacher-forcing outputs of the generator, and distractors sampled from the dataset, thereby allowing for richer feedback on the autoregressive outputs of the generator. To improve the calibration of the discriminator output, we also bootstrap the discriminator with the matching of the intermediate features of the ground truth and the generator's autoregressive output. We explore different sampling and adversarial policy optimization strategies during training in order to understand how to encourage response diversity without sacrificing relevance. Our experiments shows that adversarial bootstrapping is effective at addressing exposure bias, leading to improvement in response relevance and coherence. The improvement is demonstrated with the state-of-the-art results on the Movie and Ubuntu dialogue datasets with respect to human evaluations and BLUE, ROGUE, and distinct n-gram scores.

Figures

Figures reproduced from arXiv: 1909.00925 by the authors.

Figure 1
Figure 1. Positional Entropy of Movie and Ubuntu datasets - Applying a greedy training objective to the datasets can achieve low overall entropy just by overfitting to low entropy regions, resulting in short and generic re￾sponses. because autoregressive sampling leads to unstable training. Unfortunately, the use of MLE is incongruent with the re￾dundant nature of dialogue datasets, exacerbates the expo￾sure bias problem in d… view at source ↗
Figure 2
Figure 2. A multi-turn recurrent architecture with adversarial bootstrapping: The generator and discriminator share the same encoder (through the context state) and the same word embeddings. The generator also uses the word embeddings as the output projection weights. The encoder and the discriminator RNNs are bidirectional while the context and generator RNNs are unidirectional. Generator The generator, denoted gRNN(·), is a… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

43 extracted references · 34 canonical work pages

  1. [1]

    Bahdanau, D.; Cho, K.; and Bengio, Y. 2015. Neural machine translation by jointly learning to align and translate. In Proceedings of International Conference of Learning Representation (ICLR 2015)

    work page 2015

  2. [2]

    Banchs, R. E. 2012. Movie-dic: A movie dialogue corpus for research and development. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics , 203–--207

    work page 2012

  3. [3]

    D.; Li, W.; Song, Y.; and Bengio, Y

    Che, T.; Li, Y.; Zhang, R.; Hjelm, R. D.; Li, W.; Song, Y.; and Bengio, Y. 2017. Maximum-likelihood augmented discrete generative adversarial networks. In arXiv preprint arXiv:1702.07983

    arXiv 2017

  4. [4]

    Glorot, X., and Bengio, Y. 2010. Understanding the difficulty of training deep feedforward neural networks. In International conference on artificial intelligence and statistics

    work page 2010

  5. [5]

    Glynn, P. W. 1990. Likelihood ratio gradient estimation for stochastic systems. Communications of the ACM 33(10):75–--84

    work page 1990

  6. [6]

    Grandvalet, Y., and Bengio, Y. 2005. Semi-supervised learning by entropy minimization. In NIPS , 529–--536

    work page 2005

  7. [7]

    Grandvalet, Y., and Bengio, Y. 2006. 9 entropy regularization. mitpress: 10.7551/mitpress/9780262033589.003.0009

    arXiv 2006

  8. [8]

    G.; Grefenstette, E.; Ramalho, T.; Agapiou, J.; Badia, A

    Graves, A.; Wayne, G.; Reynolds, M.; Harley, T.; Danihelka, I.; Grabska-Barwińska, A.; Colmenarejo, S. G.; Grefenstette, E.; Ramalho, T.; Agapiou, J.; Badia, A. P.; Hermann, K. M.; Zwols, Y.; Ostrovski, G.; Cain, A.; King, H.; Summerfield, C.; Blunsom, P.; Kavukcuoglu, K.; and Hassabis, D. 2016. Hybrid computing using a neural network with dynamic externa...

    work page 2016

Show all 43 references
  1. [9]

    Graves, A.; Wayne, G.; and Danihelka, I. 2014. Neural turing machines. In arXiv preprint arXiv:1410.5401, 2014

    2014 arXiv

  2. [10]

    Jean, S.; Cho, K.; Memisevic, R.; and Bengio, Y. 2015. On using very large target vocabulary for neural machine translation. In arXiv preprint arXiv:1412.2007

    2015 arXiv

  3. [11]

    Kulis, B. 2013. Metric learning: A survey. Foundations and Trends in Machine Learning 5(4):287–--364

    2013

  4. [12]

    Lamb, A.; Goyah, A.; Zhang, Y.; Zhang, S.; Courville, A.; and Bengio, Y. 2016. Professor forcing: A new algorithm for training recurrent networks. In Proceedings of Advances in Neural Information Processing Systems (NIPS 2016)

    2016

  5. [13]

    Li, J.; Galley, M.; Brockett, C.; Gao, J.; and Dolan, B. 2016a. A diversity-promoting objective function for neural conversation models. In Proceedings of NAACL-HLT

  6. [14]

    Li, J.; Monroe, W.; Ritter, A.; Galley, M.; Gao, J.; and Jurafsky, D. 2016b. Deep reinforcement learning for dialogue generation. In arXiv preprint arXiv:1606.01541v4

    arXiv

  7. [15]

    Li, J.; Monroe, W.; Shi, T.; Ritter, A.; and Jurafsky, D. 2017. Adversarial learning for neural dialogue generation. In arXiv preprint arXiv:1701.06547

    2017 arXiv

  8. [16]

    Lin, C. Y. 2014. Rouge: a package for automatic evaluation of summaries. In Proceedings of the Workshop on Text Summarization Branches Out

    2014

  9. [17]

    Lowe, R.; Pow, N.; Serban, I.; and Pineau, J. 2015. The ubuntu dialogue corpus: A large dataset for research in unstructured multi-turn dialogue systems. In SIGDIAL

    2015

  10. [18]

    T.; Sutskever, I.; Le, Q

    Luong, M. T.; Sutskever, I.; Le, Q. V.; Vinyals, O.; and Zaremba, W. 2015. Addressing the rare word problem in neural machine translation. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics

    2015

  11. [19]

    Nakamura, R.; Sudoh, K.; Yoshino, K.; and Nakamura, S. 2019. Another diversity-promoting objective function for neural dialogue generation. In AAAI Workshop on Reasoning and Learning for Human-Machine Dialogues (DEEP-DIAL)

    2019

  12. [20]

    Olabiyi, O.; Salimov, A.; Khazane, A.; and Mueller, E. 2018. Multi-turn dialogue response generation in an adversarial learning framework. In arXiv preprint arXiv:1805.11752

    2018 arXiv

  13. [21]

    Olabiyi, O.; Khazan, A.; Salimov, A.; and Mueller, E. 2019. An adversarial learning framework for a persona-based multi-turn dialogue model. In NAACL NeuralGen Workshop

    2019

  14. [22]

    Olabiyi, O.; Khazan, A.; and Mueller, E. 2018. An adversarial learning framework for a persona-based multi-turn dialogue model. In 17th IEEE International Conference on Machine Learning and Applications (ICMLA)

    2018

  15. [23]

    Papineni, K.; Roukos, S.; Ward, T.; and Zhu, W. 2002. Bleu: A method for automatic evalution of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics , 311–--318

    2002

  16. [24]

    Radford, A.; Wu, J.; Child, R.; Luan, D.; Amodei, D.; and Sutskever, I. 2019. Language models are unsupervised multitask learners. In https://d4mucfpksywv.cloudfront.net/better-language-models

    2019

  17. [25]

    Reed, S.; Lee, H.; Anguelov, D.; Szegedy, C.; Erhan, D.; and Rabinovich, A. 2015. Training deep neural networks on noisy labels with bootstrapping. In ICLR

    2015

  18. [26]

    Serban, I.; Sordoni, A.; Bengio, Y.; Courville, A.; and Pineau, J. 2016. Building end-to-end dialogue systems using generative hierarchical neural network models. In Proceedings of The Thirtieth AAAI Conference on Artificial Intelligence (AAAI 2016) , 3776--3784

    2016

  19. [27]

    V.; Klinger, T.; Tesauro, G.; Talamadupula, K.; Zhou, B.; Bengio, Y.; and Courville, A

    Serban, I. V.; Klinger, T.; Tesauro, G.; Talamadupula, K.; Zhou, B.; Bengio, Y.; and Courville, A. 2017a. Multiresolution recurrent neural networks: An application to dialogue response generation. In Proceedings of The Thirty-first AAAI Conference on Artificial Intelligence (A...

    2017

  20. [28]

    V.; Sordoni, A.; Lowe, R.; Charlin, L.; Pineau, J.; Courville, A.; and Bengio, Y

    Serban, I. V.; Sordoni, A.; Lowe, R.; Charlin, L.; Pineau, J.; Courville, A.; and Bengio, Y. 2017b. A hierarchical latent variable encoder-decoder model for generating dialogue. In Proceedings of The Thirty-first AAAI Conference on Artificial Intelligence (AAAI 2017)

    2017

  21. [29]

    Shao, L.; Gouws, S.; Britz, D.; Goldie, A.; Strope, B.; and Kurzweil, R. 2017. Generating long and diverse responses with neural conversational models. In Proceedings of International Conference of Learning Representation (ICLR)

    2017

  22. [30]

    Sharath, T.; Tandon, S.; and Bauer, R. 2017. A dual encoder sequence to sequence model for open-domain dialogue modeling. In arXiv preprint arXiv:1710.10520

    2017 arXiv

  23. [31]

    Silver, D.; Lever, G.; Heess, N.; Degris, T.; Wierstra, D.; and Riedmiller, M. 2014. Deterministic policy gradient algorithms. In ICML

    2014

  24. [32]

    Sutskever, I.; Vinyals, O.; and Le, Q. 2014. Sequence to sequence learning with neural networks. In Proceedings of Advances in Neural Information Processing Systems (NIPS) , 3104–--3112

    2014

  25. [33]

    N.; Kaiser, L.; and Polosukhin, I

    Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, L.; and Polosukhin, I. 2017. Attention is all you need. In NIPS

    2017

  26. [34]

    Vinyals, O., and Le, Q. 2015. A neural conversational model. In Proceedings of ICML Deep Learning Workshop

    2015

  27. [35]

    J., and Zipser, D

    Williams, R. J., and Zipser, D. 1989. A learning algorithm for continually running fully recurrent neural networks. Neural computation 1(2):270–--280

    1989

  28. [36]

    Williams, R. J. 1992. Simple statistical gradientfollowing algorithms for connectionist reinforcement learning. Machine learning 8(3-4):229–--256

    1992

  29. [37]

    Xing, C.; Wu, W.; Wu, Y.; Zhou, M.; Huang, Y.; and Ma, W. 2017. Hierarchical recurrent attention network for response generation. In arXiv preprint arXiv:1701.07149

    2017 arXiv

  30. [38]

    Xu, Z.; Liu, B.; Wang, B.; Chengjie, S.; Wang, X.; Wang, Z.; and Qi, C. 2017. Neural response generation via gan with an approximate embedding layer. In EMNLP

    2017

  31. [39]

    Yu, L.; Zhang, W.; Wang, J.; and Yu, Y. 2017. Seqgan: sequence generative adversarial nets with policy gradient. In Proceedings of The Thirty-first AAAI Conference on Artificial Intelligence (AAAI 2017)

    2017

  32. [40]

    Zhang, Y.; Gan, Z.; Fan, K.; Chen, Z.; Henao, R.; Shen, D.; and Carin, L. 2017. Adversarial feature matching for text generation. In arXiv preprint arXiv:1706.03850

    2017 arXiv

  33. [41]

    Zhang, S.; Dinan, E.; Urbanek, J.; Szlam, A.; Kiela, D.; and Weston, J. 2018a. Personalizing dialogue agents: I have a dog, do you have pets too? In arXiv preprint arXiv:1801.07243v3

    arXiv

  34. [42]

    Zhang, Y.; Galley, M.; Gao, J.; Gan, Z.; Li, X.; Brockett, C.; and Dolan, B. 2018b. Generating informative and diverse conversational responses via adversarial information maximization. In NeurIPS

  35. [43]

    write newline

    " write newline "" before.all 'output.state := FUNCTION fin.entry add.period write newline FUNCTION new.block output.state before.all = 'skip after.block 'output.state := if FUNCTION new.sentence output.state after.block = 'skip output.state before.all = 'skip after.sentence '...

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.