Ensemble-Based Deep Reinforcement Learning for Chatbots

Donghyeon Lee; Heriberto Cuay\'ahuitl; Hyungtak Choi; Inchul Hwang; Jihie Kim; Satish Indurthi; Seonghan Ryu; Seunghak Yu; Sungja Choi; Yongjin Cho

REVIEW 3 major objections 4 minor 55 references

Ensemble-Based Deep Reinforcement Learning for Chatbots

T0 review · 3 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read An ensemble of 100 reinforcement-learning chatbots beats a single agent at picking human-like chitchat responses.

desk verdict A credible ensemble-DQN chatbot result whose central comparison is only as strong as an unmeasured cluster-collision assumption. read the letter →

arxiv 1908.10422 v1 pith:RSSSNYII submitted 2019-08-27 cs.AI cs.CLcs.LG

Heriberto Cuayáhuitl , Donghyeon Lee , Seonghan Ryu , Yongjin Cho , Sungja Choi , Satish Indurthi , Seunghak Yu , Hyungtak Choi

show 2 more authors

Inchul Hwang Jihie Kim

This is my paper · ORCID

classification cs.AIcs.CLcs.LG

keywords chatbotsdeepreinforcementlearningensembledialogueclusteringsentencehuman-likenessrewardopen-domainvalue-based

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that open-domain chitchat can be trained with value-based deep reinforcement learning if the infinite space of possible responses is compressed into a finite set of meaning clusters, and that an ensemble of specialised agents is better than one generalist. Each agent in the ensemble is trained on its own cluster of similar dialogues, with actions defined by sentence clusters and a reward that is +1 for choosing the cluster containing the true next human response and -1 for choosing a randomly selected distractor. On 999 held-out chitchat dialogues, the 100-agent ensemble reports a dialogue reward of -2.8882 against -6.4800 for a single agent and -5.7000 for a sequence-to-sequence baseline, and human judges rate the ensemble higher on fluency, engagingness and consistency. The authors also claim their simple reward correlates strongly (Pearson r between 0.90 and 0.91) with human ratings, even though generalisation to unseen sentences remains a difficult problem.

What carries the argument

The load-bearing machinery is a two-level clustering pipeline plus a reward predictor. Sentence embeddings are averaged word vectors, and K-means++ with $K=100$ turns every possible response into one of 100 cluster IDs, making the action set finite for a Deep Q-Network (DQN). A second K-means++ pass clusters training dialogues into 100 subsets, and one DQN agent is trained per subset; each agent's Q-function is $Q^*(s,a;\theta_i)$. At test time, a neural regressor $\hat R(\tau_i)$ predicts each agent's dialogue reward for its trajectory and the agent with the highest prediction is selected, giving $\pi^*(s) = \arg\max_a Q^*(s,a;\theta_i)$ for $i = \arg\max_i \hat R(\tau_i)$. The reward at each turn is +1 if the chosen action is the cluster containing the true human response and -1 if it is a randomly chosen distractor.

What would settle it

On the 999 held-out dialogues with $K=100$ clusters, count how often the true next response and a randomly drawn distractor share the same cluster ID; a high collision rate would make the reward too noisy for the Q-values to be meaningful and would put the reported ensemble-versus-single gap in doubt. A second check is to train the same single-agent architecture on all 17,877 dialogues with the exact same actions and reward: if it matches the ensemble's score, the ensemble advantage disappears.

Watch

Extended reading notes

Core claim

The paper's central claim is that training an ensemble of value-based deep reinforcement learning chatbots is essential for improved performance over a single agent. The claim is established by deriving a finite action set automatically through sentence clustering, partitioning the training dialogues into clusters so that each agent becomes specialised in a particular conversational style, and selecting, at test time, the agent with the highest predicted dialogue reward. In the reported experiments, the ensemble outperforms both a single DQN-based agent and a Seq2Seq model on automatic metrics and in a human evaluation, and the proposed +1/-1 human-likeness reward correlates with human judgements.

Load-bearing premise

The training signal assumes that the true human response and randomly selected distractors rarely fall into the same response cluster, so that +1 and -1 rewards actually distinguish good from bad choices.

Editorial extensions

If this is right

Value-based reinforcement learning becomes a practical option for open-domain chatbots, because clustering compresses an infinite action space into a finite set of actions.
Training many specialised agents on dialogue clusters and picking the best by predicted reward is a recipe that improves on a single generalist agent without any manually labelled dialogue data.
The proposed reward can be used as an automatic training signal for chitchat agents, since it tracks human judgements on fluency, engagingness and consistency.
The large drop from training to unseen test dialogues implies that generalisation to novel sentences is the key bottleneck; the method's gains are shown on a modest dataset.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A natural extension would be to test whether the ensemble advantage survives when sentence and dialogue clusters are built from contextual sentence embeddings instead of averaged word vectors; the paper only uses mean word vectors.
The comparison does not include a policy-gradient ensemble baseline, so the paper's ensemble-over-single result is established for value-based agents; whether the same recipe helps policy-search chatbots remains open.
The agent-selection rule assumes the predicted dialogue reward ranks agents correctly; an alternative rule such as majority voting over the agents' top actions could be tested against the paper's reported numbers.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

You should know this paper's main empirical claim—that an ensemble of 100 value-based DQN chatbots trained on dialogue clusters beats a single agent and a Seq2Seq baseline—is plausible but rests on a cluster-collision assumption the authors never measure. They use K=100 sentence clusters over 124k unique training sentences, so each cluster holds hundreds of sentences. With 20 candidate responses per turn, the true response and distractors will sometimes fall in the same cluster, and the paper's own footnote says the system then picks randomly among them. That means a +1 reward doesn't guarantee the chosen sentence is the true one, and Recall@1 is inflated whenever a distractor shares the winning cluster. The reported ensemble advantage over the single agent (Table 5) could partly reflect cluster granularity rather than better response selection.

The real novelty is worth crediting: they apply value-based DRL to open-domain chitchat by deriving finite action sets automatically through sentence clustering, and they train an ensemble where each agent specializes on a dialogue cluster. That's a clean way to get specialization without labels, and it's a genuine departure from the policy-search methods that dominate this area. They also report a human evaluation, and the dialogue reward correlates strongly (r≈0.90–0.91) with human ratings. They are honest about the generalization gap, which tempers the significance claims.

The soft spots are addressable but real. The collision problem is the biggest one; the paper acknowledges the risk in Section 5.2 but provides no measurement of collision rates or exact-sentence accuracy. That should have been a basic sanity check. The reward validation is also partly circular: the reward is defined by selecting the true response over a distractor, and then shown to correlate with human ratings of dialogues produced by exactly that selection process. It is not an independent confirmation of the reward's quality. The automatic metrics lack error bars, and the human evaluation reports significance for only one pairwise comparison. No code is provided, which makes the test-time agent selection rule hard to reproduce precisely.

None of these flaws sink the paper. The ensemble vs. single-agent comparison is directionally credible, and the method is a useful addition to the dialogue-systems toolbox. If I were refereeing, I would ask for collision-rate statistics, exact-match accuracy, and error bars on the automatic metrics. The paper deserves serious peer review rather than desk rejection, and it would be a reasonable reading-group choice for anyone working on DRL for dialogue.

Referee Report

3 major / 4 minor

Summary. The paper proposes an ensemble-based deep reinforcement learning approach for open-domain chatbots. Actions are defined as sentence clusters obtained by k-means on mean word vectors, and the training data are partitioned into dialogue clusters, with one DQN agent trained per cluster. A simple reward function assigns +1 for selecting the true human response from a candidate set and -1 for selecting a randomly drawn distractor. At test time the agent with the highest predicted dialogue reward is selected. Using the Persona-Chat dataset, the authors report automatic metrics (Dialogue Reward, F1, Recall@1) on 999 held-out dialogues and a human evaluation with 15 judges rating fluency, engagingness, and consistency. The central claim is that the ensemble of 100 agents outperforms a single DRL agent and a Seq2Seq baseline, and that the proposed reward correlates strongly (r=0.90-0.91) with human ratings.

Significance. If the claims hold, the paper makes a useful contribution by showing that value-based DRL can be applied to chit-chat with automatically induced action sets, and that an ensemble of specialised agents can improve over a single agent without manual labels. The human evaluation is a genuine strength, and the public Persona-Chat dataset makes the experiments reproducible. However, the central numerical comparison depends on a reward signal whose reliability is not verified: with K=100 sentence clusters over 124,469 unique training sentences, the paper does not quantify how often a true response and a random distractor fall into the same cluster, which would make a +1 reward achievable without selecting the true sentence. The claimed correlation between the reward and human judgments is also partly circular because the rated dialogues are generated by the same selection process that defines the reward. These issues affect the load-bearing comparison in Table 5 and need to be addressed before the ensemble claim is fully supported.

major comments (3)

[§5.2, Eq. (1), Table 5] The +1/-1 reward credits any action whose sentence cluster contains the true human response, but with K=100 clusters over 124,469 unique training sentences the average cluster contains roughly 1,245 sentences. The paper never measures how often a true response and one of the 19 distractors share a cluster. The footnote in §5.2 admits that in such cases the system selects randomly among same-cluster sentences, so a 'correct' action can output a sentence that is not the true human response while still receiving +1. Without quantifying this cluster collision rate, the Dialogue Reward and Recall@1 columns of Table 5 (Ensemble -2.8882 vs Single Agent -6.4800) may reflect cluster granularity rather than response-selection quality. Please report the collision rate on the test set and, ideally, exact-sentence accuracy as a sanity check.
[§5.4, Fig. 7] The reported Pearson correlations of 0.90-0.91 between predicted dialogue rewards and human ratings are partly circular. The reward is defined as +1 for selecting the true human response and -1 for a random distractor, and the dialogues rated by humans are generated by exactly this selection process. A dialogue with more true-response selections has a higher reward by construction, so the strong correlation does not independently validate the reward as a measure of fluency, engagingness, or consistency. An independent validation would require decoupling the reward from the selection mechanism, for example by rating fixed dialogues that vary in reward without varying the true-response content, or by controlling for exact-sentence accuracy.
[§5.3, Table 5; §5.4] The automatic evaluation reports single point estimates for Dialogue Reward, F1, and Recall@1 without error bars, confidence intervals, or significance tests. The central comparison between the Ensemble and the Single Agent is therefore not statistically supported. The human evaluation reports significance only for Ensemble vs Seq2Seq (p=0.0332 for Fluency, p<0.01 for the other metrics) and does not report a significance test for Ensemble vs Single Agent, which is the paper's headline claim. Please report pairwise significance tests and effect sizes for all system comparisons.

minor comments (4)

[§5.2] The activation function is written as 'tahn' in the GRU equations; this should be 'tanh'.
[§5.1, Table 3] The table caption refers to 'Table 5.1', but the table is numbered Table 3 in the manuscript.
[§5.3, Fig. 4] The box plots in Figure 4 do not define what the whiskers, notches, or outliers represent; please add a description.
[§5.3, Table 5] Recall@1 is used but not formally defined; please state whether it is computed at the cluster level or the exact-sentence level, as this directly affects interpretation given the cluster-collision issue.

Circularity Check

1 steps flagged · score 4.0 of 10

Reward–human-judgment validation is partly circular; the main ensemble-vs-single-agent comparison remains independently benchmarked.

self definitional [Section 4.3 (Human-Likeness Rewards) and Section 5.4 (Human Evaluation), Figure 7]
"r^d_j(a) = +1, if a is a human response in dialogue-turn j of dialogue d; -1, if a is human but randomly chosen (incoherent). ... we analysed the predictive power of dialogue rewards, derived from our reward function, against human ratings on test data. This analysis revealed positive high correlations between them ... Pearson correlation scores between 0.90 and 0.91 for all metrics (Fluency, Engagingness and Consistency). This is in favour of our proposed reward function and supports its application to training open-ended dialogue agents."

The reward being validated is defined as a tally of whether the chosen action is the true human response. A dialogue with high dialogue reward is therefore, by construction, mostly composed of human-written ground-truth sentences, up to the cluster-ID collisions admitted in Section 5.2's footnote where the system selects randomly among sentences with the same cluster ID. The human judges then rate the fluency, engagingness, and consistency of dialogues produced by that same selection process.

full rationale

The paper's central claim that an ensemble of 100 DRL agents outperforms a single DRL agent and Seq2Seq is supported by held-out test evaluation and a human evaluation, and it does not reduce by construction to the training reward. The training uses clustered actions and clustered dialogues, and the test comparison is a genuine empirical comparison on 999 unseen dialogues; no equation in the paper makes the ensemble advantage an identity. The one substantially circular piece is the validation of the reward function in Section 5.4: because the reward gives +1 exactly when the selected action contains the true human response, dialogues with high reward are built from human sentences, and human raters are then asked to judge those same dialogues. The resulting 0.90-0.91 correlation is therefore partly guaranteed by the construction, not an independent confirmation that the reward captures dialogue quality. The paper's own Section 5.2 caveat about cluster collisions is a correctness and robustness concern rather than a circularity, since it concerns noise in the reward signal rather than a definitional equivalence. Self-citations to the authors' prior work on reward prediction ([37]) are present, but the paper re-implements and evaluates the predictor; the main ensemble result does not depend solely on that citation, so it does not independently raise the circularity score.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

The central claims depend on two K-means cluster counts (both set to 100), a fixed candidate set size, and a few standard DQN hyperparameters. The reward function is a hand-designed heuristic justified by a partly circular validation. No new physical or conceptual entities are introduced; the method components (reward function, reward predictor) are algorithmic, not invented entities.

free parameters (5)

Number of sentence clusters K = 100
Chosen as a compromise between computational cost and performance (Section 5.2); defines the action set size for the DRL agents.
Number of dialogue clusters K = 100
Selected because 100 clusters gave higher average episode rewards than 20 or 50 (Section 5.2); determines the number of agents in the ensemble.
Candidate response set size = 20
Fixed number of candidate responses presented at each dialogue turn in Algorithm 1 and the automatic evaluation.
Maximum dialogue history length = 25
Number of mean sentence vectors in H used to represent the state (Section 5.2).
Discount factor gamma = 0.99
Standard DQN hyperparameter listed in Section 5.2.

assumptions (3)

domain assumption The true human response in a human-human dialogue is the gold human-like response, and randomly sampled sentences are non-human-like.
This underpins the reward function in Section 4.3, which assigns +1 to the true response and -1 to randomly chosen distractors.
domain assumption K-means with Euclidean distance on mean word vectors yields semantically coherent sentence clusters and dialogue clusters.
The action set and training data partitions are built on this assumption in Section 4.2, with only manual inspection of a few example clusters.
domain assumption The reward predictor trained on synthetically distorted dialogues generalizes to real test dialogues and human-agent interactions.
The test-time ensemble selection in Section 4.1 relies on this predictor (trained as in [37]) to choose the best agent.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Ensemble-Based Deep Reinforcement Learning for Chatbots." pith.science (2026). https://pith.science/paper/RSSSNYII

@misc{pith2026190810422,
  author       = {Pith},
  title        = {Pith review of: Ensemble-Based Deep Reinforcement Learning for Chatbots},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RSSSNYII}},
  note         = {Machine review of arXiv:1908.10422}
}

read the original abstract

Trainable chatbots that exhibit fluent and human-like conversations remain a big challenge in artificial intelligence. Deep Reinforcement Learning (DRL) is promising for addressing this challenge, but its successful application remains an open question. This article describes a novel ensemble-based approach applied to value-based DRL chatbots, which use finite action sets as a form of meaning representation. In our approach, while dialogue actions are derived from sentence clustering, the training datasets in our ensemble are derived from dialogue clustering. The latter aim to induce specialised agents that learn to interact in a particular style. In order to facilitate neural chatbot training using our proposed approach, we assume dialogue data in raw text only -- without any manually-labelled data. Experimental results using chitchat data reveal that (1) near human-like dialogue policies can be induced, (2) generalisation to unseen data is a difficult problem, and (3) training an ensemble of chatbot agents is essential for improved performance over using a single agent. In addition to evaluations using held-out data, our results are further supported by a human evaluation that rated dialogues in terms of fluency, engagingness and consistency -- which revealed that our proposed dialogue rewards strongly correlate with human judgements.

Figures

Figures reproduced from arXiv: 1908.10422 by the authors.

**Figure 2.** Example clusters of our training data using Principal Component Analysis [44] for visualisations [PITH_FULL_IMAGE:figures/full_fig_p017_2.png] view at source ↗

**Figure 3.** Learning curves of ChatDQN agents [PITH_FULL_IMAGE:figures/full_fig_p018_3.png] view at source ↗

Figures from the paper (4 more)

**Figure 4.** Figure 4: Test performance of 100 ChatDQN agents on training (blue boxes) and test data (red boxes) using [PITH_FULL_IMAGE:figures/full_fig_p019_4.png]

**Figure 5.** Figure 5: Screenshot of our dialogue evaluation tool [PITH_FULL_IMAGE:figures/full_fig_p022_5.png]

**Figure 6.** Figure 6: Human evaluation results, the higher the better [PITH_FULL_IMAGE:figures/full_fig_p022_6.png]

**Figure 7.** Figure 7: Scatter plots showing strong correlations (with Pearson coefficients of [PITH_FULL_IMAGE:figures/full_fig_p023_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

55 extracted references · 49 canonical work pages

[1]

B. J. Grosz, C. L. Sidner, Attention, intentions, and the structure of discourse, Computational Linguistics 12 (3) (1986) 175–204

work page 1986
[2]

R. S. Sutton, A. G. Barto, Reinforcement learning - an introduction, 2nd Edition, Adaptive computation and machine learning, MIT Press, 2018

work page 2018
[3]

Hastie, R

T. Hastie, R. Tibshirani, J. H. Friedman, The elements of statistical learning: data mining, inference, and prediction, 2nd Edition, Springer series in statistics, Springer, 2009

work page 2009
[4]

LeCun, Y

Y . LeCun, Y . Bengio, G. E. Hinton, Deep learning, Nature 521 (7553) (2015) 436–444. 25

work page 2015
[5]

I. V . Serban, C. Sankar, M. Germain, S. Zhang, Z. Lin, S. Subramanian, T. Kim, M. Pieper, S. Chandar, N. R. Ke, S. Rajeswar, A. de Br ´ebisson, J. M. R. Sotelo, D. Suhubdy, V . Michalski, A. Nguyen, J. Pineau, Y . Bengio, A deep reinforce- ment learning chatbot (short version), CoRR abs/1801.06700

work page Pith review arXiv
[6]

J. Li, W. Monroe, A. Ritter, D. Jurafsky, M. Galley, J. Gao, Deep reinforcement learning for dialogue generation, in: EMNLP, 2016

work page 2016
[7]

H. Cuay ´ahuitl, Simpleds: A simple deep reinforcement learning dialogue sys- tem, in: Dialogues with Social Robots - Enablements, Analyses, and Evaluation, Seventh International Workshop on Spoken Dialogue Systems, IWSDS 2016, Saariselk¨a, Finland, January 13-16, 2016, 2016, pp. 109–118

work page 2016
[8]

J. Gao, M. Galley, L. Li, Neural approaches to conversational AI, Foundations and Trends in Information Retrieval 13 (2-3) (2019) 127–298

work page 2019

Show all 55 references

[9]

Li, Deep reinforcement learning: An overview, CoRR abs/1701.07274

Y . Li, Deep reinforcement learning: An overview, CoRR abs/1701.07274

arXiv
[10]

Casanueva, P

I. Casanueva, P. Budzianowski, P. Su, S. Ultes, L. M. Rojas-Barahona, B. Tseng, M. Gasic, Feudal reinforcement learning for dialogue management in large do- mains, in: NAACL-HLT, 2018

2018
[11]

Cuay ´ahuitl, S

H. Cuay ´ahuitl, S. Yu, Deep reinforcement learning of dialogue policies with less weight updates, in: INTERSPEECH, 2017

2017
[12]

Cuay ´ahuitl, S

H. Cuay ´ahuitl, S. Yu, A. Williamson, J. Carse, Scaling up deep reinforcement learning for multi-domain dialogue systems, in: IJCNN, 2017

2017
[13]

J. D. Williams, K. Asadi, G. Zweig, Hybrid code networks: practical and efﬁcient end-to-end dialog control with supervised and reinforcement learning, in: ACL, 2017

2017
[14]

B. Peng, X. Li, L. Li, J. Gao, A. C ¸ elikyilmaz, S. Lee, K. Wong, Composite task- completion dialogue policy learning via hierarchical deep reinforcement learning, in: EMNLP, 2017. 26

2017
[15]

Cuay ´ahuitl, SimpleDS: A simple deep reinforcement learning dialogue sys- tem, in: International Workshop on Spoken Dialogue Systems (IWSDS), 2016

H. Cuay ´ahuitl, SimpleDS: A simple deep reinforcement learning dialogue sys- tem, in: International Workshop on Spoken Dialogue Systems (IWSDS), 2016

2016
[16]

J. Li, W. Monroe, T. Shi, S. Jean, A. Ritter, D. Jurafsky, Adversarial learning for neural dialogue generation, in: EMNLP, 2017

2017
[17]

Vinyals, Q

O. Vinyals, Q. V . Le, A neural conversational model, CoRR abs/1506.05869

arXiv
[18]

Sordoni, M

A. Sordoni, M. Galley, M. Auli, C. Brockett, Y . Ji, M. Mitchell, J. Nie, J. Gao, B. Dolan, A neural network approach to context-sensitive generation of conver- sational responses, in: HLT-NAACL, 2015

2015
[19]

I. V . Serban, T. Klinger, G. Tesauro, K. Talamadupula, B. Zhou, Y . Bengio, A. C. Courville, Multiresolution recurrent neural networks: An application to dialogue response generation, in: AAAI, 2017

2017
[20]

J. Li, M. Galley, C. Brockett, G. P. Spithourakis, J. Gao, W. B. Dolan, A persona- based neural conversation model, in: ACL, 2016

2016
[21]

W. Wang, M. Huang, X.-S. Xu, F. Shen, L. Nie, Chat more: Deepening and widening the chatting topic via a deep model, in: SIGIR, ACM, 2018

2018
[22]

Zhang, E

S. Zhang, E. Dinan, J. Urbanek, A. Szlam, D. Kiela, J. Weston, Personalizing dialogue agents: I have a dog, do you have pets too?, CoRR abs/1801.07243

arXiv
[23]

Sutskever, O

I. Sutskever, O. Vinyals, Q. V . Le, Sequence to sequence learning with neural networks, in: NIPS, 2014

2014
[24]

Y . Song, C. Li, J. Nie, M. Zhang, D. Zhao, R. Yan, An ensemble of retrieval-based and generation-based human-computer conversation systems, in: IJCAI, 2018

2018
[25]

Papaioannou, A

I. Papaioannou, A. C. Curry, J. L. Part, I. Shalyminov, X. Xu, Y . Yu, O. Dusek, V . Rieser, O. Lemon, An ensemble model with ranking for social dialogue, CoRR abs/1712.07558

arXiv
[26]

C. Liu, R. Lowe, I. Serban, M. Noseworthy, L. Charlin, J. Pineau, How NOT to evaluate your dialogue system: An empirical study of unsupervised evaluation metrics for dialogue response generation, in: EMNLP, 2016. 27

2016
[27]

Papineni, S

K. Papineni, S. Roukos, T. Ward, W.-J. Zhu, Bleu: A method for automatic eval- uation of machine translation, in: ACL, 2002

2002
[28]

Lavie, A

A. Lavie, A. Agarwal, Meteor: An automatic metric for mt evaluation with high levels of correlation with human judgments, in: Proceedings of the Second Work- shop on Statistical Machine Translation (StatMT), 2007

2007
[29]

L. F. D’Haro, R. E. Banchs, C. Hori, H. Li, Automatic evaluation of end-to-end dialog systems with adequacy-ﬂuency metrics, Computer Speech & Language 55 (2019) 200–215

2019
[30]

Szepesv ´ari, Algorithms for Reinforcement Learning, Morgan and Claypool Publishers, 2010

C. Szepesv ´ari, Algorithms for Reinforcement Learning, Morgan and Claypool Publishers, 2010

2010
[31]

V . Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. A. Riedmiller, A. Fidjeland, G. Ostrovski, S. Petersen, C. Beattie, A. Sadik, I. Antonoglou, H. King, D. Kumaran, D. Wierstra, S. Legg, D. Hassabis, Human-level control through deep rein...
[32]

van Hasselt, A

H. van Hasselt, A. Guez, D. Silver, Deep reinforcement learning with double Q-learning, in: AAAI, 2016

2016
[33]

Z. Wang, T. Schaul, M. Hessel, H. van Hasselt, M. Lanctot, N. de Freitas, Dueling network architectures for deep reinforcement learning, in: ICML, 2016

2016
[34]

Cuay ´ahuitl, S

H. Cuay ´ahuitl, S. Ryu, D. Lee, S. Choi, I. Hwang, J. Kim, Deep reinforcement learning for chatbots using clustered actions and human-likeness rewards, in: IJCNN, 2019

2019
[35]

M. A. Wiering, H. van Hasselt, Ensemble algorithms in reinforcement learning, Trans. Sys. Man Cyber. Part B 38 (4)
[36]

liang Chen, L

X. liang Chen, L. Cao, C.-X. Li, Z.-X. Xu, J. Lai, Ensemble network architecture for deep reinforcement learning, Mathematical Problems in Engineering Article ID 2129393. 28
[37]

Today’s Practice and Tomorrow’s Potential

H. Cuay ´ahuitl, S. Ryu, D. Lee, J. Kim, A study of dialogue reward prediction for open-ended conversational agents, in: NeurIPS Workshop on Conversational AI: “Today’s Practice and Tomorrow’s Potential”, 2018

2018
[38]

Mikolov, I

T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, J. Dean, Distributed represen- tations of words and phrases and their compositionality, in: NIPS, 2013

2013
[39]

Pennington, R

J. Pennington, R. Socher, C. D. Manning, GloVe: Global vectors for word repre- sentation, in: EMNLP, 2014

2014
[40]

Iyyer, V

M. Iyyer, V . Manjunatha, J. L. Boyd-Graber, H. D. III, Deep unordered composi- tion rivals syntactic methods for text classiﬁcation, in: ACL (1), 2015

2015
[41]

Arthur, S

D. Arthur, S. Vassilvitskii, K-means++: The advantages of careful seeding, in: SODA, SIAM, 2007

2007
[42]

A. H. Miller, W. Feng, D. Batra, A. Bordes, A. Fisch, J. Lu, D. Parikh, J. Weston, ParlAI: A dialog research software platform, in: EMNLP, 2017

2017
[43]

K. Cho, B. van Merrienboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, Y . Bengio, Learning phrase representations using RNN encoder– decoder for statistical machine translation, in: EMNLP, 2014

2014
[44]

M. E. Tipping, C. Bishop, Probabilistic principal component analysis, Journal of the Royal Statistical Society, Series B 21/3 (1999) 611622

1999
[45]

Ioffe, C

S. Ioffe, C. Szegedy, Batch normalization: Accelerating deep network training by reducing internal covariate shift, in: ICML, 2015

2015
[46]

Devlin, M

J. Devlin, M. Chang, K. Lee, K. Toutanova, BERT: pre-training of deep bidirec- tional transformers for language understanding, in: NAACL-HLT, 2019

2019
[47]

D. Cer, Y . Yang, S. Kong, N. Hua, N. Limtiaco, R. S. John, N. Constant, M. Guajardo-Cespedes, S. Yuan, C. Tar, Y . Sung, B. Strope, R. Kurzweil, Univer- sal sentence encoder, CoRR abs/1803.11175. 29

arXiv
[48]

Q. Le, T. Mikolov, Distributed representations of sentences and documents, in: ICML, 2014

2014
[49]

C. D. Manning, M. Surdeanu, J. Bauer, J. Finkel, S. J. Bethard, D. McClosky, The Stanford CoreNLP natural language processing toolkit, in: ACL, 2014

2014
[50]

Papaioannou, O

I. Papaioannou, O. Lemon, Combining chat and task-based multimodal dialogue for more engaging HRI: A scalable method using reinforcement learning, in: In- ternational Conference on Human-Robot Interaction (HRI), 2017

2017
[51]

Cuay ´ahuitl, S

H. Cuay ´ahuitl, S. Keizer, O. Lemon, Strategic dialogue managent via deep rein- forcement learning, in: NIPS Deep Reinforcement Learning Workhop, 2015

2015
[52]

Dethlefs, Domain transfer for deep natural language generation from abstract meaning representations, IEEE Comp

N. Dethlefs, Domain transfer for deep natural language generation from abstract meaning representations, IEEE Comp. Int. Mag. 12 (3) (2017) 18–28

2017
[53]

Z. Wei, Q. Liu, B. Peng, H. Tou, T. Chen, X. Huang, K. Wong, X. Dai, Task- oriented dialogue system for automatic diagnosis, in: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, Melbourne, Australia, July 15-20, 2018, V olume 2...

2018
[54]

F. Weng, P. Angkititrakul, E. Shriberg, L. P. Heck, S. Peters, J. H. L. Hansen, Conversational in-vehicle dialog systems: The past, present, and future, IEEE Signal Process. Mag. 33 (6) (2016) 49–60

2016
[55]

Cuay ´ahuitl, A data-efﬁcient deep learning approach for deployable multimodal social robots, Neurocomputing, issn 0925-2312, 2019

H. Cuay ´ahuitl, A data-efﬁcient deep learning approach for deployable multimodal social robots, Neurocomputing, issn 0925-2312, 2019. 30

2019

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

B. J. Grosz, C. L. Sidner, Attention, intentions, and the structure of discourse, Computational Linguistics 12 (3) (1986) 175–204

work page 1986

[2] [2]

R. S. Sutton, A. G. Barto, Reinforcement learning - an introduction, 2nd Edition, Adaptive computation and machine learning, MIT Press, 2018

work page 2018

[3] [3]

Hastie, R

T. Hastie, R. Tibshirani, J. H. Friedman, The elements of statistical learning: data mining, inference, and prediction, 2nd Edition, Springer series in statistics, Springer, 2009

work page 2009

[4] [4]

LeCun, Y

Y . LeCun, Y . Bengio, G. E. Hinton, Deep learning, Nature 521 (7553) (2015) 436–444. 25

work page 2015

[5] [5]

I. V . Serban, C. Sankar, M. Germain, S. Zhang, Z. Lin, S. Subramanian, T. Kim, M. Pieper, S. Chandar, N. R. Ke, S. Rajeswar, A. de Br ´ebisson, J. M. R. Sotelo, D. Suhubdy, V . Michalski, A. Nguyen, J. Pineau, Y . Bengio, A deep reinforce- ment learning chatbot (short version), CoRR abs/1801.06700

work page Pith review arXiv

[6] [6]

J. Li, W. Monroe, A. Ritter, D. Jurafsky, M. Galley, J. Gao, Deep reinforcement learning for dialogue generation, in: EMNLP, 2016

work page 2016

[7] [7]

H. Cuay ´ahuitl, Simpleds: A simple deep reinforcement learning dialogue sys- tem, in: Dialogues with Social Robots - Enablements, Analyses, and Evaluation, Seventh International Workshop on Spoken Dialogue Systems, IWSDS 2016, Saariselk¨a, Finland, January 13-16, 2016, 2016, pp. 109–118

work page 2016

[8] [8]

J. Gao, M. Galley, L. Li, Neural approaches to conversational AI, Foundations and Trends in Information Retrieval 13 (2-3) (2019) 127–298

work page 2019