Speech Driven Backchannel Generation using Deep Q-Network for Enhancing Engagement in Human-Robot Interaction

Engin Erzin; Nusrah Hussain; T. Metin Sezgin; Yucel Yemez

REVIEW 3 major objections 5 minor 28 references

Speech Driven Backchannel Generation using Deep Q-Network for Enhancing Engagement in Human-Robot Interaction

T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that a social robot can learn when to laugh by applying batch deep Q-learning to recorded human dialogues, and that the learned policy raises measured user engagement.

desk verdict First RL formulation of backchannel generation, but the engagement reward counts the agent's own laugh, so the reported gains don't show enhanced engagement. read the letter →

arxiv 1908.01618 v1 pith:QDS5ZIH5 submitted 2019-08-05 cs.AI cs.LGcs.RO

Nusrah Hussain , Engin Erzin , T. Metin Sezgin , Yucel Yemez This is my paper · ORCID

classification cs.AIcs.LGcs.RO

keywords backchannelgenerationhuman-robotinteractionengagementbatchreinforcementlearningdeepQ-networkoff-policyevaluationlaughterspeechfeatures

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a social robot can learn when to produce laughs during conversation without any live trial-and-error interaction, by training on recorded human-human dialogues as a fixed batch of experience. The problem is cast as a Markov decision process in which the state is derived from the speaker's recent speech features, the action is a binary laugh/no-laugh decision, and the reward is a quantified engagement score built from connection events such as mutual gaze, turn-taking, and backchannels. Against the neural fitted Q-iteration baseline and the original behavior policy, the proposed batch deep Q-network is reported to yield lower Bellman residuals, a higher estimated discounted return in off-policy evaluation, and laugh durations statistically closer to human laughter. The paper argues that offline batch reinforcement learning is a viable route to engaging social-robot behavior, avoiding the cost and risk of learning by trial and error with human partners.

What carries the argument

The central object is batch-DQN, a deep Q-network trained on fixed tuples $\langle s_t,a_t,r_t,s_{t+1}\rangle$ drawn from a recorded dyadic interaction, with a replay buffer initialized much smaller than the full batch and refilled only with samples that agree with the current $\epsilon$-greedy policy. The state $s_t$ is a 209-dimensional vector of MFCC and prosody statistics computed over the past one second at 25 ms steps; the action is binary (laugh or stay silent); the reward is the engagement pace $1/\text{MTBCE}$, where MTBCE is the mean time between connection events, computed over 15-second windows from mutual facial gaze, adjacency pairs, and backchannels. A target Q-network and the Bellman optimality equation drive the updates, and the claims are evaluated by Bellman residual and step-wise weighted importance sampling.

What would settle it

Compute the pace reward after removing the agent's own backchannel events from the connection-event count, then re-run the off-policy evaluation of the batch-DQN and NFQ policies on the same data; if batch-DQN no longer shows a higher estimated return than the baseline, the central engagement claim collapses.

Watch

Extended reading notes

Core claim

The paper's central claim is that a backchannel-generation policy for a social robot can be extracted from fixed offline data, without any live interaction, using a batch variant of deep Q-learning. The learned laugh/no-laugh policy is reported to achieve a lower Bellman residual (0.0371 versus 0.0571 for the NFQ baseline), a higher estimated discounted return under step-wise weighted importance sampling (27.57 versus 24.32 for NFQ and 23.15 for the behavior policy), and laugh durations more similar to human laughter (KL divergence 0.1921 versus 0.2890). On these grounds the paper argues that the robot is learning to produce laughs at moments that increase quantified user engagement.

Load-bearing premise

The load-bearing assumption is that the engagement reward is a valid measure of user engagement even though it counts the agent's own laughter as one of the connection events that increase the reward; if engagement is measured independently of the action being optimized, the reported advantage may reflect self-generated reward rather than genuine user engagement.

Editorial extensions

If this is right

If the paper is right, a social robot can be prepared for engaging interaction entirely offline, avoiding long, costly, or unsafe live training sessions with humans.
The same MDP formulation extends to backchannel types beyond laughter, since the action space is generic binary event generation.
The batch-DQN replay-buffer filter is claimed to be a better batch-RL choice than NFQ for this task, not merely an equal alternative.
Because engagement is computed automatically from connection events rather than manual annotation, the policy can in principle be updated online as the robot interacts, as the paper notes.
Laugh durations from the learned policy are closer to human duration statistics than the baseline's, which the paper treats as evidence of more natural behavior.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The strongest test the paper leaves implicit is a live human-robot study: if human raters do not report higher engagement under the batch-DQN policy, the offline OPE advantage would not establish real engagement gains.
The reward definition may be the main confound; redefining engagement to exclude the agent's own backchannel events would be a direct way to test whether the policy truly tracks user engagement.
Because the behavior policy is estimated from the same data used for training, importance-sampling estimates may be sensitive to that estimation; comparing several OPE estimators could reveal how much of the reported advantage is method-dependent.
The framework assumes offline engagement measures computed from recorded dyads transfer to a robot in real time; a testable extension is to check whether the same speech features predict engagement when the listener is a robot rather than a human.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

Here's the one-sentence version: this is the first RL formulation of backchannel generation I know of, and it's a clean statement of the offline-learning idea, but the engagement metric is constructed so the agent can reward itself by laughing, so the reported gains don't support the headline claim.

The new part is real. Treating backchannel timing as an MDP, with states from speech features and rewards from an engagement measure, and learning from a fixed human dyadic corpus, is a sensible and fairly original way to sidestep the cost of online human-robot trials. The batch-DQN with replay-buffer filtering is a small but reasonable algorithmic tweak, and the leave-one-subject-out protocol is a step up from random splits.

What doesn't hold up is the evaluation. In Section 3.5, engagement is quantified via connection events that include backchannels — laughs, smiles, nods, head-shakes. The action is the agent's own laugh. So the reward at each step is partly determined by the action the agent is learning. Section 5 confirms the problem: mean reward is higher during laugh intervals, and both policies over-generate laughs. The OPE in Table 2 then just shows that the learned policies do better at producing their own reward. That's not evidence of user engagement. The laugh-duration analysis in Table 3 is more fair, but it also shows the artifacts (NFQ producing 198-second laughs) and doesn't rescue the central claim.

The authors are honest about the over-generation and list a human-subject study as future work, which is good. The citation pattern is fine; they locate themselves against rule-based and data-driven backchannel work and batch-RL baselines. But the paper as written doesn't establish its headline. The MDP framing is worth keeping; the reward needs to be redefined to exclude the agent's own backchannels, or the work needs a real interaction experiment.

Who should read this: researchers working on RL for social robotics, especially those thinking about offline training from human interaction data. It would be a useful reading-group discussion of how reward design can lead to self-referential optimization. I'd send it to a serious venue with the explicit expectation of major revision, because the formulation is original and the flaw is fixable.

Referee Report

3 major / 5 minor

Summary. The paper proposes an offline (batch) deep reinforcement learning method for generating laughter backchannels in human-robot interaction. States are 209-dimensional speech features, the action is binary (laugh or no laugh), and the reward is an engagement metric based on the pace of connection events from Rich et al. The method is trained on the IEMOCAP dyadic interaction dataset, treating one participant as the behavior policy and the other as the environment. The batch-DQN algorithm is compared with neural fitted Q-iteration (NFQ) using Bellman residual, off-policy policy evaluation (step-wise weighted importance sampling), and laugh-duration statistics. The paper claims that batch-DQN outperforms NFQ and can learn engaging robot behaviors offline.

Significance. If the central claim were established, the paper would make a useful contribution to social robotics by showing that offline RL can leverage existing human-human interaction corpora to train backchannel-generation policies. The MDP formulation, the use of IEMOCAP as a batch source, and the attempt to apply off-policy evaluation are all interesting. The paper also has the strength of clearly describing the state-action-reward structure. However, the reward definition is self-referential: the agent's own laugh action is counted as a connection event in the engagement metric, so the reward partially measures the agent's own behavior rather than the user's engagement. This flaw undermines both the training signal and the off-policy evaluation, and the paper itself acknowledges that both policies over-generate laughs because reward is higher during laugh intervals. As a result, the title and abstract's claim of enhancing engagement is not supported.

major comments (3)

[Section 3.5] The engagement reward is circular with respect to the action. Section 3.5 defines connection events to include backchannels ( laughs, smiles, nods, head-shakes), and Section 3.2 defines the action as the binary presence or absence of a backchannel (laugh). Therefore the agent's own laugh directly creates a connection event, increases the pace metric, and raises the reward, independent of the other participant's state. Section 5 confirms this, stating that 'the mean reward is higher in the intervals where laughs are present' and that both techniques 'prefer to generate laughs more frequently than in the dataset.' Consequently, the higher OPE value for batch-DQN in Table 2 (27.57 vs 24.32 vs 23.15) may simply indicate a higher laugh rate, not enhanced user engagement. The reward should be computed from connection events that exclude the agent's own backchannels, or the claim should be restricted to maximizing this particular self-referential metric; as written, neither training nor evaluation supports the stated engagement claim.
[Section 6] The paper's central claim that the method 'maximizes the engagement of the user' is not directly tested with human participants. The conclusion states that 'an immediate extension of this work is to perform subjective evaluations with human participants,' which concedes that no such evaluation was conducted. All reported metrics (Bellman residual, OPE, laugh-duration statistics) are computed on the IEMOCAP batch and are contaminated by the reward circularity identified above. Without a human-robot interaction study or an engagement measure that is independent of the agent's own actions, the results cannot substantiate the abstract's assertion that the robot 'learn[s] to produce non-verbal backchannels ... to maximize the engagement and attention of the user.'
[Section 4.1.2 and Table 2] The off-policy policy evaluation results are reported as point estimates without confidence intervals or variance measures. Step-wise weighted importance sampling is known to have high variance, especially with long horizons (T=250) and a behavior policy estimated via approximate nearest neighbor. The differences between the estimated values (23.15, 24.32, 27.57) may not be statistically significant. Without error bars, a significance test, or at least multiple seeds/restarts, the claim that batch-DQN is superior to NFQ is not rigorously established.

minor comments (5)

[Title page] The title contains a typographical error: 'Deep Q-N ETWORK' has an unintended space in 'N ETWORK'.
[Section 4] The text says 'batch data of size 1.5 million (as described in Section 1)', but the dataset description appears in Section 3.1 and the tuple generation in Section 3.3, not in Section 1.
[Section 4] The training setup says 'train and test sets in the ratio 4:1 as leave one subject out (LOSO), hence 5 folds.' With five sessions, this is leave-one-session-out; the term 'subject' is potentially misleading, and the relationship between the 4:1 ratio and LOSO could be clarified.
[Section 3.6] Several hyperparameters are missing: the learning rate, minibatch size, target network update frequency, epsilon decay schedule, and replay buffer capacity are not reported. This hinders reproducibility of the batch-DQN method.
[Table 3] The NFQ maximum laugh duration of 198.45 seconds is drastically unnatural, but the paper does not analyze how such long durations arise from the policy dynamics (e.g., self-looping in the laugh action) or whether they are an artifact of the discretized state space.

Circularity Check

1 steps flagged · score 6.0 of 10

The engagement reward includes the agent's own laugh as a connection event, so the optimized 'engagement' is partially defined by the action being trained, and the evaluation uses the same self-referential reward.

self definitional [The relevant passages are the MDP definitions in Section 3.2, the engagement measurement in Section 3.5, and the results discussion in Section 5.]
"Action: Agent's action is a binary variable, indicating the absence or presence of the backchannel. The backchannels of the user with behavioral policy in the dataset were labeled at a rate of 40 Hz. ... we use the connection events (CE) (1) mutual facial gaze, (2) adjacency pair and (3) backchannels (that include laughs, smiles, nods and head-shakes) to quantify engagement ... MTBCE is calculated by T / (no. of CEs in T) ... we use pace = 1/MTBCE to quantify the engagement between a participant and the robot. ..."

By the paper's own definitions, the action a_t is the presence or absence of a backchannel (laugh), while the reward r_t is pace = 1/MTBCE = (#CEs in a 15 s window)/T, and backchannels are explicitly counted among the CEs. Therefore r_t is a direct, increasing function of the action variable: a policy that laughs more mechanically produces more CEs and hence higher reward, independent of the other participant's engagement. The paper admits this in Section 5: 'the mean reward is higher in the intervals where laughs are present.

full rationale

The central circularity is in the reward definition, not in the algorithm. Section 3.2 defines the action as 'the absence or presence of the backchannel' and the reward as 'engagement measures of the user.' Section 3.5 then defines engagement as pace = 1/MTBCE, where MTBCE counts connection events that explicitly include backchannels ('laughs, smiles, nods and head-shakes'). Thus the reward depends directly on the action: laughing more raises the immediate pace. The paper's own Section 5 observation that mean reward is higher in laugh intervals confirms that the learned policies are partly rewarded for producing more laughs, and the authors even state that this explains why both techniques generate laughs more frequently than the dataset. Since the same reward is used for the OPE comparison in Table 2 and for the Bellman-residual comparisons, those numbers do not establish an independent improvement in user engagement; they evaluate policies against an objective that includes the action. This is a partial, not total, self-definitional circularity because engagement also includes mutual facial gaze and adjacency-pair events, which are not fully determined by the laugh action. No separate self-citation chain or imported uniqueness theorem was found; the external citations for backchannels promoting engagement are not the load-bearing problem. The laugh-duration statistics in Table 3 are an external, independent metric and provide some non-circular evidence, though the paper notes the learned laughs remain unnatural in maximum length. Overall score 6: the main engagement claim partially reduces by construction to the reward's inclusion of the action being optimized.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central claim rests on several domain assumptions about reward validity and data transfer, plus hand-chosen hyperparameters. No new entities are introduced; the engagement metric comes from Rich et al. and the algorithm is a modification of DQN.

free parameters (6)

Discount factor gamma = 0.99
Set for all experiments (Section 4); no sensitivity analysis provided.
Reward window length = 15 seconds
Chosen in Section 3.5 for computing pace; affects the reward signal.
State window length = 1 second
Set in Section 3.2; affects state sufficiency for the MDP.
Q-network hidden sizes = 100 and 25
Architecture chosen in Section 4; no ablation study.
Epsilon-greedy schedule = Not reported
Epsilon decays with time in Section 3.6, but the exact schedule is not specified, making results hard to reproduce.
OPE trajectory length = 250 samples used; T=100 in definition
Inconsistent trajectory length in Sections 4.1.2 and 5; chosen for numerical reasons without sensitivity analysis.

assumptions (5)

domain assumption The past 1 second of speech features is a sufficient state representation for optimal backchannel timing.
Assumed in the MDP formulation (Sections 3.2 and 3.4); no state-abstraction analysis is provided.
domain assumption Sidner's connection events computed from the same dyadic recordings provide a valid reward for engagement.
Adopted in Section 3.5; includes backchannels as reward events, creating circularity with the action under evaluation.
domain assumption The behavior policy for importance sampling is well estimated by approximate nearest neighbor.
Used in Section 4.1.2; no calibration or error analysis is shown.
domain assumption IEMOCAP acted dyadic conversations transfer to human-robot interaction.
Assumed in Section 3.1; actors and robots differ from natural human-robot interaction.
domain assumption Bellman residual is a valid proxy for policy quality and engagement.
Used as a main metric in Section 4.1.1; a lower residual does not directly measure engagement.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Speech Driven Backchannel Generation using Deep Q-Network for Enhancing Engagement in Human-Robot Interaction." pith.science (2026). https://pith.science/paper/QDS5ZIH5

@misc{pith2026190801618,
  author       = {Pith},
  title        = {Pith review of: Speech Driven Backchannel Generation using Deep Q-Network for Enhancing Engagement in Human-Robot Interaction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QDS5ZIH5}},
  note         = {Machine review of arXiv:1908.01618}
}

read the original abstract

We present a novel method for training a social robot to generate backchannels during human-robot interaction. We address the problem within an off-policy reinforcement learning framework, and show how a robot may learn to produce non-verbal backchannels like laughs, when trained to maximize the engagement and attention of the user. A major contribution of this work is the formulation of the problem as a Markov decision process (MDP) with states defined by the speech activity of the user and rewards generated by quantified engagement levels. The problem that we address falls into the class of applications where unlimited interaction with the environment is not possible (our environment being a human) because it may be time-consuming, costly, impracticable or even dangerous in case a bad policy is executed. Therefore, we introduce deep Q-network (DQN) in a batch reinforcement learning framework, where an optimal policy is learned from a batch data collected using a more controlled policy. We suggest the use of human-to-human dyadic interaction datasets as a batch of trajectories to train an agent for engaging interactions. Our experiments demonstrate the potential of our method to train a robot for engaging behaviors in an offline manner.

Figures

Figures reproduced from arXiv: 1908.01618 by the authors.

**Figure 2.** Bellman residual vs training epochs 4.1.1 Bellman Residual For a Q-value function approximation network Qθ, the Bellman residual is defined as the difference between the two sides of a Bellman control equation [24]. A smaller residual error means that the learned policy is closer to the optimal policy and is a true Q-function since it follows the Bellman equation more closely. Similar to the work of [15], we compute… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

28 extracted references · 15 canonical work pages

[1]

Fostering user engagement in face-to-face human-agent interactions: a survey,

C. Clavel, A. Cafaro, S. Campano, and C. Pelachaud, “Fostering user engagement in face-to-face human-agent interactions: a survey,” inToward Robotic Socially Believable Behaving Systems-Volume II. Springer, 2016, pp. 93–120

work page 2016
[2]

Analysis of engagement and user experience with a laughter responsive social robot,

B. B. Türker, Z. Buçinca, E. Erzin, Y . Yemez, and M. Sezgin, “Analysis of engagement and user experience with a laughter responsive social robot,” inProc. 18th Annu. Conf. Int. Speech Commun. Assoc, 2017, pp. 844–848

work page 2017
[3]

Timing and entrainment of multimodal backchanneling behavior for an embodied conversational agent,

B. Inden, Z. Malisz, P. Wagner, and I. Wachsmuth, “Timing and entrainment of multimodal backchanneling behavior for an embodied conversational agent,” inProceedings of the 15th ACM on International conference on multimodal interaction. ACM, 2013, pp. 181–188

2013
[4]

Generating robot/agent backchannels during a storytelling experiment,

S. Al Moubayed, M. Baklouti, M. Chetouani, T. Dutoit, A. Mahdhaoui, J.-C. Martin, S. Ondas, C. Pelachaud, J. Urbain, and M. Yilmaz, “Generating robot/agent backchannels during a storytelling experiment,” in2009 IEEE International Conference on Robotics and Automation. IEEE, 2009, pp. 3749–3754

work page 2009
[5]

Generation of nodding, head tilting and eye gazing for human-robot dialogue interaction,

C. Liu, C. T. Ishi, H. Ishiguro, and N. Hagita, “Generation of nodding, head tilting and eye gazing for human-robot dialogue interaction,” in 2012 7th ACM/IEEE International Conference on Human-Robot Interaction (HRI) . IEEE, 2012, pp. 285–292

2012
[6]

Data-driven model of nonverbal behavior for socially assistive human-robot interactions,

H. Admoni and B. Scassellati, “Data-driven model of nonverbal behavior for socially assistive human-robot interactions,” inProceedings of the 16th international conference on multimodal interaction. ACM, 2014, pp. 196–199

work page 2014
[7]

Robot gains social intelligence through multimodal deep reinforcement learning,

A. H. Qureshi, Y . Nakamura, Y . Yoshikawa, and H. Ishiguro, “Robot gains social intelligence through multimodal deep reinforcement learning,” in2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids). IEEE, 2016, pp. 745–751

2016
[8]

Show, attend and interact: Perceivable human-robot social interaction through neural attention q-network,

——, “Show, attend and interact: Perceivable human-robot social interaction through neural attention q-network,” in 2017 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2017, pp. 1639–1645

work page 2017

Show all 28 references

[9]

Robot behavior adaptation for human-robot interaction based on policy gradient reinforcement learning,

N. Mitsunaga, C. Smith, T. Kanda, H. Ishiguro, and N. Hagita, “Robot behavior adaptation for human-robot interaction based on policy gradient reinforcement learning,”Journal of the Robotics Society of Japan, vol. 24, no. 7, pp. 820–829, 2006

2006
[10]

Neural network based reinforcement learning for audio–visual gaze control in human–robot interaction,

S. Lathuilière, B. Massé, P. Mesejo, and R. Horaud, “Neural network based reinforcement learning for audio–visual gaze control in human–robot interaction,”Pattern Recognition Letters, vol. 118, pp. 61–71, 2019

2019
[11]

Deﬁnitions of engagement in human-agent interaction,

N. Glas and C. Pelachaud, “Deﬁnitions of engagement in human-agent interaction,” in 2015 International Conference on Affective Computing and Intelligent Interaction (ACII). IEEE, 2015, pp. 944–949

2015
[12]

Poggi, Mind, hands, face and body: a goal and belief view of multimodal communication

I. Poggi, Mind, hands, face and body: a goal and belief view of multimodal communication. Weidler, 2007

2007
[13]

Recognizing engagement in human-robot interaction,

C. Rich, B. Ponsler, A. Holroyd, and C. L. Sidner, “Recognizing engagement in human-robot interaction,” in Human-Robot Interaction (HRI), 2010 5th ACM/IEEE International Conference on. IEEE, 2010, pp. 375–382

2010
[14]

Batch reinforcement learning,

S. Lange, T. Gabel, and M. Riedmiller, “Batch reinforcement learning,” inReinforcement learning. Springer, 2012, pp. 45–73

2012
[15]

Tree-based batch mode reinforcement learning,

D. Ernst, P. Geurts, and L. Wehenkel, “Tree-based batch mode reinforcement learning,” Journal of Machine Learning Research, vol. 6, no. Apr, pp. 503–556, 2005

2005
[16]

Neural ﬁtted q iteration–ﬁrst experiences with a data efﬁcient neural reinforcement learning method,

M. Riedmiller, “Neural ﬁtted q iteration–ﬁrst experiences with a data efﬁcient neural reinforcement learning method,” inEuropean Conference on Machine Learning. Springer, 2005, pp. 317–328

2005
[17]

R. S. Sutton and A. G. Barto, Introduction to reinforcement learning. MIT press Cambridge, 1998, vol. 135

1998
[18]

Human-level control through deep reinforcement learning,

V . Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski et al., “Human-level control through deep reinforcement learning,”Nature, vol. 518, no. 7540, p. 529, 2015

2015
[19]

Deep q-learning from demonstrations,

T. Hester, M. Vecerik, O. Pietquin, M. Lanctot, T. Schaul, B. Piot, D. Horgan, J. Quan, A. Sendonaris, I. Osband et al., “Deep q-learning from demonstrations,” inThirty-Second AAAI Conference on Artiﬁcial Intelligence, 2018

2018
[20]

Iemocap: Interactive emotional dyadic motion capture database,

C. Busso, M. Bulut, C.-C. Lee, A. Kazemzadeh, E. Mower, S. Kim, J. N. Chang, S. Lee, and S. S. Narayanan, “Iemocap: Interactive emotional dyadic motion capture database,”Language resources and evaluation, vol. 42, no. 4, p. 335, 2008

2008
[21]

Multimodal analysis of speech and arm motion for prosody-driven synthesis of beat gestures,

E. Bozkurt, E. Erzin, and Y . Yemez, “Multimodal analysis of speech and arm motion for prosody-driven synthesis of beat gestures,”Speech Communication, vol. 85, pp. 29–42, December 2016. 7

2016
[22]

Tracking continuous emotional trends of participants during affective dyadic interactions using body language and speech information,

A. Metallinou, A. Katsamanis, and S. Narayanan, “Tracking continuous emotional trends of participants during affective dyadic interactions using body language and speech information,”Image and Vision Computing, vol. 31, no. 2, pp. 137–152, 2013

2013
[23]

Selecting concise training sets from clean data,

M. Plutowski and H. White, “Selecting concise training sets from clean data,” IEEE Transactions on neural networks, vol. 4, no. 2, pp. 305–318, 1993

1993
[24]

Residual algorithms: Reinforcement learning with function approximation,

L. Baird, “Residual algorithms: Reinforcement learning with function approximation,” in Machine Learning Proceedings 1995. Elsevier, 1995, pp. 30–37

1995
[25]

Data-efﬁcient off-policy policy evaluation for reinforcement learning,

P. Thomas and E. Brunskill, “Data-efﬁcient off-policy policy evaluation for reinforcement learning,” inInterna- tional Conference on Machine Learning, 2016, pp. 2139–2148

2016
[26]

Importance sampling for fair policy selection

S. Doroudi, P. S. Thomas, and E. Brunskill, “Importance sampling for fair policy selection.”Grantee Submission, 2017

2017
[27]

Behaviour policy estimation in off-policy policy evaluation: Calibration matters,

A. Raghu, O. Gottesman, Y . Liu, M. Komorowski, A. Faisal, F. Doshi-Velez, and E. Brunskill, “Behaviour policy estimation in off-policy policy evaluation: Calibration matters,”arXiv preprint arXiv:1807.01066, 2018

2018 arXiv
[28]

Fast nearest neighbor search through sparse random projections and voting,

V . Hyvönen, T. Pitkänen, S. Tasoulis, E. Jääsaari, R. Tuomainen, L. Wang, J. Corander, and T. Roos, “Fast nearest neighbor search through sparse random projections and voting,” in Big Data (Big Data), 2016 IEEE International Conference on. IEEE, 2016, pp. 881–888. 8

2016

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

Fostering user engagement in face-to-face human-agent interactions: a survey,

C. Clavel, A. Cafaro, S. Campano, and C. Pelachaud, “Fostering user engagement in face-to-face human-agent interactions: a survey,” inToward Robotic Socially Believable Behaving Systems-Volume II. Springer, 2016, pp. 93–120

work page 2016

[2] [2]

Analysis of engagement and user experience with a laughter responsive social robot,

B. B. Türker, Z. Buçinca, E. Erzin, Y . Yemez, and M. Sezgin, “Analysis of engagement and user experience with a laughter responsive social robot,” inProc. 18th Annu. Conf. Int. Speech Commun. Assoc, 2017, pp. 844–848

work page 2017

[3] [3]

Timing and entrainment of multimodal backchanneling behavior for an embodied conversational agent,

B. Inden, Z. Malisz, P. Wagner, and I. Wachsmuth, “Timing and entrainment of multimodal backchanneling behavior for an embodied conversational agent,” inProceedings of the 15th ACM on International conference on multimodal interaction. ACM, 2013, pp. 181–188

2013

[4] [4]

Generating robot/agent backchannels during a storytelling experiment,

S. Al Moubayed, M. Baklouti, M. Chetouani, T. Dutoit, A. Mahdhaoui, J.-C. Martin, S. Ondas, C. Pelachaud, J. Urbain, and M. Yilmaz, “Generating robot/agent backchannels during a storytelling experiment,” in2009 IEEE International Conference on Robotics and Automation. IEEE, 2009, pp. 3749–3754

work page 2009

[5] [5]

Generation of nodding, head tilting and eye gazing for human-robot dialogue interaction,

C. Liu, C. T. Ishi, H. Ishiguro, and N. Hagita, “Generation of nodding, head tilting and eye gazing for human-robot dialogue interaction,” in 2012 7th ACM/IEEE International Conference on Human-Robot Interaction (HRI) . IEEE, 2012, pp. 285–292

2012

[6] [6]

Data-driven model of nonverbal behavior for socially assistive human-robot interactions,

H. Admoni and B. Scassellati, “Data-driven model of nonverbal behavior for socially assistive human-robot interactions,” inProceedings of the 16th international conference on multimodal interaction. ACM, 2014, pp. 196–199

work page 2014

[7] [7]

Robot gains social intelligence through multimodal deep reinforcement learning,

A. H. Qureshi, Y . Nakamura, Y . Yoshikawa, and H. Ishiguro, “Robot gains social intelligence through multimodal deep reinforcement learning,” in2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids). IEEE, 2016, pp. 745–751

2016

[8] [8]

Show, attend and interact: Perceivable human-robot social interaction through neural attention q-network,

——, “Show, attend and interact: Perceivable human-robot social interaction through neural attention q-network,” in 2017 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2017, pp. 1639–1645

work page 2017