{"id":"8db0d697-c6eb-490d-9159-d3c89462384e","arxiv_id":"1908.10398","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A humanoid robot learns to play noughts and crosses and its harder variant with a few hundred images, a dozen dialogues, and simulated games, beating DQN baselines in simulation and playing with 130 humans in the wild.","lead":"Researchers trained a Pepper humanoid robot to play tic-tac-toe and a harder variant using a modest amount of visual and dialogue data plus simulated games, and tested it with 130 visitors. The work shows a small-data recipe for teaching a social robot a new joint activity, while also revealing that imperfect visual perception is the main barrier to real-world play.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claim that the trained policy is competitive against humans is not measured: the field trial reports only perception error rates and anecdotal outcomes, and all quantitative policy results are against a random-legal-move simulator.","rationale":"The reader's weakest assumption identifies the same gap: the RL policy is trained and evaluated only against random legal moves, and the human evaluation does not report win/loss/draw statistics. This is the single most load-bearing concern because the paper's central claim is about playing games with people, not about beating a random simulator. Alternative concerns, such as ambiguity in Algorithm 2's pseudocode and missing error bars, are real but do not directly threaten the main claim as severely. The conditionality is appropriate: the paper's simulation results are suggestive, and the perception improvements are plausible, but the missing human outcome data prevents acceptance. A concrete check on existing logs would settle the transfer question; if the logs do not exist, a small targeted field study with outcome tallies would do so. Therefore the reader's CONDITIONAL verdict is unchanged.","tokens_in":14931,"tokens_out":7262,"duration_ms":80520,"concrete_test":"Extract from the field-trial logs the outcome of every game (robot win, human win, draw), split by day, by Standard vs Ultimate N&C, and by whether a game-move misrecognition occurred. Compute the robot's win/draw rate in misrecognition-free games and compare it with the 0.998 (Standard) and 0.975 (Ultimate) task-success rates reported in Table 4. If the robot's human-game win/draw rate is materially below those simulation rates, the simulated-opponent distribution is not representative. If logs are unavailable, run a fresh deployment with at least 30 naive participants using the fixed perception system and report the same outcome tallies.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires that a policy trained in simulation transfers to real human opponents. Section 4.1.6 states that the simulated opponent is 'semi-random user behaviour, i.e. from random but legal game moves.' All results in Table 4 and Figures 8-9 are obtained against this same simulated opponent; task success is defined as win/draw rate and includes draws (+1 reward). This does not establish competitive play against humans. Section 6's field trial with 130 opponents reports only game-move misrecognition rates (31%, 25%, 22%, 10%) and the statement 'In games without misrecognitions the robot ended up winning or in a draw.' No win/loss/draw counts, per-day or per-game breakdown, or comparison with simulation performance is provided. Because the paper's headline claim is successful play against unknown humans, the absence of any quantitative human outcome data is the load-bearing gap. The deployment also used the policy without temporal information, while the best simulation results used it, further widening the gap. It is possible the policy transfers well, but the current evidence does not demonstrate it.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a data-efficient deep learning pipeline for training a Pepper humanoid robot to play Noughts and Crosses and Ultimate Noughts and Crosses. The pipeline combines a supervised CNN for recognizing hand-drawn game moves (Algorithm 1) with a deep reinforcement learning policy that is trained in simulation (Algorithm 2, called 'competitive DQN'). The policy learner augments standard DQN with a one-step look-ahead that prunes actions leading to immediate loss and forces winning moves when available. The manuscript reports simulation results showing that the proposed algorithm outperforms two DQN baselines in win/draw rate and average reward, and it describes a four-day field trial in which the robot played against 130 human opponents, with game-move misrecognition rates decreasing from 31% to 10% after incremental improvements. The conclusion claims that the approach achieves high task success in both game variants and that accurate visual perception is essential for successful human-robot interaction.","tokens_in":15132,"tokens_out":6111,"duration_ms":55007,"significance":"If the claims are supported, the paper is a useful demonstration that a multimodal social robot can be trained for a joint human-robot activity with modest human effort: a few hundred images, a small set of demonstration dialogues, and automatically generated simulated games. The field deployment with 130 spontaneous human opponents is a strength and goes beyond typical lab-based or simulation-only evaluations. The algorithmic idea of using game-rules-based one-step look-ahead to constrain Deep Q-learning is clearly described and likely explains much of the reported performance difference over plain DQN. However, the manuscript's strongest claims—competitive play against humans and substantial improvement over baselines—rest on evidence that is currently incomplete: the human evaluation reports no win/loss/draw statistics, and the simulation results lack variance estimates and are measured against a random-legal-move opponent.","major_comments":[{"comment":"The human evaluation does not report any win/loss/draw statistics. The only quantitative numbers are game-move misrecognition rates per day (31%, 25%, 22%, 10%), and the statement 'In games without misrecognitions the robot ended up winning or in a draw' appears without counts and applies only to the subset of games without perception errors. Since the abstract and introduction claim successful play against unknown humans and 'competitive robot behaviours,' the absence of human-game outcomes is a load-bearing gap. In addition, the deployed policy was 'without temporal information' even though Table 4 shows temporal information improves task success in simulation, so the field trial tests a weaker configuration than the one reported as best.","section":"Section 6"},{"comment":"The simulation opponent is 'semi-random user behaviour, i.e. from random but legal game moves.' All quantitative policy results in Table 4 and Figures 8–9 are obtained against this opponent, and task success is defined as win/draw rate, with draws receiving +1 reward. A random legal opponent is a weak baseline; high win/draw rates against it do not establish competitive play against humans, whose play is likely to be more structured. The paper should either evaluate against stronger simulated opponents (e.g., minimax-based or human-data-driven) or provide quantitative human-game outcomes.","section":"Section 4.1.6 and Table 4"},{"comment":"The proposed algorithm is not a purely learning-based policy; lines 5–6 of Algorithm 2 inject the game rules by pruning all actions with negative immediate reward and forcing the winning move when one exists. The DQN baselines in the comparison do not have this action filter. Thus the performance difference in Table 4 may be attributed to the injected domain knowledge rather than to the Deep Q-learning procedure itself. An ablation is needed—for example, DQN with the same action-filtering oracle—and the paper should discuss the fraction of wins that result from the forced winning moves.","section":"Section 3.2, Algorithm 2"},{"comment":"The experimental results are based on single training runs with no random seeds, and Table 4 reports averages over 3000 test games without standard deviations or significance tests. Overlapping or small differences (e.g., 0.9868 vs 0.9980 for Standard N&C in Table 4) cannot be judged to be 'substantially' different without run-to-run variance. The claim that the proposed algorithm substantially outperforms the baselines requires repeated runs and a report of variance.","section":"Section 5.2"}],"minor_comments":[{"comment":"The loss-function text writes 'θi are the parameters of the network at iteration i, and θi are the target parameters' but the second θi should be θ̄i for the target network.","section":"Section 3.2"},{"comment":"'maxium number of actions per dialogue=100' contains a typo; it should read 'maximum.'","section":"Section 4.1.5"},{"comment":"The confusion matrix would benefit from an explicit statement that rows are true classes and columns are predicted classes; the current layout is ambiguous.","section":"Table 3"},{"comment":"The definition of TemporalInfo uses only RobotGameMoves in the denominator; it is unclear whether user moves are excluded by design and whether this asymmetry affects the learned policies.","section":"Section 4.1.1"},{"comment":"It is unclear whether the reported misrecognition rates (31%, 25%, 22%, 10%) are measured online during live games or offline after each day; please clarify the measurement procedure.","section":"Section 6"},{"comment":"The condition uses dist(P'_t, P'_{t-1}) + dist(P'_{t-1}, P'_{t-2}) > τ; the Euclidean distance is named in a footnote, but the threshold τ and the rationale for using two successive differences are not given.","section":"Algorithm 1, line 7"}],"recommendation":"major_revision","confidential_remarks":"This arXiv version includes a DOI to a Neurocomputing article (10.1016/j.neucom.2018.09.104). If this preprint is the version under review, the editor may wish to confirm whether the published version contains quantitative human-game outcomes or other additions that address the missing evidence identified in the major comments."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's what you should know: this is a systems/integration paper, not a methods breakthrough. The algorithmic delta over the author's earlier DQN variant is a 1-step look-ahead action filter plus temporal features, and the win-rate gains in simulation against a random-legal-move opponent are real but modest. What's actually valuable is the 130-person field trial of a Pepper robot playing tic-tac-toe variants in a university atrium with non-recruited visitors, plus a low-data pipeline that gets a vision system to 98.4% accuracy on noisy handwriting with a few hundred images. That deployment effort is rare in this literature and deserves credit.\n\nWhere the paper falls short: the headline claim is that the robot plays competitively against unknown humans, but the human evaluation reports only game-move misrecognition rates (31% down to 10% across days) and a single vague sentence: 'In games without misrecognitions the robot ended up winning or in a draw.' No win/loss/draw counts, no per-day breakdown, no comparison to simulation performance. The stress-test note is right: the transfer to humans is not demonstrated. It doesn't help that the deployment used the policy without temporal information, which was the best-performing version in simulation. Also, all RL results are single runs; no seeds or error bars are reported, so we can't judge variance. The pseudocode in Algorithm 2 is ambiguous (the min over r(s,a) is not well-defined, and the look-ahead mechanics are described in prose more clearly than in code).\n\nOn the positive side, the paper is honest about its limitations—online learning, learning to write, and larger language skills are all flagged as future work. The reward design is clearly stated, and the use of draws as +1 is up front, so the task-success numbers are interpretable. The field trial conditions are described candidly (varying lighting, multiple backgrounds, occasional two-opponent games). The author's earlier work is cited where relevant; the incremental extension is clear.\n\nWho benefits: researchers building deployable social robots, especially game-playing interaction; people working on data-efficient skill acquisition. The paper would be a reasonable read for a group interested in in-the-wild HRI evaluation. It deserves a serious referee—the integration and deployment are valuable—but the referee should push for the actual game outcome data and repeated runs. If the author supplies those, the paper would be acceptable; without them, the strongest claims remain unsupported. My recommendation: send to peer review with a request for major revision.","headline":"A rare in-the-wild humanoid robot deployment whose headline claim about competitive play is under-supported by the reported human data.","tokens_in":15678,"tokens_out":2675,"would_cite":false,"duration_ms":28970,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A robot can be trained to play two board games from a few hundred images, a dozen dialogues, a few demonstrations, and simulated games, using two proposed algorithms that beat DQN baselines.","keywords":["data-efficient learning","multimodal human-robot interaction","deep reinforcement learning","deep supervised learning","visual game tracking","noughts and crosses","humanoid social robots","competitive policy learning"],"falsifier":"A decisive test would be to record explicit win/loss/draw outcomes for the deployed robot against a larger sample of human opponents and compare them with the robot's performance against random-legal-move simulations; if human play differs enough that the win/draw rate collapses or the misrecognition rate stays high under varied lighting and handwriting styles, the transfer claim fails. A second, calculation-level check is to retrain the competitive DQN against a strategic (for example minimax) opponent and see whether the random-trained policy still wins.","tokens_in":14706,"feed_emoji":"🤖","tokens_out":7291,"duration_ms":70703,"temperature":0.7,"pith_summary":"This paper argues that a deployable social robot does not need massive datasets to acquire a new interactive skill. Using the two-player game Noughts and Crosses and its harder variant Ultimate Noughts and Crosses as a testbed, it claims a pipeline of deep supervised learning for vision and deep reinforcement learning for behaviour can bootstrap from a few hundred labelled images, a dozen example dialogues, a few physical demonstrations, and automatically generated simulated games. If this is right, robots in homes could be taught new joint activities by their users with modest human effort, rather than requiring millions of examples. The paper also proposes two algorithms—a hand-robust visual game tracker and a competitive DQN policy learner—and reports that both substantially outperform DQN baselines in automatic evaluation, with a 130-player field trial showing that accurate perception is the key ingredient for successful play.","feed_headline":"Robot learns two board games on a tiny data budget","feed_subtitle":"A few hundred images, a dozen dialogues, and simulations give a robot competitive play against people.","key_machinery":"The load-bearing machinery is a cascade of two learners plus a compact multimodal state representation. Algorithm 1, the game move recogniser, uses image-processing contour detection to locate the board, a CNN to classify each cell as nought, cross, or nothing, a hand-filtering step that skips frames whose transformation distance is too large, and a three-frame majority vote to commit to a move. Algorithm 2, competitive DQN learning, replaces the usual greedy action selection with a filtered rule: remove actions whose immediate reward is negative, force the winning move if one exists, and otherwise argmax over the remaining actions; the state vector also encodes when each move happened, using a normalised time step, which the paper shows improves task success. These mechanisms feed a state vector of 73 features for standard Noughts and Crosses and 289 for Ultimate, and restrict actions to legal, dialogue-plausible moves.","core_discovery":"The central claim is that a multimodal humanoid robot can learn to perceive and play two board games using only modest human-provided resources: a few hundred example images for a CNN character classifier, a dozen example dialogues for language behaviour, a few pre-recorded demonstrations of drawing moves, and simulated games for reinforcement learning. The proposed visual game recogniser (Algorithm 1) finds the board in the camera image, splits it into cells, classifies each cell with a CNN, filters out frames where the player's hand is present, and takes a majority vote over three frames before recording a move. The proposed competitive DQN policy learner (Algorithm 2) prunes actions with negative immediate reward, locks onto a winning move when one exists, and otherwise picks the highest-Q action among the remaining legal moves; it also adds temporal move-order features to the state. In automatic tests, this combination achieves higher win/draw rates and shorter dialogues than the two DQN baselines, and in a deployment against 130 human visitors the robot won or drew whenever its game-move recogniser made no misrecognitions.","pith_inferences":["Editorial inference: A natural extension not tested in the paper would be to run the same pipeline on a second game, such as Connect Four or draughts, to see whether the few-hundred-images and dozen-dialogues budget really is task-independent rather than specific to tic-tac-toe's simple symbols.","Editorial inference: The one-step look-ahead in the competitive DQN suggests a family of algorithms that interpolate between Q-learning and tree search; multi-step look-ahead or Monte Carlo tree search would likely strengthen play further at the cost of computation, which the paper explicitly leaves open.","Editorial inference: If the simulated random opponent is the bottleneck, a practical extension would be to seed simulations with human game logs or with increasingly strong self-play opponents, which would directly test whether human competitiveness improves.","Editorial inference: The observed success of hand-filtering plus temporal voting suggests a transferable design principle for any robot vision system that reads handwriting or gestures in the wild: transient occluders should be detected and excluded before classification."],"forward_implications":["A new board game or similar joint activity could be added to a robot with a few hundred labelled images, a dozen dialogue examples, a few demonstrations, and simulated practice, rather than a large curated dataset.","The proposed policy learner's one-step look-ahead and losing-action pruning yields substantially higher win/draw rates and shorter dialogues than vanilla DQN, making deep RL practical for real-time interaction.","Temporal move-order information is a cheap and effective state augmentation: including it raises win/draw rate and shortens games.","The visual tracker's hand-filtering and three-frame voting reduced game-move misrecognitions from 31% to 10% across deployment days, and the paper attributes successful human-robot games to accurate perception.","Because the approach is not game-specific—only the action set, reward function, and perception labels change—the same pipeline should extend to other social games and interactive tasks."],"supporting_citations":[{"why":"Provides the original DQN baseline that the proposed competitive DQN must beat; the paper measures task success and dialogue length against it.","marker":"[2]"},{"why":"Supplies the earlier DQN-variant baseline and the example multimodal dialogues that bootstrap the interaction behaviour.","marker":"[1]"},{"why":"Underlies the automatic simulation of user behaviour used to generate the reinforcement learning training games.","marker":"[21]"},{"why":"Supplies the convolutional neural network architecture used to classify nought, cross, and nothing in each grid cell.","marker":"[22]"},{"why":"Provides the interaction-manager infrastructure that orchestrates the robot's multimodal perception and action loop.","marker":"[26]"}],"fun_headline_variants":["Robot learns Noughts & Crosses with a few hundred images","Data-efficient deep learning lets robot play games with little data","Humanoid robot beats DQN baselines after tiny training set","Few examples, competitive play: robot learns board games"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The training sessions pit the robot only against a simulated opponent that makes random but legal moves, and the paper's central claim stands or falls on the assumption that this distribution of play is close enough to how real humans play that the learned policy will remain competitive against people.","fun_headline_variants_meta":{"raw":{"variants":["Robot learns Noughts & Crosses with a few hundred images","Data-efficient deep learning lets robot play games with little data","Humanoid robot beats DQN baselines after tiny training set","Few examples, competitive play: robot learns board games"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000813,"raw_usage":{"total_tokens":3561,"prompt_tokens":942,"completion_tokens":2619,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":558,"completion_tokens_details":{"reasoning_tokens":2549}},"tokens_in":558,"tokens_out":2619,"duration_ms":21633,"temperature":1.0,"reasoning_tokens":2549,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T10:44:23.289145+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A decisive test would be to record explicit win/loss/draw outcomes for the deployed robot against a larger sample of human opponents and compare them with the robot's performance against random-legal-move simulations; if human play differs enough that the win/draw rate collapses or the misrecognition rate stays high under varied lighting and handwriting styles, the transfer claim fails. A second, calculation-level check is to retrain the competitive DQN against a strategic (for example minimax) opponent and see whether the random-trained policy still wins.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the original DQN baseline that the proposed competitive DQN must beat; the paper measures task success and dialogue length against it."},{"cited_title":"Cuay ´ahuitl, Deep reinforcement learning for conversational robots playing games, in: IEEE-RAS International Conference on Humanoid Robots (Hu- manoids), 2017","cited_arxiv_id":null,"evidence_quote":"Supplies the earlier DQN-variant baseline and the example multimodal dialogues that bootstrap the interaction behaviour."},{"cited_title":"Cuay ´ahuitl, SimpleDS: A simple deep reinforcement learning dialogue sys- tem, in: International Workshop on Spoken Dialogue Systems (IWSDS), 2016","cited_arxiv_id":null,"evidence_quote":"Underlies the automatic simulation of user behaviour used to generate the reinforcement learning training games."},{"cited_title":"LeCun, L","cited_arxiv_id":null,"evidence_quote":"Supplies the convolutional neural network architecture used to classify nought, cross, and nothing in each grid cell."},{"cited_title":"Cuay ´ahuitl, SimpleDS: A Simple Deep Reinforcement Learning Dia- logue System, Springer, Singapore, 2017, pp","cited_arxiv_id":null,"evidence_quote":"Provides the interaction-manager infrastructure that orchestrates the robot's multimodal perception and action loop."}],"review_version":1}