REVIEW 4 major objections 4 minor 34 references
Use of Winsome Robots for Understanding Human Feedback (UWU)
T0 review · 4 major / 4 minor · reviewed 2026-08-08 · deepseek-v4-flash
Pith's one-line read The paper claims that a robot's perceived cuteness shifts human feedback toward a higher ratio of positive to negative critiques, and that a stochastic version of TAMER can compensate for this bias.
desk verdict Candid pilot study with a solid cuteness manipulation, but the central feedback-ratio claim is overstated and possibly order-confounded. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The study's central object is the baby schema face manipulation: a 'cute' face with large eyes low on the face versus a control face with smaller eyes higher up, built on the baby schema principle from developmental psychology. The outcome metric is the ratio of positive to negative keypress critiques given during playback of four prerecorded pick-and-place trajectories per robot. For the algorithmic half, the central mechanism is a user score inside Stochastic TAMER that judges whether each piece of user feedback was appropriate by comparing it against the environmental reward for that action; when the score falls below a threshold, the algorithm swaps positive feedback for negative with a probability that increases as the user score stays low.
What would settle it
Run the same within-subject procedure with a preregistered sample of at least 60 participants: if the cute robot's positive-to-negative feedback ratio is not reliably higher than the control's at $p < 0.05$, the claimed shift fails. Then run Stochastic TAMER in Wumpus World without providing the per-action environmental reward; if the user score cannot be computed or performance does not beat plain TAMER under 66 percent positive feedback, the proposed mitigation is not as described.
Extended reading notes
Core claim
The central claim is that a robot's perceived cuteness is a cause of the well-documented positive feedback bias in interactive robot learning, not just a background property of the platform. The authors establish the perception manipulation first: the baby-schema face (larger eyes lower on the face) is rated significantly cuter than the control (mean 4.0 vs 2.79, $p < 0.01$). They then report that participants gave the cute robot a higher ratio of positive to negative feedback (2.65 ± 2.26 vs 1.71 ± 1.23), with a one-tailed paired t-test at $p = 0.061$, which they interpret as a trend warranting further study. The paper further claims that this skewed feedback degrades learning in TAMER-style algorithms and that a proposed Stochastic TAMER, which stochastically converts positive feedback into negative when a user score indicates poor feedback quality, can mitigate the problem while retaining identical performance when feedback is optimal.
Load-bearing premise
The Stochastic TAMER user score assumes the algorithm has access to the true environmental reward for each action so it can tell whether a human's praise was deserved, which is exactly the ground-truth signal that TAMER-style learning is designed to do without.
Editorial extensions
If this is right
- Aesthetic design choices are not neutral for robot learning: making a robot cuter can inflate the praise signal a learning algorithm receives.
- Systems that learn from human feedback should estimate and compensate for the user's positive feedback bias rather than treating every critique as ground truth.
- The Stochastic TAMER modification, if validated, would let an agent break out of positive reward circuits without needing a change in the human's behavior.
- Researchers designing user studies for interactive robot learning should control for robot appearance as a variable, not just as a confound to mention.
- Pilot results at $p = 0.061$ justify larger preregistered studies rather than dismissing the effect.
Reading between the lines
- The cuteness effect likely generalizes beyond faces to other baby-schema cues such as body proportions, voice, and movement, since the mechanism is affective response, not visual pixels.
- The Stochastic TAMER design as described has an oracle problem, since it requires the very environmental reward that TAMER is meant to replace, so a practical version would need to estimate feedback appropriateness from trajectory outcomes or user consistency.
- A between-subjects design with more participants and a washout period could separate the order and anchoring effects the authors observed from the true cuteness effect.
- If the effect holds, deploying cuter robots in real feedback-driven learning systems will systematically bias collected data, which RLHF-style pipelines in consumer robots should account for.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper reports a within-subject pilot study (n=14) investigating whether a robot's perceived cuteness affects the ratio of positive to negative feedback that users give while critiquing pick-and-place trajectories. The authors find that their baby-schema face manipulation is effective (H1: t(13)=3.19, p<0.01), and they report a higher positive/negative feedback ratio for the cute robot than the control (2.65 vs 1.71) with a one-tailed paired t-test reported as t(14)=1.66, p=0.061. They report that H2 is not supported at the conventional threshold in Section IV.B, but the conclusion states that perceived cuteness "does have a statistical impact" on feedback ratio. The paper also proposes a Stochastic TAMER variant that judges the optimality of user feedback using the environmental reward and stochastically swaps positive feedback for negative when a user score falls below a threshold, with a small Wumpus World evaluation across three trials.
Significance. If the main empirical claim were established, the paper would address a relevant and understudied question in HRI and interactive robot learning: whether aesthetic cuteness systematically skews the reward signal that humans provide to learning agents. The manuscript is honest in reporting the primary result as p=0.061, and the cuteness manipulation is well supported by the data. The use of prerecorded trajectories controls for variability in robot behavior, and the authors explicitly enumerate confounds in the limitations section. However, the central new empirical claim rests on a single borderline p-value with an inconsistent degrees-of-freedom report and a plausible order confound that is acknowledged but not resolved. The Stochastic TAMER contribution is preliminary and rests on a reward-availability assumption that conflicts with the original TAMER setup. The paper is suitable as a pilot-study report if the claims are tempered and the load-bearing issues are addressed, but not as definitive evidence of a cuteness effect on feedback.
major comments (4)
- [§III.C and §V] The order in which participants saw the two robots is a direct threat to the internal validity of the feedback-ratio comparison, yet the Methods do not state whether the cute/control order was counterbalanced or randomized across participants. Section V reports that the first-seen robot received a higher overall rating (3.86 vs 3.35) and that cuteness ratings shifted systematically with order. If the cute robot was not equally often seen first and second, the observed difference in positive/negative feedback ratio (2.65 vs 1.71) could be an artifact of order, anchoring, or fatigue rather than cuteness. Please report the order assignment, provide the feedback-ratio comparison separately by order, and either control for order in the analysis or clearly restrict the conclusion to an exploratory trend.
- [§IV.B] The reported degrees of freedom for the feedback-ratio test are inconsistent with the sample size. The paper states t(14)=1.66 for a paired test with n=14 participants; the correct degrees of freedom are 13, not 14. This discrepancy could affect the exact p-value and should be corrected. In addition, the paper should report the effect size and a confidence interval for the mean difference, since the p-value alone does not convey the magnitude or precision of the claimed shift.
- [§VII vs §IV.B] The conclusion overstates the evidence for H2. Section IV.B explicitly says "we cannot confidently state that our second hypothesis (H2) is supported," but Section VII states that "perceived cuteness does have a statistical impact on a human's ratio of positive to negative feedback." These statements are contradictory. The conclusion must be aligned with the reported p=0.061 and the acknowledged confounds, or new supporting evidence must be provided.
- [§III.D and §IV.C] The Stochastic TAMER user-score mechanism assumes that the algorithm can access the environmental reward for each action when determining whether user feedback was appropriate. This assumption conflicts with the core premise of TAMER described in Section II.A, which is to learn without an environmental reward function. The simulation evaluation is also very limited: only three trials are reported, with no statistical comparison across runs or across simulated users, and the statement that the optimal-feedback condition has "standard deviation of zero" is not a substitute for an analysis of variability. Please either reframe the algorithm as an illustrative proof-of-concept that relaxes TAMER's assumptions, or provide a more thorough evaluation and discuss the reward-availability limitation.
minor comments (4)
- [§III.C.1] The text says the two faces are "as seen in Figure 2," but Figure 2 shows the pick-and-place task; the faces appear in Figure 3. Please correct the cross-reference.
- [§III.C.2] The design is described as giving each participant a random sample of four prerecorded trajectories, but the earlier text says half of the 12 trajectories are successes and half are failures. It is unclear whether each participant always received exactly two successes and two failures, or whether the random sample could produce an uneven split. Please clarify.
- [§IV.C] The Wumpus World experiments lack essential details: the state/action space, the user feedback model that generates the specified positive-feedback rates, the number of episodes, and the aggregation across the three trials. Adding these details would make the simulation reproducible.
- [§III.C.1] The figure captions for Figure 2 and Figure 3 are interchanged in content: Figure 2 is labeled as an illustration of the task but is referenced in the text for the face display, while Figure 3 shows the faces. Please ensure each figure number matches its caption.
Circularity Check
No circularity; the cuteness-feedback effect is an empirical measurement and the stochastic TAMER result, while relying on ground-truth reward, is not a fitted or self-referential prediction.
full rationale
No significant circularity found. The paper's central empirical claim—that perceived cuteness shifts the ratio of positive to negative feedback—is an experimental measurement. The cute and control faces are independently manipulated according to the baby schema, perceived cuteness is verified by a paired t-test (Section IV.A), and the feedback ratio is compared between the two conditions (Section IV.B). Neither quantity is fitted to the other: the cuteness stimulus is not defined in terms of the feedback ratio, and the measured feedback ratio is not used to construct the cuteness stimulus. The reported effect is weak (p = 0.061) and the paper's own Section V reports a first-robot order effect (overall rating 3.86 vs 3.35) that could confound the within-subject comparison, but this is a threats-to-validity concern, not a circular derivation. The stochastic TAMER addition (Section III.D) does use the environmental reward to evaluate whether user feedback is appropriate, and then stochastically swaps positive feedback for negative when the user score is low; in the Wumpus World simulations this makes the algorithm's improved returns follow fairly directly from having access to ground-truth reward. That is a design assumption and a practical limitation—it conflicts with TAMER's reward-free premise—but it is not a fitted parameter renamed as a prediction, and no simulation outcome is used to define the mechanism post hoc. The self-citations in the background section (e.g., references [16], [18], [19]) are not load-bearing for the paper's empirical conclusions, and no uniqueness theorem or prior result by the same authors is invoked to force a choice. The derivation chain is therefore self-contained with respect to circularity, even though the study has statistical and validity weaknesses.
Assumptions & free parameters
free parameters (2)
- Stochastic TAMER threshold
- Initial swap probability p
assumptions (3)
- domain assumption Baby schema features (large eyes, high forehead) increase perceived cuteness.
- ad hoc to paper The environmental reward is available to compute user score optimality.
- standard math Paired t-test assumptions apply to the ratio of positive to negative feedback.
Cite this review
Pith. "Pith review of Use of Winsome Robots for Understanding Human Feedback (UWU)." pith.science (2026). https://pith.science/paper/DI3J6BUE
@misc{pith2026250205118,
author = {Pith},
title = {Pith review of: Use of Winsome Robots for Understanding Human Feedback (UWU)},
year = {2026},
howpublished = {\url{https://pith.science/paper/DI3J6BUE}},
note = {Machine review of arXiv:2502.05118}
}
read the original abstract
As social robots become more common, many have adopted cute aesthetics aiming to enhance user comfort and acceptance. However, the effect of this aesthetic choice on human feedback in reinforcement learning scenarios remains unclear. Previous research has shown that humans tend to give more positive than negative feedback, which can cause failure to reach optimal robot behavior. We hypothesize that this positive bias may be exacerbated by the robot's level of perceived cuteness. To investigate, we conducted a user study where participants critique a robot's trajectories while it performs a task. We then analyzed the impact of the robot's aesthetic cuteness on the type of participant feedback. Our results suggest that there is a shift in the ratio of positive to negative feedback when perceived cuteness changes. In light of this, we experiment with a stochastic version of TAMER which adapts based on the user's level of positive feedback bias to mitigate these effects.
Figures
Figures from the paper (4 more)
Reference graph
Works this paper leans on
-
[1]
Die angeborenen Formen m ¨oglicher Erfahrung,
K. Lorenz, “Die angeborenen Formen m ¨oglicher Erfahrung,” Zeitschrift f¨ur Tierpsychologie, vol. 5, no. 2, pp. 235–409, 1943, doi: https://doi.org/10.1111/j.1439- 0310.1943.tb00655.x
-
[2]
The (Ir)relevance of Robot Cuteness: An Exploratory Study of Emotionally Durable Robot Design,
C. Caudwell, C. Lacey, and E. B. Sandoval, “The (Ir)relevance of Robot Cuteness: An Exploratory Study of Emotionally Durable Robot Design,” in *Proceedings of the 31st Australian Conference on Human-Computer Interaction (OzCHI ’19)*, Association for Computing Machinery, New York, NY , USA, 2020, pp. 64–72
work page 2020
-
[3]
C.-H. Chen and X. Jia, “Research on the influence of the baby schema ef- fect on the cuteness and trustworthiness of social robot faces,” *International Journal of Advanced Robotic Systems*, vol. 20, no. 3, 2023
work page 2023
-
[4]
Human-robot interaction: the impact of robotic aesthetics on anticipated human trust,
J. Pinney, F. Carroll, and P. Newbury, “Human-robot interaction: the impact of robotic aesthetics on anticipated human trust,” PeerJ Computer Science, vol. 8, p. e837, Jan. 2022, doi: https://doi.org/10.7717/peerj-cs.837
-
[5]
Reinforcement learning from human reward: Discounting in episodic tasks,
W. B. Knox and P. Stone, “Reinforcement learning from human reward: Discounting in episodic tasks,” in *2012 IEEE RO-MAN: The 21st IEEE International Symposium on Robot and Human Interactive Communication*, Paris, France, 2012, pp. 878-885
work page 2012
-
[6]
How humans teach agents: A new experimental perspective,
W. B. Knox, B. D. Glass, B. C. Love, W. T. Maddox, and P. Stone, “How humans teach agents: A new experimental perspective,” *International Journal of Social Robotics*, vol. 4, no. 4, pp. 409–421, 2012
work page 2012
-
[7]
Teachable robots: Understanding human teaching behavior to build more effective robot learners,
A. L. Thomaz and C. Breazeal, “Teachable robots: Understanding human teaching behavior to build more effective robot learners,” *Artificial Intelligence*, vol. 172, no. 6–7, pp. 716-737, 2008
work page 2008
-
[8]
Learning and Comfort in Human–Robot Interaction: A Review,
W. Wang, Y . Chen, R. Li, and Y . Jia, “Learning and Comfort in Human–Robot Interaction: A Review,” *Appl. Sci.*, vol. 9, no. 5152, 2019
work page 2019
Show all 34 references
-
[9]
Recent advances in robot learning from demonstration,
H. Ravichandar, A. S. Polydoros, S. Chernova, and A. Billard, “Recent advances in robot learning from demonstration,” *Annual Review of Control, Robotics, and Autonomous Systems*, vol. 3, no. 1, pp. 297–330, 2020
2020
-
[10]
A survey of robot learning from demonstration,
B. D. Argall, S. Chernova, M. Veloso, and B. Browning, “A survey of robot learning from demonstration,” *Robotics and Autonomous Systems*, vol. 57, no. 5, pp. 469–483, 2009
2009
-
[11]
Robot learning from demonstration,
C. G. Atkeson and S. Schaal, “Robot learning from demonstration,” in *Proceedings of ICML ’97*, vol. 97, pp. 12–20, July 1997
1997
-
[12]
Robot learning by demonstration,
A. Billard and D. Grollman, “Robot learning by demonstration,” *Schol- arpedia*, vol. 8, no. 12, p. 3824, 2013
2013
-
[13]
Learning from demonstration,
S. Schaal, “Learning from demonstration,” in *Advances in Neural Information Processing Systems 9*, 1996
1996
-
[14]
A robot learning from demonstration framework to perform force-based manipulation tasks,
L. Rozo, P. Jim´enez, and C. Torras, “A robot learning from demonstration framework to perform force-based manipulation tasks,” *Intelligent Service Robotics*, vol. 6, no. 1, pp. 33–51, 2013
2013
-
[15]
Interactive and Explainable Robot Learning: A Comprehensive Review,
E. Seraj, K. M. Lee, Z. Zaidi, et al., “Interactive and Explainable Robot Learning: A Comprehensive Review,” *Foundations and Trends in Robotics*, vol. 12, no. 2–3, pp. 75–349, 2024
2024
-
[16]
Human-Robot Alignment through Interactivity and Interpretability: Don’t Assume a ’Spherical Human’,
M. Gombolay, “Human-Robot Alignment through Interactivity and Interpretability: Don’t Assume a ’Spherical Human’,” in *Proceedings of the International Joint Conference on Artificial Intelligence*, 2024
2024
-
[17]
Extrapolating beyond suboptimal demonstrations via inverse reinforcement learning from observations,
D. Brown, W. Goo, P. Nagarajan, and S. Niekum, “Extrapolating beyond suboptimal demonstrations via inverse reinforcement learning from observations,” in *International Conference on Machine Learning*, PMLR, May 2019, pp. 783–792
2019
-
[18]
Learning from suboptimal demonstration via self-supervised reward regression,
L. Chen, R. Paleja, and M. Gombolay, “Learning from suboptimal demonstration via self-supervised reward regression,” in *Conference on Robot Learning*, PMLR, Oct. 2021, pp. 1262–1277
2021
-
[19]
Droid: Learning from offline heterogeneous demonstrations via reward-policy distillation,
S. Jayanthi, L. Chen, N. Balabanska, et al., “Droid: Learning from offline heterogeneous demonstrations via reward-policy distillation,” in *Conference on Robot Learning*, PMLR, Dec. 2023, pp. 1547–1571
2023
-
[20]
Beyond Success: Quantifying Demonstration Quality in Learning from Demonstration,
M. Bilal, N. Lipovetzky, D. Oetomo, and W. Johal, “Beyond Success: Quantifying Demonstration Quality in Learning from Demonstration,” in *2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)*, IEEE, Oct. 2024, pp. 5120–5127
2024
-
[21]
TAMER: Training an Agent Manually via Evaluative Reinforcement,
W. B. Knox and P. Stone, “TAMER: Training an Agent Manually via Evaluative Reinforcement,” International Conference on Development and Learning, Oct. 2008, doi: https://doi.org/10.1109/devlrn.2008.4640845
2008
-
[22]
Deep TAMER: Interactive agent shaping in high-dimensional state spaces,
G. Warnell, N. Waytowich, V . Lawhern, and P. Stone, “Deep TAMER: Interactive agent shaping in high-dimensional state spaces,” in *Proceedings of the AAAI Conference on Artificial Intelligence*, vol. 32, no. 1, Apr. 2018
2018
-
[23]
COACH: Learning continuous actions from corrective advice communicated by humans,
C. Celemin and J. Ruiz-del-Solar, “COACH: Learning continuous actions from corrective advice communicated by humans,” in *2015 International Conference on Advanced Robotics (ICAR)*, IEEE, July 2015, pp. 581–586
2015
-
[24]
Deep reinforcement learning from policy-dependent human feedback,
D. Arumugam, J. K. Lee, S. Saskin, and M. L. Littman, “Deep reinforcement learning from policy-dependent human feedback,” *arXiv preprint arXiv:1902.04257*, 2019
1902 arXiv
-
[25]
Open problems and fundamental limitations of reinforcement learning from human feedback,
S. Casper, X. Davies, C. Shi, T. K. Gilbert, J. Scheurer, J. Rando, R. Freedman, T. Korbak, D. Lindner, P. Freire, and T. Wang, “Open problems and fundamental limitations of reinforcement learning from human feedback,” *arXiv preprint arXiv:2307.15217*, 2023
2023 arXiv
-
[26]
Learning optimal advantage from preferences and mistaking it for reward,
W. B. Knox, S. Hatgis-Kessell, S. O. Adalgeirsson, S. Booth, A. Dragan, P. Stone, and S. Niekum, “Learning optimal advantage from preferences and mistaking it for reward,” in *Proceedings of the AAAI Conference on Artificial Intelligence*, vol. 38, no. 9, Mar. 2024, pp. 10066–10073
2024
-
[27]
Safe RLHF: Safe reinforcement learning from human feedback,
J. Dai, X. Pan, R. Sun, J. Ji, X. Xu, M. Liu, Y . Wang, and Y . Yang, “Safe RLHF: Safe reinforcement learning from human feedback,” *arXiv preprint arXiv:2310.12773*, 2023
2023 arXiv
-
[28]
RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMs,
S. Chaudhari, P. Aggarwal, V . Murahari, T. Rajpurohit, A. Kalyan, K. Narasimhan, A. Deshpande, and B. C. da Silva, “RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMs,” 2024
2024
-
[29]
C. L. Breazeal, *Designing Sociable Robots*. MIT Press, 2004
2004
-
[30]
Fostering Cross-Cultural Research by Cross-Cultural Student Teams: A Case Study Related to (Cute) Robot Design,
D. Berque, H. Chiba, M. Ohkura, P. Sripian, and M. Sugaya, “Fostering Cross-Cultural Research by Cross-Cultural Student Teams: A Case Study Related to (Cute) Robot Design,” in *Cross-Cultural Design. User Expe- rience of Products, Services, and Intelligent Environments (HCII 2...
2020
-
[31]
Austermann, S
A. Austermann, S. Yamada, K. Funakoshi and M. Nakano, ”Does the appearance of a robot affect users’ ways of giving Commands and feedback?,” 19th International Symposium in Robot and Human Interactive Communication, Viareggio, Italy, 2010, pp. 234-239, doi: 10.1109/ROMAN.2010.5598628
2010
-
[32]
Baby Schema in Infant Faces Induces Cuteness Perception and Motivation for Caretaking in Adults,
M. L. Glocker, D. D. Langleben, K. Ruparel, J. W. Loughead, R. C. Gur, and N. Sachser, “Baby Schema in Infant Faces Induces Cuteness Perception and Motivation for Caretaking in Adults,” Ethology, vol. 115, no. 3, pp. 257–263, Mar. 2009, doi: https://doi.org/10.1111/j.1439-0310...
2009
-
[33]
Isn’t it cute: An evolutionary perspective of baby-schema effects in visual product designs,
L. Miesler, H. Leder, and A. Herrmann, “Isn’t it cute: An evolutionary perspective of baby-schema effects in visual product designs,” *International Journal of Design*, vol. 5, no. 3, pp. 17–30, 2011
2011
-
[34]
Interactively shaping agents via human reinforcement: the TAMER framework,
W. B. Knox and P. Stone, “Interactively shaping agents via human reinforcement: the TAMER framework,” in *Proceedings of the Fifth International Conference on Knowledge Capture (K-CAP ’09)*, Association for Computing Machinery, New York, NY , USA, 2009, pp. 9–16
2009
Reviewed August 8, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.