REVIEW 2 major objections 5 minor 48 references
Learning to Sit: Synthesizing Human-Chair Interactions via Hierarchical Control
T0 review · 2 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read A learned meta controller that switches among imitation-learned motion skills lets a simulated human sit on a chair from varied starting poses.
desk verdict Learned meta-controller over mocap subtasks is a real step, but the non-hierarchical baseline never gets the task reward, so the headline comparison is underdetermined. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is a two-level policy hierarchy: four subtask controllers, each an actor-critic policy trained with proximal policy optimization to imitate one short motion-capture clip, and a meta controller that runs at a slower rate and outputs both a discrete choice of subtask and a continuous two-dimensional walking target. The meta controller's reward only asks the pelvis to approach and contact the seat surface, so it must learn task progress through the subtask switches. Subtask controllers are fine-tuned on poses sampled from the end states of other subtasks so that transitions such as walk-to-turn are executable. The hierarchy is what carries the argument: it turns one hard long-horizon control problem into small imitation problems plus a simple selection problem.
What would settle it
Design an evaluation in which the humanoid starts directly facing the chair at close range, a configuration the paper identifies as needing a 'back up' move. If the full model's success rate in that configuration stays at zero even after extended meta-controller training, then the claim that hierarchical skill selection generalizes to the sitting task is falsified, and the skill vocabulary would have to be expanded. A complementary test is to record whether any failure trajectory involves the humanoid oscillating between skills with no progress toward the seat.
Extended reading notes
Core claim
The central claim is that interactive human-object motion can be synthesized by decomposing it into reusable mocap-imitated subtask controllers and learning a meta controller that selects among them in real time. Evaluated on the task of sitting onto a chair, the full hierarchical model reaches a success rate of 31.61 percent with a mean minimum pelvis-to-seat distance of 0.33 meters in the easy setting, outperforming a fixed walk-turn-sit order, single-turn variants, and non-hierarchical kinematic and physics-based imitators. In harder starting zones the method still succeeds occasionally, and a curriculum that begins training near the chair before moving to harder spawn points raises success in the rear zone from 4.05 to 7.05 percent. The authors also argue the same framework can predict sitting motion from a single image when paired with 3D scene reconstruction.
Load-bearing premise
The sitting task must be fully decomposable into the four skills of walking, left turn, right turn, and sitting; if a successful strategy needs any other move, such as backing up when directly facing the chair, the meta controller cannot produce it and the method fails.
Editorial extensions
If this is right
- A physics-based character can handle a class of interactive tasks with variable human-object configurations, not just a single repeating motion pattern.
- Fixed-order hierarchical plans fail when the right sequence depends on the start pose; a learned meta controller that can switch freely outperforms them.
- Pretrained reusable subtask skills make a new interactive task trainable with only the meta controller learning from sparse task rewards.
- Curriculum learning from easy to hard spawn locations improves success in the harder parts of the task.
- The same subtask repertoire can be driven from visual input, suggesting a path from a single image to plausible physical interaction.
Reading between the lines
- A natural next test is to enlarge the skill vocabulary, for example adding a backup or sidestep skill; the paper's own failure analysis predicts this would raise the success ceiling, since the current meta controller cannot recover when the chair blocks the direct path.
- The approach should transfer to other sit-able furniture of varying seat height and orientation if the sit subtask state feature includes the seat pose; this is a direct extension the authors do not evaluate.
- The reported numbers suggest that success rate is dominated by subtask execution reliability and transition quality, so improving low-level imitation accuracy may matter more than meta-policy architecture.
- Because the meta controller is trained separately from the skills, swapping in a new chair model or a new skill requires retraining only one level, which is a practical advantage for embodied agents.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a hierarchical reinforcement learning framework for the task of a physics-based character sitting onto a chair. The lower level consists of subtask policies (walk, left turn, right turn, sit) trained to imitate short CMU mocap clips via PPO, while a higher-level meta controller learns to select among these subtasks and to set the walking target. The meta controller is trained with a curriculum over spawn zones, and the whole system is evaluated by success rate (pelvis contact with the seat for 3 seconds) and mean minimum distance in Easy and Hard settings. The paper reports 31.61% success for the full model in the Easy setting, outperforming fixed-order hierarchical baselines, ablated single-turn variants, and two non-hierarchical baselines; it also shows that curriculum learning improves Hard-setting success. An image-based application is demonstrated qualitatively.
Significance. If the comparative claim holds, the paper makes a useful contribution to physics-based character animation by showing that a hierarchical decomposition into mocap-imitated subtasks can handle a task with variable human-chair configurations. The paper is transparent about its low absolute success rates, provides subtask-level diagnostics, reports transition statistics, and includes several ablation-style baselines. The explicit self-acknowledged limitation of an insufficient subtask repertoire is a sign of honest reporting. However, the central claim that hierarchy outperforms non-hierarchical approaches is currently under-supported because the only non-hierarchical physics baseline is an imitation policy that never optimizes the task reward; and the margin over the best ablated hierarchical baseline is small and reported without statistical support.
major comments (2)
- [Results, Table 1] The non-hierarchical physics baseline is a DeepMimic-style policy trained solely to imitate one holistic mocap clip (walk+turn+sit), and it never receives the meta-task reward of Eq. 8. Its 0.00% success rate and 1.33 m minimum distance therefore show that blind imitation fails, not that a flat non-hierarchical RL policy trained on the same task reward would fail. The paper's central claim that hierarchical control beats non-hierarchical control requires a control experiment: a flat PPO policy trained on the same 57-d state, same 21-d action space, same reward, and same Easy/Hard curriculum. Without this baseline, the advantage of the hierarchy could be an artifact of giving the hierarchical method task-reward supervision while denying it to the flat baseline. Please add this baseline or explicitly restrict the claim to 'hierarchy over imitation-only non-hierarchical baselines.'
- [Results, Table 1] The full model's advantage over the best ablated hierarchical baseline (31.61% vs 29.38%) is small, and the paper reports no confidence intervals, significance tests, or multiple training seeds. Given the low absolute success rate and the paper's own analysis listing subtask execution failures, transition failures, and an insufficient subtask repertoire, the claim that the full model 'outperforms' these baselines needs statistical support. At minimum, the authors should report standard errors across multiple training runs or a paired significance test on the 10,000 evaluation trials.
minor comments (5)
- [Analysis, final paragraph] The authors transparently acknowledge that a failure case requires a 'back up' move and that a more diverse subtask skill set is future work. This is an explicit scope limitation: the success rates are upper bounds for the chosen four-subtask vocabulary, not for the general sitting task. I recommend stating this caveat in the abstract or conclusion, since the current abstract's phrase 'flexible motion patterns' is broader than what the method can deliver.
- [Introduction] There is a typo in the Introduction: 'sythensized' should be 'synthesized'.
- [Appendix, Additional Data] In the appendix, 'hyerparamters' should be 'hyperparameters'.
- [Table 1] For the kinematics baseline, the success rate is reported as '--' and only distance is given; please clarify in the caption that success cannot be computed for a purely kinematic replay because contact is not simulated, or explain why success is not reported.
- [Results, Easy Setting] The definition of the 'Easy' setting states that Zone 1 is roughly 2 meters away on the front half plane, but the curriculum figure and Table 3 also use Zones 2 and 3; please make the zone definitions and spawn distributions consistent in one place to avoid ambiguity.
Circularity Check
No significant circularity: empirical RL evaluation against external baselines; reward/evaluation alignment is standard task specification.
full rationale
This paper makes no first-principles derivation claim. The central claim—that hierarchical control with mocap-imitated subtasks outperforms non-hierarchical and fixed-order baselines—is supported by the Table 1 experiments. The meta-controller reward (Eq. 8: 1 if pelvis-seat contact else 0.5(-V_sit)) and the evaluation metrics (contact for 3 seconds, minimum pelvis-seat distance) share the same quantities, but this is the task specification used consistently across all methods, not a parameter fitted to a subset of data and renamed as a prediction. The only self-citation (Chao et al. 2017) is a literature pointer in related work and is not load-bearing. The acknowledged limitation that a 'back up' skill is missing from the subtask repertoire is an assumption about task decomposition, not a circular step. The absence of a flat task-reward PPO baseline is a legitimate experimental-design concern, because the comparative claim would be stronger with such a control, but it does not make any result true by construction; it is a correctness risk rather than circularity. No circular step can be exhibited from the paper's own equations. Score 0.
Assumptions & free parameters
free parameters (3)
- Imitation reward weights in Eq. 2-3 =
ωp=0.5, ωv=0.05, αp=1, αv=10
- Early termination thresholds =
root height 0.78 m for walk/turn, 0.54 m for sit, yaw deviation 45 degrees for turn
- PPO hyperparameters =
nsteps 8192/64, nminibatches 32/8, noptepochs 4/2, lr 1e-4
assumptions (4)
- ad hoc to paper The sitting task can be decomposed into the four subtasks walk, left turn, right turn, and sit, and all successful strategies from the tested start zones can be composed from them.
- domain assumption Mocap clips retargeted to the Bullet humanoid provide sufficiently faithful reference motions for learning natural walking, turning, and sitting.
- domain assumption Bullet physics contact and torque actuation model human-chair interaction well enough for the reported success criterion, pelvis contact for 3 seconds, to be meaningful.
- standard math PPO with the stated hyperparameters reliably converges to useful subtask and meta policies.
Cite this review
Pith. "Pith review of Learning to Sit: Synthesizing Human-Chair Interactions via Hierarchical Control." pith.science (2026). https://pith.science/paper/MAHT4FYD
@misc{pith2026190807423,
author = {Pith},
title = {Pith review of: Learning to Sit: Synthesizing Human-Chair Interactions via Hierarchical Control},
year = {2026},
howpublished = {\url{https://pith.science/paper/MAHT4FYD}},
note = {Machine review of arXiv:1908.07423}
}
read the original abstract
Recent progress on physics-based character animation has shown impressive breakthroughs on human motion synthesis, through imitating motion capture data via deep reinforcement learning. However, results have mostly been demonstrated on imitating a single distinct motion pattern, and do not generalize to interactive tasks that require flexible motion patterns due to varying human-object spatial configurations. To bridge this gap, we focus on one class of interactive tasks -- sitting onto a chair. We propose a hierarchical reinforcement learning framework which relies on a collection of subtask controllers trained to imitate simple, reusable mocap motions, and a meta controller trained to execute the subtasks properly to complete the main task. We experimentally demonstrate the strength of our approach over different non-hierarchical and hierarchical baselines. We also show that our approach can be applied to motion prediction given an image input. A supplementary video can be found at https://youtu.be/3CeN0OGz2cA.
Figures
Figures from the paper (4 more)
Reference graph
Works this paper leans on
-
[1]
, " * write output.state after.block = add.period write newline
ENTRY address author booktitle chapter doi edition editor eid howpublished institution isbn issn journal key month note number organization pages publisher school series title type url volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.all := #1...
-
[2]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
-
[3]
Agrawal, S.; and van de Panne , M. 2016. Task-based locomotion. In SIGGRAPH
work page 2016
-
[4]
u tepage, J.; Black, M. J.; Kragic, D.; and Kjellstr\
B\" u tepage, J.; Black, M. J.; Kragic, D.; and Kjellstr\" o m, H. 2017. Deep representation learning for human motion prediction and classification. In CVPR
work page 2017
-
[5]
Chang, A. X.; Funkhouser, T.; Guibas, L.; Hanrahan, P.; Huang, Q.; Li, Z.; Savarese, S.; Savva, M.; Song, S.; Su, H.; Xiao, J.; Yi, L.; and Yu, F. 2015. Shape N et: An Information-Rich 3 D Model Repository. arXiv preprint arXiv:1512.03012
arXiv 2015
-
[6]
Chao, Y.-W.; Yang, J.; Price, B.; Cohen, S.; and Deng, J. 2017. Forecasting Human Dynamics from Static Images. In CVPR
work page 2017
-
[7]
Clegg, A.; Yu, W.; Tan, J.; Liu, C. K.; and Turk, G. 2018. Learning to Dress: Synthesizing Human Dressing Motion via Deep Reinforcement Learning. In SIGGRAPH Asia
work page 2018
-
[8]
CMU . 2003. CMU G raphics L ab M otion C apture D atabase. http://mocap.cs.cmu.edu
work page 2003
Show all 48 references
-
[9]
2016--2019
Coumans, E.; and Bai, Y. 2016--2019. Py B ullet, a P ython module for physics simulation for games, robotics and machine learning. http://pybullet.org
2016
-
[10]
F.; Laptev, I.; Sivic, J.; Gupta, A.; and Efros, A
Delaitre, V.; Fouhey, D. F.; Laptev, I.; Sivic, J.; Gupta, A.; and Efros, A. A. 2012. Scene Semantics from Long-Term Observation of People. In ECCV
2012
-
[11]
Dhariwal, P.; Hesse, C.; Klimov, O.; Nichol, A.; Plappert, M.; Radford, A.; Schulman, J.; Sidor, S.; Wu, Y.; and Zhokhov, P. 2017. O pen AI B aselines. https://github.com/openai/baselines
2017
-
[12]
Fragkiadaki, K.; Levine, S.; Felsen, P.; and Malik, J. 2015. Recurrent Network Models for Human Dynamics. In ICCV
2015
-
[13]
Ghosh, P.; Song, J.; Aksan, E.; and Hilliges, O. 2017. Learning Human Motion Models for Long-Term Predictions. In 3DV
2017
-
[14]
Gui, L.-Y.; Wang, Y.-X.; Liang, X.; and Moura, J. M. F. 2018 a . Adversarial Geometry-Aware Human Motion Prediction. In ECCV
2018
-
[15]
Gui, L.-Y.; Wang, Y.-X.; Ramanan, D.; and Moura, J. M. F. 2018 b . Few-Shot Human Motion Prediction via Meta-Learning. In ECCV
2018
-
[16]
A.; and Hebert, M
Gupta, A.; Satkin, S.; Efros, A. A.; and Hebert, M. 2011. From 3 D Scene Geometry to Human Workspace. In CVPR
2011
-
[17]
Heess, N.; TB, D.; Sriram, S.; Lemmon, J.; Merel, J.; Wayne, G.; Tassa, Y.; Erez, T.; Wang, Z.; Eslami, S. M. A.; Riedmiller, M.; and Silver, D. 2017. Emergence of Locomotion Behaviours in Rich Environments. arXiv preprint arXiv:1707.02286
2017 arXiv
-
[18]
Heess, N.; Wayne, G.; Tassa, Y.; Lillicrap, T.; Riedmiller, M.; and Silver, D. 2016. Learning and Transfer of Modulated Locomotor Controllers. arXiv preprint arXiv:1610.05182
2016 arXiv
-
[19]
Holden, D.; Komura, T.; and Saito, J. 2017. Phase-Functioned Neural Networks for Character Control. In SIGGRAPH
2017
-
[20]
Holden, D.; Saito, J.; and Komura, T. 2016. A Deep Learning Framework for Character Motion Synthesis and Editing. In SIGGRAPH
2016
-
[21]
Huang, S.; Qi, S.; Zhu, Y.; Xiao, Y.; Xu, Y.; and Zhu, S.-C. 2018. Holistic 3 D Scene Parsing and Reconstruction from a Single RGB Image. In ECCV
2018
-
[22]
R.; Savarese, S.; and Saxena, A
Jain, A.; Zamir, A. R.; Savarese, S.; and Saxena, A. 2016. Structural- RNN : Deep Learning on Spatio-Temporal Graphs. In CVPR
2016
-
[23]
Kovar, L.; Gleicher, M.; and Pighin, F. 2002. Motion Graphs. In SIGGRAPH
2002
-
[24]
D.; Narasimhan, K.; Saeedi, A.; and Tenenbaum, J
Kulkarni, T. D.; Narasimhan, K.; Saeedi, A.; and Tenenbaum, J. 2016. Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation. In NIPS
2016
-
[25]
S.; and Lee, G
Li, C.; Zhang, Z.; Lee, W. S.; and Lee, G. H. 2018. Convolutional Sequence to Sequence Model for Human Dynamics. In CVPR
2018
-
[26]
Liu, L.; and Hodgins, J. 2017. Learning to Schedule Control Fragments for Physics-Based Characters Using Deep Q -Learning. ToG 36(3)
2017
-
[27]
Liu, L.; and Hodgins, J. 2018. Learning Basketball Dribbling Skills Using Trajectory Optimization and Deep Reinforcement Learning. In SIGGRAPH
2018
-
[28]
Liu, L.; van de Panne , M.; and Yin, K. 2016. Guided Learning of Control Graphs for Physics-Based Characters. ToG 35(3): 29:1--29:14
2016
-
[29]
J.; ; and Romero, J
Martinez, J.; Black, M. J.; ; and Romero, J. 2017. On human motion prediction using recurrent neural networks. In CVPR
2017
-
[30]
Merel, J.; Ahuja, A.; Pham, V.; Tunyasuvunakool, S.; Liu, S.; Tirumala, D.; Heess, N.; and Wayne, G. 2019 a . Hierarchical Visuomotor Control of Humanoids. In ICLR
2019
-
[31]
W.; and Heess, N
Merel, J.; Hasenclever, L.; Galashov, A.; Ahuja, A.; Pham, V.; Wayne, G.; Teh, Y. W.; and Heess, N. 2019 b . Neural Probabilistic Motor Primitives for Humanoid Control. In ICLR
2019
-
[32]
Merel, J.; Tassa, Y.; TB, D.; Srinivasan, S.; Lemmon, J.; Wang, Z.; Wayne, G.; and Heess, N. 2017. Learning human behaviors from motion capture by adversarial imitation. arXiv preprint arXiv:1707.02201
2017 arXiv
-
[33]
O pen AI . 2017. O pen AI R oboschool. https://blog.openai.com/roboschool/
2017
-
[34]
B.; Abbeel, P.; Levine, S.; and van de Panne , M
Peng, X. B.; Abbeel, P.; Levine, S.; and van de Panne , M. 2018 a . Deep M imic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills. In SIGGRAPH
2018
-
[35]
B.; Berseth, G.; Yin, K.; and van de Panne , M
Peng, X. B.; Berseth, G.; Yin, K.; and van de Panne , M. 2017. Deep L oco: Developing Locomotion Skills Using Hierarchical Deep Reinforcement Learning. In SIGGRAPH
2017
-
[36]
B.; Chang, M.; Zhang, G.; Abbeel, P.; and Levine, S
Peng, X. B.; Chang, M.; Zhang, G.; Abbeel, P.; and Levine, S. 2019. MCP : Learning Composable Hierarchical Control with Multiplicative Compositional Policies. In NeurIPS
2019
-
[37]
B.; Kanazawa, A.; Malik, J.; Abbeel, P.; and Levine, S
Peng, X. B.; Kanazawa, A.; Malik, J.; Abbeel, P.; and Levine, S. 2018 b . SFV : Reinforcement Learning of Physical Skills from Videos. In SIGGRAPH Asia
2018
-
[38]
Schulman, J.; Wolski, F.; Dhariwal, P.; Radford, A.; and Klimov, O. 2017. Proximal Policy Optimization Algorithms. arXiv preprint arXiv:1707.06347
2017 arXiv
-
[39]
J.; and Mannor, S
Tessler, C.; Givony, S.; Zahavy, T.; Mankowitz, D. J.; and Mannor, S. 2017. A Deep Hierarchical Approach to Lifelong Learning in M inecraft. In AAAI
2017
-
[40]
Villegas, R.; Yang, J.; Ceylan, D.; and Lee, H. 2018. Neural Kinematic Networks for Unsupervised Motion Retargetting. In CVPR
2018
-
[41]
Walker, J.; Marino, K.; Gupta, A.; and Hebert, M. 2017. The Pose Knows: Video Forecasting by Generating Pose Futures. In ICCV
2017
-
[42]
Wang, X.; Girdhar, R.; and Gupta, A. 2017. Binge Watching: Scaling Affordance Learning From Sitcoms. In CVPR
2017
-
[43]
J.; and Hodgins, J
Yamane, K.; Kuffner, J. J.; and Hodgins, J. K. 2004. Synthesizing Animations of Human Manipulation Tasks. In SIGGRAPH
2004
-
[44]
Yan, X.; Rastogi, A.; Villegas, R.; Sunkavalli, K.; Shechtman, E.; Hadap, S.; Yumer, E.; and Lee, H. 2018. MT - VAE : Learning Motion Transformations to Generate Multimodal Human Dynamics. In ECCV
2018
-
[45]
Yao, T.; Wang, M.; Ni, B.; Wei, H.; and Yang, X. 2018. Multiple Granularity Group Interaction Prediction. In CVPR
2018
-
[46]
Zaremba, W.; and Sutskever, I. 2014. Learning to Execute. arXiv preprint arXiv:1410.4615
2014 arXiv
-
[47]
Zhou, Y.; Li, Z.; Xiao, S.; He, C.; Huang, Z.; and Li, H. 2018. Auto-Conditioned Recurrent Networks for Extended Complex Human Motion Synthesis. In ICLR
2018
-
[48]
Zhu, Y.; Zhao, Y.; and Zhu, S.-C. 2015. Understanding Tools: Task-Oriented Object Modeling, Learning and Recognition. In CVPR
2015
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.