REVIEW 4 major objections 5 minor 28 references
Interactive Trajectory Adaptation through Force-guided Bayesian Optimization
T0 review · 4 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read The paper claims that physical forces applied by a human can guide a robot to locally adapt its demonstration-learned trajectory through Bayesian optimization, converging in about five to six trials.
desk verdict A clean integration of local Bayesian optimization with HSMM trajectory distributions that adapts spatial and temporal patterns from force cues, but the evidence is too thin to support the human-interaction claims. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is a hidden semi-Markov model (HSMM), a probabilistic encoding of demonstrated trajectories that includes explicit lognormal state-duration distributions alongside per-state Gaussian observation models. It carries the argument in two roles: it supplies the nominal trajectory distribution via a smooth trajectory-retrieval step, and it identifies which state should be locally adapted when a force cue arrives, which restricts Bayesian optimization to a low-dimensional parameter vector. The adaptation itself is driven by expected-improvement Bayesian optimization with a Gaussian process surrogate, minimizing the norm of the difference between the reference and sensed forces.
What would settle it
Run the same framework with human subjects pushing the end-effector in a direction that increases rather than decreases the force norm (for example, pressing against the robot's preferred path to indicate a detour). If the adapted trajectory does not follow the human's indicated direction, or if the disagreement force is not driven to near zero because the human's intent is not encoded as force minimization, the central claim fails. A real-human dataset with noisy, non-monotonic force readings relative to the intended target would settle the assumption.
Extended reading notes
Core claim
The central claim is that interaction forces, interpreted as noisy cues to an unobservable objective the human is trying to optimize, can drive a local, data-efficient adaptation of a learned trajectory model. Concretely, the paper proposes to minimize the disagreement between the reference force distribution and sensed forces by running Bayesian optimization over the mean of the motor commands and the mean duration of the HSMM state in which the adaptation is triggered. Because the active state is identified from the HSMM forward variable, the parameter vector remains tiny, and the search domain is automatically bounded by the learned state covariance. The paper reports that this local search converges in five or six iterations and yields reference trajectory distributions that deform only in the adapted region, whereas optimizing all HSMM means at once fails due to the curse of dimensionality.
Load-bearing premise
The load-bearing premise is that the force a human applies is a trustworthy, informative signal of the intended trajectory change, so that minimizing the measured force disagreement is the same as fulfilling the human's intent; the paper assumes this rather than testing it with real human forces.
Editorial extensions
If this is right
- If the method is correct, a robot can incorporate a human's physical correction into a demonstration-learned skill without retraining the model from scratch.
- Because the adaptation is local, the trajectory returns to the nominal plan when the human stops pushing, which suits tasks with fixed start and end constraints.
- Since the search domain is inferred from the model's own covariance, the method avoids an explicit handcrafted reward function and needs only a handful of physical trials.
- Spatial and temporal patterns can be adapted together, because the optimized parameters include both the state mean and the state duration.
- The Bayesian optimization backbone leaves room to add constraints, such as limiting deviation from the nominal plan or penalizing jerky motion.
Reading between the lines
- A natural next step, which the paper does not test, is to run the method with real human force data rather than a synthetic force profile; the assumption that force magnitude is monotonically related to disagreement would then face a harder test.
- The local-state selection via the forward variable could be extended to simultaneous adaptation of several neighboring states, since a human push often spans more than one HSMM state.
- If the force-to-intent assumption holds, the same mechanism could serve as a negotiation protocol in shared tasks: the robot resists deviation from the nominal plan unless the human's physical input is consistent enough to shift the model.
- Adapting covariance parameters is left open; methods that respect positive-definiteness, such as optimizing in a transformed space, would let the robot also change the variance of the trajectory distribution.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a framework for adapting a demonstration-learned trajectory distribution through physical human-robot interaction. A hidden semi-Markov model (HSMM) encodes the nominal task plan, and a smooth reference trajectory distribution is retrieved by weighted least-squares over static and dynamic features. During an adaptation phase, the robot minimizes disagreement forces by running Bayesian optimization locally on the mean motor-command and duration parameters of the currently active HSMM state, with the search domain extracted from the model covariances. Experiments on synthetic 2D C- and L-shape pick-and-place tasks, using a simulated external force with a fixed direction and profile, show convergence in five to six BayesOpt iterations and local spatial and temporal deformations of the reference trajectory. The paper claims that this yields data-efficient adaptation consistent with human-intended changes, but real human-in-the-loop experiments are explicitly left to future work.
Significance. If the central claim holds, the paper makes a useful contribution to learning-from-demonstration and human-robot collaboration: it combines a probabilistic task model with force-guided Bayesian optimization in a way that preserves the nominal plan while incorporating local corrections. The HSMM forward computation and the weighted least-squares trajectory retrieval are standard and are presented correctly. The local BayesOpt formulation with expected improvement is also standard, and the use of the active HSMM state to reduce dimensionality is a sensible and clearly explained choice. These strengths make the framework plausible and worth developing. However, the empirical evidence is limited to a single scripted force profile in simulation, so the paper's central claim about adaptation to human-intended changes is not yet supported.
major comments (4)
- [Section IV-C / Section III-B] The only experimental instantiation of the human is a simulated external force with a fixed direction and profile (Fig. 4), and Section VI explicitly states that real scenarios are left for future work. In the objective D_n = {(θ_i,j, ||f_hat - f_s||_j)} defined in Section III-B, the GP stationarity assumption of Section III-A requires the force to be an exogenous, repeatable function of θ. In a closed-loop physical interaction, the force is generated by the coupled human-robot system and depends on the human's reaction, so the same θ can produce different force readings. The presented evidence therefore does not support the abstract's claim that the adapted distribution is consistent with the changes introduced by the human. A concrete remedy is a human-subjects study with time-varying intentions, or a substantially softened claim that restricts the result to open-loop force corrections.
- [Section III-B] The force-minimization objective is underdetermined as a model of human intent. As the robot deforms its trajectory toward the human's preferred motion, the interaction force tends to zero, so many different deformed trajectories are equally force-free. Minimizing ||f_hat - f_s|| cannot by itself identify which of these force-free trajectories encodes the intended change. The paper assumes in the opening of Section III that force cues provide information about an unobservable reward function, but it neither formalizes nor tests the needed identifiability or monotonicity conditions. Please provide a formal statement of the assumptions under which the force error uniquely identifies the intended adaptation, and add experiments with multiple force directions and profiles that correspond to distinct intended changes.
- [Section IV-C] The comparison with full-parameter BayesOpt over θ ∈ R^18 is reported as not converging, and this comparison underpins the claimed benefit of local search. The paragraph does not state the number of iterations, the GP kernel or hyperparameters, the acquisition-function settings, or the initialization for this baseline. Without these details the reader cannot judge whether the failure is due to dimensionality alone or to an unfair or poorly tuned baseline. Please report a controlled comparison.
- [Section III-C] The time window T_w and the re-computation of the state sequence s_{t:T_w} are central to the online adaptation behavior, but the paper does not specify how T_w was chosen or how sensitive the results are to this choice and to the search bounds (±2σ_i). Since these quantities define the local search space, please report a sensitivity analysis or at least a justification for the chosen values.
minor comments (5)
- [Section II-A] The parameter d_max in Eq. (1) is used before being defined; please state its meaning and how it is selected.
- [Section III-A] In Eq. (17), the threshold τ is not defined precisely; please state whether it is the incumbent value or a user-specified target, and how it is updated during the optimization.
- [Section IV-A] The text says six synthetic demonstrations were collected, but no details are given about the generation process, sensor noise, or number of independent runs; please add these experimental details for reproducibility.
- [Section IV-C / Fig. 5] Figure 5 shows GP posterior means and credible intervals, but the locations of the queried points and the order of evaluations are not shown; adding them would make the reported convergence to six and five iterations more transparent.
- [General] There are several typos, including 'leaded' in Section IV-C and 'forge-guided' in Section IV-A; please proofread the manuscript.
Circularity Check
No significant circularity; the force objective is an external input and the optimization is a fitting procedure, not a self-referential derivation.
full rationale
The derivation chain is self-contained: the HSMM is trained on demonstrations; BayesOpt minimizes an external objective (the L2 norm of disagreement force ||hat f - f_s||) over local state-mean parameters theta_i; the adapted trajectory is then computed from the optimized parameters via Eqs. (11)-(12). The quantity being minimized is not the same as the quantity claimed as a prediction: the force readings are exogenous inputs (in the experiments, a scripted 'simulated external force', Section IV-C), and the deformed trajectory is an output of the model update. Optimizing parameters to drive a measured error to zero is a fitting/control procedure, not a circular derivation. The only self-citations ([15], [27], [28]) supply modeling choices or pointers to future work and are not used to forbid alternatives or to justify the core force-to-adaptation mapping. The paper's limitation is external validity: real closed-loop human forces are not tested (Section VI states 'we will evaluate the proposed framework in different real scenarios'), and the assumption that forces encode human intent (Section III) is untested. These are correctness/validity concerns, not circularity. Score 0.
Assumptions & free parameters
free parameters (3)
- Local search bounds +/- 2 sigma_i
- Time window T_w =
5 time steps
- GP kernel and hyperparameters
assumptions (4)
- domain assumption Human interaction forces are a reliable, noise-corrupted signal of the desired trajectory adaptation.
- domain assumption The robot is equipped with impedance control that can be exploited for physical interaction.
- domain assumption The disagreement force is well represented by the L2 norm of the difference between reference and sensed forces.
- standard math The trajectory retrieval via weighted least squares (Eq. 11) produces a valid reference distribution.
Cite this review
Pith. "Pith review of Interactive Trajectory Adaptation through Force-guided Bayesian Optimization." pith.science (2026). https://pith.science/paper/6QUULUJV
@misc{pith2026190807263,
author = {Pith},
title = {Pith review of: Interactive Trajectory Adaptation through Force-guided Bayesian Optimization},
year = {2026},
howpublished = {\url{https://pith.science/paper/6QUULUJV}},
note = {Machine review of arXiv:1908.07263}
}
read the original abstract
Flexible manufacturing processes demand robots to easily adapt to changes in the environment and interact with humans. In such dynamic scenarios, robotic tasks may be programmed through learning-from-demonstration approaches, where a nominal plan of the task is learned by the robot. However, the learned plan may need to be adapted in order to fulfill additional requirements or overcome unexpected environment changes. When the required adaptation occurs at the end-effector trajectory level, a human operator may want to intuitively show the robot the desired changes by physically interacting with it. In this scenario, the robot needs to understand the human intended changes from noisy haptic data, quickly adapt accordingly and execute the nominal task plan when no further adaptation is needed. This paper addresses the aforementioned challenges by leveraging LfD and Bayesian optimization to endow the robot with data-efficient adaptation capabilities. Our approach exploits the sensed interaction forces to guide the robot adaptation, and speeds up the optimization process by defining local search spaces extracted from the learned task model. We show how our framework quickly adapts the learned spatial-temporal patterns of the task, leading to deformed trajectory distributions that are consistent with the nominal plan and the changes introduced by the human.
Figures
Figures from the paper (2 more)
Reference graph
Works this paper leans on
-
[1]
Physical collaboration of human-human and human-robot teams,
K. Reed and M. Peshkin, “Physical collaboration of human-human and human-robot teams,” IEEE Transactions on Haptics, vol. 1, no. 2, pp. 108–120, 2008
work page 2008
-
[2]
Math- ematical models of adaptation in human-robot collaboration,
S. Nikolaidis, J. Forlizzi, D. Hsu, J. Shah, and S. Srinivasa, “Math- ematical models of adaptation in human-robot collaboration,” ArXiv e-prints, July 2017
work page 2017
-
[3]
Tactile guidance for policy refinement and reuse,
B. Argall, E. L. Sauser, and A. Billard, “Tactile guidance for policy refinement and reuse,” in Joint IEEE Intl. Conf. on Development and Learning and Epigenetic Robotics , Ann Arbor, USA, August 2010, pp. 7–12
work page 2010
-
[4]
Learning haptic affordances from demonstration and human-guided exploration,
V . Chu, B. Akgun, and A. Thomaz, “Learning haptic affordances from demonstration and human-guided exploration,” in IEEE Haptics Symposium (HAPTICS), Philadelphia, USA, April 2016, pp. 119–125
work page 2016
-
[5]
Analyzing differences between teachers when learning object affordances via guided-exploration,
V . Chu and A. Thomaz, “Analyzing differences between teachers when learning object affordances via guided-exploration,” Intl. Journal of Robotics Research, vol. 36, no. 5–7, pp. 739–758, 2017
work page 2017
-
[6]
Directing policy search with interactively taught via-points,
Y . Schroecker, H. B. Amor, and A. Thomaz, “Directing policy search with interactively taught via-points,” in Intl. Conf. on Autonomous Agents and Multiagent Systems (AAMAS) , Singapore, May 2016, pp. 1052–1059
work page 2016
-
[7]
Taking the human out of the loop: A review of Bayesian optimiza- tion,
B. Shahriari, K. Swersky, Z. Wang, R. P. Adams, and N. de Freitas, “Taking the human out of the loop: A review of Bayesian optimiza- tion,” Proceedings of the IEEE , vol. 104, no. 1, pp. 148–175, 2016
work page 2016
-
[8]
Robots that can adapt like animals,
A. Cully, J. Clune, D. Tarapore, and J. B. Mouret, “Robots that can adapt like animals,” Nature, vol. 521, pp. 503–507, 2015
work page 2015
Show all 28 references
-
[9]
Automatic lqr tuning based on gaussian process global optimization,
A. Marco, P. Hennig, J. Bohg, S. Schaal, and S. Trimpe, “Automatic lqr tuning based on gaussian process global optimization,” in IEEE Intl. Conf. on Robotics and Automation (ICRA) , Stockholm, Sweden, May 2016, pp. 270–277
2016
-
[10]
Deep kernels for optimizing locomotion controllers,
R. Antonova, A. Rai, and C. Atkeson, “Deep kernels for optimizing locomotion controllers,” in Conference on Robot Learning (CoRL) , California, USA, November 2017, pp. 1–10
2017
-
[11]
Constrained Bayesian op- timization of combined interaction force/task space controllers for manipulations,
D. Drie, P. Englert, and M. Toussaint, “Constrained Bayesian op- timization of combined interaction force/task space controllers for manipulations,” in IEEE Intl. Conf. on Robotics and Automation (ICRA), Singapore, May 2017, pp. 902–907
2017
-
[12]
A sensorimotor reinforcement learning framework for physical human-robot interaction,
A. Ghadirzadeh, J. B ¨utepage, A. Maki, D. Kragic, and M. Bj ¨orkman, “A sensorimotor reinforcement learning framework for physical human-robot interaction,” inIEEE/RSJ Intl. Conf. on Intelligent Robots and Systems (IROS) , Daejeon, Korea, October 2016, pp. 2682–2688
2016
-
[13]
Learning dynamic robot-to-human object handover from human feedback,
A. Kupcsik, D. Hsu, and S. Lee, “Learning dynamic robot-to-human object handover from human feedback,” in Intl. Symp. on Robotics Research, Sestri Levante, Italy, September 2015
2015
-
[14]
Hidden semi-Markov models,
S. Yu, “Hidden semi-Markov models,” Artificial Intelligence, vol. 174, no. 2, pp. 215–243, 2010
2010
-
[15]
Learning controllers for reactive and proactive behaviors in human-robot col- laboration,
L. Rozo, J. Silv ´erio, S. Calinon, and D. G. Caldwell, “Learning controllers for reactive and proactive behaviors in human-robot col- laboration,” Frontiers in Robotics and AI , vol. 3, no. 30, pp. 1–11, June 2016, specialty Section Robotic Control Systems
2016
-
[16]
Trajectory deformations from physical human–robot interaction,
S. Losey and M. Malley, “Trajectory deformations from physical human–robot interaction,” IEEE Transactions on Robotics , vol. 34, no. 1, pp. 126–138, 2018
2018
-
[17]
Safe controller opti- mization for quadrotors with Gaussian processes,
F. Berkenkamp, A. Schoellig, and A. Krause, “Safe controller opti- mization for quadrotors with Gaussian processes,” in IEEE Intl. Conf. on Robotics and Automation (ICRA) , Stockholm, Sweden, May 2016, pp. 491–496
2016
-
[18]
Robot learning from demonstra- tion of force-based tasks with multiple solution trajectories,
L. Rozo, P. Jim ´enez, and C. Torras, “Robot learning from demonstra- tion of force-based tasks with multiple solution trajectories,” in IEEE Intl. Conf. on Advanced Robotics (ICAR) , Tallin, Estonia, June 2011, pp. 124–129
2011
-
[19]
A robot learning from demonstration framework to per- form force-based manipulation tasks,
——, “A robot learning from demonstration framework to per- form force-based manipulation tasks,” Journal of Intelligent Ser- vice Robotics, Special Issue on Artificial Intelligence Techniques for Robotics: Sensing, Representation and Action, Part 2 , vol. 6, no. 1, pp. 33–51, 2013
2013
-
[20]
Small-variance asymptotics for non- parametric online robot learning,
A. K. Tanwani and S. Calinon, “Small-variance asymptotics for non- parametric online robot learning,” Intl. Journal of Robotics Research , vol. 38, no. 1, pp. 3–22, 2019
2019
-
[21]
An experience-driven robotic assistant acquiring human knowledge to improve haptic cooperation,
J. Medina, M. Lawitzky, A. Mortl, D. Lee, and S. Hirche, “An experience-driven robotic assistant acquiring human knowledge to improve haptic cooperation,” in IEEE/RSJ Intl. Conf. on Intelligent Robots and Systems (IROS) , San Francisco, USA, September 2011, pp. 2416–2422
2011
-
[22]
A tutorial on hidden Markov models and selected appli- cations in speech recognition,
L. Rabiner, “A tutorial on hidden Markov models and selected appli- cations in speech recognition,” in Proceedings of the IEEE , 1989, pp. 257–286
1989
-
[23]
Practical implementation of an efficient forward-backward algorithm for an explicit-duration hidden Markov model,
S. Yu and T. Kobayashi, “Practical implementation of an efficient forward-backward algorithm for an explicit-duration hidden Markov model,” IEEE Trans. on Signal Processing , vol. 54, no. 5, pp. 1947– 1951, 2006
1947
-
[24]
Learning, generation, and recognition of motions by reference-point-dependent probabilistic models,
K. Sugiura, N. Iwahashi, H. Kashioka, and S. Nakamura, “Learning, generation, and recognition of motions by reference-point-dependent probabilistic models,” Advanced Robotics, vol. 25, no. 6–7, pp. 825– 848, 2011
2011
-
[25]
On Bayesian methods for seeking the extremum,
J. Mo ˇckus, “On Bayesian methods for seeking the extremum,” in Optimization Techniques IFIP Technical Conference , 1975, pp. 400– 404
1975
-
[26]
Incorporating human expertise in robot motion learning and synthesis,
H. Yin, “Incorporating human expertise in robot motion learning and synthesis,” Ph.D. dissertation, ´Ecole Polytechnique F ´ed´erale De Laussane, 2018
2018
-
[27]
Force-based vari- able impedance learning for robotic manipulation,
F. J. Abu-Dakka, L. Rozo, and D. G. Caldwell, “Force-based vari- able impedance learning for robotic manipulation,” Robotics and Autonomous Systems, vol. 109, pp. 156–167, 2018
2018
-
[28]
Learning manipulability ellipsoids for task compatibility in robot manipulation,
L. Rozo, N. Jaquier, S. Calinon, and D. G. Caldwell, “Learning manipulability ellipsoids for task compatibility in robot manipulation,” in IEEE/RSJ Intl. Conf. on Intelligent Robots and Systems (IROS) , Vancouver, Canada, September 2017, pp. 3183–3189
2017
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.