{"id":"98a81c58-1b0b-4b4f-b06e-19d3a1a019c7","arxiv_id":"1908.02077","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"A Lyapunov-based critic weight update for SNAC kinematic control of robot manipulators is proposed and tested on a UR10, yet the optimality claim is not proven.","lead":"The paper offers a new rule for updating the critic network in an adaptive optimal controller for robot arms, applied to reaching and trajectory tracking. A UR10 arm experiment shows the scheme works, but the claimed guarantee of convergence to the optimal cost is not established by the mathematics in the paper.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The critic update (19)/(38) has no HJB/Bellman residual and no Q-dependence; it only cancels a Lyapunov cross-term, so the claimed convergence of the critic to the optimal cost is not established.","rationale":"The reader correctly flags Assumption 1 and the loose Barbalat step as serious gaps. My stress-test identifies a more direct failure: even if Assumption 1 holds exactly, the weight update in (19)/(38) is not derived from the HJB equation and does not drive W_hat toward W*. The exact scalar counterexample is a constructive disproof of the claimed convergence-to-optimal-cost under every condition the proof assumes. Since the theoretical contribution of the paper is precisely that convergence guarantee, the gap is load-bearing. I therefore agree with the REJECT verdict, though my main reason differs from the reader's weakest-assumption pick.","tokens_in":9920,"tokens_out":9790,"duration_ms":107658,"concrete_test":"Run the paper's controller on the scalar integrator x_dot = u with Q=R=1, feature sigma(e)=e^2, and the proposed laws (15) and (19): u = -W_hat*e and W_hat_dot = 2*alpha*e^2. Pick e(0)=1, W_hat(0)=0, alpha=1. The exact solution gives W_hat(t) -> sqrt(2) and e(t) -> 0, whereas the HJB-optimal weight is W* = 1 and V* = e^2. Check whether W_hat equals 1 and whether the Hamiltonian residual from (9) goes to zero; if not, the critic is not converging to the optimal cost and the main claim fails even under Assumption 1.","verdict_should_be":"REJECT","load_bearing_attack":"For the central claim it is not enough that e(t) goes to zero; the critic weights must converge to the HJB cost, i.e. W_hat -> W*. Equation (19) is W_hat_dot = alpha * grad_sigma_c * J * R^{-1} * J^T * grad_Js(e). With Js = e^T e / 2, this is an algebraic function of e alone; it contains no Hamiltonian or Bellman error from (9) and no Q. In the derivation (22)-(24), it is chosen so that the W_tilde term cancels; as a result, L_dot in (24) no longer contains W_tilde. Even if Barbalat were applied correctly, L_dot -> 0 cannot imply W_tilde -> 0 without persistence of excitation, which is neither assumed nor present. Moreover, in the exactly representable scalar case n=m=1, J=1, Q=R=1, sigma(e)=e^2, the true HJB solution is V* = e^2 and W* = 1, and Assumption 1 holds because e_dot* = -e. Yet (19) gives W_hat_dot = 2*alpha*e^2, independent of W_hat. With e_dot = -W_hat*e, the closed-form relation e^2 = e0^2 - W_hat^2/(2*alpha) gives W_hat -> e0*sqrt(2*alpha), generally not 1. Thus even with Assumption 1 granted, the proposed update does not learn the optimal cost; the abstract's convergence claim is unsupported and demonstrably false in this minimal setting.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a single-network adaptive critic (SNAC) approach to optimal kinematic control of a robot manipulator. Regulation to a fixed target and tracking of a time-varying trajectory are formulated as infinite-horizon optimal control problems for the velocity-level kinematics ẋ = J u. The value function is approximated by a single-layer neural network, and a new critic weight update law is proposed in Eq. (19) for regulation and Eq. (38) for tracking. Stability and convergence to the optimal cost are claimed via a Lyapunov analysis and Barbalat's lemma. The scheme is validated in simulations and on a real UR10 manipulator, and a statistical comparison is made against an RNN-based method and the SVF method.","tokens_in":10227,"tokens_out":9653,"duration_ms":95092,"significance":"If the central convergence claim were correct, the paper would provide a simple, experimentally validated critic update for optimal kinematic control. The hardware experiments and the large-sample comparison on the UR10 are useful engineering contributions. However, the main theoretical claim, that the proposed update drives the critic to the HJB optimal cost while stabilizing the closed loop, is unsupported by the derivation and is false in a minimal exactly representable example. The contribution as stated is therefore not valid, although the experimental part could be reframed as a heuristic kinematic controller with empirical evaluation.","major_comments":[{"comment":"Eq. (19) contains no HJB or Bellman residual and no dependence on Q; it is an algebraic function of e alone and is constructed to cancel the \\tilde W_c term in the Lyapunov derivative. Consequently the update does not drive \\hat W_c toward W*. In the scalar exactly representable case n=m=1, J=1, Q=R=1, σ_c(e)=e^2, the HJB solution is V*=e^2, so W*=1, and Assumption 1 holds with ˙e*=-e. The update (19) becomes \\dot{\\hat W}_c=2α e^2, independent of \\hat W_c. With \\dot e=-\\hat W_c e, one obtains e^2=e_0^2-\\hat W_c^2/(2α), so \\hat W_c converges to e_0√(2α), which is not 1 in general. Thus the claimed convergence to the optimal cost is false in this minimal setting, and the same objection applies to the tracking law (38).","section":"Section III-B, Eq. (19)"},{"comment":"After the construction cancels the weight-error term, \\dot L in (24) contains no \\tilde W_c dependence. Even if e(t)→0 and \\dot L→0, this cannot imply \\tilde W_c→0 without a persistence-of-excitation condition, which is neither assumed nor established. The Barbalat step is also not justified: (25)-(26) give \\dot L≤0 only outside a ball, not globally, and Remark 2 simply asserts that \\ddot L is bounded rather than proving it. The same issues are repeated in the tracking stability analysis in Section IV-D.","section":"Section III-C, Eqs. (24)-(28)"},{"comment":"Assumption 1 is a strong structural assumption: it states that the optimal closed-loop error dynamics make J_s(e)=e^T e/2 satisfy ∇J_s^T ˙e* = -∇J_s^T M∇J_s, i.e., that the quadratic tracking error is a strict Lyapunov function for the optimal closed loop. This is exactly the desired stability property, and Remark 1's invocation of [23] does not verify it for the UR10 Jacobian or for the chosen trajectories. If Assumption 1 fails, the negative definite term in (25) disappears and the claimed stability bound is unsupported. For tracking, Remark 3 assumes the same property for the augmented system, but no argument is given that the reference dynamics in (29) satisfy it.","section":"Section III-B, Assumption 1"}],"minor_comments":[{"comment":"The notation ∇V* is used in the Hamiltonian before the optimal value function is introduced; the Hamiltonian should be written with ∇V^T and the optimal value function should be defined first.","section":"Section III-B, Eq. (7)"},{"comment":"The quantities ˙e* and ¨e* are not defined as functions of the state, and the boundedness claim in Remark 2 is asserted without derivation.","section":"Section III-C, Eq. (28)"},{"comment":"The learning-rate schedule α=α_initial(tanh(n−k))+α_final is not clearly specified; the roles of n and k and the time dependence are unclear, and no values are given for all experiments.","section":"Section V-B, V-C"},{"comment":"The comparison reports only mean trajectory costs; no error bars, confidence intervals, or implementation details for the RNN [3] and SVF [26] baseline are provided, so the large cost differences in Table I are difficult to evaluate.","section":"Section V-E, Table I"},{"comment":"The individual subfigures are referenced only as groups, and the text does not explain the cost plots or how they relate to the HJB optimal cost.","section":"Figures 3 and 4"}],"recommendation":"reject","confidential_remarks":"The paper's central theoretical claim is contradicted by a simple closed-form counterexample using the paper's own update law. This is not a matter of missing proof details; the algorithm as written does not learn the HJB value function. The experimental section may still be useful as an empirical study, but the submission's stated contribution would need to be substantially revised, likely beyond the scope of a major revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things you should know before reading this one. First, the headline result is not established: the weight update law in (19)/(38) does not actually learn the optimal cost. Second, the paper is not a waste of time—there is a real UR10 experiment and a coherent SNAC framework, but the proof is the weak load-bearing wall.\n\nWhat's new is the specific critic update law and its Lyapunov analysis. The authors correctly note that standard SNAC uses backprop without convergence guarantees, and they try to fix that with a law that cancels the weight-error cross term in the Lyapunov derivative. That is a clever trick, and the regulation/tracking setup is standard. The hardware validation on a 6-DOF UR10, plus the 1000-sample comparison against RNN and SVF baselines, is genuine effort.\n\nThe problem is that the update law has no dependence on the cost matrix Q and no Bellman or HJB residual. It is just an algebraic function of the current error. In the minimal scalar case (n=m=1, Q=R=1, sigma(e)=e^2) the true optimal value is V*=e^2 with W*=1, but the proposed law gives W_hat_dot = 2 alpha e^2, independent of W_hat. Solving the closed-loop dynamics gives W_hat converging to e0*sqrt(2 alpha), generally not 1. So the abstract's claim—convergence to the desired optimal cost—is demonstrably false even when Assumption 1 holds.\n\nThe Lyapunov analysis also has softer but still serious gaps. Assumption 1 is stronger than what [23] actually provides; it assumes the optimal closed-loop makes e^T e a strict Lyapunov function, which is not verified for the UR10 Jacobian or the chosen trajectories. Then the Barbalat step is asserted: L_dot <= 0 and bounded L_ddot do not imply W_tilde -> 0 without persistence of excitation, and L_dot -> 0 is not even shown rigorously. The experimental comparison is a minor additional concern: no error bars, and the reported cost includes an extra derivative term that the theoretical cost does not.\n\nWho is this for? Readers working on adaptive critic or SNAC-based kinematic control will find the framework and the negative result useful. But as a research claim, it is not publishable in its current form. The authors need to either derive an update that genuinely minimizes the HJB error and prove its convergence, or substantially weaken the claims to practical convergence of the tracking error without weight convergence.\n\nFor peer review: send it to referees, because the topic is active and the hardware work deserves scrutiny, but expect heavy revision or rejection unless the main theorem is reworked. I would not accept it as is.","headline":"The paper's central claim—that the proposed critic weight update converges to the optimal HJB cost—is not supported and fails in a minimal scalar example, though the hardware experiments and the SNAC formulation itself have merit.","tokens_in":10813,"tokens_out":2253,"would_cite":false,"duration_ms":26212,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"By choosing a critic weight update that cancels the weight error against the control error, this paper gives a single-network adaptive critic with Lyapunov stability and convergence to the HJB optimal cost for robot kinematic control.","keywords":["optimal kinematic control","adaptive critic","single network adaptive critic","Hamilton-Jacobi-Bellman equation","Lyapunov stability","robot manipulator","trajectory tracking","UR10"],"falsifier":"Compute $e(t)^T J(\\theta) u^*(t)$ during the regulation experiments, or on the kinematic model, from the paper's random seed poses; if this quantity is ever positive, the optimal closed-loop dynamics do not make $J_s$ a strict Lyapunov function, and the negative-definite identity in Assumption 1 fails. A static version is to evaluate $\\nabla J_s(e)^T J u^*$ on a grid of error states within the robot's reachable workspace for the UR10 Jacobian and check that no value is positive.","tokens_in":9665,"feed_emoji":"🦾","tokens_out":12218,"duration_ms":142476,"temperature":0.7,"pith_summary":"This paper tries to make the single-network adaptive critic (SNAC), a neural network that learns the value function of an optimal control problem, into a tool with guarantees for robot kinematic control. It proposes a critic weight-update law that, in the regulation case, is $\\dot{\\hat{W}}_c=\\alpha\\nabla\\sigma_c J R^{-1}J^T\\nabla J_s$ with $J_s=e^Te/2$, and an analogous law for tracking on an augmented state. The claim is that this update drives the estimated cost toward the optimal cost of the Hamilton-Jacobi-Bellman equation while keeping the closed-loop error dynamics stable in the Lyapunov sense. If true, a robot arm can be steered by a learned critic that provably approaches the optimal velocity profile, without solving the HJB equation offline. The paper supports the claim with simulations, experiments on a real six-degree-of-freedom UR10 arm, and a quantitative comparison against two existing kinematic controllers.","feed_headline":"One weight rule gives a robot arm provably stable optimal control","feed_subtitle":"A single adaptive-critic network now gets stability and optimal-cost convergence, tested on a real six-axis arm.","key_machinery":"The carrying mechanism is the critic weight-update law (19)/(38) together with the quadratic Lyapunov function $J_s$. The update is deliberately chosen so that, inside the Lyapunov derivative, the term containing the unknown weight error $\\tilde{W}_c$ cancels against the corresponding term generated by the difference between optimal and estimated control; what remains is $\\nabla J_s^T\\dot{e}^*$ plus a bounded function-approximation error. Assumption 1, which asserts that the optimal closed-loop dynamics satisfy $\\nabla J_s(e)^T\\dot{e}^*=-\\nabla J_s(e)^TM\\nabla J_s(e)$, turns the remaining term into a negative definite quadratic bound, and Barbalat's lemma upgrades the resulting boundedness to asymptotic stability. For tracking the same machinery is applied to the augmented state $\\xi=[e^T,x_d^T]^T$.","core_discovery":"The central claim is that optimal kinematic control of a manipulator reduces to choosing the critic weights by $\\dot{\\hat{W}}_c = \\alpha\\nabla\\sigma_c J R^{-1} J^T \\nabla J_s(e)$ for regulation and $\\dot{\\hat{W}}_c = \\alpha\\nabla\\varphi_c G R^{-1} G^T \\nabla J_s(\\xi)$ for tracking, where $\\xi=[e^T,x_d^T]^T$ augments the tracking error with the reference state and $\\nabla J_s(e)=e$, $\\nabla J_s(\\xi)=\\xi$. With this law the estimated control $\\hat{u}^* = -\\tfrac12 R^{-1}J^T\\nabla\\sigma_c^T\\hat{W}_c$ is shown to converge to a bounded neighborhood of the true HJB-optimal control, and the Lyapunov function $L=\\tfrac{1}{4\\alpha}\\tilde{W}_c^T\\tilde{W}_c+J_s$ has a negative-semidefinite derivative once $\\|\\nabla J_s\\|$ is large enough. Barbalat's lemma is then used to conclude asymptotic stability and convergence of the weight estimation error to zero. The same construction is carried out for tracking on the augmented system.","pith_inferences":["A testable extension the paper leaves open is applying the same weight law to redundant or mobile manipulators, since the update only uses $JR^{-1}J^T$ and never requires pseudo-inverting the Jacobian.","The proof would become a fully practical guarantee if Assumption 1 were checked numerically on the UR10 Jacobian over the sampled workspace; the paper asserts it from a cited result rather than verifying it for its own trajectories.","Because the experimental cost is a pre-chosen quadratic form, the optimality achieved in practice is the best approximation to the true value function within that function-approximation class; comparing the learned critic against the LQR solution of a linearized arm model would quantify that gap."],"forward_implications":["A robot arm can be driven to a fixed target as an optimal regulation problem whose critic converges to the HJB-optimal cost, so the motion follows a near-optimal trajectory instead of an arbitrary inverse-kinematics solution.","A time-varying task-space trajectory can be followed optimally by solving an optimal tracking problem on the augmented state $\\xi=[e^T,x_d^T]^T$, with a control law that naturally combines feedforward and feedback action.","The closed loop is stable in the Lyapunov sense, and under Barbalat's lemma the weight estimation error and tracking error converge to zero, giving an analytical guarantee that earlier back-propagation-based SNAC updates only suggested through simulations.","In the quantitative comparison over 1,000 random regulation targets and 1,000 elliptical tracking trajectories, the proposed controller reports normalized trajectory costs of 5.78 and 3.30, compared with 49.74/48.32 and 14.39/32.21 for the two baselines."],"supporting_citations":[{"why":"Introduces the single-network adaptive critic architecture that this paper tunes with a new weight law.","marker":"[10]"},{"why":"Earlier SNAC-based kinematic control with back-propagation weight updates, the approach the proposed law replaces.","marker":"[11]"},{"why":"Supplies the augmented-system construction for optimal tracking that the paper uses for time-varying trajectories.","marker":"[22]"},{"why":"Underpins Assumption 1, the negative-definiteness condition on optimal closed-loop error dynamics that the stability proof relies on.","marker":"[23]"},{"why":"Cited for Barbalat's lemma, which upgrades the Lyapunov boundedness result to asymptotic stability in regulation.","marker":"[24]"},{"why":"Neural-network kinematic controller used as a baseline in the quantitative cost comparison.","marker":"[3]"},{"why":"Singular-value-filtering kinematic controller used as the second baseline in the quantitative cost comparison.","marker":"[26]"}],"fun_headline_variants":["One weight rule gives robot arms optimal control with guaranteed stability","Adaptive critic weight update ensures stable optimal kinematic control for robots","Lyapunov-guaranteed optimal kinematic control via single adaptive critic","Optimal robot arm control with stability from a novel critic weight law"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is Assumption 1: under the optimal control law the quadratic error $J_s=e^Te/2$ is a strict Lyapunov function, meaning $\\nabla J_s^T\\dot e^* = -\\nabla J_s^TM\\nabla J_s$ for some positive definite $M$; the paper cites [23] for this but never verifies it on the UR10 Jacobian or the chosen trajectories.","fun_headline_variants_meta":{"raw":{"variants":["One weight rule gives robot arms optimal control with guaranteed stability","Adaptive critic weight update ensures stable optimal kinematic control for robots","Lyapunov-guaranteed optimal kinematic control via single adaptive critic","Optimal robot arm control with stability from a novel critic weight law"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000834,"raw_usage":{"total_tokens":3688,"prompt_tokens":1041,"completion_tokens":2647,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":657,"completion_tokens_details":{"reasoning_tokens":2575}},"tokens_in":657,"tokens_out":2647,"duration_ms":22275,"temperature":1.0,"reasoning_tokens":2575,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T14:55:25.554043+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute $e(t)^T J(\\theta) u^*(t)$ during the regulation experiments, or on the kinematic model, from the paper's random seed poses; if this quantity is ever positive, the optimal closed-loop dynamics do not make $J_s$ a strict Lyapunov function, and the negative-definite identity in Assumption 1 fails. A static version is to evaluate $\\nabla J_s(e)^T J u^*$ on a grid of error states within the robot's reachable workspace for the UR10 Jacobian and check that no value is positive.","supporting_citations":[{"cited_title":"A single network adaptive critic (snac) architecture for optimal control synthesis for a class of nonlinear systems,","cited_arxiv_id":null,"evidence_quote":"Introduces the single-network adaptive critic architecture that this paper tunes with a new weight law."},{"cited_title":"A single network adaptive critic-based redundancy resolution scheme for robot manip- ulators,","cited_arxiv_id":null,"evidence_quote":"Earlier SNAC-based kinematic control with back-propagation weight updates, the approach the proposed law replaces."},{"cited_title":"Neural network robust tracking control with adaptive critic framework for uncertain nonlinear systems,","cited_arxiv_id":null,"evidence_quote":"Supplies the augmented-system construction for optimal tracking that the paper uses for time-varying trajectories."},{"cited_title":"Optimal control of affine nonlinear continuous-time systems,","cited_arxiv_id":null,"evidence_quote":"Underpins Assumption 1, the negative-definiteness condition on optimal closed-loop error dynamics that the stability proof relies on."},{"cited_title":"Vamvoudakis and S","cited_arxiv_id":null,"evidence_quote":"Cited for Barbalat's lemma, which upgrades the Lyapunov boundedness result to asymptotic stability in regulation."},{"cited_title":"Kinematic control of redundant manipu- lators using neural networks,","cited_arxiv_id":null,"evidence_quote":"Neural-network kinematic controller used as a baseline in the quantitative cost comparison."},{"cited_title":"Redundant inverse kinematics: Experimen- tal comparative review and two enhancements,","cited_arxiv_id":null,"evidence_quote":"Singular-value-filtering kinematic controller used as the second baseline in the quantitative cost comparison."}],"review_version":1}