{"id":"03c4171e-6e7a-4137-af30-ac23a23c64a7","arxiv_id":"2509.00571","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":6,"one_line_summary":"A gray-box computed torque controller with eight learned parameters trained by TD3 tracks differential-drive robot trajectories after just 11 short episodes in simulation.","lead":"This paper tunes the gains of a computed torque controller with reinforcement learning for a differential-drive robot, claiming stable tracking after only 11 five-second episodes. Robotics engineers interested in sample-efficient learning-based control might find this gray-box approach worth testing.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Stability/critical-damping guarantee rests on exact parameter cancellation; learned parameters are not shown to match the plant, and no robustness analysis is provided.","rationale":"The reader's weakest_assumption identifies exactly the same load-bearing issue: the stability and critical-damping claims assume exact parameter reproduction, which contradicts the method's purpose of learning unknown parameters. My reading of the paper confirms this. The derivation of Eq. (52) is only valid for perfect model-plant matching; with learned parameters, a mismatch term appears in the error dynamics. No robust stability analysis, sensitivity bound, or quantitative evaluation is supplied. The experimental section contains only trajectory plots, not error metrics or parameter estimates, so there is no evidence that the learned parameters approach the true values. The paper does have a structurally plausible idea—using a model-based policy class to reduce sample complexity—and the constrained parameterization and tanh reparameterization are sensible engineering choices. However, the formal guarantee is the paper's headline contribution and it is not established. I agree with the reader's REJECT verdict; no adjustment is needed.","tokens_in":11732,"tokens_out":3748,"duration_ms":47928,"concrete_test":"Implement the GCTC in the MuJoCo simulation with the described TD3 training for 11 episodes on the training sinusoid. After training, deliberately replace the learned sigma_i, cV, cD by values within their constrained ranges but differing from the true plant parameters (e.g., the centers of the ranges, or the initial guesses). Run the three test trajectories and measure the tracking error and the closed-loop error response. If the error does not converge to a small neighborhood of zero, or if the response becomes oscillatory/overshooting on any trajectory, then the exact-cancellation assumption behind Eq. (52) fails and the stability/critical-damping guarantee is not valid.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that GCTC guarantees a stable, critically damped closed-loop response while learning controller parameters from only 11 short episodes (Abstract, §3.3). That guarantee is derived by substituting the control law (48) into the plant (43) and obtaining the linear error equation (52). This cancellation requires the controller's M, C, cV, cD to exactly equal the true plant parameters. But in GCTC these are trainable variables (Section 3.3), learned from a reward signal that penalizes tracking error and control effort, not from a system-identification objective. Nothing guarantees convergence to the true dynamics. With estimates M_hat, C_hat, cV_hat, cD_hat, the actual error dynamics become (from (43) and (48)): M e_ddot + M Kd e_dot + M Kp e + M Ki ∫e = (M - M_hat) p_dd + (C - C_hat) + (cV - cV_hat) Λ p_dot + (cD - cD_hat) sign(Λ p_dot), up to sign conventions. The right-hand side is a persistent, state- and acceleration-dependent disturbance whenever parameters are imperfect. Thus (52) does not hold, asymptotic convergence is not guaranteed, and the critical-damping conclusion based on (54) is void. The paper provides no bound on this disturbance, no Lyapunov or ISS analysis for the mismatch case, and no quantitative experimental data to show the learned parameters actually coincide with the plant. The conclusion (§5) that the constraints 'guarantee a stable and critically damped closed-loop time response' is therefore unsupported. This is the single most load-bearing weakness: without exact parameter recovery, the formal guarantee collapses, and the method reduces to an unverified parameter-tuning heuristic.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Gray-box Computed Torque Control (GCTC) for differential-drive mobile robot tracking. The method replaces the neural-network policy of a TD3 reinforcement-learning agent with a structured computed-torque controller whose physical parameters (mass/inertia-like terms, viscous and Coulomb friction coefficients) and feedback gains are learned. The controller is parameterized so that the learned gains enforce a triple-pole, critically damped error response when substituted into the nominal dynamics. The authors report training on a single sinusoidal trajectory for only 11 episodes of 5 s and evaluating on three other trajectories in MuJoCo, comparing with a CTC using exact parameters and a kinematic controller. The central claims are that GCTC finds near-optimal controller parameters with very few episodes and that the added constraints guarantee a stable, critically damped closed-loop response.","tokens_in":12199,"tokens_out":4668,"duration_ms":59728,"significance":"If the central claims were fully supported, the paper would make a useful contribution: it gives a compact analytic CTC parameterization for a DDMR with friction, embeds physically motivated box constraints via a tanh parameterization, and demonstrates that a structured policy can be learned much more sample-efficiently than a black-box neural policy. The algebraic derivation of the nominal error dynamics is a strength, and the use of a gray-box policy in TD3 is a sensible direction. However, the principal stability guarantee is derived under an exact-parameter cancellation assumption that is not satisfied by the learned controller, and the experimental section does not report the quantitative data needed to verify the claimed performance or sample efficiency. The significance of the paper therefore rests on an unproven and, as stated, incorrect guarantee.","major_comments":[{"comment":"The derivation of the error dynamics (52) requires that the matrices M, C and friction coefficients c_V, c_D in the control law exactly match the true plant parameters. In GCTC these are trainable and are never shown to coincide with the plant. With estimates M_hat, C_hat, cV_hat, cD_hat, substituting (48) into (43) gives a persistent disturbance on the right-hand side, e.g. terms such as (M_hat - M) p_dd, (C_hat - C), and (cV_hat - cV) Lambda p_dot. Equation (52) therefore does not hold, the asymptotic convergence argument is void, and the critical-damping conclusion based on (54) is unsupported. The paper provides no bound on the mismatch disturbance, no Lyapunov/ISS analysis, and no experimental evidence that the learned parameters converge to the true values. The abstract and Section 5 claim that the constraints 'guarantee a stable and critically damped closed-loop time response'; th","section":"Sections 3.2 and 3.3, Eqs. (48) and (52)"},{"comment":"The experimental section contains no numerical tracking-error values, no learning curves, no control-effort data, and no quantitative comparison with the baselines. The claims that GCTC 'outperforms' the raw CTC with exact parameters and the kinematic controller are not supported by the figures alone, which show only trajectories without axes labels, legends, or error statistics. This is especially problematic because the exact-parameter CTC should, by Eq. (52), produce zero tracking error; any comparison that claims GCTC is better requires careful numerical reporting. Without such data, the central performance claim is unverifiable.","section":"Section 4, Figures 2-4"},{"comment":"The proof of Lemma 1 is not correct as written. In Eq. (59) the term |F(t)| need not remain bounded away from zero for large t: if the coefficient of the monomial corresponding to the largest eigenvalue alpha_tilde is zero for the particular initial condition, then |F(t)| decays to zero rather than satisfying |F(t)| >= M1 > 0. The argument therefore does not establish the claimed fastest non-oscillatory property for all initial conditions. Since this lemma is the basis for the triple-pole placement in Eqs. (65)-(67), the critical-damping rationale needs to be either rigorously repaired or replaced by a direct and correct argument.","section":"Lemma 1, Eqs. (59)-(60)"},{"comment":"The paper does not report the reward weight matrices H_e and H_u, the parameter ranges (centers and radii), TD3 hyperparameters, initial parameter values, episode termination thresholds, torque limits, or the number of random seeds. The sample-efficiency claim ('only a few short learning episodes') is central to the paper and cannot be assessed without these details. The absence of quantitative learning curves and seed statistics makes the reported 11-episode training result irreproducible.","section":"Section 4 and Algorithm 1"}],"minor_comments":[{"comment":"The notation is inconsistent in places: the policy is denoted mu_Pi and also Pi, and the parameter vector Pi is used both for the policy parameters and for the matrix in Eq. (39). Please unify the notation.","section":"Notation throughout"},{"comment":"Taking vector powers elementwise in Eq. (54) is nonstandard and should be stated more explicitly, or the scalar equations for the two axes should be separated.","section":"Section 3.2, Eq. (54)"},{"comment":"The phrase 'fastest non-oscillatory time-response' is defined only through Lemma 1, whose proof has a gap (see Major Comment 3). Consider rephrasing the pole-placement criterion as a design choice supported by numerical evidence.","section":"Section 3.2, Eq. (65)"},{"comment":"The friction model in Eq. (45) uses sign(Lambda p_dot), but at p_dot = 0 the sign function is discontinuous; the paper does not discuss the well-posedness of the closed-loop dynamics at zero wheel velocity.","section":"Section 2.3, Eq. (43)"},{"comment":"The trajectory plots lack axis labels and legends. It is impossible to read the scale of the tracking errors or distinguish the controllers clearly. A zoomed-in error plot would be much more informative.","section":"Figures 2-4"},{"comment":"Reference [5] is cited as 'Mobile Robots', which appears to be a book chapter; the full bibliographic information is missing. Please provide the complete citation.","section":"References"},{"comment":"The while condition uses ||e(t)|| but the infinity norm or a specific threshold is not defined. Please specify how the tracking-error threshold is measured.","section":"Algorithm 1"}],"recommendation":"reject","confidential_remarks":"The core problem is that the paper's headline guarantee—stability and critical damping under learned parameters—is derived only for exact parameter cancellation and is unsupported for the actual learned controller. The experimental evaluation, as reported, cannot rescue the claim because no numerical results are provided. If the authors were to resubmit, they would need to either add a robustness analysis with explicit mismatch bounds or substantially weaken the guarantee claims and supply rigorous empirical validation. As written, the central contribution is not established."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First, the good news: the idea is genuinely interesting. The author reparameterizes DDMR dynamics into a compact form with four positive sigma constants, then uses TD3 to tune those plus friction coefficients and controller gains. That is a clean gray-box alternative to a black-box policy, and the tanh reparameterization to keep parameters in physical ranges is sensible. The derivation of the CTC law is internally consistent.\n\nThe problem is the central stability claim. The paper says the constraints \"guarantee a stable and critically damped closed-loop time response.\" That conclusion comes from substituting the control law into the exact plant, which only works if the learned parameters exactly match the true dynamics. But the whole point is that these are learned from a reward, not from system identification. There is no robustness analysis for parameter mismatch. The actual error dynamics then include a disturbance term that depends on the state and the desired acceleration. Without a bound or Lyapunov/ISS analysis, the guarantee collapses. So the method is at best an unverified parameter-tuning heuristic.\n\nThe experimental section does not help: it shows trajectory plots but reports no tracking errors, no learning curves, no quantitative comparisons, and no information about whether the learned parameters actually converge to the true values. With only 11 short episodes and a single training trajectory, it is hard to judge generalizability.\n\nWhat the paper does well: the pole-placement via equal negative eigenvalues is standard, and the lemma on fastest non-oscillatory response is intuitive, though the proof has gaps. The MuJoCo setup with viscous and Coulomb friction is a reasonable testbed.\n\nWho is this for: people working on learning-based control for mobile robots, particularly those combining structured models with RL. The idea deserves a serious referee, but the paper needs major revision: either significantly weaken the stability claim or analyze the mismatch case, and provide quantitative experimental results.\n\nMy recommendation: send it to peer review, but the reviewers should be told to focus on the robustness issue and the missing empirical validation. It is a promising work-in-progress, not a validated research claim.","headline":"A neat structured-policy idea for DDMR control, but the stability guarantee is only formal under exact parameter knowledge and the experiments lack numbers; worth review after major revision.","tokens_in":790,"tokens_out":1525,"would_cite":false,"duration_ms":46698,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A gray-box computed torque controller learns a differential-drive robot's dynamics and tracking gains in 11 short episodes, using a reinforcement learning agent to tune only eight physical parameters instead of a neural-network policy.","keywords":["differential-drive mobile robot","computed torque control","gray-box control","reinforcement learning","TD3","trajectory tracking","critical damping","MuJoCo"],"falsifier":"Run the trained GCTC on a real differential-drive robot or in a higher-fidelity simulator with actuator delay, then intentionally perturb one learned parameter (for example, cV) by increasing amounts and record the tracking-error response; the guarantee would be disproved if small perturbations produce overshoot, oscillation, or instability that the error equation (52) does not predict.","tokens_in":11679,"feed_emoji":"🤖","tokens_out":4922,"duration_ms":57121,"temperature":0.7,"pith_summary":"This paper proposes a middle path between model-based computed torque control and model-free deep reinforcement learning for differential-drive mobile robots. The controller keeps the structure of computed torque—a model-based law that can cancel the robot's nonlinear dynamics—but leaves eight physical parameters to be learned by the TD3 reinforcement learning algorithm instead of requiring them to be known in advance. The paper claims this gray-box design can identify the system and tune the controller within 11 episodes of five seconds each, and that hard constraints on the learned parameters keep them physically plausible while the chosen pole configuration guarantees a stable, critically damped tracking error. In simulation, the learned controller tracks three test trajectories better than a Lyapunov-based kinematic controller and nearly as well as a computed torque controller given exact parameters. If the claim holds, it means accurate dynamic tracking can be obtained with very little interaction data and without a prior parameter-identification step.","feed_headline":"11 five-second episodes tune a full robot tracking controller","feed_subtitle":"Gray-box computed torque learns the robot's dynamics and stabilizes tracking without prior identification.","key_machinery":"The central object is the gray-box policy: a computed torque control law whose eight tunable parameters (σ1..σ4, cV, cD, α, β) replace the actor network of TD3. The physics is encoded in M(θ) and C(ω) derived from the DDMR equations of motion, and in the friction terms τ_V and τ_D; the learning is encoded in a tanh reparameterization that keeps σ and friction parameters inside physically plausible ranges while leaving the optimization unconstrained. The pole-placement formulas kp = 3(α²+ε)², ki = (α²+ε)³, kd = 3(α²+ε), and their β counterparts, enforce the equal-pole configuration that Lemma 1 shows gives the fastest non-oscillatory response. Substitution of the control law into the plant dy","core_discovery":"The central claim is that replacing the neural-network policy of a reinforcement learning agent with a reparameterized computed torque controller—the gray-box GCTC—lets a single learning process identify the robot's dynamic parameters and synthesize a high-performance tracking controller. The control law takes the form u = M(θ)(p̈_d + Kp e + Ki ∫e + Kd ė) + C(ω) + viscous and Coulomb friction terms, where M, C, and the friction coefficients are expressed through four positive lumped parameters σ1..σ4 and two friction coefficients cV, cD. Only these plus two pole-placement parameters α, β are learned. Because the controller structure cancels the modeled nonlinearities, the tracking error obey","pith_inferences":["The stability and critical-damping guarantee is exact only in the nominal case: under parameter mismatch the error equation gains extra terms, so the practical claim is conditional on learned parameters closely matching the true ones; the paper does not quantify the mismatch region.","A direct validation would be to compare the learned σi, cV, cD against independently identified physical parameters; strong agreement would confirm the gray-box premise, while disagreement would indicate the performance comes from something other than exact cancellation.","The same gray-box treatment could be applied to other mechanical systems with known computed-torque structure, such as manipulators, whenever the unknown parameters enter the control law in a structured way.","The Lemma 1 argument relies on linear time-invariant error dynamics; if unmodeled effects such as wheel slip or actuator dynamics become significant, the guaranteed critical damping may not hold on real hardware."],"forward_implications":["A single short training budget of 11 episodes of 5 seconds on one sinusoid yields a controller that transfers to qualitatively different trajectories, suggesting structured parameter search is far more sample-efficient than black-box policy search.","The exact-parameter CTC is treated as an upper bound; GCTC approaches it, implying that dynamic model knowledge can be recovered online during control rather than through a separate identification experiment.","Because the trained controller is just the computed torque law with fixed coefficients, it can run at high frequency on low-cost hardware in real time.","The pole-placement construction gives closed-loop tracking error dynamics that are independent of the trajectory shape, as long as the learned parameters are correct.","Adding integral action and friction compensation reduces steady-state error and improves tracking of high-velocity trajectories compared to kinematic control.","The learning process simultaneously identifies the plant and designs the controller, so no prior identification step is needed.","The learned controller can be deployed on a low-cost microcontroller because the resulting policy has low computational complexity, unlike a neural-network policy."],"supporting_citations":[{"why":"Provides the computed torque method that the gray-box controller adapts to mobile robots.","marker":"[13]"},{"why":"Supplies the DDMR dynamic equations and constants Ci used to derive M(θ) and C(ω).","marker":"[12]"},{"why":"Source for the CTC structure, the viscous and Coulomb friction models, and the diagonal gain matrices.","marker":"[14]"},{"why":"Supplies the benchmark Lyapunov-based kinematic controller and the tracking-error formulation used throughout.","marker":"[5]"},{"why":"Defines the TD3 actor-critic algorithm used to learn the gray-box parameters.","marker":"[23]"},{"why":"Provides the MuJoCo physics engine used for simulated training and evaluation with friction.","marker":"[30]"},{"why":"Provides the tanh change-of-variables that enforces physically plausible parameter ranges during optimization.","marker":"[38]"}],"fun_headline_variants":["Gray-box RL tunes robot controller in five-second episodes","Five-second episodes train a full tracking controller","Hybrid model-based RL stabilizes mobile robot tracking fast","Computed torque meets RL for sample-efficient robot control","Few episodes learn robot dynamics for stable tracking"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The guarantee of stability and critical damping depends on the learned parameters exactly reproducing the true dynamic and friction coefficients; if they do not, the linear error equation that justifies the guarantee does not hold, and the paper does not analyze this mismatch.","fun_headline_variants_meta":{"raw":{"variants":["Gray-box RL tunes robot controller in five-second episodes","Five-second episodes train a full tracking controller","Hybrid model-based RL stabilizes mobile robot tracking fast","Computed torque meets RL for sample-efficient robot control","Few episodes learn robot dynamics for stable tracking"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000873,"raw_usage":{"total_tokens":3600,"prompt_tokens":712,"completion_tokens":2888,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":456,"completion_tokens_details":{"reasoning_tokens":2815}},"tokens_in":456,"tokens_out":2888,"duration_ms":25996,"temperature":1.0,"reasoning_tokens":2815,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T13:26:27.071867+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the trained GCTC on a real differential-drive robot or in a higher-fidelity simulator with actuator delay, then intentionally perturb one learned parameter (for example, cV) by increasing amounts and record the tracking-error response; the guarantee would be disproved if small perturbations produce overshoot, oscillation, or instability that the error equation (52) does not predict.","supporting_citations":[{"cited_title":"Resolved -acceleration control of mechanical manipulators,","cited_arxiv_id":null,"evidence_quote":"Provides the computed torque method that the gray-box controller adapts to mobile robots."},{"cited_title":"Dynamic Modelling of Differential -Drive Mobile Robots using Lagrange and Newton-Euler Methodologies: A Unified Framework,","cited_arxiv_id":null,"evidence_quote":"Supplies the DDMR dynamic equations and constants Ci used to derive M(θ) and C(ω)."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Source for the CTC structure, the viscous and Coulomb friction models, and the diagonal gain matrices."},{"cited_title":"An algebraic and suboptimal solution of constrained model predictive control via tangent hyperbolic function,","cited_arxiv_id":null,"evidence_quote":"Provides the tanh change-of-variables that enforces physically plausible parameter ranges during optimization."}],"review_version":1}