{"id":"5f1058f7-13ac-41c7-9327-b2bc260754ec","arxiv_id":"2512.12649","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"Bayesian optimization with a Gaussian-process surrogate tuned four coupled controller gains on a real three-wheeled robot, improving a composite tracking cost from 2076 to 1888 in 32 lap trials.","lead":"A robotics team used Bayesian optimization to tune four coupled gains of a Lyapunov-based path-following controller on Honda's AI-Formula three-wheeled robot. Within 32 real-world lap trials, the method found gains that beat a manually tuned baseline, though the reported gains are modest and lack repeated-run error bars.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Headline improvement may be a selection artifact: best of 32 noisy one-lap evaluations vs a single baseline draw, with no repeats and no random-search control.","rationale":"I read the paper as making an empirical claim: within 32 real-vehicle trials, BO found a gain setting with lower path-following cost than the manual baseline, demonstrating data-efficient tuning on a nonlinear controller. That claim requires that the reported cost gap reflects true controller performance rather than noise and selection. The paper's own Section IV-B admits substantial run-to-run disturbances and says misrankings cannot be corrected within the budget; this is precisely the condition under which a single-lap minimum-of-32 comparison is unreliable. The missing repeated evaluations of at least the baseline and the final best are therefore load-bearing. The stability proof in Section II-C also has gaps (singular control at α=0, ρ=0 and no LaSalle argument), but those are secondary: the central contribution is the empirical demonstration, and even a fully rigorous proof would not fix the statistical support. I do not see this as misconduct; the hardware work appears genuine and the BO implementation is standard. The concern is that the headline effect size (188.7 cost units, about 9%) is the same order as admitted run-to-run variability and could be explained by the minimum-of-many effect. A targeted repeat experiment would settle it. Since the reader already marked the paper CONDITIONAL on this type of issue, my verdict is unchanged.","tokens_in":16299,"tokens_out":6339,"duration_ms":66324,"concrete_test":"A single decisive check: perform five interleaved full-lap evaluations of the manual baseline gain vector and the Iteration 31 gain vector, all other settings fixed, and compute the paired mean difference and its 95% confidence interval. If the CI includes zero or the observed single-run gap of 188.7 is within two pooled within-configuration standard deviations, the claimed improvement is not statistically established; additionally, take the minimum of 32 bootstrap samples drawn from the baseline repeats and check whether it reaches 1887.65 with non-negligible probability.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Central claim: BO's best value bJBO=1887.65 (Iteration 31) improves on the manual baseline 2076.35 (Iteration 1). The weakest point is not the BO math but the experiment design. Each gain vector is evaluated exactly once, and 1887.65 is the minimum of 32 noisy draws, whereas 2076.35 is a single draw. Even if all evaluations are unbiased, the minimum of 32 draws is expected to sit below a single draw, so part (possibly all) of the reported 9% improvement can be selection bias. The paper itself concedes in Section IV-B that disturbances 'can misrank gain vectors' and that with a limited budget 'these misjudgements cannot be corrected through repeated trials.' No error bars, repeated trials, or confidence intervals are given. Table II shows only modest raw-error differences (lateral RMS 0.459 vs 0.478 m; mean lateral 0.415 vs 0.424 m), consistent with run-to-run noise. A further missing control: the warm-start space-filling sample at Iteration 9 already reached 1985.14, only ~5% above the BO best, so the specific contribution of acquisition-driven search over a few random samples is not demonstrated. Thus the strongest claim—that BO 'identifies well-performing configurations with far fewer trials'—is not yet supported by the reported evidence.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a Bayesian optimization (BO) framework for tuning the four gains of a Lyapunov-based path-following controller on Honda's AI-Formula three-wheeled robot. The controller gains are treated as black-box inputs; a Gaussian-process surrogate with an ARD Matérn-5/2 kernel and expected-improvement acquisition is used to select gain vectors within a bounded search domain. After 15 warm-start evaluations (including a manually tuned baseline) and 17 acquisition-driven evaluations, the best observed performance is reported at Iteration 31 with cost 1887.65 versus the baseline cost 2076.35, and the paper claims that BO efficiently locates well-performing gain configurations with a small real-world evaluation budget.","tokens_in":16698,"tokens_out":5200,"duration_ms":58922,"significance":"If the empirical claim is supported, the paper would be a useful real-vehicle demonstration of BO tuning for a geometric/nonlinear path-following controller, extending earlier BO studies that focus mainly on MPC formulations. The paper has clear strengths: the BO machinery is standard and correctly described, the Lyapunov-derivative correction is explicit, the penalty formulation for unstable laps is a reasonable practical device, and the authors state that code is available. However, the central validation is currently weak. The reported improvement rests on single-run evaluations without repeated trials, error bars, or a random-search control, and the cost function includes a per-lap normalization that can itself distort comparisons. These issues make the headline contribution—that BO is reliably data-efficient for this controller—not yet established.","major_comments":[{"comment":"The headline improvement is not supported by the experimental design because every gain vector is evaluated in exactly one full-lap run. The best cost (1887.65 at Iteration 31) is the minimum of 32 noisy one-lap evaluations, while the baseline (2076.35) is a single draw. Even if all settings have identical expected cost, the minimum of 32 noisy draws is expected to lie below a single draw, so part or all of the reported 9% gap may be a selection artifact. The manuscript itself concedes in Section IV-B that disturbances 'can misrank gain vectors' and that with limited budget 'these misjudgements cannot be corrected through repeated trials.' The authors should provide repeated laps at least at the baseline and at the final candidate (e.g., 3–5 laps each), report mean and standard deviation, and quantify the selection effect via a permutation test or a random-search comparison.","section":"Section IV-B, Fig. 5, Table II"},{"comment":"The warm-start initialization already appears to find near-optimal settings: Iteration 9 yields cost 1985.14, only about 5% above the final best, and Iteration 7 yields 2019.88. Since the initial 15 evaluations are described as essentially a hybrid space-filling/randomized search, it is unclear whether the acquisition-driven BO loop adds value over the initial sample. The paper does not provide a control of 32 space-filling or random evaluations, nor does it report the full set of 32 points and costs so that the later iterations can be assessed independently. The conclusion that BO 'identifies well-performing configurations with far fewer trials' would require evidence that the GP/EI-guided iterations improve on the warm-start phase beyond random or space-filling selection.","section":"Section IV-B, Iterations 1–15 vs 16–32"},{"comment":"The cost normalization in Eqs. (22)–(23) is problematic. Because each lap's error series is divided by its own median, J_lat is approximately N * mean(|e_lat|) / median(|e_lat|). A lap with large absolute errors has a large denominator and is thereby penalized less, while a lap with uniformly small errors has a small denominator and is penalized more. This per-lap normalization makes J incomparable across gain vectors and can bias the GP toward regions with larger tracking errors. The authors should justify this normalization or replace it with a fixed reference normalization (e.g., using the baseline lap's median or absolute units with a known scale factor). As written, the cost values in Fig. 5 and Table II may not faithfully rank the tested gains.","section":"Section II-D, Eqs. (21)–(23)"}],"minor_comments":[{"comment":"The abstract says the framework is 'assessed through repeated full-lap experiments,' but every gain setting is evaluated only once. Rephrase to avoid implying replicated trials.","section":"Abstract and Section IV"},{"comment":"The hybrid space-filling strategy for Iterations 2–15 is described only verbally. Please provide the exact sampling method, bounds, and any random seed so the warm-start set is reproducible.","section":"Section III-D"},{"comment":"The GP hyperparameters (lengthscales, signal variance, noise variance) are said to be obtained by marginal-likelihood maximization but their fitted values are not reported. Given that each θ is observed once, σ_n² is not identifiable without further structure; reporting fitted values or fixing σ_n from separate experiments would improve reproducibility.","section":"Section III-B"},{"comment":"The angular velocity command contains sinα in the denominator, which is singular at α=0. The implementation must handle this division; please state the safeguard used on the robot (e.g., a small-angle approximation, a lower clamp, or a modified formula).","section":"Section II-C, Eq. (18)"},{"comment":"The upper and lower subplots have the same x-label but the lower plot's y-range is not explicitly described in the caption. Add a caption note that the lower panel is a zoomed view and mark the baseline, Iteration 9, and Iteration 31 for clarity.","section":"Fig. 5"}],"recommendation":"major_revision","confidential_remarks":"The paper's BO methodology is standard and the hardware demonstration is potentially valuable, but the empirical evidence for the central claim is not yet convincing. The missing repeated trials, absence of a random-search control, and the questionable per-lap normalization are load-bearing. I would recommend major revision: if the authors can add repeated baseline/final evaluations, a control comparison, and a corrected cost normalization, the paper could become a solid application contribution. If these cannot be provided, the paper may be better positioned as a preliminary demonstration rather than a validation of BO's efficiency."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is a genuine hardware demonstration of BO tuning a four-gain Lyapunov-based path-following controller on a real three-wheeled robot. That alone is worth something: prior BO tuning work for path following focused on MPC formulations, so this extends the evidence base to a different controller class. The corrected derivative in Remark 1 is a useful detail, and the BO implementation (log-space gains, warm start, instability penalty) is sensible and described clearly. The authors also put code and data in a public repo, which helps reproducibility.\n\nWhat the paper does not establish is its headline claim: that BO efficiently finds better configurations than manual tuning within 32 trials. The reported best (1887.65 at Iteration 31) is the minimum of 32 noisy one-lap evaluations, while the manual baseline (2076.35) is a single draw. Even with unbiased noise, the minimum of 32 draws will typically sit below a single draw, so part—possibly all—of the 9% improvement can be selection artifact. The paper itself admits in Section IV-B that disturbances can misrank gain vectors and that misjudgements cannot be corrected through repeated trials. No repeats, no error bars, no confidence intervals. A warm-start sample at Iteration 9 already reached 1985.14, only about 5% above the BO best, so the marginal value of the acquisition-driven search is not demonstrated. Without a random-search or initialization-only baseline, the specific contribution of BO remains unclear.\n\nThere is also a gap in the stability proof in Section II-C. The derivative \\dot{V}_1 in equation (16) contains a positive term that does not vanish unless α→0, so the claim that \\dot{V}_1 is negative semi-definite near equilibrium is local at best, and the argument does not establish global convergence. This is a real flaw in the proof, though the controller evidently works in practice on the hardware.\n\nThe trajectory comparisons are qualitatively suggestive, and Table II shows modest reductions in lateral error, but those differences are within plausible run-to-run noise. The paper is honest about the noise problem, which I credit—but an honest acknowledgment of a limitation does not remove the need for an experimental design that controls for it.\n\nWho is this for? People doing real-world controller tuning with BO will find the engineering details useful. The contribution is modest but real. It deserves a serious referee: the experiment design can be improved with repeated trials or error bars, and the stability proof needs fixing. I would not desk-reject it, but I would send it back with requests for a proper baseline and uncertainty quantification.","headline":"Solid real-world BO tuning demo undermined by weak statistical evidence—single noisy laps, no repeats, and a stability proof gap.","tokens_in":17139,"tokens_out":1599,"would_cite":true,"duration_ms":18544,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper shows that Bayesian optimization, treating the vehicle as a black box, can tune four strongly coupled controller gains in 32 real-vehicle laps, improving the lap cost from 2076 to 1888 and cutting peak lateral error by about 18 pe","keywords":["Bayesian optimization","parameter tuning","Gaussian process","Expected Improvement","Lyapunov-based control","path following","mobile robot","real-world experiments"],"falsifier":"Rerun the baseline gains and the Iteration-31 gains for several additional laps each under identical conditions and compare the distributions of lap costs; if the observed 1887.65 vs 2076.35 difference is smaller than the run-to-run spread, the reported improvement is not distinguishable from measurement noise.","tokens_in":16212,"feed_emoji":"🤖","tokens_out":6780,"duration_ms":59724,"temperature":0.7,"pith_summary":"The paper sets out to show that Bayesian optimization is a practical way to tune the four coupled gains of a Lyapunov-based path-following controller on a real three-wheeled robot, where each evaluation costs one full lap. Using a Gaussian-process surrogate and expected-improvement acquisition, it claims to find gains within 32 trials (15 of them warm-start) that lower the lap-based tracking cost from 2076.35 to 1887.65 and reduce peak lateral error by around 18 percent. The result matters because manual tuning of such coupled nonlinear gains is slow and often blind, and simulation-only tuning does not transfer to hardware. If the claim holds, BO becomes a credible default tool for data-efficient controller tuning on real platforms.","feed_headline":"Bayesian optimization tunes path-following gains in 32 laps","feed_subtitle":"Four coupled gains drop the lap cost from 2076 to 1888 on a real robot.","key_machinery":"The load-bearing mechanism is the Bayesian optimization loop: a Gaussian-process surrogate with an ARD Matérn-5/2 kernel models the mapping from the four gain values to the lap cost, and expected improvement selects the next gain vector by balancing predicted gain against model uncertainty. Two engineering details make it work on hardware: gains are optimized in logarithmic space because their effective ranges differ by orders of magnitude, and any lap lost to instability is retained with a penalty proportional to the uncompleted distance, so the surrogate learns which regions are unsafe without discarding those trials.","core_discovery":"We demonstrate that Bayesian optimization, with a Gaussian-process surrogate, expected-improvement acquisition, logarithmic gain scaling, and a penalty for incomplete unstable laps, can tune the four coupled gains of a Lyapunov-based kinematic path-following controller on a real three-wheeled robot. Within a budget of 32 full-lap evaluations, the best BO-selected gains reach a performance cost of 1887.65 versus the manually tuned baseline of 2076.35, with the maximum lateral error reduced by about 18 percent and visibly smoother yaw-rate and speed profiles, especially on curved sections. This is presented as evidence that BO is a practical, data-efficient tool for black-box controller tuning","pith_inferences":["The headline improvement (about 9% in cost) is measured on one lap per setting; given the paper's own acknowledgment of run-to-run disturbance, the practical gain may be smaller or even indistinguishable from noise until repeated trials confirm it.","The same framework could be extended to multi-objective costs, such as minimizing both tracking error and control effort, without changing the surrogate—suggesting a path to more general tuning policies.","A natural falsifier is to freeze the search after finding the best gain and rerun it multiple times to estimate the noise floor; if the noise floor exceeds the measured improvement, a different comparison protocol (e.g., paired laps or averaged costs) is needed before claiming superiority over manual tuning.","The paper's own future direction—safety-aware BO with a conservative, expanding safe region—could turn this tuning tool into an online adaptation layer that re-tunes gains as track or platform conditions drift."],"forward_implications":["If the central claim holds, engineers can tune nonlinear geometric controllers with a few dozen hardware experiments instead of hundreds of manual trials.","The same loop—GP surrogate, expected improvement, log-scale gains, penalty for failed runs—should transfer to other black-box closed-loop tuning tasks, such as MPC weights or PID gains on vehicles.","The warm-start and penalization design means unsafe gain regions can be mapped and avoided without discarding the information from unstable laps, reducing the risk of damaging hardware experiments.","The observed error reductions concentrate in curved track sections, indicating that BO-tuned gains improve transient tracking rather than only steady-state offset.","Because each trial is one lap, the approach is affordable on platforms where running a full experiment is costly but not destructive, opening the door to on-site retuning as conditions change."],"fun_headline_variants":["BO cuts track cost 9% in 32 trials on real robot","32 laps: Bayesian optimization beats manual tuning","Gaussian-process tuning drops cost 1888 vs 2076","Data-efficient BO: 32 laps find better robot gains","Bayesian optimization tunes 4 gains in 32 real laps"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"A single full-lap evaluation of each gain setting reliably ranks gain vectors, even though the paper acknowledges that run-to-run disturbances can misrank vectors and that repeated trials cannot correct these misjudgements within the budget.","fun_headline_variants_meta":{"raw":{"variants":["BO cuts track cost 9% in 32 trials on real robot","32 laps: Bayesian optimization beats manual tuning","Gaussian-process tuning drops cost 1888 vs 2076","Data-efficient BO: 32 laps find better robot gains","Bayesian optimization tunes 4 gains in 32 real laps"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000906,"raw_usage":{"total_tokens":3720,"prompt_tokens":721,"completion_tokens":2999,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":465,"completion_tokens_details":{"reasoning_tokens":2915}},"tokens_in":465,"tokens_out":2999,"duration_ms":20258,"temperature":1.0,"reasoning_tokens":2915,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T16:34:34.053032+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Rerun the baseline gains and the Iteration-31 gains for several additional laps each under identical conditions and compare the distributions of lap costs; if the observed 1887.65 vs 2076.35 difference is smaller than the run-to-run spread, the reported improvement is not distinguishable from measurement noise.","supporting_citations":[],"review_version":1}