{"id":"5ce5979a-b0ce-4bd9-b703-1d16d079a033","arxiv_id":"2502.01329","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"HPIPM with a sparse MPC formulation is the fastest solver, and the Jetson ARM board is the most power-efficient platform according to the new Solve Frequency per Watt metric.","lead":"This paper benchmarks quadratic programming solvers and dense versus sparse problem formulations for controlling a simulated four-legged robot, and introduces a Watts-based efficiency metric. It recommends specific solver and hardware combinations for onboard walking controllers, with the ARM-based Jetson board winning on efficiency.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The solver ranking ignores solution accuracy: HPIPM speed_abs is faster but explicitly less accurate, and no KKT-residual or constraint-violation metric is reported; the speed comparison may be an artifact of the speed-accuracy tradeoff.","rationale":"The reader's concern about 10 Hz power sampling is well-taken and affects the hardware-efficiency conclusion. However, the paper's most generalizable contribution is the solver/formulation ranking, and that ranking is vulnerable to an uncontrolled speed-accuracy tradeoff before any power measurement is involved. The text itself acknowledges that speed_abs is less accurate, yet the benchmark reports no accuracy metric. Because solve time alone is the independent variable in the headline claim, the claim is only meaningful if solution quality is comparable across solvers. A fast, inaccurate solver can win on time but be unsuitable for deployment; the simulation failure criterion does not protect against this because the controller and integrator can tolerate small errors. The concrete test above would settle the issue directly. If HPIPM speed_abs remains fastest at matched accuracy, the recommendation stands and the paper's usefulness increases; if not, the paper should at least rank solvers on a Pareto frontier of time vs accuracy. Either way, the power-sampling issue, while real, is secondary.","tokens_in":11539,"tokens_out":4952,"duration_ms":44146,"concrete_test":"Re-run the MPC benchmark (N=10 and N=20, all three platforms) while logging the KKT residual and maximum constraint violation for every QP solution. Then impose a common accuracy threshold, e.g., relative KKT residual <= 1e-4 and max constraint violation <= 1e-6, by adjusting solver tolerances; recompute mean solve times only for solutions meeting the threshold. If HPIPM speed_abs no longer dominates under matched accuracy, the Sec. IV-A recommendation must be weakened. A weaker intermediate check: use the available benchmark code to compare the final residuals of speed_abs vs balanced on identical QP instances; an order-of-magnitude residual gap would confirm the speed-accuracy confound.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The strongest claim—'For all computer architectures, condensing levels, and horizons, the best-performing solver is HPIPM speed abs' (Sec. IV-A)—rests entirely on solve time, with no reported measure of solution quality. The paper itself states that HPIPM's 'balanced' mode 'provides more accurate results' while speed_abs 'focuses on speed' (Sec. III-B), and the discussion concedes that speed_abs 'does not provide accurate results for some applications' (Sec. V). If speed_abs returns solutions with looser tolerances or larger constraint violations, it can win on time for the wrong reason. The only recorded failure criterion is the robot falling in simulation, which is far too coarse to detect inaccurate stabilizing solutions. Thus the solver recommendation may be an artifact of an uncontrolled speed-accuracy tradeoff, not a genuine performance advantage. This is more fundamental than the 10 Hz power-sampling issue: even with perfect power measurements, the solver ranking would remain unsupported without accuracy control.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript benchmarks QP formulations and solvers for MPC and WBC in dynamic quadrupedal walking on three computing platforms (Jetson Orin NX, LattePanda Alpha, Desktop PC). It compares sparse, partially condensed, and fully condensed MPC formulations across prediction horizons N=10 and N=20, and two WBC variants (reduced and full TSID), using solvers HPIPM (balanced and speed_abs), OSQP, qpOASES, DAQP, PROXQP, and Eiquadprog. It introduces Solve Frequency per Watt (SFPW) as a cross-hardware efficiency metric. Experiments use a simulated Unitree Go2 in Drake with a standard MPC+WBC controller in trotting and standing scenarios. Main findings are that HPIPM speed_abs is fastest across all architectures, condensing levels, and horizons; sparse formulations outperform dense ones for MPC; all tested WBC solvers perform adequately; and the Jetson Orin is the most power-efficient platform. The benchmark code is made open source.","tokens_in":11701,"tokens_out":5249,"duration_ms":44975,"significance":"The paper addresses a practical gap: there is no systematic comparison of QP formulations, solvers, and hardware for legged MPC/WBC. If the findings are robust, they give actionable defaults to practitioners and highlight that WBC formulation choice matters more than solver choice. Strengths include the reproducible open-source implementation (controller plus benchmark), coverage of partial condensing levels via acados, inclusion of multiple solver methods (IPM, ADMM, ASM, ALM), and a novel efficiency metric (SFPW) that, despite measurement issues, is a useful step. The qualitative trends (sparse better for longer horizons, IPM robust to changing problem structure) are plausible and consistent with prior work. However, the quantitative claims require stronger experimental controls, especially regarding solution accuracy and power measurement.","major_comments":[{"comment":"The headline result that \"For all computer architectures, condensing levels, and horizons, the best-performing solver is HPIPM speed abs\" is based exclusively on wall-clock solve time, with no measure of solution quality (e.g., KKT residual, primal/dual constraint violation, or cost gap). The manuscript itself states that HPIPM's balanced mode \"provides more accurate results\" while speed_abs \"focuses on speed\" (Section III-B) and later concedes that speed_abs \"does not provide accurate results for some applications\" (Section V). Because speed_abs is admitted to trade away accuracy, its speed advantage may be an artifact of looser termination tolerances rather than a genuine algorithmic gain. The robot-falling failure criterion is too coarse to detect inaccurate but stabilizing MPC solutions. Please report accuracy metrics per solver and mode, and either restrict the ranking to solutions meeting a common accuracy threshold or discuss how the speed-accuracy tradeoff affects each recommendation.","section":"§III-B, §IV-A"},{"comment":"The SFPW metric divides mean solve time by CPU power consumption sampled at 10 Hz, while individual QP solves take microseconds to milliseconds. As a result, the power sample may be dominated by background load or miss the solve burst entirely, so the cross-platform efficiency conclusion (\"Jetson Orin performs best... more than twice as efficient as the LattePanda and around three times as efficient as the desktop PC\") is not robust to measurement error. No error bars or confidence intervals are provided for the SFPW values in Table IV. Please either synchronize power measurement with the benchmark workload, report integrated energy over a fixed number of solves, or provide a sensitivity analysis showing that the ranking is stable under plausible power-measurement offsets.","section":"§III-A, §IV-C, Table IV"},{"comment":"The reported \"mean MPC solve time\" includes condensing time, stated as \"less than 1 ms.\" Since some of the fastest solves are sub-millisecond, condensing time can be a large fraction of the reported total; moreover, dense-interface solvers (qpOASES, DAQP) necessarily include different preprocessing than sparse-interface solvers with partial condensing. The paper therefore conflates formulation/condensing cost with solver performance. Please report solver-only time and condensing/setup time separately for each condensing level, and report distributions (e.g., percentiles) in addition to means.","section":"§IV-A, Fig. 3"},{"comment":"The experimental protocol lacks statistical detail: no number of trials per configuration, no error bars, and no explicit handling of failed runs beyond marking \"Exp. failed\" and the two-retry rule if the robot falls. This matters because several comparisons are described as marginal (e.g., HPIPM balance vs. OSQP on x86 for N=20) or as \"similar\" (Jetson vs. LattePanda), yet no dispersion or significance measure is given. Please add per-configuration repetitions with standard deviations or percentiles and state the number of trials.","section":"§III-D, §IV"}],"minor_comments":[{"comment":"The sentence \"the impact of QP formulation ... on performance used has yet to be comprehensively examined\" is grammatically broken and hard to parse; please revise.","section":"§I"},{"comment":"The sentence \"both qpOASES and DAQP perform almost as well as OSQP with Np = 20 (sparse problem)\" appears to be a typo; for the N=10 comparison, the sparse problem has Np=10, not Np=20.","section":"§IV-A"},{"comment":"Several numeric entries contain stray spaces (e.g., \"8 .82\", \"67.30\", \"753 .73\"); please fix the formatting.","section":"Table IV"},{"comment":"The text says \"we use the subscript a to account only for the actuated joints,\" but the equation notation tau_m, tau_M is not subscripted; please clarify the relationship between S, tau, and the torque limits.","section":"§II-B, Eq. (2e)"},{"comment":"The red circles marking failed experiments may be hard to see in grayscale; consider using distinct markers or adding a legend entry that remains legible in print.","section":"Fig. 3"}],"recommendation":"major_revision","confidential_remarks":"The central concern is not novelty or circularity but experimental control: the solver ranking needs accuracy normalization and the efficiency ranking needs better power measurement. The paper is otherwise a solid engineering contribution with open code, and I believe it can be revised to address these issues. The reader's power-sampling concern and the skeptic's accuracy concern are both legitimate; the accuracy issue is the more fundamental one because it affects the primary solver recommendation even if power measurements were perfect."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"What you should know: this is a genuinely useful engineering benchmark, but its headline recommendations are not fully supported because the solver ranking ignores solution accuracy. The reader's take and the stress-test both land.\n\nThe paper does something new: it combines QP formulation (sparse/partial condensing), five solvers, three computers, and two walking tasks into one benchmark, and introduces SFPW as a cross-hardware efficiency metric. The code is open source, the controller is standard, and the qualitative findings—sparse formulations win for longer horizons, IPM is robust to task changes, WBC solver choice matters less than formulation—are consistent with prior theory from Axehill and Frison et al. That consistency is a strength, not a weakness.\n\nSoft spots, in order. First, the solver ranking \"HPIPM speed_abs is best\" rests entirely on solve time. The paper itself says speed_abs \"focuses on speed\" and \"does not provide accurate results for some applications.\" No KKT residual, constraint violation, or cost suboptimality is reported. The only failure criterion is the robot falling, which is too coarse to detect a stabilizing but inaccurate solution. So the speed advantage may partly be a relaxed-tolerance artifact. This is a real gap and should be fixed before the recommendation is trusted.\n\nSecond, SFPW divides by CPU power sampled at 10 Hz while solves take microseconds. The power reading can be dominated by background load or miss the solve burst. The paper's hardware-efficiency claim (ARM is more efficient) depends on this. It is a legitimate concern, though not as fundamental as the accuracy issue.\n\nThird, the reported solve times include condensing time. That is defensible for end-to-end MPC, but it should be separated or justified so readers can compare with other benchmarks. Also, no error bars or statistics are reported, which makes the quantitative ratios shaky. The qualitative rankings are plausible; the exact efficiency ratios are not.\n\nWho this is for: practitioners choosing a solver, formulation, and hardware for legged MPC. The benchmark is a useful starting point despite the caveats. It deserves a serious referee; I would send it out and ask for accuracy metrics and power measurement details. With those fixed, it would be a solid accept.","headline":"Useful engineering benchmark whose solver and hardware recommendations are not yet fully supported: the solver ranking ignores solution accuracy, and the power metric is sampled too coarsely.","tokens_in":12211,"tokens_out":2019,"would_cite":true,"duration_ms":18223,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that in dynamic quadrupedal walking, the sparse MPC formulation solved with HPIPM speed abs is fastest on all tested hardware, and the ARM-based Jetson Orin is the most energy-efficient platform.","keywords":["quadratic programming","model predictive control","whole-body control","quadrupedal walking","sparse formulations","solver benchmark","solve frequency per watt","embedded computing"],"falsifier":"Run the same benchmark with power measured at kilohertz rate or with direct current sensing and compare the SFPW ranking; if the Jetson Orin no longer gives roughly 2-3x the efficiency of the desktop or LattePanda, the hardware-efficiency claim fails. Separately, finding any condensing level and horizon on one tested computer where a solver other than HPIPM speed abs achieves the lowest mean solve time would falsify the solver-dominance claim.","tokens_in":11354,"feed_emoji":"🦿","tokens_out":5636,"duration_ms":51392,"temperature":0.7,"pith_summary":"This paper sets out to give practitioners a defensible default choice for the quadratic programs (QPs) at the heart of quadrupedal walking control. It claims that for model predictive control (MPC), the sparse problem formulation solved with the interior-point solver HPIPM in its speed-focused mode is the fastest option across every tested hardware platform, condensing level, and horizon. For whole-body control (WBC), it claims solver choice matters little; the formulation and setup dominate. It also introduces a metric, solve frequency per watt, and reports that the ARM-based embedded platform outperforms both x86 systems in energy efficiency for this workload, making it the recommended choice for battery-limited robots.","feed_headline":"HPIPM speed-abs wins robot-walking QP benchmark","feed_subtitle":"Sparse interior-point MPC wins on speed, and ARM's Jetson Orin delivers 2-3x the efficiency per watt.","key_machinery":"Two mechanisms carry the argument. First, the SFPW metric, defined as $(\\text{solve time})^{-1}$ divided by CPU power consumption, with CPU power sampled at 10 Hz through Intel RAPL on x86 and tegrastats on ARM; it converts raw solve times into a hardware-independent efficiency ranking. Second, the partial-condensing formulation, which splits the prediction horizon N into Np blocks and eliminates state variables inside each block, so the benchmark can sweep continuously from fully sparse to fully dense QPs. The solver set spans interior-point, active-set, ADMM/ALM, and dual active-set methods, with HPIPM providing both the sparse solver and the condensing routines.","core_discovery":"The paper's central claim is that a single solver and formulation combination dominates for MPC on walking robots: the sparse QP formulation using HPIPM's speed-abs interior-point mode. In their experiments this combination delivered the lowest mean solve time for all three computers, for both horizons (N=10 and N=20), and across all partial-condensing levels; dense formulations became impractical at N=20 for several solvers. For WBC, all tested solvers stayed below 0.5 ms, with Eiquadprog fastest in every comparison, and the larger full TSID formulation increases time only slightly. The paper further claims that the Jetson Orin is the most efficient platform: more than twice as efficient as the LattePanda and about three times as efficient as the desktop PC for MPC, and roughly four times as efficient for WBC, as measured by the new solve-frequency-per-watt metric.","pith_inferences":["The 10 Hz power sampling may undercount brief solve bursts; if so, the ARM-efficiency conclusion could change under a higher-rate power measurement, so the hardware ranking should be re-tested before being treated as universal.","The benchmark uses standard solver hyperparameters and a simulated trotting and standing task; per-application tuning or real-hardware power draw could shift the ranking.","Because the MPC problem has input dimension close to state dimension, sparse wins; the paper's own reasoning implies condensing could win for robots with far fewer actuators, a testable prediction.","The SFPW metric could be adopted more broadly as a standard for comparing control algorithms on embedded robots, not just QP solvers."],"forward_implications":["MPC users on legged robots can default to a sparse formulation with HPIPM's speed-abs mode rather than hand-tuned dense QPs, especially at longer horizons.","For WBC, engineering effort is better spent on the problem formulation than on solver selection, since all tested solvers are comfortably fast.","Energy-constrained deployments should favor ARM-based embedded boards such as the Jetson Orin, which delivered roughly 2-3x the solve frequency per watt of x86 systems.","At short horizons (N=10), active-set solvers like qpOASES and DAQP are competitive, so dense formulations remain viable for smaller problems.","Interior-point solvers show more stable solve-time distributions across changing contact schedules, which matters for dynamic gait."],"supporting_citations":[{"why":"Supplies the fastest solver (HPIPM) and the partial-condensing routines that the benchmark's central comparison depends on.","marker":"[10]"},{"why":"Establishes the standard convex MPC formulation for quadrupedal locomotion that this benchmark implements and extends.","marker":"[13]"},{"why":"Provides the MPC plus whole-body control architecture whose WBC QP is benchmarked here.","marker":"[14]"},{"why":"Introduces the concept of controlling the level of sparsity in MPC, which the partial-condensing sweep operationalizes.","marker":"[12]"},{"why":"Gives the efficient partial-condensing implementation used to generate the different condensing levels.","marker":"[21]"},{"why":"Supplies OSQP as a comparison sparse ADMM-type solver baseline.","marker":"[8]"},{"why":"Supplies qpOASES as a comparison dense parametric active-set solver baseline.","marker":"[7]"},{"why":"Supplies the WBC formulation variant used in the reduced TSID benchmark.","marker":"[22]"},{"why":"Provides the MPC software interface through which the solvers and condensing levels are benchmarked.","marker":"[25]"},{"why":"Supplies the quadruped robot platform used for the simulated evaluation.","marker":"[6]"}],"fun_headline_variants":["Sparse QP with HPIPM speed-abs wins quadruped MPC race","Jetson Orin triples QP efficiency per watt for walking robots","New SFPW metric benchmarks QP solvers across hardware","HPIPM and Eiquadprog lead QP solver benchmarks for quadrupeds"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The efficiency ranking rests on CPU power measurements sampled at 10 Hz being accurate for QP solves that take microseconds to milliseconds; a noisy or background-dominated power sample could flip the ordering between platforms.","fun_headline_variants_meta":{"raw":{"variants":["Sparse QP with HPIPM speed-abs wins quadruped MPC race","Jetson Orin triples QP efficiency per watt for walking robots","New SFPW metric benchmarks QP solvers across hardware","HPIPM and Eiquadprog lead QP solver benchmarks for quadrupeds"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00052,"raw_usage":{"total_tokens":2514,"prompt_tokens":940,"completion_tokens":1574,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":556,"completion_tokens_details":{"reasoning_tokens":1499}},"tokens_in":556,"tokens_out":1574,"duration_ms":11802,"temperature":1.0,"reasoning_tokens":1499,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T15:39:26.394486+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same benchmark with power measured at kilohertz rate or with direct current sensing and compare the SFPW ranking; if the Jetson Orin no longer gives roughly 2-3x the efficiency of the desktop or LattePanda, the hardware-efficiency claim fails. Separately, finding any condensing level and horizon on one tested computer where a solver other than HPIPM speed abs achieves the lowest mean solve time would falsify the solver-dominance claim.","supporting_citations":[{"cited_title":"HPIPM: a high-performance quadratic programming framework for model predictive control *,","cited_arxiv_id":null,"evidence_quote":"Supplies the fastest solver (HPIPM) and the partial-condensing routines that the benchmark's central comparison depends on."},{"cited_title":"Controlling the level of sparsity in MPC,","cited_arxiv_id":null,"evidence_quote":"Introduces the concept of controlling the level of sparsity in MPC, which the partial-condensing sweep operationalizes."},{"cited_title":"An efficient implementation of partial condensing for Nonlinear Model Predictive Control,","cited_arxiv_id":null,"evidence_quote":"Gives the efficient partial-condensing implementation used to generate the different condensing levels."},{"cited_title":"Implementing Torque Control with High-Ratio Gear Boxes and Without Joint-Torque Sensors,","cited_arxiv_id":null,"evidence_quote":"Supplies the WBC formulation variant used in the reduced TSID benchmark."},{"cited_title":"Unitree Go2 Quadruped","cited_arxiv_id":null,"evidence_quote":"Supplies the quadruped robot platform used for the simulated evaluation."}],"review_version":1}