{"id":"ecafeeac-9813-450e-a2f2-db20e33ae625","arxiv_id":"2412.09989","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"An observation-conditioned reachability network predicts safety value functions online, enabling a controller-agnostic safety filter that adapts to new obstacles and dynamics uncertainty for quadruped navigation.","lead":"This paper introduces a safety filter for quadruped robots that predicts safe regions from LiDAR scans and estimated dynamics uncertainty, overriding unsafe commands from any navigation controller. It is a step toward a single plug-in safety layer for legged robots across different controllers and unknown environments.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The online disturbance-bound estimator (Eq. 14) is not a genuine upper bound, so the reachability safety guarantee can be void when true disturbance spikes; the 2-s adaptation window leaves an unprotected latency period in the claimed robustness regime.","rationale":"The single most load-bearing concern is the validity of the online disturbance-bound estimate, because the entire safety argument for 'robust to unmodeled dynamics' depends on it. HJ reachability's guarantee (Section IV) assumes the disturbance set D is known and contains all actual disturbances. The deployment-time estimator in Eq. (14) computes a trimmed mean ± 2σ over a 2-second sliding window; this is a descriptive statistic, not a bound. It intentionally ignores the most extreme 10% on each side (c=0.8) and has no coverage or worst-case guarantee. If the true disturbance exceeds the estimated bound, the OCR-VN's predicted value function is not conservative, and the filter in Eq. (15) may fail to intervene. The 2-s window also creates an adaptation delay that is explicitly acknowledged in Section VIII. The paper's empirical collision rates, especially in hard/slippery conditions (e.g., 8-9% collision for ABS-Agile with OCR), are consistent with this gap. The reader identified the same weakest assumption; I agree. The concrete test I propose directly measures whether the assumption holds by logging true disturbances in simulation and checking coverage, especially around sudden friction changes. If the test shows the estimated bound is violated at a non-negligible rate or that collisions occur during the adaptation window, then the central claim is not supported and the verdict would need to become REJECT (or at least require a fundamentally different disturbance estimator). If the test shows the bound is virtually always valid and the latency is small, the concern is mitigated and the CONDITIONAL verdict stands. Given the current evidence, I believe UNCHANGED (CONDITIONAL) is the right verdict: the paper is promising and transparent, but the disturbance-bound hypothesis needs empirical validation.","tokens_in":25278,"tokens_out":7914,"duration_ms":88861,"concrete_test":"In Isaac Sim, log the ground-truth reduced-order disturbance d_r = (x_r^i+1 - x_r^i)/η - f_r(x_r^i, ω^i) at each control step during the hard-condition scenarios, and compare it with the online bound ¯d_r from Eq. (14). Include the sudden-friction-change setup of Fig. 6(b) (or a step change in friction coefficient) and record: (i) the fraction of control steps where ||d_px,py|| exceeds ¯d_px,py or |d_pθ| exceeds ¯d_pθ; (ii) the latency between the friction change and the time when the estimated bound first covers the true disturbance; (iii) whether any collision occurs during that latency. If the violation fraction is non-negligible (e.g., >1%) or a collision occurs in the latency window, the disturbance-bound assumption is empirically falsified and the central robustness claim is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central safety claim depends on HJ reachability theory, which requires the disturbance set D to be a known, fixed bound on the actual disturbance (Section IV-A). In deployment, the OCR filter uses the online estimate from Eq. (14): a trimmed mean ± 2σ over the middle 80% of a 2-second sliding window of recent tracking errors. This is a heuristic location-scale summary, not an upper bound; by construction it excludes 20% of observed disturbances and has no statistical or worst-case coverage guarantee. Whenever the true disturbance (e.g., from a sudden friction change, a slip, or a tracking-error burst) exceeds the estimated bound, the value function predicted by OCR-VN can be larger than the true safety value, so the filter in Eq. (15) may classify an unsafe state as safe and fail to intervene. Furthermore, the estimator reacts to the most recent 2 seconds of history, so a sudden disturbance spike will not be reflected in the bound until it has persisted in the window; during that latency the robot is effectively unprotected. The authors acknowledge this in Section VIII: 'the system may experience disturbances that exceed the estimated disturbance bounds' and there is an 'unavoidable delay.' This is not a peripheral failure mode: the entire claim of robustness to unmodeled dynamics rests on the disturbance bound being valid and timely. The observed nonzero collision rates in Tables III and IV, especially in hard and slippery conditions, are consistent with this gap.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an observation-conditioned reachability (OCR) safety-filter framework for quadrupedal navigation. A value network (OCR-VN) is trained offline, via supervised learning on Hamilton-Jacobi reachability value functions, to predict the safety value function from LiDAR observations, the reduced-order state, and an estimated disturbance bound. At deployment, an online disturbance estimator (Eq. 13-14) computes disturbance bounds from recent state-action history, and a quadratic-program-based filter (Eq. 15-16) overrides nominal high-level twist commands when the predicted value falls below a calibrated threshold. The authors evaluate the framework in Isaac Sim and on a Unitree Go1 quadruped across several nominal planners and locomotion policies, including model-based, learning-based, and teleoperated controllers, and compare against the ABS baseline and ablations without disturbance estimation or calibration. The central claim is that the filter provides reusable, policy-agnostic safety in unknown environments without a priori access to controllers or environments.","tokens_in":25525,"tokens_out":4478,"duration_ms":62481,"significance":"If the empirical claims hold, this is a valuable step toward reusable safety layers for legged locomotion: the value function is policy-independent, the ground-truth data are computed by HJ reachability rather than derived from the safety outcomes, and the ablations give credible evidence that disturbance estimation and calibration each contribute to the observed success rates. The simulation study is reasonably large (100 trials per condition), the hardware experiments cover multiple controllers and a slippery-floor condition, and the paper is unusually candid about residual collision rates and known failure modes. However, the central robustness claim is stated more strongly than the guarantees support: the online disturbance bound is a heuristic location-scale summary rather than an upper bound, the filter uses a slack-softened constraint, and the calibration guarantee does not transfer to deployment. These issues do not invalidate the empirical contribution, but they require a re-framing of what is claimed and, ideally, additional analysis of the residual risk.","major_comments":[{"comment":"The online disturbance bound is not an upper bound on the actual disturbance. Equation (14) computes the trimmed mean plus/minus two standard deviations over only the middle 80% (c = 0.8) of a two-second sliding window; by construction it discards 20% of observed disturbances and has no distribution-free or worst-case coverage guarantee. The HJ reachability guarantee in Section IV-A (Eqs. 5-6) requires the disturbance set D to be a known, fixed bound on the true disturbance. When the true disturbance exceeds the estimated bound, the predicted value can exceed the true safety value, and the filter (Eq. 15) can classify an unsafe state as safe. The authors acknowledge this in Section VIII, but the acknowledgment is placed in the limitations rather than in the framing of the central claim, which states robustness to unmodeled dynamics. The observed nonzero collision rates in Tables III, IV, VI, and VII are consistent with this gap. I request a quantitative characterization: for the reported trials, report how often the true disturbance exceeded the estimated bound, the resulting time-to-collision during the latency window, or a conservative tail bound that can be used to restore a formal guarantee.","section":"Section V-B1, Eq. (14)"},{"comment":"The deployed filter does not implement the smooth least-restrictive filter of Eqs. (8)-(9), so the reachability-based safety guarantee does not directly apply. Equation (16) replaces the hard constraint DtV + min_d ∇V·f = 0 with the softened constraint ... ≥ -s, and the slack variable s is only penalized with a finite weight λ = 10^3 (rather than enforced to zero). Even with a perfect value function and exact disturbance bounds, a positive slack allows the value function to decrease along the filtered trajectory, so the superlevel set is not invariant and the statement in Section V-B2 that the filter is 'grounded in reachability theory' is only approximate. The paper should either enforce s = 0 when feasible and characterize when slack is active, or explicitly describe the resulting behavior as a heuristic safety filter and provide an analysis of the residual risk from slack activation.","section":"Section V-B2, Eqs. (15)-(16)"},{"comment":"The conformal calibration guarantee in Theorem 1 is stated over the calibration distribution P on the validation dataset, and Section VIII correctly notes that the guarantees cannot be extrapolated to the real world because of distribution shift. This is a load-bearing caveat: the threshold δ = 0.49 m used in Eq. (15) is drawn from Table II, but the paper does not quantify how likely it is that the true deployment-time error exceeds this threshold. The result is that the 'safety margin' from calibration is not a safety margin in any formally guaranteed sense. I would like to see either an online calibration scheme, a sensitivity analysis of δ, or an explicit statement that the calibrated threshold is a heuristic tuning parameter rather than a probabilistic safety certificate during deployment.","section":"Section V-A4 and Section VIII"},{"comment":"The claim that a single filter is deployed across all settings is tempered by per-setting modifications. In Section VII-B5, the authors state that for highly cluttered indoor environments they discard LiDAR readings outside a front-facing cone of π/2 rad and use the uncalibrated output of the OCR-VN. Similarly, Section VI-E and Figure 7(b) report using OCR \\ C in a hand-designed dead-end environment. These are not merely implementation details: they change the network input and the activation threshold, and they are motivated by a distribution shift from the training data. The 'one filter to deploy them all' framing should therefore be scoped to the settings in which the same configuration is used, or the paper should justify that the modifications are automatic and not environment-specific tuning.","section":"Section VII-B5 and Section VI-E"}],"minor_comments":[{"comment":"There is a typo in 'cluttered anda priori unknown environments'; it should read 'cluttered and a priori unknown environments'.","section":"Section I"},{"comment":"The disturbance estimation parameters (c = 0.8, b = 2, η·k = 2 s, η·φ = 2 s) are introduced without a sensitivity study. Since the central robustness claim depends on these values, a small ablation or a discussion of how the results change with c and b would be helpful.","section":"Section V-B1"},{"comment":"The hardware results report only 10 trials per condition, and the simulation tables do not include confidence intervals or standard errors. Given that several comparisons (e.g., ABS-Agile in the slippery condition) involve small counts, reporting binomial confidence intervals would strengthen the empirical claims.","section":"Tables III, IV, VI, VII"},{"comment":"The caption says 'Right top-row' and 'Right bottom-row' but it is clearer to label the panels directly, since the left panel shows the LiDAR observation and the right panels show ground truth and prediction.","section":"Figure 3 caption"},{"comment":"The phrases 'We believe this is necessary' and 'We theorize that' are informal; they should be supported by a quantitative comparison or an explicit hypothesis test, otherwise the statements read as speculation.","section":"Section VII-B5"},{"comment":"The quantile definition in the proof of Theorem 1 uses N−k/N, while standard split conformal prediction often uses a slightly different finite-sample correction. The paper should either justify this choice or reconcile it with the cited reference [79].","section":"Appendix A"}],"recommendation":"major_revision","confidential_remarks":"The stress-test concern about Eq. (14) is real and is, in my view, the most important technical gap. The authors already acknowledge the underlying issue in Section VIII, but the manuscript's abstract and introduction frame the method as providing robust safety in a way that the current analysis does not fully support. With a re-framing of the guarantees, an explicit residual-risk analysis, and a tightening of the claims around Eq. (15)-(16), the paper could become a solid contribution. The experimental breadth is a genuine strength, and the candid limitations section is a point in the authors' favor."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Worth a serious look: this is a real step toward controller-agnostic safety filters for quadrupeds, and the experiments back it up. But the title is doing heavy lifting. The online disturbance bound in Eq. (14) is a heuristic location-scale summary, not a worst-case upper bound, so the formal reachability guarantee does not actually hold when true disturbances spike. The authors know this—Section VIII admits exceedance and latency—but it means the central claim should be softened from 'robust' to 'practically robust.'\n\nWhat is genuinely new: the observation-conditioned reachability value network, trained against ground-truth HJ value functions from randomly generated obstacle fields, queried with LiDAR and an online disturbance estimate. That combination is new, and the ablations (OCR \\ DE, OCR \\ C) show both ingredients earn their keep. The evaluation is substantial: three low-level policies, several high-level planners, sim and hardware on a Go1, plus a slippery-floor condition where the OCR filter clearly beats the ABS baseline. The paper is also unusually candid about its own failure modes.\n\nThe soft spots are real but proportionate. The estimator in Eq. (14) trims the middle 80% of a 2-second window and uses mean ± 2σ; it excludes 20% of observed disturbances by construction and has no coverage guarantee. The reactivity delay is also real. These are not peripheral implementation details—they are the load-bearing part of the 'robustness to unmodeled dynamics' claim. The nonzero collision rates in Tables III and IV are consistent with this gap. Second, the 'no a priori environment knowledge' claim is undercut by the cluttered-environment adaptations: cropping the LiDAR to a front-facing cone and dropping calibration are per-setting tuning, done because of distribution shift. That is honest but not 'deploy them all.'\n\nThe experiment reporting would be stronger with confidence intervals on the success/collision rates and with code/data released. The current numbers look good, but exact reproduction is impossible. None of this makes the paper a bad paper. As a practical safety layer, the OCR filter is clearly effective and the framework is well-engineered. It deserves peer review and likely acceptance after revisions that either tighten the estimator (or reframe the guarantee) and release artifacts. Serious referee needed, yes.","headline":"Real step forward with honest experiments, but the disturbance-bound heuristic undercuts the formal safety claim and the 'one filter' promise has per-setting caveats.","tokens_in":26065,"tokens_out":2776,"would_cite":true,"duration_ms":30782,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"One observation-conditioned safety filter can protect many quadruped controllers across unseen environments and dynamics.","keywords":["Hamilton-Jacobi reachability","safety filter","quadrupedal navigation","observation-conditioned value network","disturbance estimation","conformal prediction","LiDAR","safety-critical control"],"falsifier":"Run the filter on a quadruped that steps onto a low-friction patch that appears suddenly, such as an oil-soaked tarp placed just before a wall, and record whether the robot collides before the two-second disturbance window has registered the slip; if the robot enters the failure set while the estimated bound is still low, the central claim fails for that regime.","tokens_in":25033,"feed_emoji":"🤖","tokens_out":6996,"duration_ms":74417,"temperature":0.7,"pith_summary":"This paper tries to establish that safety can be made a transportable layer for legged navigation: one filter, trained once, can protect many different quadruped controllers in environments it has never seen. The claim is that a value network conditioned on a LiDAR scan and on an online estimate of dynamics uncertainty can reproduce the optimal reachability-based safety value function well enough to serve as a safety filter. If true, users would no longer need to re-derive safety certificates whenever they swap a locomotion policy or enter a new obstacle configuration. The paper supports the claim with simulation and hardware trials showing high success rates across learning-based and model-based controllers, including under slippery terrain and external pushes.","feed_headline":"One safety filter guards diverse quadruped controllers in new terrain","feed_subtitle":"A LiDAR-conditioned reachability network adapts to fresh obstacles and slippery ground without retraining per controller.","key_machinery":"The load-bearing object is the Observation-Conditioned Reachability Value Network (OCR-VN), a neural network that maps a reduced state $x_r=(p_x,p_y,p_\\theta)$, a 100-ray LiDAR observation $o^e$, and a disturbance bound $\\bar d^e_r$ to a predicted safety value $V_\\psi(x_r,\\bar d^e_r,o^e)$ and its spatial gradient. It is trained by supervised regression against ground-truth Hamilton-Jacobi reachability value functions on 1,000 randomly generated obstacle environments, using sinusoidal activations so that the gradients the filter needs are modeled accurately. The companion mechanism is the online disturbance estimator, which computes the difference between the actual state history and the disturbance-free reduced-order prediction over a sliding window of about two seconds and converts the spread of those residuals into position and heading disturbance bounds. A conformal-calibration shift $\\delta$ is subtracted from the network output so that overestimation of safety is controlled to a chosen violation rate. Together these pieces turn the classical backward reachable tube computation, normally too heavy for online use, into a query that can be answered at control rate.","core_discovery":"The central claim is that the optimal control-theoretic safety value function for a quadruped — the function whose sign tells whether the robot can avoid a failure set under worst-case disturbance — can be predicted at deployment time from onboard sensing and recent experience, rather than recomputed for each new controller and environment. The authors model the robot as a reduced-order 3D Dubins car with additive adversarial disturbances, compute ground-truth Hamilton-Jacobi reachability value functions over 1,000 randomly generated obstacle fields and disturbance bounds, and train the OCR-VN to output both the value and its spatial gradients from a LiDAR observation, the reduced state, and the estimated disturbance bound. The predicted value function is wrapped in an adaptive least-restrictive safety filter: while the predicted value exceeds a calibrated threshold the nominal controller runs unmodified, and once the value drops to the threshold the filter solves a quadratic program to minimally adjust the twist command so that the value function's derivative constraint is satisfied. The paper reports that this single filter safeguards a range of high-level planners and low-level policies without retraining, and outperforms a policy-specific safety-critic method, especially when ground friction and payloads vary.","pith_inferences":["The two-second sliding-window disturbance estimate implicitly assumes disturbances change slowly; a terrain-aware estimator that predicts friction or slip from exteroception before the robot enters a patch could close the adaptation lag the authors acknowledge.","The same conditioning recipe — a reduced-order model plus a network that maps observations and uncertainty bounds to a reachability value function — should transfer to other twist-command robots, such as wheeled bases, drones, or manipulators with a low-level tracker.","The filter's conservatism in cluttered, walled environments suggests a testable extension: instead of discarding LiDAR readings outside a front cone, one could train with wall-like obstacles or learn an observation-conditioned calibration level to reduce stalling.","Because the filter operates at the twist level and only needs a backup low-level policy, it could serve as a generic safety wrapper for end-to-end RL locomotion policies that do not natively separate planning from control."],"forward_implications":["A single trained OCR-VN can be attached in zero-shot fashion to a new hierarchical controller (high-level planner plus low-level locomotion policy) with no retraining or policy-specific tuning, provided the controller emits twist commands.","Because the filter only intervenes when the calibrated value function falls to the threshold, it preserves nominal agility and minimizes deviation; in cluttered settings it can be made less conservative by dropping calibration or limiting LiDAR to a front-facing cone.","Online disturbance estimation is what extends safety from nominal dynamics to real-world slips, payload changes, and tracking errors; without it, success rates drop sharply under hard dynamic conditions.","The hardware results on the quadruped show the same pattern as simulation: the filter raises success from zero for a naive goal-seeking controller to roughly nine out of ten trials, even on a slippery floor."],"supporting_citations":[{"why":"Supplies the policy-specific safety-critic baseline against which the proposed filter is compared on the same nominal policy.","marker":"[29]"},{"why":"Provides the Hamilton-Jacobi reachability safety-filter construction and the least-restrictive filtered-policy guarantee that the adaptive filter extends.","marker":"[68]"},{"why":"Computes the ground-truth value functions used as training targets for the OCR-VN.","marker":"[77]"},{"why":"Provides the split conformal prediction result that the calibration theorem applies to the value network.","marker":"[79]"},{"why":"Provides one of the learned low-level locomotion policies the filter is tested with.","marker":"[21]"},{"why":"Provides the model-based MPC low-level policy used as another nominal controller.","marker":"[74]"},{"why":"Supplies the LiDAR-based SLAM state estimation used during deployment for the reduced state.","marker":"[81]"},{"why":"Motivates sinusoidal activations so the network can model the spatial gradients needed by the filter.","marker":"[78]"}],"fun_headline_variants":["Single safety filter adapts to new obstacles and dynamic uncertainties","LiDAR-powered safety filter guards any quadruped controller","One reachability filter adapts to novel terrain and controllers","Adaptive safety filter for quadruped navigation without retraining","Universal safety filter for legged robots in unknown environments"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole guarantee rests on the assumption that the disturbance bound computed from the last two seconds of state-action history really upper-bounds the disturbance the robot is about to experience, and that the estimate updates faster than the environment changes.","fun_headline_variants_meta":{"raw":{"variants":["Single safety filter adapts to new obstacles and dynamic uncertainties","LiDAR-powered safety filter guards any quadruped controller","One reachability filter adapts to novel terrain and controllers","Adaptive safety filter for quadruped navigation without retraining","Universal safety filter for legged robots in unknown environments"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000448,"raw_usage":{"total_tokens":2297,"prompt_tokens":1021,"completion_tokens":1276,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":637,"completion_tokens_details":{"reasoning_tokens":1197}},"tokens_in":637,"tokens_out":1276,"duration_ms":10389,"temperature":1.0,"reasoning_tokens":1197,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T16:28:18.988306+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the filter on a quadruped that steps onto a low-friction patch that appears suddenly, such as an oil-soaked tarp placed just before a wall, and record whether the robot collides before the two-second disturbance window has registered the slip; if the robot enters the failure set while the estimated bound is still low, the central claim fails for that regime.","supporting_citations":[{"cited_title":"hj reachability: Hamilton-jacobi reachability analysis in jax","cited_arxiv_id":null,"evidence_quote":"Computes the ground-truth value functions used as training targets for the OCR-VN."},{"cited_title":"unitree ros","cited_arxiv_id":null,"evidence_quote":"Provides the model-based MPC low-level policy used as another nominal controller."},{"cited_title":"tinyslam: A slam algorithm in less than 200 lines c-language program,","cited_arxiv_id":null,"evidence_quote":"Supplies the LiDAR-based SLAM state estimation used during deployment for the reduced state."}],"review_version":1}