{"id":"6dce1144-50e5-479b-89f5-2194874b07d6","arxiv_id":"2505.19214","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":11,"one_line_summary":"Omni-Perception is an end-to-end RL policy for legged robots that processes raw LiDAR point clouds with PD-RiskNet to achieve omnidirectional collision avoidance, validated in simulation and on a Unitree G1.","lead":"Omni-Perception trains a legged robot to avoid moving, hanging, and ground obstacles by feeding raw LiDAR point clouds directly into a reinforcement learning policy. It also contributes a fast GPU LiDAR simulator for training and reports real-robot trials showing better avoidance than the Unitree native system.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The zero-shot sim-to-real claim rests on one qualitative visual comparison (Fig. 4), while the paper's own distal-average-downsampling limitation predicts the small-obstacle deficit (83% vs 100%) seen in Table 4.","rationale":"The central claim has three load-bearing dependencies: novelty, the end-to-end architecture, and sim-to-real transfer. Novelty cannot be settled from the manuscript alone. The architecture is internally consistent, and the Table 3 ablation shows PD-RiskNet outperforming simpler point-cloud encoders under the same reward, so the strongest residual threat is the transfer claim. The reader flagged the same dependency. I make the concern sharper: the fidelity validation is purely qualitative, and the paper's own limitation anticipates the small-obstacle degradation. A paired real/sim sensor measurement plus a closed-loop success comparison would either confirm that the perceptual gap is small enough for zero-shot transfer or expose the point at which the policy's advantage disappears. This does not warrant rejection; it is precisely the condition for accepting the high-fidelity simulator and zero-shot transfer contributions, so the conditional verdict remains appropriate.","tokens_in":13085,"tokens_out":14623,"duration_ms":105030,"concrete_test":"Record paired real and simulated Livox Mid-360 scans from identical robot poses in the same physical scene, then compute per-ray range error and point-cloud recall (e.g., Chamfer distance); additionally run the trained policy in both simulation and reality on a fixed, randomized 100-trial obstacle suite. If the mean range error exceeds roughly 5-10 cm, or if the real-vs-sim success gap exceeds roughly 10 percentage points, the zero-shot transfer and high-fidelity simulator claims should be downgraded to conditional.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4.1 validates the LiDAR simulator only via the side-by-side visual in Fig. 4. No quantitative metric compares per-ray range error, point-cloud recall, scan-pattern coverage, or self-occlusion between the simulated Livox Mid-360 and the physical sensor. This matters because the policy does not consume raw scans directly: Section 3.3.1 applies FPS for proximal points, average downsampling for distal points, and deterministic spherical sorting before the GRUs. Any bias in the simulated scan is therefore propagated into the policy observation. The paper's own Limitations section and Fig. 8 state that averaging the distant point cloud destroys the features of very small or thin objects, and Table 4 shows exactly that class underperforming: 83% success on small obstacles versus 100% for the non-perceptive native Unitree system. The central claim of a high-fidelity LiDAR simulator enabling effective zero-shot transfer thus depends on an unmeasured perceptual gap, while the method's own design predicts failure on the obstacle class where the real-robot numbers deteriorate. Without a quantitative fidelity metric or a closed-loop sim-vs-real comparison, the reported 70%/90% successes cannot be cleanly attributed to a faithful sensor model rather than to scenario-specific artifacts.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Omni-Perception, an end-to-end reinforcement learning framework for legged locomotion that consumes raw spatio-temporal LiDAR point clouds through a new perception module, PD-RiskNet, and outputs joint position targets. The method partitions each scan into proximal and distal clouds, processes them with FPS/GRU and average-downsampling/GRU pathways, and trains with a reward that combines velocity tracking with a hand-designed sector-based avoidance velocity and a LiDAR distance-maximization term. The authors also contribute a custom GPU-accelerated LiDAR simulator and report a simulation ablation plus real-robot trials across static, aerial, small, and moving-human obstacle scenarios.","tokens_in":13451,"tokens_out":4838,"duration_ms":40215,"significance":"If the sim-to-real transfer is robust, the paper would be a meaningful step toward using direct 3D LiDAR point clouds in learned legged locomotion, avoiding intermediate elevation maps and depth-camera limitations. The custom LiDAR simulation toolkit with fast parallel raycasting and cross-platform support is a practical contribution, as is the public code repository. The real-robot demonstrations show that a raw-point-cloud policy can produce reactive avoidance behaviors that the native Unitree system lacks. However, the quantitative evidence for the central zero-shot transfer claim is currently thinner than the narrative suggests: the LiDAR fidelity evaluation is qualitative, the real-world results are point estimates from 30 trials without statistical support, and the paper's own limitations predict the observed small-obstacle deficit. The significance is therefore real but conditional on stronger validation.","major_comments":[{"comment":"The LiDAR simulator fidelity claim, which underpins the zero-shot sim-to-real transfer result, rests on a single qualitative side-by-side comparison in Figure 4. No quantitative metric is reported for per-ray range error, point-cloud recall, scan-pattern coverage, or self-occlusion between the physical Livox Mid-360 and the simulator. This is load-bearing because the policy does not consume raw scans directly: Section 3.3.1 applies FPS, average downsampling, and spherical sorting before the GRUs, so any bias in the simulated scan propagates into the policy observation. The paper's own Limitations section and Figure 8 state that averaging the distant point cloud destroys the features of very small or thin objects, and Table 4 shows exactly that class underperforming (83% vs. 100% for the non-perceptive Unitree system). Please add a quantitative sim-vs-real fidelity comparison or a closed-loop sim-vs-real benchmark, and discuss how the distal downsampling interacts with the fidelity claim.","section":"4.1"},{"comment":"The real-world success rates in Table 4 are point estimates from only 30 trials per scenario, with no confidence intervals, no statistical significance tests, and no description of the trial protocol (obstacle dimensions, approach speeds, initial configurations, or success criteria). For example, 21/30 successes for aerial obstacles has an exact 95% confidence interval of roughly 51% to 85%, which is much wider than the 70% point estimate suggests. Please report confidence intervals or additional repetitions, and specify the trial protocol so the reader can assess whether the differences are robust and reproducible.","section":"4.4"},{"comment":"In Table 3, the reported success rates and collision rates are not mutually exclusive and sum to more than 100% for several rows (e.g., FPS+MLP: 33.3% + 93.3% = 126.6%; Ours: 76.7% + 56.7% = 133.4%). If the collision rate is the fraction of episodes with at least one collision, this must be stated explicitly; if a trial can both succeed and collide, the current presentation makes it difficult to interpret the ablation's advantage. Please clarify the definition and, ideally, report a single outcome per trial (success, collision, timeout) or a confusion-style breakdown.","section":"4.3"},{"comment":"Several parameters needed to reproduce PD-RiskNet and the reward are missing. In particular, the vertical angle threshold θ in Section 3.3.1, the FPS sample count, the distal downsampling factor, the distance cap d_max, the avoidance coefficient α_avoid, the velocity-tracking weight β_va, and the privileged-height supervision loss or weight are not specified. Nhist=10 appears only in Appendix B, and dthresh=1m and Nsec=36 appear in the main text. Please provide a complete hyperparameter table or list all numeric values in one place, otherwise the core architecture cannot be reimplemented from the paper alone.","section":"3.3 / Appendix A-B"}],"minor_comments":[{"comment":"In the contribution list on page 2, item 2 is incomplete: '2. Novel LiDAR Perception Network (PD-RiskNet):' is followed by no description. Please complete the sentence.","section":"1"},{"comment":"The caption contains a typo, 'Omni-Perceptio', which should read 'Omni-Perception'.","section":"Figure 1 caption"},{"comment":"The sentence 'You can find more lidar model scan patterns in the Appendix.??' contains a stray '??' placeholder and should be completed or removed.","section":"4.1"},{"comment":"Please clarify which robot platform(s) were used in the real-world experiments: Figure 2 shows the Unitree Go2, while Figure 4 shows the Unitree G1. The text should state the exact hardware used for each reported trial.","section":"4.4"},{"comment":"The privileged-height supervision is mentioned as a training signal, but no loss function, prediction target, or loss weight is described. Please specify how this supervision is applied.","section":"3.3.1 / B.1"},{"comment":"The reward function already contains a hand-designed LiDAR-based avoidance velocity, while the paper claims that the policy learns directly from raw point clouds. This is not circular, but the relationship should be stated explicitly: the observation is raw, yet the reward injects a LiDAR-derived prior, so the learned perception is shaped by that prior. Please add a clarifying sentence.","section":"3.3.2"}],"recommendation":"major_revision","confidential_remarks":"The central idea is plausible and the real-robot demonstrations are promising, but the evidence for the zero-shot sim-to-real claim is weaker than the paper's framing suggests. The most important fixes are quantitative LiDAR fidelity metrics and statistical support for the real-world success rates. I do not see an internal inconsistency that would require rejection; the issues are fixable within the manuscript's scope. One additional point for the editor: the 'first framework' claim should be checked carefully against prior raycasting-based end-to-end locomotion works, since the distinction between 'raw point clouds' and 'ray distances' is the main novelty and must be crisply drawn."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nOmni-Perception makes a plausible first claim: an end-to-end legged locomotion policy trained on raw LiDAR point clouds, with real-world avoidance of aerial and human obstacles where the native Unitree controller fails. The central idea is sound and the paper is worth a serious referee, but the evidence for the sim-to-real transfer story is thinner than the abstract suggests.\n\nWhat's genuinely good: raw LiDAR is a sensible exteroception choice for 360-degree awareness, and the PD-RiskNet proximal-distal split is a reasonable way to handle near-field density and far-field sparsity. The custom LiDAR simulator looks like a real engineering contribution—fast, cross-platform, supports non-repetitive scan patterns and self-occlusion, and the ablation (Table 3) shows the full architecture clearly beats FPS+MLP and FPS+GRU baselines. The real-robot numbers, while small, are encouraging: 70% on aerial obstacles and 90% on moving humans versus 0% for the native system. The limitations section is candid, which earns trust.\n\nSoft spots, in proportion. The evaluation is statistically thin: 30 trials per scenario, no error bars, no significance tests, no comparison to a learned depth-camera policy. The sim-to-real fidelity claim rests on one qualitative side-by-side (Fig. 4); there's no per-ray range error or point-cloud recall metric. That matters because the policy doesn't consume raw scans—it downsamples and sorts, so any simulator bias propagates. The paper's own limitation about averaging distal points destroying small-object features predicts exactly the small-obstacle deficit in Table 4 (83% vs 100%). So the zero-shot transfer claim is plausible but not nailed down. There's also a mild circularity: the avoidance rewards are computed from the same LiDAR ray distances that define the evaluation, so the policy is being rewarded for maximizing the metric it's scored on. That doesn't make the result fake, but it means the success rates are partly trading on the reward design.\n\nBottom line: this is a useful contribution with a reusable simulator and a first demonstration. A reader working on legged perception or sim-to-real transfer will get something out of it. It deserves peer review, but a referee should push for proper statistics and at least one quantitative sim-to-real fidelity check before the zero-shot claim is accepted.","headline":"First raw-LiDAR end-to-end legged locomotion policy with real-world promise, but sim-to-real fidelity is asserted more than measured.","tokens_in":14006,"tokens_out":2621,"would_cite":true,"duration_ms":24396,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Omni-Perception claims the first end-to-end legged locomotion policy trained directly on raw LiDAR point clouds, achieving omnidirectional collision avoidance in dynamic 3D scenes.","keywords":["Legged Robot","Locomotion","Reinforcement Learning","LiDAR Perception","Collision Avoidance","Sim-to-Real","Point Cloud","PD-RiskNet"],"falsifier":"Measure the perceptual gap by placing the same static scene in front of the real LiDAR and the simulator, computing the Chamfer distance between the two point clouds, then running the policy on a thin rod whose diameter is below the averaging kernel. If the Chamfer distance is large but the policy still transfers, or if the policy collides with thin rods that were clearly visible in simulation, the high-fidelity-transfer claim is weakened.","tokens_in":12900,"feed_emoji":"🤖","tokens_out":5514,"duration_ms":37544,"temperature":0.7,"pith_summary":"The paper sets out to show that a quadruped can navigate dynamic, three-dimensional surroundings using only raw spatio-temporal LiDAR point clouds, learned end-to-end by reinforcement learning, without elevation maps or decoupled planning. It introduces PD-RiskNet, a hierarchical network that splits each scan into near and far regions, processes each with a recurrent unit, and feeds the result into a locomotion policy. To train this policy, the authors build a high-fidelity LiDAR simulator with realistic noise and non-repetitive scan patterns. On a real quadruped, the resulting policy avoids aerial obstacles and moving humans at success rates of 70 and 90 percent respectively, where the native system scores zero.","feed_headline":"LiDAR-trained quadruped dodges aerial and human obstacles","feed_subtitle":"An end-to-end policy skips elevation maps and hits 70-90 percent real-robot avoidance, versus zero for the native system.","key_machinery":"The load-bearing component is PD-RiskNet, the Proximal-Distal Risk-Aware Hierarchical Network, a perception module that partitions each raw LiDAR frame into a proximal set of dense near-field points and a distal set of sparse far-field points. Each set is downsampled, ordered by spherical coordinates, and processed by its own gated recurrent unit, with the proximal unit receiving privileged height supervision during training. The resulting embeddings are concatenated with proprioceptive history and the velocity command, then passed to an MLP actor. A second mechanism is the sector-based avoidance velocity: the horizontal plane is divided into 36 angular sectors, each sector contributes a repulsive velocity whose magnitude decays exponentially with the distance of the nearest point in that sector, and the reward encourages tracking the command velocity plus this avoidance term.","core_discovery":"The central claim is that end-to-end policies can consume raw LiDAR point clouds directly and achieve omnidirectional collision avoidance in dynamic 3D environments, something depth-camera pipelines and map-based methods do not deliver. The discovery is that a proximal-distal hierarchical processing scheme makes this tractable: dense near-field points are downsampled by farthest-point sampling, sparse far-field points by averaging, and each stream is fed to a separate gated recurrent unit trained with privileged height supervision. Combined with a sector-based avoidance velocity in the reward, this yields a reactive policy that tracks velocity commands while pushing away from nearby obstacles in all directions. The paper reports real-robot success rates of 100 percent on static obstacles, 70 percent on aerial obstacles, 83 percent on small obstacles, and 90 percent on moving humans, compared to zero for the native system on aerial and human obstacles.","pith_inferences":["The proximal-distal split is a generic way to handle non-uniform point density; it could be applied to other irregular point-cloud streams such as UAV navigation or manipulation where near-field detail matters more than far-field density.","A quantitative benchmark of LiDAR simulator fidelity, such as Chamfer distance or occupancy IoU between simulated and real scans, would let the community separate perception gap from control policy quality; the current paper only offers a qualitative comparison.","Extending the paper's failure analysis, adding semantic segmentation could address the dense-grass failure case, since geometric features alone misclassify vegetation as danger.","The avoidance-velocity reward is a reactive heuristic; a learned or predictive variant might improve behavior in corridors where the robot is forced to walk close to obstacles."],"forward_implications":["If the claim holds, intermediate representations like elevation maps become optional for reactive legged navigation, since raw LiDAR provides lighting-invariant 3D awareness at control frequency.","The 70 percent aerial and 90 percent moving-human success rates imply that end-to-end LiDAR policies can handle threats that proprioception-only systems miss entirely.","The LiDAR simulator's speed, more than an order of magnitude faster than one commercial baseline on static scenes, makes large-scale parallel training of perception-locomotion policies practical.","The same policy transfers to a real robot without additional adaptation, indicating that zero-shot sim-to-real transfer is feasible for point-cloud-based legged locomotion."],"supporting_citations":[{"why":"Supplies the base reward structure that Omni-Perception adapts for velocity tracking and auxiliary terms.","marker":"[13]"},{"why":"Provides the physical domain randomization parameters used to improve sim-to-real transfer.","marker":"[7]"},{"why":"Inspires the GPU-accelerated, parallel LiDAR rendering approach for the custom simulator.","marker":"[58]"},{"why":"Supplies farthest point sampling, used in the proximal pathway to downsample dense near-field points.","marker":"[62]"},{"why":"Provides the PPO algorithm used to train the end-to-end policy.","marker":"[63]"},{"why":"Serves as the massively parallel training simulator platform on which the policy is learned.","marker":"[55]"}],"fun_headline_variants":["Quadruped learns to dodge directly from LiDAR point clouds","Raw LiDAR only: quadruped avoids obstacles in all directions","LiDAR-to-action: end-to-end policy beats map-based navigation","Quadruped dodges aerial and human obstacles with LiDAR only"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole sim-to-real transfer rests on the assumption that the simulated LiDAR scans faithfully reproduce the real sensor's noise, scan pattern, and self-occlusion; the paper supports this with a single qualitative side-by-side comparison and domain randomization, but offers no quantitative error metrics.","fun_headline_variants_meta":{"raw":{"variants":["Quadruped learns to dodge directly from LiDAR point clouds","Raw LiDAR only: quadruped avoids obstacles in all directions","LiDAR-to-action: end-to-end policy beats map-based navigation","Quadruped dodges aerial and human obstacles with LiDAR only"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000779,"raw_usage":{"total_tokens":3462,"prompt_tokens":982,"completion_tokens":2480,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":598,"completion_tokens_details":{"reasoning_tokens":2406}},"tokens_in":598,"tokens_out":2480,"duration_ms":15962,"temperature":1.0,"reasoning_tokens":2406,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T14:17:54.716094+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure the perceptual gap by placing the same static scene in front of the real LiDAR and the simulator, computing the Chamfer distance between the two point clouds, then running the policy on a thin rod whose diameter is below the averaging kernel. If the Chamfer distance is large but the policy still transfers, or if the policy collides with thin rods that were clearly visible in simulation, the high-fidelity-transfer claim is weakened.","supporting_citations":[{"cited_title":"Cheng, K","cited_arxiv_id":null,"evidence_quote":"Supplies the base reward structure that Omni-Perception adapts for velocity tracking and auxiliary terms."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the physical domain randomization parameters used to improve sim-to-real transfer."},{"cited_title":"Kulkarni, W","cited_arxiv_id":null,"evidence_quote":"Inspires the GPU-accelerated, parallel LiDAR rendering approach for the custom simulator."}],"review_version":1}