Safe Navigation using Neural Radiance Fields via Reachable Sets

Malarvizhi Sankaranarayanasamy; Omanshu Thapliyal; Ravigopal Vennelakanti

arxiv: 2604.26899 · v2 · submitted 2026-04-29 · 📡 eess.SY · cs.RO· cs.SY

Safe Navigation using Neural Radiance Fields via Reachable Sets

Omanshu Thapliyal , Malarvizhi Sankaranarayanasamy , Ravigopal Vennelakanti This is my paper

Pith reviewed 2026-05-07 10:43 UTC · model grok-4.3

classification 📡 eess.SY cs.ROcs.SY

keywords safe navigationneural radiance fieldsreachable setsconstrained optimal controlpath planningautonomous robotslinear matrix inequalities

0 comments

The pith

Reachable sets from robot dynamics combined with NeRF obstacle volumes turn path planning into a constrained optimal control problem that keeps trajectories safe.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper shows how to encode a robot's possible future positions as reachable sets in state space and to represent nearby obstacles or the robot itself as three-dimensional volumes extracted from neural radiance fields. These two representations are then turned into linear matrix inequality constraints inside a standard optimal control solver. Simulations in two cluttered scenarios demonstrate that the resulting trajectories remain collision-free. The approach matters because it gives a concrete way to enforce safety guarantees without relying on simplified bounding boxes or hand-crafted margins.

Core claim

Safe navigation is demonstrated through using reachable sets in the corresponding constrained optimal control problems, where neural radiance fields supply the volumetric obstacle models needed to build the linear-matrix-inequality constraints.

What carries the argument

Reachable-set representations of the robot's reachable states in a fixed time horizon, paired with NeRF-derived volumetric obstacle models, which together produce the linear matrix inequality constraints solved inside the optimal control problem.

Load-bearing premise

Reachable sets can be computed and enforced fast enough for real-time use while the NeRF volumes remain accurate enough that the resulting inequality constraints truly prevent collisions.

What would settle it

A real-time experiment in which the robot, following the computed trajectory, collides with an obstacle whose NeRF model was updated at the same rate as the reachable-set calculation.

Figures

Figures reproduced from arXiv: 2604.26899 by Malarvizhi Sankaranarayanasamy, Omanshu Thapliyal, Ravigopal Vennelakanti.

**Figure 1.** Figure 1: The polytope resulting from the intersection of all the view at source ↗

**Figure 1.** Figure 1: Polytopic Approximation of Reachable Sets: Evolution of a selected Hyperplane under dynamics view at source ↗

**Figure 2.** Figure 2: Reconstructing a 3D object using NeRFs at the given viewing direction for the object. In this case, the input data to the NeRF is a sequence of images across different camera angles Ii ∈ R 3×R 2 , where Ii = x, y, z, θ, ϕ represents the 3D point and viewing angle. The neural radiance output function F : R 3 ×R 2 → R 4 maps to RGB color and intensity. The core NeRF architecture is a feedforward neural netwo… view at source ↗

**Figure 5.** Figure 5: Trained NeRF object: 3D pointcloud view at source ↗

**Figure 6.** Figure 6: Convex hull of the trained NeRF object to describe robot geometry view at source ↗

**Figure 4.** Figure 4: Training data for the NeRF along different camera views view at source ↗

**Figure 7.** Figure 7: NeRF object represents robot geometry, and reachable sets to impose view at source ↗

**Figure 8.** Figure 8: NeRF objects represents goal set, and reachable sets to impose obstacle view at source ↗

read the original abstract

Safe navigation in cluttered environments is an important challenge for autonomous systems. Robots navigating through obstacle ridden scenarios need to be able to navigate safely in the presence of obstacles, goals, and ego objects of varying geometries. In this work, reachable set representations of the robot's real-time capabilities in the state space can be utilized to capture safe navigation requirements. While neural radiance fields (NeRFs) are utilized to compute, store, and manipulate the volumetric representations of the obstacles, or ego vehicle, as needed. Constrained optimal control is employed to represent the resulting path planning problem, involving linear matrix inequality constraints. We present simulation results for path planning in the presence of numerous obstacles in two different scenarios. Safe navigation is demonstrated through using reachable sets in the corresponding constrained optimal control problems.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

The paper shows a NeRF-plus-reachable-sets pipeline for safe simulated navigation but provides little evidence on how the volumetric models translate to valid safety constraints.

read the letter

The main point is that the authors integrate neural radiance fields to represent obstacles volumetrically with reachable sets enforced through linear matrix inequality constraints in an optimal control setup for path planning. They show simulation results for two scenarios with multiple obstacles and claim this achieves safe navigation. This approach has some merit in bringing together scene reconstruction and formal safety methods. NeRFs are good for capturing detailed 3D structure from images, which could help in environments where obstacles have irregular shapes. Pairing that with reachable sets gives a way to reason about the robot's possible states over time without needing to simulate every possible trajectory. The use of constrained optimal control to solve the planning problem is a standard but effective choice here. The weaker part is the missing explanation of how the NeRF data gets turned into the LMI constraints. NeRFs produce implicit representations, so steps like extracting surfaces or creating conservative bounds are probably needed. Any such step risks either making the feasible set too small or allowing paths that actually collide if the approximation isn't tight enough. The simulations might look fine, but without reported metrics on modeling error or comparisons showing the safety margin, it's hard to judge if the central claim holds up beyond the specific cases. This kind of work is for engineers and researchers focused on autonomous navigation systems that need both perception and control guarantees. A reader looking for ideas on using modern 3D models in safety-critical planning would find the examples useful. I would bring this to a reading group to talk about the implementation hurdles. I do not see myself citing it soon, given the application nature and limited validation shown. It should go to peer review so the full methods and any proofs can be evaluated properly.

Referee Report

2 major / 2 minor

Summary. The manuscript proposes a framework for safe robot navigation in cluttered environments that combines reachable-set representations of the system's real-time capabilities with neural radiance fields (NeRFs) for volumetric obstacle modeling. Path planning is cast as a constrained optimal control problem whose safety requirements are encoded via linear matrix inequality (LMI) constraints derived from the NeRF geometry; simulation results are presented for two scenarios with multiple obstacles, and the authors conclude that safe navigation is demonstrated.

Significance. If the conversion from NeRF density fields to LMI constraints can be shown to preserve reachable-set invariance without introducing unquantified modeling error, the work would offer a concrete bridge between modern implicit scene representations and control-theoretic safety certificates. The simulation-based demonstration, while preliminary, illustrates applicability to non-convex obstacle geometries that are difficult to handle with traditional convex approximations.

major comments (2)

[Constrained optimal control formulation] The description of the constrained optimal control problem (abstract and method outline) states that NeRF volumetric representations are converted into LMI constraints, yet supplies neither the explicit transformation (e.g., level-set extraction, conservative bounding, or convexification steps) nor any error bounds on the resulting feasible set. This conversion is load-bearing for the central safety claim; without it, one cannot verify that trajectories remain inside the true occupied volume rather than an inner or outer approximation.
[Simulation results] The simulation results section asserts that safe navigation is demonstrated in two scenarios but reports no quantitative validation metrics (minimum distance to obstacles, violation rates, computation times for reachable-set/LMI solves, or comparison against a baseline without NeRF). In the absence of these data, the claim that the LMI-enforced reachable sets remain valid under NeRF-induced modeling error cannot be assessed.

minor comments (2)

Notation for the reachable-set representation and the precise form of the LMI constraints should be introduced with explicit equations rather than descriptive prose only.
The abstract would benefit from a brief statement of the robot dynamics assumed and the specific NeRF implementation (e.g., density threshold or marching-cubes resolution) used to generate the obstacle geometry.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive and detailed feedback. The comments highlight important aspects of the safety claims and validation that require clarification and strengthening. We address each major comment below and commit to revisions that will improve the manuscript without misrepresenting the current work.

read point-by-point responses

Referee: [Constrained optimal control formulation] The description of the constrained optimal control problem (abstract and method outline) states that NeRF volumetric representations are converted into LMI constraints, yet supplies neither the explicit transformation (e.g., level-set extraction, conservative bounding, or convexification steps) nor any error bounds on the resulting feasible set. This conversion is load-bearing for the central safety claim; without it, one cannot verify that trajectories remain inside the true occupied volume rather than an inner or outer approximation.

Authors: We agree that the explicit mapping from NeRF density fields to LMI constraints is central to the safety argument and that the manuscript currently provides only a high-level outline. The full paper describes the use of reachable-set LMI constraints derived from NeRF geometry but does not detail the level-set extraction, bounding, or error analysis. In the revised manuscript we will add the precise transformation steps, including how conservative inner approximations are formed to preserve invariance, and any available bounds on the introduced modeling error. This addition will directly address the verifiability concern. revision: yes
Referee: [Simulation results] The simulation results section asserts that safe navigation is demonstrated in two scenarios but reports no quantitative validation metrics (minimum distance to obstacles, violation rates, computation times for reachable-set/LMI solves, or comparison against a baseline without NeRF). In the absence of these data, the claim that the LMI-enforced reachable sets remain valid under NeRF-induced modeling error cannot be assessed.

Authors: The referee is correct that the simulation section relies on qualitative demonstration without reporting quantitative metrics. The current manuscript shows trajectories in two multi-obstacle scenarios but does not include minimum distances, violation counts, timing data, or baseline comparisons. We will revise the results section to incorporate these metrics (minimum clearance, solve times, and a simple baseline comparison) while noting the preliminary, simulation-only nature of the study. This will allow better assessment of the LMI constraints under NeRF modeling error. revision: yes

Circularity Check

0 steps flagged

No circularity: method combines external concepts without self-referential reduction

full rationale

The paper presents a synthesis of reachable-set safety representations, NeRF-based volumetric obstacle modeling, and LMI-constrained optimal control for path planning. No derivation chain, equation, or claim reduces by construction to a quantity defined inside the paper itself. The abstract describes the approach as utilizing established external techniques (reachable sets for state-space safety, NeRFs for geometry, constrained optimal control for planning) and reports simulation results. No fitted parameters are renamed as predictions, no self-citations form load-bearing premises, and no ansatz or uniqueness result is smuggled in. The central demonstration therefore remains independent of internal tautologies or redefinitions.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

The central claim rests on two domain assumptions about the representational power of reachable sets and NeRFs; no free parameters or new entities are identified in the abstract.

axioms (2)

domain assumption Reachable set representations of the robot's real-time capabilities can capture safe navigation requirements
Explicitly invoked in the abstract as the basis for the constrained optimal control formulation.
domain assumption Neural radiance fields can be utilized to compute, store, and manipulate the volumetric representations of the obstacles or ego vehicle
Stated as the mechanism for handling geometry of varying obstacles and the robot itself.

pith-pipeline@v0.9.0 · 5445 in / 1415 out tokens · 89328 ms · 2026-05-07T10:43:57.522588+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

20 extracted references · 2 canonical work pages · 1 internal anchor

[1]

Nerf: Representing scenes as neural radiance fields for view synthesis,

B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoorthi, and R. Ng, “Nerf: Representing scenes as neural radiance fields for view synthesis,”Communications of the ACM, vol. 65, no. 1, pp. 99–106, 2021

2021
[2]

Plenoctrees for real-time rendering of neural radiance fields,

A. Yu, R. Li, M. Tancik, H. Li, R. Ng, and A. Kanazawa, “Plenoctrees for real-time rendering of neural radiance fields,” inProceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 5752–5761

2021
[3]

NeRF: Neural Radiance Field in 3D Vision: A Comprehensive Review (Updated Post-Gaussian Splatting)

K. Gao, Y . Gao, H. He, D. Lu, L. Xu, and J. Li, “Nerf: Neural radiance field in 3d vision, a comprehensive review,”arXiv preprint arXiv:2210.00379, 2022

work page internal anchor Pith review arXiv 2022
[4]

3d ultrasound spine imaging with application of neural radiance field method,

H. Li, H. Chen, W. Jing, Y . Li, and R. Zheng, “3d ultrasound spine imaging with application of neural radiance field method,” in2021 IEEE International Ultrasonics Symposium (IUS). IEEE, 2021, pp. 1–4

2021
[5]

Vision-only robot navigation in a neural radiance world,

M. Adamkiewicz, T. Chen, A. Caccavale, R. Gardner, P. Culbertson, J. Bohg, and M. Schwager, “Vision-only robot navigation in a neural radiance world,”IEEE Robotics and Automation Letters, vol. 7, no. 2, pp. 4606–4613, 2022

2022
[6]

Renderable neural radiance map for visual navigation,

O. Kwon, J. Park, and S. Oh, “Renderable neural radiance map for visual navigation,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 9099–9108

2023
[7]

Hamilton-jacobi reachability: A brief overview and recent advances,

S. Bansal, M. Chen, S. Herbert, and C. J. Tomlin, “Hamilton-jacobi reachability: A brief overview and recent advances,” in2017 IEEE 56th Annual Conference on Decision and Control (CDC). IEEE, 2017, pp. 2242–2253

2017
[8]

Hamilton–jacobi reachability: Some recent theoretical advances and applications in unmanned airspace manage- ment,

M. Chen and C. J. Tomlin, “Hamilton–jacobi reachability: Some recent theoretical advances and applications in unmanned airspace manage- ment,”Annual Review of Control, Robotics, and Autonomous Systems, vol. 1, pp. 333–358, 2018

2018
[9]

Reachability-based safety and goal satisfaction of unmanned aerial platoons on air highways,

M. Chen, Q. Hu, J. F. Fisac, K. Akametalu, C. Mackin, and C. J. Tomlin, “Reachability-based safety and goal satisfaction of unmanned aerial platoons on air highways,”Journal of Guidance, Control, and Dynamics, vol. 40, no. 6, pp. 1360–1373, 2017

2017
[10]

Approximate reachability for koopman systems using mixed monotonicity,

O. Thapliyal and I. Hwang, “Approximate reachability for koopman systems using mixed monotonicity,”IEEE Access, vol. 10, pp. 84 754– 84 760, 2022

2022
[11]

Approximating reachable sets for neural network-based models in real time via optimal control,

——, “Approximating reachable sets for neural network-based models in real time via optimal control,”IEEE transactions on control systems technology, vol. 31, no. 4, pp. 1901–1908, 2023

1901
[12]

Embed- ding safety requirements into learning-based controllers for urban air mobility applications,

O. Thapliyal, M. Sankaranarayanasamy, and R. Vennelakanti, “Embed- ding safety requirements into learning-based controllers for urban air mobility applications,” inAIAA SCITECH 2024 Forum, 2024, p. 2395

2024
[13]

Deepreach: A deep learning approach to high-dimensional reachability,

S. Bansal and C. J. Tomlin, “Deepreach: A deep learning approach to high-dimensional reachability,” in2021 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2021, pp. 1817–1824

2021
[14]

Reachability-based trajectory design with neural implicit safety constraints,

J. Michaux, Q. Chen, Y . Kwon, and R. Vasudevan, “Reachability-based trajectory design with neural implicit safety constraints,”arXiv preprint arXiv:2302.07352, 2023

work page arXiv 2023
[15]

Path planning for a network of robots with distributed multi-objective linear programming,

O. Thapliyal and I. Hwang, “Path planning for a network of robots with distributed multi-objective linear programming,” in2021 American Control Conference (ACC). IEEE, 2021, pp. 4643–4648

2021
[16]

Reach set computation using optimal control,

P. Varaiya, “Reach set computation using optimal control,” inVerification of Digital and Hybrid Systems. Springer, 2000, pp. 323–331. [17]Hybrid Motion Planning Using Minkowski Sums, 2009, pp. 97–104

2000
[17]

Fast collision checking for intelligent vehicle motion planning,

J. Ziegler and C. Stiller, “Fast collision checking for intelligent vehicle motion planning,” in2010 IEEE intelligent vehicles symposium. IEEE, 2010, pp. 518–522

2010
[18]

On translational motion planning in 3- space,

B. Aronov and M. Sharir, “On translational motion planning in 3- space,” inProceedings of the tenth annual symposium on Computational geometry, 1994, pp. 21–30

1994
[19]

Hybrid motion planning using minkowski sums,

J.-M. Lien, “Hybrid motion planning using minkowski sums,”Proceed- ings of robotics: science and systems IV, 2008

2008
[20]

Nerfstudio: A modular framework for neural radiance field development,

M. Tancik, E. Weber, E. Ng, R. Li, B. Yi, T. Wang, A. Kristoffersen, J. Austin, K. Salahi, A. Ahujaet al., “Nerfstudio: A modular framework for neural radiance field development,” inACM SIGGRAPH 2023 Conference Proceedings, 2023, pp. 1–12

2023

[1] [1]

Nerf: Representing scenes as neural radiance fields for view synthesis,

B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoorthi, and R. Ng, “Nerf: Representing scenes as neural radiance fields for view synthesis,”Communications of the ACM, vol. 65, no. 1, pp. 99–106, 2021

2021

[2] [2]

Plenoctrees for real-time rendering of neural radiance fields,

A. Yu, R. Li, M. Tancik, H. Li, R. Ng, and A. Kanazawa, “Plenoctrees for real-time rendering of neural radiance fields,” inProceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 5752–5761

2021

[3] [3]

NeRF: Neural Radiance Field in 3D Vision: A Comprehensive Review (Updated Post-Gaussian Splatting)

K. Gao, Y . Gao, H. He, D. Lu, L. Xu, and J. Li, “Nerf: Neural radiance field in 3d vision, a comprehensive review,”arXiv preprint arXiv:2210.00379, 2022

work page internal anchor Pith review arXiv 2022

[4] [4]

3d ultrasound spine imaging with application of neural radiance field method,

H. Li, H. Chen, W. Jing, Y . Li, and R. Zheng, “3d ultrasound spine imaging with application of neural radiance field method,” in2021 IEEE International Ultrasonics Symposium (IUS). IEEE, 2021, pp. 1–4

2021

[5] [5]

Vision-only robot navigation in a neural radiance world,

M. Adamkiewicz, T. Chen, A. Caccavale, R. Gardner, P. Culbertson, J. Bohg, and M. Schwager, “Vision-only robot navigation in a neural radiance world,”IEEE Robotics and Automation Letters, vol. 7, no. 2, pp. 4606–4613, 2022

2022

[6] [6]

Renderable neural radiance map for visual navigation,

O. Kwon, J. Park, and S. Oh, “Renderable neural radiance map for visual navigation,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 9099–9108

2023

[7] [7]

Hamilton-jacobi reachability: A brief overview and recent advances,

S. Bansal, M. Chen, S. Herbert, and C. J. Tomlin, “Hamilton-jacobi reachability: A brief overview and recent advances,” in2017 IEEE 56th Annual Conference on Decision and Control (CDC). IEEE, 2017, pp. 2242–2253

2017

[8] [8]

Hamilton–jacobi reachability: Some recent theoretical advances and applications in unmanned airspace manage- ment,

M. Chen and C. J. Tomlin, “Hamilton–jacobi reachability: Some recent theoretical advances and applications in unmanned airspace manage- ment,”Annual Review of Control, Robotics, and Autonomous Systems, vol. 1, pp. 333–358, 2018

2018

[9] [9]

Reachability-based safety and goal satisfaction of unmanned aerial platoons on air highways,

M. Chen, Q. Hu, J. F. Fisac, K. Akametalu, C. Mackin, and C. J. Tomlin, “Reachability-based safety and goal satisfaction of unmanned aerial platoons on air highways,”Journal of Guidance, Control, and Dynamics, vol. 40, no. 6, pp. 1360–1373, 2017

2017

[10] [10]

Approximate reachability for koopman systems using mixed monotonicity,

O. Thapliyal and I. Hwang, “Approximate reachability for koopman systems using mixed monotonicity,”IEEE Access, vol. 10, pp. 84 754– 84 760, 2022

2022

[11] [11]

Approximating reachable sets for neural network-based models in real time via optimal control,

——, “Approximating reachable sets for neural network-based models in real time via optimal control,”IEEE transactions on control systems technology, vol. 31, no. 4, pp. 1901–1908, 2023

1901

[12] [12]

Embed- ding safety requirements into learning-based controllers for urban air mobility applications,

O. Thapliyal, M. Sankaranarayanasamy, and R. Vennelakanti, “Embed- ding safety requirements into learning-based controllers for urban air mobility applications,” inAIAA SCITECH 2024 Forum, 2024, p. 2395

2024

[13] [13]

Deepreach: A deep learning approach to high-dimensional reachability,

S. Bansal and C. J. Tomlin, “Deepreach: A deep learning approach to high-dimensional reachability,” in2021 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2021, pp. 1817–1824

2021

[14] [14]

Reachability-based trajectory design with neural implicit safety constraints,

J. Michaux, Q. Chen, Y . Kwon, and R. Vasudevan, “Reachability-based trajectory design with neural implicit safety constraints,”arXiv preprint arXiv:2302.07352, 2023

work page arXiv 2023

[15] [15]

Path planning for a network of robots with distributed multi-objective linear programming,

O. Thapliyal and I. Hwang, “Path planning for a network of robots with distributed multi-objective linear programming,” in2021 American Control Conference (ACC). IEEE, 2021, pp. 4643–4648

2021

[16] [16]

Reach set computation using optimal control,

P. Varaiya, “Reach set computation using optimal control,” inVerification of Digital and Hybrid Systems. Springer, 2000, pp. 323–331. [17]Hybrid Motion Planning Using Minkowski Sums, 2009, pp. 97–104

2000

[17] [17]

Fast collision checking for intelligent vehicle motion planning,

J. Ziegler and C. Stiller, “Fast collision checking for intelligent vehicle motion planning,” in2010 IEEE intelligent vehicles symposium. IEEE, 2010, pp. 518–522

2010

[18] [18]

On translational motion planning in 3- space,

B. Aronov and M. Sharir, “On translational motion planning in 3- space,” inProceedings of the tenth annual symposium on Computational geometry, 1994, pp. 21–30

1994

[19] [19]

Hybrid motion planning using minkowski sums,

J.-M. Lien, “Hybrid motion planning using minkowski sums,”Proceed- ings of robotics: science and systems IV, 2008

2008

[20] [20]

Nerfstudio: A modular framework for neural radiance field development,

M. Tancik, E. Weber, E. Ng, R. Li, B. Yi, T. Wang, A. Kristoffersen, J. Austin, K. Salahi, A. Ahujaet al., “Nerfstudio: A modular framework for neural radiance field development,” inACM SIGGRAPH 2023 Conference Proceedings, 2023, pp. 1–12

2023