REVIEW 4 major objections 5 minor 18 references
The paper presents Neurosim, a simulator that turns rendered intensity frames into event-camera streams at thousands of frames per second, and claims this removes the data bottleneck for neuromorphic robot perception.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
Neurosim simulates event cameras and other robot sensors at multi-kilohertz rates on a GPU and streams the data to training and control pipelines without touching disk.
T0 review reviewed 2026-08-02 challenge →
load-bearing objection A genuinely fast event-camera simulator with a clean streaming architecture, but the sim-to-real motivation is untested and the benchmarks need reproducibility artifacts. the 4 major comments →
Neurosim: A Fast Simulator for Neuromorphic Robot Perception
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
Core claim
The central claim is that a warp-synchronous CUDA event-generation kernel removes the event-simulation bottleneck: it processes intensity frames at over 31 kHz for VGA and 23 kHz for HD, so the full simulation stack—rendering, dynamics, and event generation—runs at about 2,700 FPS on a desktop GPU. This is roughly 8–13× faster than other GPU event simulators and 55–121× faster than CPU implementations. Together with Cortex, this sustains event streams that train a monocular depth network entirely online, and it supports closed-loop control at controller rates up to 5 kHz with sub-millisecond latency.
What carries the argument
The key mechanism is the warp-synchronous event-generation kernel. For each pixel, it holds the intensity at the last triggered event and compares the current rendered intensity against a contrast threshold, producing an event when the log-intensity change exceeds the threshold. Threads in a warp cooperate to build a bitmask of active pixels, count events, and perform a single atomic operation to reserve a contiguous block in the output buffer; each thread then writes its event into that block. This reduces global atomic contention from one per event to roughly one per warp, letting event generation run in a single kernel launch and occupy only about 8% of a simulation step.
Load-bearing premise
The load-bearing premise is that event streams generated by Neurosim from rendered intensity frames are faithful enough to real event-camera output that perception and control algorithms trained on them will transfer to physical hardware; the paper asserts this motivationally (Section 1) but offers no sim-to-real evaluation.
What would settle it
Train a monocular depth network on Neurosim events, then run the same network on a real event camera observing the same scene geometry; a large drop in depth accuracy compared with the simulated test set would refute the sim-to-real transfer claim. A simpler check is comparing event-rate statistics and background noise between Neurosim output and a real DVS sensor under identical motion.
If this is right
- Perception and control algorithms for event cameras can be trained on simulated data faster than real time, without multi-terabyte datasets or disk I/O.
- Closed-loop experiments can be run at the edge of a quadrotor's performance envelope—flips, fast obstacle avoidance—without risking hardware.
- Because event generation costs only about 8% of a simulation step, the renderer becomes the bottleneck, so future rendering speedups translate directly to higher simulation rates.
- Online training on streaming simulated data can saturate a second GPU's ingestion, making simulation a scalable replacement for static dataset curation.
- Cortex's low latency (<0.7 ms) makes it possible to run the controller on separate hardware (e.g., an embedded board) while the simulator runs on a desktop GPU.
Where Pith is reading between the lines
- The absence of sim-to-real validation is the key open question; the natural next test is to deploy a Neurosim-trained depth or control policy on a physical event camera in the same environment and measure the performance gap.
- The warp-synchronous event-generation pattern could be reused for other sparse, asynchronous sensor streams, such as LiDAR point clouds or spike-based neural sensors, where output is sparse and variable-size.
- Because the simulator stresses temporal resolution over photorealism, domain randomization over scenes and trajectories may be more important than photometric realism for closing the sim-to-real gap; the paper's noise-model list in the discussion points in that direction.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces Neurosim, a GPU-accelerated modular simulator for neuromorphic robot perception, together with Cortex, a ZeroMQ-based communication middleware. Neurosim integrates Habitat-Sim for rendering, RotorPy for multirotor dynamics, and a custom CUDA event-generation kernel. The authors report that the full stack runs at ~2700 FPS on an NVIDIA 4090 and that the event backend processes intensity frames at >31 kHz (VGA) and >23 kHz (HD), substantially outperforming ESIM, VID2E, and AirSim. They also present Cortex benchmarks showing low-latency, high-throughput messaging, a closed-loop control latency analysis, and an online training demonstration in which an F3-based monocular depth network is trained on-the-fly from Neurosim data. The stated motivation is that simulated event data can reduce the sim-to-real gap, but the paper provides no validation of event realism or hardware transfer.
Significance. If the performance claims hold, Neurosim and Cortex could be valuable infrastructure for high-throughput simulation of event cameras and multi-modal sensors, enabling online training and closed-loop evaluation that would otherwise require terabytes of disk storage. The warp-synchronous CUDA event-generation design is a concrete engineering contribution, and the open-source release is a practical benefit to the community. However, the paper's central value proposition — that simulated event streams are faithful enough for sim-to-real transfer — is not demonstrated. The benchmark numbers are plausible but lack statistical rigor and reproducibility artifacts, and the only learned model is evaluated qualitatively on simulated data. The strength of the paper is therefore in its reported speed and architecture, not yet in its claimed realism.
major comments (4)
- [Sec. 2 vs Sec. 5] There is an internal contradiction about sensor non-idealities. Sec. 2 states that 'Real-world event sensors exhibit non-idealities such as noise, bandwidth limitations, and sensor saturation effects. Neurosim simulates these artifacts,' yet Sec. 5 lists as future work 'adding sensory noise models (shot/thermal noise, bandwidth saturation, refractory effects, rolling shutter).' This makes it unclear which non-idealities are actually implemented. Since realism is a core motivation, please resolve this discrepancy, specify the current noise/fidelity model precisely, and validate it.
- [Sec. 1, Desideratum 1; Sec. 4] The paper's motivating claim is that 'the sim-to-real gap for systems trained on simulated event data is smaller.' No experiment in Sec. 4 tests this. The only learning demonstration trains and evaluates entirely on simulated data, showing a single qualitative depth map (Fig. 6B(v)), and the Discussion explicitly defers hardware-in-the-loop and sensory noise models to future work. This is a load-bearing gap: the tool's primary use case is to train algorithms that transfer to physical robots. The authors should either add a quantitative comparison of Neurosim's event statistics (e.g., event rate, temporal distribution, noise) against real event-camera data (e.g., M3ED) or a sim-to-real transfer study, or explicitly rescope the paper's claims to simulation infrastructure without the sim-to-real promise.
- [Sec. 2, Fig. 3, Fig. 4] The central speed and throughput claims are reported as single-point averages with no error bars, number of trials, or variance information. The ~2700 FPS full-stack figure, the 31/23 kHz event-backend rates, and the Cortex throughput plots in Fig. 4 would be much more convincing with mean±std over repeated runs. Additionally, the manuscript does not provide a commit hash, container image, or pinned software versions (Habitat-Sim, CUDA, PyTorch, etc.), so the numbers are not independently verifiable. Please supply these reproducibility artifacts and statistical details.
- [Sec. 4, Fig. 6B] The online training demonstration lacks any quantitative evaluation. The paper claims the F3-based depth network was trained online and that the pipeline saturates the training GPU, but no training curves, validation losses, or standard depth metrics (e.g., RMSE, abs rel, delta thresholds) are reported. Without these, the reader cannot assess whether the training actually produced a useful perception model. Please include quantitative results on a held-out set of simulated trajectories and scenes.
minor comments (5)
- [Sec. 2] The list of design principles is misnumbered: it reads '(1) modularity ...; (2) low-latency ...; (2) real-time streaming ...; (4) Pythonic accessibility.' The second '(2)' should be '(3)'.
- [Eqs. (1)-(2)] The regularizer is denoted R(˜d, ˜d*) in Eq. (1) but R(d,d*) in Eq. (2), and the subscripted d_sigma is not explicitly defined as a downsampled version of d. Please unify notation.
- [Sec. 2, Fig. 3] The paper states 'ESIM Backends (left to right in each plot)' but the bars are not individually labeled in the text; please clarify which bar corresponds to each backend and whether the CUDA baseline is ESIM or another implementation.
- [References] The statement in Sec. 1 that 'Works like [2] have obtained remarkable results by training entirely on simulated event datasets' may not be an accurate characterization of Klenk et al.'s Deep Event Visual Odometry. Please verify the citation and adjust the claim.
- [Sec. 4, Fig. 6A] The closed-loop latency analysis uses a VGA RGB camera, not an event camera. Since event streams have variable-length, asynchronous payloads, the reported <0.7 ms latency and ~2.3 kHz sensor rate may not directly apply to the event-based setting that motivates the paper. Consider adding a similar analysis with event streams or stating the limitation explicitly.
Circularity Check
No significant circularity: speed and throughput claims are externally benchmarked; self-cited F3 is not load-bearing.
full rationale
The paper's central performance claims are measured against external baselines: Fig. 3 benchmarks Neurosim's CUDA event backend against ESIM [4], AirSim [6], and a PyTorch GPU implementation, and the ~2700 FPS end-to-end figure is tied to Habitat-Sim's independently reported rendering rate. These benchmarks do not depend on the authors' own F3 model. The only self-references are (i) using the authors' F3 representation in the Sec. 4 monocular-depth demonstration and (ii) citing Das et al. [10] for event-rate context; neither feeds back into the simulator's speed, event-generation, or throughput measurements. The paper's motivating claim that simulated event data transfers to real hardware (Sec. 1, Desideratum 1: 'the sim-to-real gap for systems trained on simulated event data is smaller') is not validated, and Sec. 5 explicitly defers sensory-noise models and hardware-in-the-loop experiments to future work. That is an unverified assumption and a correctness risk, not a circular derivation. No equation defines a predicted quantity in terms of a fitted input, and no uniqueness theorem or ansatz is imported from the authors' prior work. Consequently, no circular step can be exhibited under the quoted-evidence standard.
Axiom & Free-Parameter Ledger
axioms (3)
- domain assumption ESIM-style contrast-threshold event model is an adequate model of real event-camera behavior for the purpose of training downstream perception and control algorithms.
- domain assumption Rendering intensity images at multi-kilohertz rates with Habitat-Sim captures the photometric dynamics needed for event generation; missed-event probability is negligible at 2700 FPS.
- domain assumption The RotorPy dynamics model faithfully represents agile multirotor flight up to the performance envelope used in the demonstrations.
Cite this review
Pith. "Pith review of Neurosim: A Fast Simulator for Neuromorphic Robot Perception." pith.science (2026). https://pith.science/paper/EUELXZW2
@misc{pith2026260215018,
author = {Pith},
title = {Pith review of: Neurosim: A Fast Simulator for Neuromorphic Robot Perception},
year = {2026},
howpublished = {\url{https://pith.science/paper/EUELXZW2}},
note = {Machine review of arXiv:2602.15018}
}
read the original abstract
Neurosim is a fast, real-time, high-performance library for simulating sensors such as dynamic vision sensors, RGB cameras, depth sensors, and inertial sensors. It can also simulate agile dynamics of multi-rotor vehicles in complex and dynamic environments. Neurosim can achieve frame rates as high as ~2700 FPS on a desktop GPU. Neurosim integrates with a ZeroMQ-based communication library called Cortex to facilitate seamless integration with machine learning and robotics workflows. Cortex provides a high-throughput, low-latency message-passing system for Python and C++ applications, with native support for NumPy arrays and PyTorch tensors. This paper discusses the design philosophy behind Neurosim and Cortex. It demonstrates how they can be used to (i) train neuromorphic perception and control algorithms, e.g., using self-supervised learning on time-synchronized multi-modal data, and (ii) test real-time implementations of these algorithms in closed-loop. Neurosim and Cortex are available at https://github.com/grasp-lyrl/neurosim .
Figures
Reference graph
Works this paper leans on
-
[1]
Ani Hsieh, Christopher Korpela, Vijay Kumar, Camillo J
Kenneth Chaney, Fernando Cladera, Ziyun Wang, Anthony Bisulco, M. Ani Hsieh, Christopher Korpela, Vijay Kumar, Camillo J. Taylor, and Kostas Daniilidis. M3ed: Multi-robot, multi-sensor, multi- environment event dataset. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pages 4015–4022, June 2023
2023
-
[2]
Deep event visual odometry
Simon Klenk, Marvin Motzet, Lukas Koestler, and Daniel Cremers. Deep event visual odometry. In International Conference on 3D Vision, 3DV 2024, Davos, Switzerland, March 18-21, 2024, pages 739–749. IEEE, 2024
2024
-
[3]
Tartanair: A dataset to push the limits of visual slam
Wenshan Wang, Delong Zhu, Xiangwei Wang, Yaoyu Hu, Yuheng Qiu, Chen Wang, Yafei Hu, Ashish Kapoor, and Sebastian Scherer. Tartanair: A dataset to push the limits of visual slam. 2020
2020
-
[4]
Esim: an open event camera simulator
Henri Rebecq, Daniel Gehrig, and Davide Scaramuzza. Esim: an open event camera simulator. In Aude Billard, Anca Dragan, Jan Peters, and Jun Morimoto, editors,Proceedings of The 2nd Conference on Robot Learning, volume 87 ofProceedings of Machine Learning Research, pages 969–982. PMLR, 29–31 Oct 2018. 12
2018
-
[5]
CARLA: An open urban driving simulator
Alexey Dosovitskiy, German Ros, Felipe Codevilla, Antonio Lopez, and Vladlen Koltun. CARLA: An open urban driving simulator. InProceedings of the 1st Annual Conference on Robot Learning, pages 1–16, 2017
2017
-
[6]
Airsim: High-fidelity visual and physical simulation for autonomous vehicles
Shital Shah, Debadeepta Dey, Chris Lovett, and Ashish Kapoor. Airsim: High-fidelity visual and physical simulation for autonomous vehicles. InField and Service Robotics, 2017
2017
-
[7]
Rerun: A visualization sdk for multimodal data, 2024
Rerun Development Team. Rerun: A visualization sdk for multimodal data, 2024. Available from https://www.rerun.io/ and https://github.com/rerun-io/rerun
2024
-
[8]
Habitat: A Platform for Embodied AI Research
Manolis Savva, Abhishek Kadian, Oleksandr Maksymets, Yili Zhao, Erik Wijmans, Bhavana Jain, Julian Straub, Jia Liu, Vladlen Koltun, Jitendra Malik, Devi Parikh, and Dhruv Batra. Habitat: A Platform for Embodied AI Research. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2019
2019
-
[9]
Pybullet, a python module for physics simulation for games, robotics and machine learning, 2016
Erwin Coumans and Yunfei Bai. Pybullet, a python module for physics simulation for games, robotics and machine learning, 2016
2016
-
[10]
Fast feature field (F3): A predictive representation of events.arXiv preprint, 2025
Richeek Das, Kostas Daniilidis, and Pratik Chaudhari. Fast feature field (F3): A predictive representation of events.arXiv preprint, 2025. arXiv:2509.25146
arXiv 2025
-
[11]
Davison, J¨org Conradt, Kostas Daniilidis, and Davide Scaramuzza
Guillermo Gallego, Tobi Delbr¨ uck, Garrick Orchard, Chiara Bartolozzi, Brian Taba, Andrea Censi, Stefan Leutenegger, Andrew J. Davison, J¨org Conradt, Kostas Daniilidis, and Davide Scaramuzza. Event-based vision: A survey.IEEE Trans. Pattern Anal. Mach. Intell., 44(1):154–180, January 2022
2022
-
[12]
Spencer Folk, James Paulos, and Vijay Kumar. RotorPy: A python-based multirotor simulator with aerodynamics for education and research.arXiv preprint arXiv:2306.04485, 2023
Pith/arXiv arXiv 2023
-
[13]
Robot operating system 2: Design, architecture, and uses in the wild.Science Robotics, 7(66):eabm6074, 2022
Steven Macenski, Tully Foote, Brian Gerkey, Chris Lalancette, and William Woodall. Robot operating system 2: Design, architecture, and uses in the wild.Science Robotics, 7(66):eabm6074, 2022
2022
-
[14]
Zeromq: An open-source universal messaging library
The ZeroMQ Project. Zeromq: An open-source universal messaging library. https://github.com/ zeromq
-
[15]
Huang, Edwin Olson, and David C
Albert S. Huang, Edwin Olson, and David C. Moore. Lcm: Lightweight communications and marshalling. In2010 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 4057–4062, 2010
2010
-
[16]
Minimum snap trajectory generation and control for quadrotors
Daniel Mellinger and Vijay Kumar. Minimum snap trajectory generation and control for quadrotors. In 2011 IEEE International Conference on Robotics and Automation, pages 2520–2525, 2011
2011
-
[17]
Depth anything v2
Lihe Yang, Bingyi Kang, Zilong Huang, Zhen Zhao, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything v2. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang, editors,Advances in Neural Information Processing Systems, volume 37, pages 21875–21911. Curran Associates, Inc., 2024
2024
-
[18]
Unreal engine.https://www.unrealengine.com
Epic Games. Unreal engine.https://www.unrealengine.com. Technical Report. 13
This paper was first reviewed by deepseek-v4-flash on August 2, 2026.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.