Pith. sign in

REVIEW 4 major objections 5 minor 18 references

The paper presents Neurosim, a simulator that turns rendered intensity frames into event-camera streams at thousands of frames per second, and claims this removes the data bottleneck for neuromorphic robot perception.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Neurosim simulates event cameras and other robot sensors at multi-kilohertz rates on a GPU and streams the data to training and control pipelines without touching disk.

T0 review reviewed 2026-08-02 challenge →

load-bearing objection A genuinely fast event-camera simulator with a clean streaming architecture, but the sim-to-real motivation is untested and the benchmarks need reproducibility artifacts. the 4 major comments →

arxiv 2602.15018 v2 pith:EUELXZW2 submitted 2026-02-16 cs.RO cs.CV

Neurosim: A Fast Simulator for Neuromorphic Robot Perception

classification cs.RO cs.CV
keywords event camera simulationneuromorphic visionsim-to-real transferparallel computingCUDA kernelmultirotor simulationonline trainingrobot perception
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Neurosim generates event-camera data by rendering scenes at very high frame rates and converting intensity changes into events in a single CUDA kernel. The authors claim this makes event simulation fast enough to stream directly into training pipelines and closed-loop control loops, with no disk storage. They pair it with Cortex, a low-latency messaging layer that moves NumPy and PyTorch tensors between processes. If correct, this makes it practical to train event-based perception and control policies entirely on simulated data, and to test them on maneuvers too risky for real hardware.

Core claim

The central claim is that a warp-synchronous CUDA event-generation kernel removes the event-simulation bottleneck: it processes intensity frames at over 31 kHz for VGA and 23 kHz for HD, so the full simulation stack—rendering, dynamics, and event generation—runs at about 2,700 FPS on a desktop GPU. This is roughly 8–13× faster than other GPU event simulators and 55–121× faster than CPU implementations. Together with Cortex, this sustains event streams that train a monocular depth network entirely online, and it supports closed-loop control at controller rates up to 5 kHz with sub-millisecond latency.

What carries the argument

The key mechanism is the warp-synchronous event-generation kernel. For each pixel, it holds the intensity at the last triggered event and compares the current rendered intensity against a contrast threshold, producing an event when the log-intensity change exceeds the threshold. Threads in a warp cooperate to build a bitmask of active pixels, count events, and perform a single atomic operation to reserve a contiguous block in the output buffer; each thread then writes its event into that block. This reduces global atomic contention from one per event to roughly one per warp, letting event generation run in a single kernel launch and occupy only about 8% of a simulation step.

Load-bearing premise

The load-bearing premise is that event streams generated by Neurosim from rendered intensity frames are faithful enough to real event-camera output that perception and control algorithms trained on them will transfer to physical hardware; the paper asserts this motivationally (Section 1) but offers no sim-to-real evaluation.

What would settle it

Train a monocular depth network on Neurosim events, then run the same network on a real event camera observing the same scene geometry; a large drop in depth accuracy compared with the simulated test set would refute the sim-to-real transfer claim. A simpler check is comparing event-rate statistics and background noise between Neurosim output and a real DVS sensor under identical motion.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Perception and control algorithms for event cameras can be trained on simulated data faster than real time, without multi-terabyte datasets or disk I/O.
  • Closed-loop experiments can be run at the edge of a quadrotor's performance envelope—flips, fast obstacle avoidance—without risking hardware.
  • Because event generation costs only about 8% of a simulation step, the renderer becomes the bottleneck, so future rendering speedups translate directly to higher simulation rates.
  • Online training on streaming simulated data can saturate a second GPU's ingestion, making simulation a scalable replacement for static dataset curation.
  • Cortex's low latency (<0.7 ms) makes it possible to run the controller on separate hardware (e.g., an embedded board) while the simulator runs on a desktop GPU.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The absence of sim-to-real validation is the key open question; the natural next test is to deploy a Neurosim-trained depth or control policy on a physical event camera in the same environment and measure the performance gap.
  • The warp-synchronous event-generation pattern could be reused for other sparse, asynchronous sensor streams, such as LiDAR point clouds or spike-based neural sensors, where output is sparse and variable-size.
  • Because the simulator stresses temporal resolution over photorealism, domain randomization over scenes and trajectories may be more important than photometric realism for closing the sim-to-real gap; the paper's noise-model list in the discussion points in that direction.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces Neurosim, a GPU-accelerated modular simulator for neuromorphic robot perception, together with Cortex, a ZeroMQ-based communication middleware. Neurosim integrates Habitat-Sim for rendering, RotorPy for multirotor dynamics, and a custom CUDA event-generation kernel. The authors report that the full stack runs at ~2700 FPS on an NVIDIA 4090 and that the event backend processes intensity frames at >31 kHz (VGA) and >23 kHz (HD), substantially outperforming ESIM, VID2E, and AirSim. They also present Cortex benchmarks showing low-latency, high-throughput messaging, a closed-loop control latency analysis, and an online training demonstration in which an F3-based monocular depth network is trained on-the-fly from Neurosim data. The stated motivation is that simulated event data can reduce the sim-to-real gap, but the paper provides no validation of event realism or hardware transfer.

Significance. If the performance claims hold, Neurosim and Cortex could be valuable infrastructure for high-throughput simulation of event cameras and multi-modal sensors, enabling online training and closed-loop evaluation that would otherwise require terabytes of disk storage. The warp-synchronous CUDA event-generation design is a concrete engineering contribution, and the open-source release is a practical benefit to the community. However, the paper's central value proposition — that simulated event streams are faithful enough for sim-to-real transfer — is not demonstrated. The benchmark numbers are plausible but lack statistical rigor and reproducibility artifacts, and the only learned model is evaluated qualitatively on simulated data. The strength of the paper is therefore in its reported speed and architecture, not yet in its claimed realism.

major comments (4)
  1. [Sec. 2 vs Sec. 5] There is an internal contradiction about sensor non-idealities. Sec. 2 states that 'Real-world event sensors exhibit non-idealities such as noise, bandwidth limitations, and sensor saturation effects. Neurosim simulates these artifacts,' yet Sec. 5 lists as future work 'adding sensory noise models (shot/thermal noise, bandwidth saturation, refractory effects, rolling shutter).' This makes it unclear which non-idealities are actually implemented. Since realism is a core motivation, please resolve this discrepancy, specify the current noise/fidelity model precisely, and validate it.
  2. [Sec. 1, Desideratum 1; Sec. 4] The paper's motivating claim is that 'the sim-to-real gap for systems trained on simulated event data is smaller.' No experiment in Sec. 4 tests this. The only learning demonstration trains and evaluates entirely on simulated data, showing a single qualitative depth map (Fig. 6B(v)), and the Discussion explicitly defers hardware-in-the-loop and sensory noise models to future work. This is a load-bearing gap: the tool's primary use case is to train algorithms that transfer to physical robots. The authors should either add a quantitative comparison of Neurosim's event statistics (e.g., event rate, temporal distribution, noise) against real event-camera data (e.g., M3ED) or a sim-to-real transfer study, or explicitly rescope the paper's claims to simulation infrastructure without the sim-to-real promise.
  3. [Sec. 2, Fig. 3, Fig. 4] The central speed and throughput claims are reported as single-point averages with no error bars, number of trials, or variance information. The ~2700 FPS full-stack figure, the 31/23 kHz event-backend rates, and the Cortex throughput plots in Fig. 4 would be much more convincing with mean±std over repeated runs. Additionally, the manuscript does not provide a commit hash, container image, or pinned software versions (Habitat-Sim, CUDA, PyTorch, etc.), so the numbers are not independently verifiable. Please supply these reproducibility artifacts and statistical details.
  4. [Sec. 4, Fig. 6B] The online training demonstration lacks any quantitative evaluation. The paper claims the F3-based depth network was trained online and that the pipeline saturates the training GPU, but no training curves, validation losses, or standard depth metrics (e.g., RMSE, abs rel, delta thresholds) are reported. Without these, the reader cannot assess whether the training actually produced a useful perception model. Please include quantitative results on a held-out set of simulated trajectories and scenes.
minor comments (5)
  1. [Sec. 2] The list of design principles is misnumbered: it reads '(1) modularity ...; (2) low-latency ...; (2) real-time streaming ...; (4) Pythonic accessibility.' The second '(2)' should be '(3)'.
  2. [Eqs. (1)-(2)] The regularizer is denoted R(˜d, ˜d*) in Eq. (1) but R(d,d*) in Eq. (2), and the subscripted d_sigma is not explicitly defined as a downsampled version of d. Please unify notation.
  3. [Sec. 2, Fig. 3] The paper states 'ESIM Backends (left to right in each plot)' but the bars are not individually labeled in the text; please clarify which bar corresponds to each backend and whether the CUDA baseline is ESIM or another implementation.
  4. [References] The statement in Sec. 1 that 'Works like [2] have obtained remarkable results by training entirely on simulated event datasets' may not be an accurate characterization of Klenk et al.'s Deep Event Visual Odometry. Please verify the citation and adjust the claim.
  5. [Sec. 4, Fig. 6A] The closed-loop latency analysis uses a VGA RGB camera, not an event camera. Since event streams have variable-length, asynchronous payloads, the reported <0.7 ms latency and ~2.3 kHz sensor rate may not directly apply to the event-based setting that motivates the paper. Consider adding a similar analysis with event streams or stating the limitation explicitly.

Circularity Check

0 steps flagged

No significant circularity: speed and throughput claims are externally benchmarked; self-cited F3 is not load-bearing.

full rationale

The paper's central performance claims are measured against external baselines: Fig. 3 benchmarks Neurosim's CUDA event backend against ESIM [4], AirSim [6], and a PyTorch GPU implementation, and the ~2700 FPS end-to-end figure is tied to Habitat-Sim's independently reported rendering rate. These benchmarks do not depend on the authors' own F3 model. The only self-references are (i) using the authors' F3 representation in the Sec. 4 monocular-depth demonstration and (ii) citing Das et al. [10] for event-rate context; neither feeds back into the simulator's speed, event-generation, or throughput measurements. The paper's motivating claim that simulated event data transfers to real hardware (Sec. 1, Desideratum 1: 'the sim-to-real gap for systems trained on simulated event data is smaller') is not validated, and Sec. 5 explicitly defers sensory-noise models and hardware-in-the-loop experiments to future work. That is an unverified assumption and a correctness risk, not a circular derivation. No equation defines a predicted quantity in terms of a fitted input, and no uniqueness theorem or ansatz is imported from the authors' prior work. Consequently, no circular step can be exhibited under the quoted-evidence standard.

Axiom & Free-Parameter Ledger

0 free parameters · 3 axioms · 0 invented entities

The paper makes no fitted-parameter derivations; its performance claims depend on hardware and on the fidelity assumptions listed above. No new theoretical entities are introduced; Cortex is a software component, not a physical postulate.

axioms (3)
  • domain assumption ESIM-style contrast-threshold event model is an adequate model of real event-camera behavior for the purpose of training downstream perception and control algorithms.
    The entire utility argument rests on synthetic events transferring to real hardware (Sec. 1, Desideratum 1; Sec. 4 application), but no sim-to-real validation is provided.
  • domain assumption Rendering intensity images at multi-kilohertz rates with Habitat-Sim captures the photometric dynamics needed for event generation; missed-event probability is negligible at 2700 FPS.
    Sec. 2 states high frame rate 'reduces the likelihood of large intensity changes... that could lead to missed events', but no quantitative bound or validation.
  • domain assumption The RotorPy dynamics model faithfully represents agile multirotor flight up to the performance envelope used in the demonstrations.
    Sec. 2 relies on RotorPy for 'fast and accurate' dynamics; accuracy is asserted from the prior work [12], not re-validated here.

reviewed 2026-08-02 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Neurosim: A Fast Simulator for Neuromorphic Robot Perception." pith.science (2026). https://pith.science/paper/EUELXZW2

@misc{pith2026260215018,
  author       = {Pith},
  title        = {Pith review of: Neurosim: A Fast Simulator for Neuromorphic Robot Perception},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EUELXZW2}},
  note         = {Machine review of arXiv:2602.15018}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Neurosim is a fast, real-time, high-performance library for simulating sensors such as dynamic vision sensors, RGB cameras, depth sensors, and inertial sensors. It can also simulate agile dynamics of multi-rotor vehicles in complex and dynamic environments. Neurosim can achieve frame rates as high as ~2700 FPS on a desktop GPU. Neurosim integrates with a ZeroMQ-based communication library called Cortex to facilitate seamless integration with machine learning and robotics workflows. Cortex provides a high-throughput, low-latency message-passing system for Python and C++ applications, with native support for NumPy arrays and PyTorch tensors. This paper discusses the design philosophy behind Neurosim and Cortex. It demonstrates how they can be used to (i) train neuromorphic perception and control algorithms, e.g., using self-supervised learning on time-synchronized multi-modal data, and (ii) test real-time implementations of these algorithms in closed-loop. Neurosim and Cortex are available at https://github.com/grasp-lyrl/neurosim .

Figures

Figures reproduced from arXiv: 2602.15018 by Pratik Chaudhari, Richeek Das.

Figure 1
Figure 1. Figure 1: Event cameras capture the scene without temporal aliasing, with high dynamic range, and consume very [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the design of Neurosim [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Neurosim event simulator backend is optimized to prevent bottlenecking of the full simulation pipeline. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Cortex provides high-throughput, low-latency, and scalable communication in Python. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Examples of a quadrotor tracking randomly generated MinSnap trajectories in a variety of indoor scenes. [PITH_FULL_IMAGE:figures/full_fig_p009_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Neurosim can be used for closed-loop controller testing and online data generation for learning. (A) [PITH_FULL_IMAGE:figures/full_fig_p011_6.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

18 extracted references · 1 linked inside Pith

  1. [1]

    Ani Hsieh, Christopher Korpela, Vijay Kumar, Camillo J

    Kenneth Chaney, Fernando Cladera, Ziyun Wang, Anthony Bisulco, M. Ani Hsieh, Christopher Korpela, Vijay Kumar, Camillo J. Taylor, and Kostas Daniilidis. M3ed: Multi-robot, multi-sensor, multi- environment event dataset. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pages 4015–4022, June 2023

  2. [2]

    Deep event visual odometry

    Simon Klenk, Marvin Motzet, Lukas Koestler, and Daniel Cremers. Deep event visual odometry. In International Conference on 3D Vision, 3DV 2024, Davos, Switzerland, March 18-21, 2024, pages 739–749. IEEE, 2024

  3. [3]

    Tartanair: A dataset to push the limits of visual slam

    Wenshan Wang, Delong Zhu, Xiangwei Wang, Yaoyu Hu, Yuheng Qiu, Chen Wang, Yafei Hu, Ashish Kapoor, and Sebastian Scherer. Tartanair: A dataset to push the limits of visual slam. 2020

  4. [4]

    Esim: an open event camera simulator

    Henri Rebecq, Daniel Gehrig, and Davide Scaramuzza. Esim: an open event camera simulator. In Aude Billard, Anca Dragan, Jan Peters, and Jun Morimoto, editors,Proceedings of The 2nd Conference on Robot Learning, volume 87 ofProceedings of Machine Learning Research, pages 969–982. PMLR, 29–31 Oct 2018. 12

  5. [5]

    CARLA: An open urban driving simulator

    Alexey Dosovitskiy, German Ros, Felipe Codevilla, Antonio Lopez, and Vladlen Koltun. CARLA: An open urban driving simulator. InProceedings of the 1st Annual Conference on Robot Learning, pages 1–16, 2017

  6. [6]

    Airsim: High-fidelity visual and physical simulation for autonomous vehicles

    Shital Shah, Debadeepta Dey, Chris Lovett, and Ashish Kapoor. Airsim: High-fidelity visual and physical simulation for autonomous vehicles. InField and Service Robotics, 2017

  7. [7]

    Rerun: A visualization sdk for multimodal data, 2024

    Rerun Development Team. Rerun: A visualization sdk for multimodal data, 2024. Available from https://www.rerun.io/ and https://github.com/rerun-io/rerun

  8. [8]

    Habitat: A Platform for Embodied AI Research

    Manolis Savva, Abhishek Kadian, Oleksandr Maksymets, Yili Zhao, Erik Wijmans, Bhavana Jain, Julian Straub, Jia Liu, Vladlen Koltun, Jitendra Malik, Devi Parikh, and Dhruv Batra. Habitat: A Platform for Embodied AI Research. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2019

  9. [9]

    Pybullet, a python module for physics simulation for games, robotics and machine learning, 2016

    Erwin Coumans and Yunfei Bai. Pybullet, a python module for physics simulation for games, robotics and machine learning, 2016

  10. [10]

    Fast feature field (F3): A predictive representation of events.arXiv preprint, 2025

    Richeek Das, Kostas Daniilidis, and Pratik Chaudhari. Fast feature field (F3): A predictive representation of events.arXiv preprint, 2025. arXiv:2509.25146

  11. [11]

    Davison, J¨org Conradt, Kostas Daniilidis, and Davide Scaramuzza

    Guillermo Gallego, Tobi Delbr¨ uck, Garrick Orchard, Chiara Bartolozzi, Brian Taba, Andrea Censi, Stefan Leutenegger, Andrew J. Davison, J¨org Conradt, Kostas Daniilidis, and Davide Scaramuzza. Event-based vision: A survey.IEEE Trans. Pattern Anal. Mach. Intell., 44(1):154–180, January 2022

  12. [12]

    RotorPy: A python-based multirotor simulator with aerodynamics for education and research.arXiv preprint arXiv:2306.04485, 2023

    Spencer Folk, James Paulos, and Vijay Kumar. RotorPy: A python-based multirotor simulator with aerodynamics for education and research.arXiv preprint arXiv:2306.04485, 2023

  13. [13]

    Robot operating system 2: Design, architecture, and uses in the wild.Science Robotics, 7(66):eabm6074, 2022

    Steven Macenski, Tully Foote, Brian Gerkey, Chris Lalancette, and William Woodall. Robot operating system 2: Design, architecture, and uses in the wild.Science Robotics, 7(66):eabm6074, 2022

  14. [14]

    Zeromq: An open-source universal messaging library

    The ZeroMQ Project. Zeromq: An open-source universal messaging library. https://github.com/ zeromq

  15. [15]

    Huang, Edwin Olson, and David C

    Albert S. Huang, Edwin Olson, and David C. Moore. Lcm: Lightweight communications and marshalling. In2010 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 4057–4062, 2010

  16. [16]

    Minimum snap trajectory generation and control for quadrotors

    Daniel Mellinger and Vijay Kumar. Minimum snap trajectory generation and control for quadrotors. In 2011 IEEE International Conference on Robotics and Automation, pages 2520–2525, 2011

  17. [17]

    Depth anything v2

    Lihe Yang, Bingyi Kang, Zilong Huang, Zhen Zhao, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything v2. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang, editors,Advances in Neural Information Processing Systems, volume 37, pages 21875–21911. Curran Associates, Inc., 2024

  18. [18]

    Unreal engine.https://www.unrealengine.com

    Epic Games. Unreal engine.https://www.unrealengine.com. Technical Report. 13

This paper was first reviewed by deepseek-v4-flash on August 2, 2026.