{"id":"6d4341d8-44a6-40b6-8c29-b99ec0ada535","arxiv_id":"2608.08522","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"EsaacSim is an Isaac Sim add-on that generates online grayscale and Bayer RGGB event streams at up to 960 Hz effective rate, with synchronized RGB, APS, depth, and IMU outputs via ROS 2.","lead":"This paper presents EsaacSim, a software add-on that simulates event cameras inside NVIDIA's Isaac Sim robotics platform. It provides synchronized event, RGB, depth, and IMU outputs over ROS 2, letting robotics researchers generate event-camera data without physical hardware.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Equation (4) appears time-reversed relative to Eq. (5): with backward motion vectors it returns the current frame at α=0 and the previous frame at α=1, while the assigned timestamps run the opposite way; the 240–960 Hz effective-rate claim rests on these intermediate frames.","rationale":"The paper is a systems contribution, and its strongest claim is that the add-on provides online, configurable event camera simulation with synchronized multimodal outputs and effective rates up to 960 Hz through motion-guided frame synthesis. The performance numbers in Fig. 8 and Table 3 are informative, and the paper openly lists its limitations: a frame-based event model, no quantitative comparison against real event camera recordings, and a single consumer GPU. Those limitations alone do not falsify the central claim, because a frame-based simulator with an unvalidated event model can still be a useful tool and the reported event-generation times are mostly below a 30 Hz frame budget. The load-bearing risk I see is more specific: the motion-guided synthesis as written in Eq. (4) is internally time-reversed relative to the timestamps defined in Eq. (5). The synthetic intermediate frames are the entire basis for the claimed 240–960 Hz effective temporal sampling, and if the equation is implemented as written, the events generated from those frames will have incorrect timestamps and polarity ordering. This is a correctness risk rather than only a fidelity risk, and it can be checked without real event camera hardware. Because the code release is still pending, we cannot tell whether the implementation already uses the corrected warp; the manuscript should either correct the equation or provide this verification. The reader's weakest assumption about fidelity to real event cameras is valid but broader; my concern sharpens it to a concrete internal check. I therefore keep the verdict conditional, as the reader did, but with the condition now anchored to the motion-guided synthesis equation.","tokens_in":10939,"tokens_out":18085,"duration_ms":198142,"concrete_test":"Render a planar textured object moving with known constant image-space velocity in Isaac Sim at 30 Hz, and additionally render ground-truth frames at each of the N=8 subdivision timestamps t_{k,n} (i.e., at 240 Hz). Compute the mean per-pixel intensity error between each Eq. (4)-synthesized I_{k,n} and the ground-truth frame at the same timestamp. If Eq. (4) is correct, errors should be at the level of bilinear interpolation artifacts; if the synthesis is reversed, the n=1 synthesized frame will match the n=7 ground-truth frame instead of the n=1 frame. Also compare the event stream generated from the synthesized frames against the event stream generated directly from the 240 Hz ground-truth renders to check timestamp order and polarity consistency.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing flaw is internal to the motion-guided frame-gap synthesis. Let v_b be the backward motion vector field, with v_b(x) pointing from a pixel's current position to its position in the previous rendered frame. Then Eq. (4) at α=0 returns I_k(x), the current frame, while Eq. (5) assigns α=0 to t_{k−1}. At α=1, Eq. (4) returns I_k(x−v_b), which is the previous frame's content at that pixel, while Eq. (5) assigns α=1 to t_k. The synthesized frames are therefore ordered in reverse along the interpolation interval; for N=2 the midpoint is symmetric, which may be why the error is easy to miss, but for the reported N=8, 16, and 32 the early frames contain content from near the current frame and vice versa. Because each synthesized frame is passed through the same log-intensity event model, the events inherit incorrect timestamps and polarity ordering for the actual motion. This is independent of the acknowledged lack of real-camera validation (Section 5.3) and directly affects the paper's core contribution of effective temporal sampling up to 960 Hz. If the implementation actually uses I_k(x + (1−α)v_b) (or an equivalent forward-warping formulation), then the manuscript's equation should be corrected; as printed, the algorithm description is internally inconsistent.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents EsaacSim, an add-on for NVIDIA Isaac Sim that generates online event-camera streams from rendered images, supporting grayscale and Bayer RGGB event modes together with synchronized RGB, APS, depth, and IMU outputs exposed through ROS 2. The event generation uses a standard frame-based log-intensity threshold model, and a motion-guided frame-gap synthesis method is proposed to increase the effective temporal sampling rate up to 960 Hz without increasing the rendering frequency. Experiments on an RTX 4060 report event-generation times of 6.98--27.28 ms per rendered-frame interval for grayscale and 7.58--29.16 ms for Bayer RGGB across five sensor resolutions, with less than 400 MB of additional GPU memory. The authors acknowledge that the simulated event streams are not yet quantitatively validated against real event camera data and that the physics-based backend is not implemented.","tokens_in":11277,"tokens_out":6282,"duration_ms":69093,"significance":"If the method is correct and the generated event streams are faithful, EsaacSim would fill a practical gap: an integrated, ROS 2-ready multimodal event camera simulator inside Isaac Sim. The paper makes a concrete engineering contribution by combining configurable sensor models, Bayer RGGB event generation, synchronized multimodal outputs, and a consumer-GPU performance evaluation. The performance measurements, while limited to a single GPU and without repeated-run statistics, are useful and clearly reported. The central algorithmic question, however, is the correctness of the motion-guided frame-gap synthesis, and the lack of any fidelity validation against real event cameras leaves the usefulness of the generated data for robotics perception unestablished.","major_comments":[{"comment":"The motion-guided frame-gap synthesis appears internally time-reversed under the standard convention for backward motion vectors. If v(x,y) points from a pixel's current position to its previous position, then Eq. (4) gives I_k(x) at α=0 and I_k(x−v), i.e., the previous frame's content, at α=1. Eq. (5), however, assigns α=0 to t_{k−1} and α=1 to t_k. The synthesized intermediate images are therefore ordered in reverse along the interpolation interval. For N=2 the single midpoint is symmetric and the error is hidden, but for the reported N=8, 16, and 32, early synthesized samples contain content near the current frame while late samples contain content near the previous frame. Because each synthesized frame is passed through the same log-intensity event model, the events inherit incorrect timestamps and reversed polarity ordering for a given direction of scene motion. This directly affects the paper's central claim of effective temporal sampling rates up to 960 Hz. Please correct Eq. (4) (e.g., use I_k(x+(1−α)v) under the current-to-previous convention, or equivalently I_{k−1}(x+αv)) or, if the implementation already uses the corrected form, amend the equation to match the implementation and state the motion-vector convention explicitly.","section":"§4.2, Eqs. (4) and (5)"},{"comment":"The paper reports no quantitative comparison of the generated event streams against real event camera recordings. The authors themselves state in Section 5.3 that 'establishing this correspondence is necessary to assess the fidelity of the generated event streams.' Since the stated purpose of the tool is to support robotics research with synthetic event data, computational efficiency alone does not establish that the events are useful. I request at least a basic fidelity evaluation: for example, known-motion scenes with quantitative event-rate and polarity checks, comparison of event counts versus contrast threshold against a real DAVIS or GenX sensor, or comparison with established simulators such as ESIM or V2E. If no such validation is available, the claims should be explicitly scoped to 'efficient online event generation' rather than 'faithful event camera simulation.'","section":"§5.3 and Conclusions"},{"comment":"The performance evaluation reports ranges over resolutions and interpolation factors but does not give repeated-run statistics, standard deviations, or the number of trials for each configuration. Because the conclusions about online operation and the 'less than 400 MB' memory bound depend on these numbers, please report means and variances over multiple runs, describe the measurement procedure (warm-up, number of rendered frames, timing synchronization), and state the GPU clock/memory state where relevant.","section":"§5.3, Fig. 8 and Table 3"}],"minor_comments":[{"comment":"The phrase 'enables supports online multimodal event-camera simulation' appears to contain a typo; it should be 'enables online multimodal event-camera simulation.'","section":"Abstract"},{"comment":"The DAVIS346 resolution is given inconsistently as 346×260 in Figure 8 and Section 5.1 but as 260×346 in Table 2. Please standardize the orientation notation.","section":"Table 2 and §5.1"},{"comment":"The GenX320 row at ×32 reports 399.3 MB (grayscale) and 399.7 MB (Bayer). The claim 'less than 400 MB' is technically true but very close to the bound; consider reporting a margin or stating the measurement precision.","section":"Table 3"},{"comment":"The term 'backward warping' should be defined in one sentence, specifying the direction of the motion-vector field and the convention used by Isaac Sim, so readers can verify Eq. (4) independently.","section":"§4.2"},{"comment":"The paper states that the add-on 'will be publicly released' but gives no repository URL, version, or installation details. Including a link would improve reproducibility and is important for a tool paper.","section":"Conclusions"},{"comment":"The per-pixel reference state update is described only as 'updated before processing the next sample.' Please clarify whether the reference is updated after every rendered and synthesized frame or after every event batch, since this affects the event-generation behavior when multiple intermediate samples are processed.","section":"§4.2, Eqs. (1)--(3)"}],"recommendation":"major_revision","confidential_remarks":"The time-reversal issue in Eqs. (4)--(5) is the key technical concern; if the implementation is correct and only the equation is wrong, the paper can be repaired locally. The absent fidelity validation is a scope issue the authors already acknowledge; I would expect at least a minimal comparison or a explicitly reduced claim before acceptance. The paper is otherwise within scope for a robotics systems venue."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: EsaacSim is a genuinely useful integration—event camera simulation inside Isaac Sim with synchronized RGB, depth, IMU, events, and ROS 2 output—and the performance data are credible for a consumer GPU. But the motion-guided interpolation as described in Eq. (4)–(5) is time-reversed. That is not a cosmetic issue; it directly affects the 240–960 Hz effective-rate claims.\n\nWhat's actually new: not the event model (standard log-intensity thresholding from ESIM/V2E), but the packaging. The authors attach to Isaac Sim render products, support Bayer RGGB patterns, give configurable sensor parameters, and output synchronized multimodal streams over ROS 2. That is a useful tool for robotics people who want synthetic event data in a modern simulator. The architecture is clean, the limitations section is honest, and the single-GPU benchmark with memory figures is a nice concrete datapoint. The self-citation to their earlier color emulator is appropriate.\n\nThe soft spots, in order of severity. First, Eqs. (4)–(5) don't agree. With the backward motion field described in the text, I_k(x − α v) at α=0 is the current frame and at α=1 is the previous frame's content, while Eq. (5) assigns α=0 to t_{k−1} and α=1 to t_k. So the synthesized frames are temporally reversed. For N=2 the midpoint is symmetric, so it's easy to miss; the paper uses N=8, 16, 32 for the rate claims. If the implementation actually uses I_k(x − (1−α)v) or equivalent, the equation is simply misprinted—but as written, the algorithm description is internally inconsistent and the published claim isn't verifiable. Second, there is no quantitative comparison against real event camera data. The authors acknowledge this in Section 5.3, but for a simulator paper, it's a significant gap: you can't assess whether the generated events are actually useful for perception. Third, the performance numbers are single-run ranges without error bars, and the code release is promised but no link appears in the paper. Both are minor next to the equation issue.\n\nWho gets value? Robotics researchers who want a working event camera in Isaac Sim and are willing to use it as a qualitative development tool. The paper deserves a serious referee—this is a real integration effort with useful infrastructure—but it needs major revision before the temporal interpolation claim can be trusted. I'd send it back, ask for a corrected equation (or a clear motion-vector convention), real-camera validation or at least a downstream task, and the actual code release.","headline":"Useful Isaac Sim event-camera integration, but the motion-guided interpolation equation is time-reversed and the effective-rate claims are not trustworthy until that is fixed and real-camera validation appears.","tokens_in":11743,"tokens_out":5135,"would_cite":false,"duration_ms":47503,"reading_group":"yes","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that an add-on for the Isaac Sim robotics simulator can generate synchronized event, RGB, APS, depth, and IMU streams online, with effective event rates up to 960 Hz, using motion-guided interpolation and a frame-based…","keywords":["Event camera simulation","Isaac Sim","Neuromorphic vision","Robotics simulation","ROS 2","Bayer RGGB events","Motion-guided interpolation","Synthetic data generation"],"falsifier":"Record a real scene with an event camera and replay the same camera trajectory and lighting in EsaacSim; if the generated events miss systematic structure such as high-dynamic-range details, motion blur, or noise, or if downstream perception performance diverges markedly from real data, the central usefulness claim would be undermined. A concrete quantitative check is to compare event-rate distributions and contrast-threshold histograms between the real and simulated streams under identical motion.","tokens_in":10795,"feed_emoji":"🎥","tokens_out":10185,"duration_ms":90559,"temperature":0.7,"pith_summary":"This paper claims that an add-on for the Isaac Sim robotics simulator can produce synchronized event, RGB, APS, depth, and IMU streams online, directly from the simulator's existing render pipeline. The central move is to attach a frame-based log-intensity event model to rendered HDR images, with motion-guided synthesis of intermediate frames that raises the effective event rate to up to 960 Hz without increasing the render frequency. Benchmarks across five event-camera resolutions report 6.98–27.28 ms per rendered-frame interval for grayscale events and 7.58–29.16 ms for Bayer RGGB events, using less than 400 MB of extra GPU memory on a consumer-grade graphics card. If these numbers hold, event-based perception such as SLAM, optical flow, and object detection can be developed and tested in a photorealistic simulator with closed-loop ROS 2 integration, rather than on scarce physical event cameras.","feed_headline":"Event-camera sim runs online in Isaac Sim at up to 960 Hz","feed_subtitle":"Synchronized RGB, depth, IMU, and grayscale or Bayer events from one simulation pipeline, fast enough for closed-loop robotics.","key_machinery":"The load-bearing mechanism is the pair formed by a frame-based log-intensity event model and motion-guided frame-gap synthesis. The event model computes $L_k(x,y)=\\log(S_k(x,y)+1)$, keeps a per-pixel reference level $L_{\\mathrm{ref}}$, and emits ON/OFF events when $\\Delta L_k(x,y)$ reaches $C_{\\mathrm{ON}}$ or $C_{\\mathrm{OFF}}$. The synthesis step takes two consecutive rendered frames with the renderer's dense motion-vector field $v(x,y)=(u(x,y),v(x,y))$ and forms intermediate samples by backward warping $I_{k,n}(x,y)=I_k(x-\\alpha u(x,y), y-\\alpha v(x,y))$ with bilinear interpolation, where $\\alpha=n/N$; timestamps are placed uniformly between the two frame times. This lets the simulator produce high-temporal-resolution event streams from a low render cadence, reusing the same event model on real and synthesized frames.","core_discovery":"EsaacSim's central claim is that online, configurable event camera simulation can be layered on top of a GPU-accelerated robotics simulator without altering the renderer. It consumes the camera's HDR render product, converts it to log-intensity (grayscale, weighted luminance, or Bayer RGGB samples), and emits an asynchronous event when the accumulated log-intensity change at a pixel crosses a positive or negative contrast threshold. Sensor parameters such as thresholds, threshold mismatch, refractory period, latency, timestamp quantization, shot noise, and leak noise are runtime-configurable per camera. To increase temporal resolution, the add-on synthesizes intermediate images by backward-warping the current rendered image along the renderer's motion vectors for $N$ fractional steps between frames; both rendered and synthesized images go through the same event model. The result is a unified asynchronous event stream with effective sampling rates up to 960 Hz, delivered alongside RGB, APS, depth, and IMU data through native ROS 2 interfaces.","pith_inferences":["A natural next test would be to record the same scene with a real event camera and compare event-rate statistics, threshold distributions, and downstream odometry error; the paper's public release would make this validation possible.","If the motion-warped interpolation proves faithful, the same synthesis approach could be reused for other high-frame-rate camera emulation and for data augmentation in event-based learning.","The architecture's separation between rendering and event backend suggests the physics-based backend currently stubbed out could later be swapped in without changing the ROS 2 interfaces, but no evidence yet shows it would match real sensor behavior.","Closed-loop training of event-based policies in simulation could become common practice before hardware deployment, assuming fidelity holds; this would reduce reliance on expensive physical event cameras."],"forward_implications":["A 30 Hz render loop can yield event streams at effective sampling rates of 240, 480, or 960 Hz, so high-speed motion can be studied without raising the rendering cost.","Synchronized RGB, APS, depth, IMU, and event data are available through native ROS 2 messages, making the add-on directly usable by robot perception and control stacks.","Five common event-camera resolutions (DVS128, DAVIS240, ATIS, DAVIS346, GenX320) are supported with configurable thresholds, latency, refractory periods, and noise, so multiple sensor profiles can be emulated in one scene.","The measured per-frame event-generation time stays below about 30 ms on a consumer-grade graphics card with under 400 MB of extra GPU memory, which is consistent with online operation alongside the simulator's own rendering.","Both grayscale and Bayer RGGB event generation are handled by the same backend, so color event camera configurations can be simulated without a separate pipeline."],"supporting_citations":[{"why":"Supplies the standard frame-based log-intensity event model that EsaacSim's backend adopts.","marker":"[26]"},{"why":"The existing Isaac Sim event plugin whose coverage EsaacSim extends with multimodal outputs and ROS 2 integration.","marker":"[32]"},{"why":"Represents the frame-based DVS emulation approach that motivates EsaacSim's design and serves as a comparison point.","marker":"[16]"},{"why":"Prior color event camera emulator that supports the Bayer RGGB event generation path and ROS-based online use.","marker":"[4]"},{"why":"YCB object set used for the qualitative evaluation of synchronized multimodal outputs.","marker":"[5]"},{"why":"Provides the event camera principles and notation (log-intensity, contrast threshold, polarity) that the event model relies on.","marker":"[9]"}],"fun_headline_variants":["Event camera add-on for Isaac Sim hits 960 Hz online","Multimodal event sim for Isaac Sim at up to 960 Hz","Online event camera simulation in Isaac Sim: 960 Hz max","EsaacSim brings configurable event cameras to Isaac Sim","Event stream at 960 Hz from Isaac Sim with EsaacSim"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that thresholded log-intensity changes computed from rendered HDR frames, including the motion-warped synthesized frames, faithfully reproduce the event stream a real event camera would produce for the same scene; the paper reports no quantitative comparison against real event camera recordings.","fun_headline_variants_meta":{"raw":{"variants":["Event camera add-on for Isaac Sim hits 960 Hz online","Multimodal event sim for Isaac Sim at up to 960 Hz","Online event camera simulation in Isaac Sim: 960 Hz max","EsaacSim brings configurable event cameras to Isaac Sim","Event stream at 960 Hz from Isaac Sim with EsaacSim"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000651,"raw_usage":{"total_tokens":2998,"prompt_tokens":971,"completion_tokens":2027,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":587,"completion_tokens_details":{"reasoning_tokens":1937}},"tokens_in":587,"tokens_out":2027,"duration_ms":12857,"temperature":1.0,"reasoning_tokens":1937,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T04:32:01.388961+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Record a real scene with an event camera and replay the same camera trajectory and lighting in EsaacSim; if the generated events miss systematic structure such as high-dynamic-range details, motion blur, or noise, or if downstream perception performance diverges markedly from real data, the central usefulness claim would be undermined. A concrete quantitative check is to compare event-rate distributions and contrast-threshold histograms between the real and simulated streams under identical motion.","supporting_citations":[{"cited_title":"In: Billard, A., Dragan, A., Peters, J., Morimoto, J","cited_arxiv_id":null,"evidence_quote":"Supplies the standard frame-based log-intensity event model that EsaacSim's backend adopts."},{"cited_title":"EVIS: A Physics-Grounded Event Camera Plugin for NVIDIA Isaac Sim","cited_arxiv_id":"2607.08098","evidence_quote":"The existing Isaac Sim event plugin whose coverage EsaacSim extends with multimodal outputs and ROS 2 integration."},{"cited_title":"In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)","cited_arxiv_id":null,"evidence_quote":"Represents the frame-based DVS emulation approach that motivates EsaacSim's design and serves as a comparison point."},{"cited_title":"In: International Conference on Robotics, Computer Vision and Intelligent Systems","cited_arxiv_id":null,"evidence_quote":"Prior color event camera emulator that supports the Bayer RGGB event generation path and ROS-based online use."},{"cited_title":"IEEE Transactions on Pattern Analysis and Machine Intelligence 44(1), 154–180 (2022) 1, 4","cited_arxiv_id":null,"evidence_quote":"Provides the event camera principles and notation (log-intensity, contrast threshold, polarity) that the event model relies on."}],"review_version":1}