{"id":"5c90f63c-dc97-4a80-8691-adedaafffb06","arxiv_id":"2412.06770","paper_version":4,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Dynamic EventNeRF reconstructs dynamic 4D scenes from sparse multi-view event cameras and sparse RGB frames using time-conditioned, multi-segment NeRF models with event-based losses.","lead":"The paper introduces Dynamic EventNeRF, a method for reconstructing moving 3D scenes from six synchronized event cameras plus a few blurry RGB frames, and releases a new multi-view event dataset. A generalist reader might care because it demonstrates a path for 3D capture in dark, fast-motion conditions where ordinary cameras fail.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Event accumulation decay (b=0.93) in Eq. 11 makes the Eq. 6 target non-physical: the model is trained to match a damped event sum rather than the true log-intensity change, and the paper's only justification is an unquantified claim that the MLP compensates.","rationale":"The reader's weakest_assumption centered on CRF calibration and the manually chosen cylinder. I find a more fundamental issue in the event loss itself: the decay factor in Eq. 11 makes the target of Eq. 6 inconsistent with the physical event model of Eq. 1. This is not merely a calibration constant that can be absorbed; it is a systematic, time-dependent bias that is explicitly acknowledged in Appx. I but never quantified. The paper's empirical results are still credible and the ablation study is extensive, so the concern does not overturn the conditional verdict. However, it adds a specific technical condition that must be verified: the decay must not materially distort the legitimate event signal. The proposed test isolates the bias by using noise-free synthetic data and comparing decayed, undecayed, and exact event losses. If the bias is small, the method's claim of physically grounded event supervision survives; if it is large, the reported improvements may partly stem from a heuristic temporal regularizer rather than from correct event modeling. I therefore keep the reader's CONDITIONAL verdict unchanged, but I disagree with the reader's identification of the most load-bearing assumption.","tokens_in":22453,"tokens_out":4795,"duration_ms":50155,"concrete_test":"On a synthetic scene with known ground-truth intensities and noise-free events, train three variants: (A) full model with b=0.93, (B) same with b=1.0 (no decay), and (C) an exact event loss that accumulates per-event thresholds without decay, matching Eq. 1. Compare PSNR/SSIM on novel views and directly measure the regression of predicted F(log C(t1) - log C(t0)) against the true ground-truth log-intensity change and against E(t0,t1). If variant (A) systematically under-predicts intensity changes, or if (B)/(C) achieve comparable or better metrics in the noiseless setting, then the decay bias is real and the event supervision is miscalibrated. Report results split by window duration to show whether the bias grows with the decay of older events.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The event supervision in Eq. 6 compares rendered log-intensity differences, F(log C(t1) - log C(t0)), to the accumulated event tensor E(t0,t1). But E(t0,t1) is defined in Eq. 11 with a decay factor b^... where b=0.93, so older events are downweighted. According to the physical event model in Eq. 1, the true log-intensity difference is the undecayed sum of per-event thresholds. Thus Eq. 6 does not enforce the physical event constraint; it enforces a biased target that systematically understates intensity changes when legitimate events are old within the sampled window. The paper acknowledges this in Appx. I: decay 'does still negatively impact the legitimate events' and relies on 'the underlying MLP ... can compensate for that sufficiently,' but no experiment isolates this bias from noise. The existing ablation 'w/o decay' (Tab. 3) is run on noisy real data, where removing decay introduces noise instability, and it even shows slightly higher PSNR (27.119 vs 27.048), so it does not demonstrate that the bias is negligible. Since the central claim is that direct event supervision yields high-quality reconstruction, a miscalibrated event loss weakens the argument that the method is using the physical event signal rather than a high-pass temporal regularizer. This is a load-bearing concern independent of the CRF calibration issue flagged by the reader.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Dynamic EventNeRF, a NeRF-based method for reconstructing general dynamic scenes from sparse multi-view event streams and sparse RGB frames. The method splits a recorded sequence into short overlapping temporal segments, trains a separate time-conditioned MLP NeRF per segment, and supervises each model with an event-accumulation loss, an RGB loss at sparse reference frames, an accumulation loss, a sparsity loss, and a hand-set cylindrical volume clipping. The authors contribute a synthetic benchmark and a real six-view event-RGB dataset recorded in dim lighting, and report state-of-the-art novel-view synthesis, outperforming RGB-only, E2VID/EDI-to-RGB, and deformation-based baselines on both synthetic and real sequences.","tokens_in":22735,"tokens_out":8136,"duration_ms":92151,"significance":"If the central claims hold, this is a meaningful step: it demonstrates that multi-view event streams can drive volumetric reconstruction of general dynamic scenes in conditions where RGB frames suffer from blur and noise, and it provides a public dataset and codebase that should facilitate follow-up work. The paper is thorough in its ablations, covering backbone choices, loss components, number of views, supporting-RGB frame rate, and design choices, and it includes a formal variance-bound argument for the decayed event accumulation in Appx. I. The main caveat is that the event-supervision loss, as defined, does not exactly enforce the physical event-accumulation constraint because of the decay factor; this issue needs to be resolved or convincingly quantified before the central quantitative claims can be taken at face value.","major_comments":[{"comment":"The event target E(t0,t1) in Eq. (11) is a decayed accumulation with b=0.93, so the event loss in Eq. (6) does not enforce the physical ESI constraint from Eq. (1); it matches a biased quantity in which old legitimate events are discounted. The paper explicitly acknowledges in Appx. I that decay \"does still negatively impact the legitimate events\" and relies on the MLP to compensate, but no experiment isolates this bias from noise. The \"w/o decay\" ablation in Table 3 is on a single real sequence, is confounded by noise, and even improves PSNR (27.119 vs 27.048). Since the central claim is that direct event supervision recovers physical intensity changes, the authors should either remove the bias from the supervised target or provide a controlled experiment, e.g., on noise-free synthetic events, that quantifies the distortion and shows it is negligible for the reported improvements.","section":"Sec. 4.4, Eq. (11), and Appx. I"},{"comment":"As printed, the decay exponent in Eq. (11) is iend - istart - i. For the most recent event i=iend this gives the weight b^{-istart}, which depends on the global index of the first event and can exceed one; this is inconsistent with the stated purpose that \"events that are far in the past weigh less\" and is not obviously compatible with the prefix-query formula in Eq. (12). The authors should correct the exponent to a relative index (presumably iend - i) and define the index origin unambiguously. They should also report the event threshold Cp used in Eq. (11), since the scale of the event loss in Eq. (6) depends on it and no value is given.","section":"Sec. 4.4, Eq. (11), and Eq. (12)"},{"comment":"It is not specified how the background is obtained for the held-out views used in evaluation. In Eq. (4), the final image is alpha-blended with the captured background image Ak of a training view. For the real-data evaluation, the held-out view is one of the six cameras, so its background image is available; if it is used during evaluation, the full-image PSNR values in Table 1 do not test novel-view generalization for the background. For the synthetic hold-out views the situation is also unclear. The foreground ROI results mitigate this concern, but the full-image numbers and the 6-DOF novel-view claim require a precise statement of what background is used at test time.","section":"Sec. 4.1, Eq. (4), and Sec. 5.4"}],"minor_comments":[{"comment":"On the real \"Sword\" sequence, the full model is not the PSNR best: w/o decay, w/o Levent, and only Lacc all give higher PSNR than the full model (27.119, 27.620, and 27.754 vs 27.048). The text says SSIM and LPIPS favor the full model, which is true, but the PSNR discrepancy should be stated explicitly and discussed.","section":"Table 3"},{"comment":"The notation \"δ^a a\" in the definition of the event accumulation is malformed and should be rewritten, e.g., as a Kronecker delta with explicit arguments.","section":"Sec. 3.2, Eq. (3)"},{"comment":"The label \"GT RGB [37] + Dyn-NeRF\" is confusing because [37] is the EDI paper; clarify that \"GT RGB\" means the original non-blurry rendered frames and [37] refers to the EDI deblurring used for real data.","section":"Table 1"},{"comment":"The CRF calibration description does not say how many pixels were used for the linear fits, whether the fits were per camera, or what the residual error of the linear approximation is; adding these numbers would make the calibration reproducible.","section":"Appx. B"},{"comment":"The real benchmark evaluates only 3 of the 16 recorded sequences; the authors should state the selection criterion for these sequences and note whether the remaining sequences are excluded for motion, calibration, or other reasons.","section":"Sec. 5.2"}],"recommendation":"major_revision","confidential_remarks":"The paper is a solid systems contribution with a valuable public dataset and extensive ablations. The main technical risk is the event-loss calibration: the decayed accumulation in Eq. (11) makes the supervised target non-physical, and the current evidence does not isolate the resulting bias from noise. If the authors can correct Eq. (11), report the threshold/CRF parameters, and supply a noise-free synthetic ablation of the decay, I would be willing to accept a revised version."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a well-engineered first for multi-view event-based dynamic reconstruction, and the central claim — that direct event supervision beats RGB-only and event-to-RGB baselines — holds up. The paper deserves a real referee.\n\nWhat's new: Dynamic EventNeRF is the first to train time-conditioned NeRF segments from sparse multi-view event streams plus sparse RGB frames, and it ships a six-view real event dataset. The multi-segment cross-fading design is sensible; the ablations are thorough, especially the view-count and RGB-FPS studies. The cleanest comparison, full model vs. the same model with only RGB, shows the event supervision earns its keep (26.99 vs 25.80 PSNR synthetic; 22.51 vs 19.76 ROI real). Code and data are released.\n\nSoft spots, in order: The event loss uses a decayed event sum (b=0.93), so the training target is not the physical log-intensity change but a damped proxy. Appendix I motivates decay as a noise bound, and the ablation shows SSIM/LPIPS improve with decay, so the choice is defensible, but the paper does not quantify how much the bias distorts legitimate events. A sensitivity study on b would settle it. Second, evaluation is on three real sequences with no error bars; the reported margins look plausible but not bulletproof. Third, the closest event-based dynamic NeRF works, DE-NeRF and EvDNeRF, are cited but not compared quantitatively. The settings differ enough to excuse this, but a direct comparison on one sequence would strengthen the positioning. Fourth, cylinder radius and vertical extent are hand-set, and the method fails to converge on real data without clipping; that dependence should be stated as a limitation and sensitivity-tested.\n\nNone of these are fatal. The central contribution — a working multi-view event-based dynamic reconstruction pipeline with benchmark data — is solid and useful. The paper is honest about its limitations and provides formal justification for the one nonstandard design choice. I'd send it to review.","headline":"A solid, well-engineered first for multi-view event-based dynamic reconstruction; the central claim holds, with caveats about decay bias and thin real-data evaluation.","tokens_in":23340,"tokens_out":2409,"would_cite":true,"duration_ms":23742,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Dynamic EventNeRF is the first method that reconstructs general dynamic scenes in 4D from sparse multi-view event streams and a few RGB frames, reporting novel-view quality above RGB-based baselines in fast, dim-lit motion.","keywords":["event cameras","dynamic scene reconstruction","neural radiance fields","novel view synthesis","multi-view event streams","sparse-view reconstruction","event accumulation","4D reconstruction"],"falsifier":"Re-measure the camera response function at a different illumination level (say 50 lx instead of the recorded 7–11.5 lx) and check whether the one-time linear fit with fixed shift $3\\times10^{-2}$ still maps every pixel's brightness to the observed event polarities; if the mapping drifts, the event loss of Eq. 6 is miscalibrated and the reported margin over RGB baselines should shrink or vanish, and the same experiment can be repeated with a subject moving outside the preset cylinder radius to test the general-scenes claim.","tokens_in":22182,"feed_emoji":"⚡","tokens_out":14729,"duration_ms":130385,"temperature":0.7,"pith_summary":"This paper claims that a scene's motion can be reconstructed in 4D from streams of events—per-pixel brightness changes recorded asynchronously—plus a handful of blurry RGB frames, in conditions where ordinary cameras fail: fast motion and dim lighting. The proposed Dynamic EventNeRF trains a short time-conditioned neural radiance field per recording segment, cross-fading neighbouring models so the whole sequence stays continuous, and supervises each segment with event, RGB, accumulation, and sparsity losses. On synthetic and real six-camera recordings, it reports higher novel-view rendering accuracy than RGB-based baselines, including pipelines that first convert events into video, and quality barely drops when RGB support is thinned to one frame every two seconds. If the results hold, multi-view event capture becomes a practical route to high-speed dynamic scene capture where RGB alone blurs or darkens out.","feed_headline":"Event streams beat blurry RGB for 4D dynamic scenes","feed_subtitle":"Events plus a few blurry frames render novel views of fast motion in near-darkness, beating RGB-only pipelines.","key_machinery":"The load-bearing identity is the Event-based Single Integral (ESI): the difference of log-intensities at two times equals the polarity sum of the events in between, $\\log I(t_1)-\\log I(t_0)=E(t_0,t_1)$. Dynamic EventNeRF converts this identity into a training signal by substituting rendered NeRF colours $\\hat{C}_k(t)$ for $I(t)$, producing the event loss of Eq. 6. Around it stand four supporting pieces: (i) a sequence of cross-faded time-conditioned MLPs, $\\hat{C}(t)=(1-\\alpha(t))\\hat{C}_i(t)+\\alpha(t)\\hat{C}_{i+1}(t)$ over 10%-overlap segments, so no single network must hold the whole motion; (ii) fast event accumulation with decay, $E_{x,y}(t_0,t_1)=\\sum_i p_i C_{p_i} b^{i_{\\mathrm{end}}-i_{\\mathrm{start}}-i}$ with $b=0.93$ in Eq. 11, queried per pixel in $O(\\log N)$ time via prefix arrays and binary search; (iii) an annealed sparsity loss plus a manually set cylinder outside which density is clamped to zero, the prior that makes six views converge; and (iv) $\\alpha$-blending of a separately captured background image instead of learning it. The ESI identity is the hook that lets a sensor measuring change supervise a renderer predicting absolute colour.","core_discovery":"The central claim is that general dynamic scenes—arbitrary non-rigid motion, not just faces, bodies, or hands—can be reconstructed from sparse multi-view event streams plus sparse RGB frames, and that supervising a shared volume directly with events beats feeding blurry or event-reconstructed video to RGB-based dynamic NeRFs. Concretely, the paper reports an average synthetic PSNR of 26.99 against 25.80 for the strongest RGB baseline and a real-data foreground-masked PSNR of 22.51 against 19.76 for the EDI-plus-Dyn-NeRF pipeline. To get there, the sequence is split into overlapping short segments, each represented by a time-conditioned MLP radiance field; segments overlap by 10% and are cross-faded, and each one is trained by comparing rendered log-intensities with accumulated events through the Event-based Single Integral identity, anchored by deblurred reference frames through accumulation and RGB losses. An annealed sparsity term, a cylindrical clipping of the reconstruction volume, and a decayed event accumulation with $b=0.93$ (whose drift under noise is shown bounded in the appendix) complete the scheme. The paper also contributes a six-camera event-plus-RGB rig and an 18-minute benchmark of dim-lit, fast motions.","pith_inferences":["The bounded-drift proof for decayed accumulation suggests the same damping trick could make event supervision robust to other noise sources (hot pixels, sensor shake) without retraining—a generalisation the paper does not explore.","The near-independence from RGB implies a cheaper capture rig: commodity low-rate cameras, or even one flash frame per segment, could replace the deblurred 5 FPS stream.","The cross-faded segment curriculum is representation-agnostic; porting it to a faster backbone (the authors point at 3D Gaussian splatting as future work) could cut the 60 GPU-hours per sequence, provided the temporal-sharing failure of grid-based encoders is addressed.","The hand-set cylinder is a ceiling on generality; learning the bounding volume from the event stream itself is a natural next step toward scenes of arbitrary size."],"forward_implications":["Fast, dimly lit scenes that defeat RGB-only capture become reconstructable, because events supply temporal resolution and dynamic range where long exposures blur and darken.","Supervising a shared volume directly with events beats reconstructing video first: E2VID- and EDI-style pipelines introduce per-view inconsistencies and artefacts that lower novel-view quality.","The RGB stream is nearly optional: cutting supporting frames from 100 FPS to 0.5 FPS costs little quality, so 4D capture driven mostly by events is feasible.","Quality rises with the number of event views—from about 16 to 33 PSNR between 2 and 24 synthetic views—so building larger multi-view event rigs directly buys fidelity.","Because no canonical-volume deformation is assumed, the method handles motions that deformation-based dynamic NeRFs cannot, as the failure of the NR-NeRF baseline shows."],"supporting_citations":[{"why":"Supplies the Event-based Single Integral supervision, positive/negative ray sampling, the base rendering implementation, and the synthetic event simulator.","marker":"[49]"},{"why":"Accelerates event accumulation queries via per-pixel prefix arrays and binary search (1–2 ms versus 4–8 s) and deblurs the supporting RGB frames with Fast EDI.","marker":"[21]"},{"why":"Defines the Event-based Double Integral blur model used to deblur training frames and names the EDI-plus-Dyn-NeRF comparison baseline.","marker":"[37]"},{"why":"Provides the E2VID event-to-video baseline whose per-view inconsistencies the method claims to overcome.","marker":"[46]"},{"why":"The NeRF volumetric rendering and positional encoding backbone on which the time-conditioned MLP is built.","marker":"[27]"},{"why":"The deformation-based dynamic NeRF baseline that fails on fast, large motion, supporting the claim that avoiding canonical volumes widens applicability.","marker":"[52]"},{"why":"The FreeNeRF sparse-view RGB baseline used for comparisons on blurry and event-reconstructed video.","marker":"[61]"},{"why":"HexPlane, the grid-based dynamic backbone ablated in the design study; its failure to propagate details through time justifies the MLP choice.","marker":"[4]"},{"why":"NGP, the hash-grid backbone that diverges in the sparse-view setting during ablations.","marker":"[31]"},{"why":"TensoRF-CP, the third ablated backbone whose spatio-temporal grid factorisation underperforms the MLP.","marker":"[7]"}],"fun_headline_variants":["Events beat blurry RGB for fast 4D scenes in near-darkness","Sparse multi-view events reconstruct general dynamic scenes","Event streams unlock dark scene 4D reconstruction","Multi-view events beat RGB for dynamic scenes in darkness","Events enable 4D NeRF from sparse views in low light"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the event camera's response is one fixed linear function with a constant offset—calibrated once per camera and assumed valid for every pixel and every lighting level—so that rendered log-intensities and measured event polarities live on the same scale; a second structural reliance is the hand-set cylinder, since the model fails to converge on real data without that clipping.","fun_headline_variants_meta":{"raw":{"variants":["Events beat blurry RGB for fast 4D scenes in near-darkness","Sparse multi-view events reconstruct general dynamic scenes","Event streams unlock dark scene 4D reconstruction","Multi-view events beat RGB for dynamic scenes in darkness","Events enable 4D NeRF from sparse views in low light"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001138,"raw_usage":{"total_tokens":4776,"prompt_tokens":1045,"completion_tokens":3731,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":661,"completion_tokens_details":{"reasoning_tokens":3649}},"tokens_in":661,"tokens_out":3731,"duration_ms":27575,"temperature":1.0,"reasoning_tokens":3649,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T19:18:53.982178+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-measure the camera response function at a different illumination level (say 50 lx instead of the recorded 7–11.5 lx) and check whether the one-time linear fit with fixed shift $3\\times10^{-2}$ still maps every pixel's brightness to the observed event polarities; if the mapping drifts, the event loss of Eq. 6 is miscalibrated and the reported margin over RGB baselines should shrink or vanish, and the same experiment can be repeated with a subject moving outside the preset cylinder radius to test the general-scenes claim.","supporting_citations":[{"cited_title":"Eventnerf: Neural radiance fields from a single colour event camera","cited_arxiv_id":null,"evidence_quote":"Supplies the Event-based Single Integral supervision, positive/negative ray sampling, the base rendering implementation, and the synthetic event simulator."},{"cited_title":"Fast event-based double integral for real-time robotics","cited_arxiv_id":null,"evidence_quote":"Accelerates event accumulation queries via per-pixel prefix arrays and binary search (1–2 ms versus 4–8 s) and deblurs the supporting RGB frames with Fast EDI."},{"cited_title":"Bringing a blurry frame alive at high frame-rate with an event camera","cited_arxiv_id":null,"evidence_quote":"Defines the Event-based Double Integral blur model used to deblur training frames and names the EDI-plus-Dyn-NeRF comparison baseline."},{"cited_title":"High speed and high dynamic range video with an event camera","cited_arxiv_id":null,"evidence_quote":"Provides the E2VID event-to-video baseline whose per-view inconsistencies the method claims to overcome."},{"cited_title":"Srinivasan, Matthew Tancik, Jonathan T","cited_arxiv_id":null,"evidence_quote":"The NeRF volumetric rendering and positional encoding backbone on which the time-conditioned MLP is built."},{"cited_title":"Non- rigid neural radiance fields: Reconstruction and novel view synthesis of a dynamic scene from monocular video","cited_arxiv_id":null,"evidence_quote":"The deformation-based dynamic NeRF baseline that fails on fast, large motion, supporting the claim that avoiding canonical volumes widens applicability."},{"cited_title":"Freenerf: Im- proving few-shot neural rendering with free frequency regu- larization","cited_arxiv_id":null,"evidence_quote":"The FreeNeRF sparse-view RGB baseline used for comparisons on blurry and event-reconstructed video."},{"cited_title":"Hexplane: A fast representation for dynamic scenes","cited_arxiv_id":null,"evidence_quote":"HexPlane, the grid-based dynamic backbone ablated in the design study; its failure to propagate details through time justifies the MLP choice."},{"cited_title":"Instant neural graphics primitives with a multireso- lution hash encoding","cited_arxiv_id":null,"evidence_quote":"NGP, the hash-grid backbone that diverges in the sparse-view setting during ablations."},{"cited_title":"Tensorf: Tensorial radiance fields","cited_arxiv_id":null,"evidence_quote":"TensoRF-CP, the third ablated backbone whose spatio-temporal grid factorisation underperforms the MLP."}],"review_version":1}