{"id":"d01c7472-e5c1-460d-9438-79569972e729","arxiv_id":"2411.16180","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Event cameras and deformable 3D Gaussians are combined with jointly optimized thresholds and dynamic-static decomposition, achieving state-of-the-art dynamic scene reconstruction on a new benchmark.","lead":"This paper combines event cameras, which capture motion at microsecond resolution, with deformable 3D Gaussian Splatting to reconstruct dynamic scenes from sparse RGB frames and continuous events. It introduces a joint optimization of event thresholds and scene Gaussians plus a dynamic-static decomposition, reporting large quality gains on a new 8-scene synthetic and 4-scene real benchmark.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"GTJM's per-pixel, per-time-bin threshold maps can in principle make the event loss a recoding of RGB supervision; without regularization or an event-corruption test, the central 2.73 dB event gain is not yet established as coming from independent event information.","rationale":"The reader's weakest_assumption identifies the same mechanism, and I agree. My formulation adds the specific degeneracy: because C_hat is unregularized and per-interval, Eqs. (7)-(10) admit a solution in which L_event is satisfied by copying RGB/pseudo-frame changes. This makes the central event-boost claim conditional on an identifiability check, not on the reported MSE alone. The synthetic threshold validation mitigates but does not eliminate the concern, since it is only on training intervals and the same C_hat participates in GS optimization. I see no other concern as load-bearing: the DSD speed-up is internally consistent, the baselines are reasonable except DE-NeRF is not compared, and missing code/data weakens reproducibility but is not a mathematical flaw. I recommend keeping the CONDITIONAL verdict: the paper's argument is plausible and the synthetic results are encouraging, but the event signal should be demonstrated to be genuinely informative. The proposed corruption experiment is the minimal test that would settle this.","tokens_in":12834,"tokens_out":6331,"duration_ms":105305,"concrete_test":"On the synthetic dataset, run the full pipeline twice with identical RGB training frames but different event streams: (A) the original ESIM events, and (B) events with timestamps/polarity randomly shuffled within each RGB interval. If held-out-view PSNR of (B) is within about 0.5 dB of (A), the GTJM threshold maps are absorbing the corruption and the event loss is not an independent signal; additionally, compute the optimized C_hat against ESIM ground-truth thresholds on held-out time bins to check whether C_hat survives outside the RGB-supervised intervals. A substantial PSNR drop in (B) plus stable threshold MSE would clear the concern.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that event streams supply independent inter-frame motion constraints beyond RGB. The GTJM module makes this attribution difficult to audit. In Eqs. (7)-(10), the learnable threshold C_hat_{t,f} in R^{B x P x H x W} is a per-pixel, per-polarity, per-time-bin scalar map. It is optimized jointly with the 3D-GS using L_thres (RGB brightness differences), L_event (differences involving images rendered by the very 3D-GS being trained), and L_rgb. Nothing in the paper constrains C_hat to be smooth, positive, or shared across time bins. A trivial solution exists: for each interval, set C_hat so that the accumulated event count maps reproduce the RGB/pseudo-frame log-brightness changes. Then L_event reduces to another photometric consistency term between rendered and GT/pseudo-frame images, and the 'event' loss contributes no independent temporal geometry; its reported benefit would be a form of RGB overfitting through the high-capacity threshold maps. Threshold accuracy is validated only on ESIM synthetic data (Table 1), where GT thresholds are known; the real-world benchmark has no GT threshold, so the same audit cannot be performed. Without an experiment that breaks the link between event content and RGB supervision, the 2.73 dB gain over Event-4DGS cannot be attributed to event information rather than to extra parameters in C_hat.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Event-boosted Deformable 3D Gaussians for dynamic scene reconstruction, integrating event camera streams into deformable 3D Gaussian Splatting. The main components are a GS-threshold joint modeling (GTJM) strategy that optimizes per-pixel, per-polarity, per-time-bin event threshold maps together with the 3D Gaussians, and a dynamic-static decomposition (DSD) strategy that separates static and dynamic Gaussians to improve rendering speed and dynamic-region fidelity. The authors also introduce a new event-inclusive 4D benchmark with eight synthetic and four real-world scenes. Reported results show an average 2.73 dB PSNR gain over their Event-4DGS baseline on synthetic scenes and an average 1.71x speedup over 4D-GS, alongside consistent gains on real-world scenes.","tokens_in":13225,"tokens_out":4959,"duration_ms":51720,"significance":"If the central attribution claim holds, this is a meaningful contribution: it is the first work to bring event supervision into deformable 3D Gaussian Splatting, it demonstrates that threshold modeling matters for event-based 3D reconstruction, and the proposed benchmark could be a useful community resource. The DSD component is a clean idea with a clear speed-quality trade-off, and the ablation results in Table 4 and sensitivity analysis in Fig. 12 are consistent. However, the paper's central claim that event streams provide independent inter-frame geometric information is not yet established, because the learnable threshold maps are fitted with RGB and rendered-image supervision and are then used inside the event loss. The decisive issue is auditability: without a control that breaks the link between event content and RGB supervision, the reported event-boosted gains could be explained by RGB or pseudo-frame information recoded through high-capacity threshold maps. This concern is concrete and testable, and the current paper does not provide the needed experiment.","major_comments":[{"comment":"The central claim that event streams provide independent motion information is not yet supported, because the learned threshold maps C_hat_{t,f} in R^{B x P x H x W} are per-pixel, per-polarity, per-time-bin scalars optimized jointly with the 3D Gaussians using L_thres on RGB brightness differences and L_event on images rendered by the very Gaussians being trained. With millions of unregularized threshold parameters, the event loss in Eq. (4) can in principle be driven down by encoding RGB/pseudo-frame log-brightness changes into C_hat rather than by exploiting the temporal geometry of events. The 2.73 dB gain over Event-4DGS reported in Table 2 is therefore not yet attributable to event information. I ask the authors to (i) state explicitly whether and how C_hat appears in Levent in Eq. (4), and (ii) add at least one control experiment that breaks the event-RGB link, such as training with shuffled or randomly relabeled events of the same statistics, training with events from a different scene, or restricting C_hat to a smooth low-dimensional parameterization. Without such a control, the attribution of the reported gain to events remains unresolved.","section":"Sec. 3.2, Eqs. (7)-(10); Tables 1, 4"},{"comment":"Threshold accuracy is validated only on ESIM synthetic sequences where ground-truth thresholds are available (Table 1); for the four real-world scenes no ground-truth threshold exists, so the same audit cannot be performed. The real-world PSNR gains in Table 3 could therefore still reflect RGB-recoding through C_hat rather than independent event supervision. Please add a real-world proxy or a controlled experiment, for example: (i) evaluate on a scene with a static camera and pure event noise, where a correctly functioning event loss should not improve rendering; (ii) ablate event supervision by dropping or randomly corrupting event counts while keeping RGB identical; or (iii) measure how much of L_event is minimized by threshold map fitting alone relative to the contribution from improved Gaussians. This would make the central claim testable on the actual benchmark rather than only on synthetic data.","section":"Sec. 4.1, Sec. 4.2, Table 1, Table 3"}],"minor_comments":[{"comment":"The notation E(t, t+Delta t) is not fully specified. Eq. (2) includes a threshold C, but Eq. (4) omits it; please state whether Levent uses the learned threshold map C_hat, a fixed constant threshold, or raw event counts, and clarify the relationship between Eqs. (2), (3), and (4).","section":"Eq. (4)"},{"comment":"The row/column structure of Table 1 is hard to parse. Please define \"Stage1\" and \"Stage2\" explicitly and state which quantities are frozen or fine-tuned in each cell, so that the mutual-boosting claims can be checked against the exact optimization protocols.","section":"Table 1"},{"comment":"The real-world capture setup would benefit from a reported synchronization accuracy between the event camera and the frame camera, and from any calibration between the two sensors; this is important for assessing how much event-frame misalignment could affect the results.","section":"Sec. 4.1"},{"comment":"The average improvements are computed over the listed scenes, but no standard deviation or number of runs is reported. Since dynamic-scene optimization can be sensitive to initialization and hyperparameters, reporting repeated-seed statistics would strengthen the comparisons.","section":"Sec. 4.2 / Tables 2, 3"},{"comment":"The benchmark is promised for public release but is not available at the time of review. Please state whether the code and data are available and, if so, provide an anonymized link or a precise release plan; this is important for reproducibility of the claimed benchmark contribution.","section":"Sec. 4.1"}],"recommendation":"major_revision","confidential_remarks":"The stress-test concern is on target. The manuscript is well written and the method is coherent, but the core claim that event streams supply independent geometric information is not yet auditable because of the fitted threshold-map cycle. The required controls are feasible within the paper's scope and should be added before acceptance. I do not see this as a fatal flaw: the approach is plausible and the ablations are consistent, but the current evidence is insufficient to rule out RGB recoding."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Wenhao Xu et al. combine event cameras with deformable 3D Gaussian splatting and add a learned per-pixel threshold map. That combination is new, and the dynamic-static decomposition via buffer-based soft assignment is a sensible way to cut deformation cost. The new synthetic and real-world event-inclusive 4D benchmark is the kind of resource the field needs. Reported gains over the Event-4DGS baseline are large (about 2.7 dB PSNR on synthetic), and the ablations are consistent.\n\nThe strongest part is the threshold modeling. Rather than assume a constant threshold, they fit a per-polarity, per-pixel, per-time-bin threshold with supervision from RGB brightness changes and then jointly optimize it with the Gaussians. On synthetic data where the true threshold is known, they show their learned threshold has low MSE. That is real evidence the mechanism works.\n\nThe soft spots are about attribution and reproducibility. First, the learned threshold map is high-capacity and unregularized. Nothing stops it from absorbing the RGB signal and making the event loss a recoding of photometric consistency. The synthetic threshold-MSE check mitigates this, but on real scenes there is no ground truth, so the 2.73 dB gain could partly be extra parameters rather than independent event information. A clean experiment that corrupts or randomly shuffles events, or ablates the event loss while keeping threshold capacity, would settle this. Second, the paper does not compare with existing event-based 3D-GS methods (Ev-GS, Event3DGS, E2GS) because they are for static or deblurring; that is a fair excuse, but a simple adaptation would strengthen the SOTA claim. Third, no error bars, and code/data are promised but not yet released.\n\nMinor: the DSD decomposition depends on VGG cosine similarity and Otsu; the sensitivity analysis only covers buffer size, not the VGG threshold. The blur extension is only one scene.\n\nOverall, this is a serious piece of work. The core pipeline is sensible, the benchmark is useful, and the synthetic threshold validation is a point in its favor. The main unresolved question—whether events are truly adding independent temporal geometry—is not a fatal flaw; it is a missing control. I would send it to peer review and ask for that control, plus code/data release.\n\nFor a reader: someone working on event-based vision or dynamic Gaussian splatting will find this worth reading. I'd bring it to a reading group if the code is out.","headline":"Solid engineering with a real new benchmark; the event-gain attribution needs one clean experiment before the 2.73 dB claim is fully credible.","tokens_in":13693,"tokens_out":3423,"would_cite":true,"duration_ms":62045,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Event streams lift dynamic 3D Gaussian rendering by 2.73 dB","keywords":["event camera","3D Gaussian splatting","dynamic scene reconstruction","threshold modeling","dynamic-static decomposition","event-inclusive 4D benchmark","novel view synthesis"],"falsifier":"Use the same pipeline on a real event camera with independently calibrated per-pixel, per-polarity thresholds, and compare reconstruction PSNR when the scene is trained with those fixed calibrated thresholds versus with the learned maps; if the fixed-calibrated version matches or exceeds the learned version, the paper's central mechanism is not adding independent motion information.","tokens_in":12681,"feed_emoji":"🎥","tokens_out":3623,"duration_ms":32583,"temperature":0.7,"pith_summary":"Deformable 3D Gaussian Splatting reconstructs dynamic scenes from sparse RGB frames, but misses motion between frames. This paper argues that event cameras -- sensors that fire per-pixel brightness-change pulses at microsecond resolution -- supply exactly that missing motion information, and that using them well requires modeling the event threshold, which varies across polarity, space, and time. The paper introduces a GS-threshold joint modeling strategy, in which threshold maps and the 3D Gaussian scene are optimized together so each improves the other, and a dynamic-static decomposition that renders static regions without deformation. On the first event-inclusive 4D benchmark, with eight synthetic and four real-world scenes, the method outperforms an event-extended baseline by 2.73 dB PSNR on average and runs 1.71 times faster than 4D-GS.","feed_headline":"Event cameras lift dynamic 3D rendering by 2.73 dB","feed_subtitle":"Joint threshold modeling and static/dynamic split make Gaussian splatting sharper and faster on a new 4D benchmark.","key_machinery":"The central object is the learnable threshold map $\\hat{C}_{t,f}\\in\\mathbb{R}^{B\\times P\\times H\\times W}$, one scalar per time bin, polarity, and pixel, used to convert accumulated event counts into a predicted log-brightness change. It is optimized with a threshold loss against RGB frame brightness differences in Eq. (7) and a rendering loss against pseudo-frames rendered by the same 3D Gaussian scene in Eq. (4), and the scene, in turn, is supervised by the threshold-weighted events. The other mechanism is the dynamic-static decomposition: rendered-versus-ground-truth VGG cosine-similarity maps, binarized by Otsu's method, are unprojected to 3D, and Gaussians inside radius $r_1$ are marked dynamic, beyond $r_2$ static, with a pruned buffer zone in between.","core_discovery":"The paper's central claim is that event supervision, properly handled, is the missing ingredient for high-fidelity dynamic scene reconstruction with 3D Gaussian Splatting. With a constant event threshold, opposing-polarity events cancel imperfectly and produce trajectory artifacts and colored haze; the paper shows that optimizing per-pixel, per-polarity, per-time-bin threshold maps jointly with the Gaussians removes these artifacts and improves reconstruction. A second claim is that modeling the whole scene as deformable wastes capacity: by first reconstructing with static Gaussians only, the method locates regions the static model cannot represent, extends the 2D mask to 3D via unprojection, and applies a buffer-based soft decomposition so only genuinely dynamic Gaussians pass through the deformation field. The result is both higher fidelity and faster rendering.","pith_inferences":["A direct test of whether the threshold maps add independent geometric information would be a real event camera with independently calibrated per-pixel thresholds; if substituting those calibrated thresholds matches or beats the learned maps, the gain may come mainly from fitting RGB brightness rather than from new motion constraints.","The same joint-threshold recipe should transfer to event-based deblurring of Gaussian scenes, since motion blur and threshold variation both corrupt event integration.","For high-speed robot egomotion, where RGB frames are extremely sparse, the dynamic-static split could let a deformation field focus on moving agents while static background is rendered at full speed.","If threshold maps transfer across scenes captured with the same sensor, they could be pretrained, reducing the per-scene optimization burden."],"forward_implications":["Events can substitute for dense RGB sampling in dynamic scenes, recovering inter-frame motion without raising camera frame rate.","Threshold non-uniformity is not a nuisance to ignore; modeling it is necessary for clean event-based supervision of Gaussian scenes.","Splitting the scene into dynamic and static Gaussians accelerates rendering and improves dynamic-region fidelity by not wasting deformation capacity.","The new event-inclusive 4D benchmark (eight synthetic, four real-world scenes) gives the community a standard testbed for event-based dynamic reconstruction.","Joint optimization of threshold and scene is mutually reinforcing: better thresholds give better event supervision, and better scenes give better pseudo-frames for threshold refinement."],"supporting_citations":[{"why":"Supplies the base 3D Gaussian Splatting representation and differentiable rasterizer on which everything else is built.","marker":"[17]"},{"why":"Supplies the deformation field and the RGB-only baseline that Event-4DGS extends with event rendering.","marker":"[43]"},{"why":"A dynamic-scene Gaussian baseline used for quality and speed comparison and for motivating the decomposition.","marker":"[39]"},{"why":"Prior event+RGB NeRF method that assumes a constant threshold, motivating the threshold-modeling contribution.","marker":"[4]"},{"why":"Closest prior work combining RGB and events for dynamic scenes; provides dataset context and motivates the new benchmark.","marker":"[25]"},{"why":"ESIM generates the synthetic event streams used to build the new benchmark.","marker":"[31]"},{"why":"E-NeRF establishes the event generation model and discusses threshold variation affecting event supervision.","marker":"[18]"}],"fun_headline_variants":["Event cameras sharpen dynamic 3D scenes","Event-boosted Gaussians capture motion faster","Event data makes dynamic 3D renders sharper and faster","Joint threshold modeling upgrades dynamic Gaussian splatting"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that per-pixel, per-polarity, per-time-bin learnable thresholds, trained against RGB brightness changes and pseudo-frames rendered by the very Gaussian scene being optimized, capture real threshold variation rather than merely re-encoding the RGB supervision.","fun_headline_variants_meta":{"raw":{"variants":["Event cameras sharpen dynamic 3D scenes","Event-boosted Gaussians capture motion faster","Event data makes dynamic 3D renders sharper and faster","Joint threshold modeling upgrades dynamic Gaussian splatting"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000556,"raw_usage":{"total_tokens":2620,"prompt_tokens":892,"completion_tokens":1728,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":508,"completion_tokens_details":{"reasoning_tokens":1668}},"tokens_in":508,"tokens_out":1728,"duration_ms":12517,"temperature":1.0,"reasoning_tokens":1668,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T13:24:32.505820+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Use the same pipeline on a real event camera with independently calibrated per-pixel, per-polarity thresholds, and compare reconstruction PSNR when the scene is trained with those fixed calibrated thresholds versus with the learned maps; if the fixed-calibrated version matches or exceeds the learned version, the paper's central mechanism is not adding independent motion information.","supporting_citations":[{"cited_title":"Deformable 3d gaussians for high- fidelity monocular dynamic scene reconstruction","cited_arxiv_id":null,"evidence_quote":"Supplies the deformation field and the RGB-only baseline that Event-4DGS extends with event rendering."},{"cited_title":"4d gaussian splatting for real-time dynamic scene rendering","cited_arxiv_id":null,"evidence_quote":"A dynamic-scene Gaussian baseline used for quality and speed comparison and for motivating the decomposition."},{"cited_title":"Mitigating motion blur in neural radiance fields with events and frames","cited_arxiv_id":null,"evidence_quote":"Prior event+RGB NeRF method that assumes a constant threshold, motivating the threshold-modeling contribution."},{"cited_title":"Esim: an open event camera simulator","cited_arxiv_id":null,"evidence_quote":"ESIM generates the synthetic event streams used to build the new benchmark."},{"cited_title":"E-nerf: Neural radiance fields from a mov- ing event camera","cited_arxiv_id":null,"evidence_quote":"E-NeRF establishes the event generation model and discusses threshold variation affecting event supervision."}],"review_version":1}