{"id":"4fd1bcfe-36ca-4e2d-b27f-0ff06e80a601","arxiv_id":"2412.06299","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"SaRO-GS models dynamic scenes with 4D Gaussians plus a scale-aware residual field and adaptive per-Gaussian optimization, achieving state-of-the-art PSNR at real-time frame rates on D-NeRF and Plenoptic Video datasets.","lead":"This paper presents SaRO-GS, a method for rendering dynamic 3D scenes in real time by representing the scene as 4D Gaussians with a learned residual field. It reports higher reconstruction quality and faster rendering than prior dynamic-scene methods on standard monocular and multi-view benchmarks.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The CDF approximation in Eq. 21/27 is not the standard normal CDF—Q(0)≈0.632 and Q(t)<0 for t≲−0.6—so the per-Gaussian temporal integral I_i in Eq. 23 is miscalculated for primitives near observation boundaries, undermining the Adaptive Optimization contribution as written.","rationale":"Reader's verdict is CONDITIONAL, and my concern reinforces that rather than changing it, so verdict_should_be is UNCHANGED. I considered the reader's weakest_assumption (single symmetric Gaussian visibility) but do not find it the most load-bearing: disjoint or repeated visibility can be represented by allocating multiple primitives, and the evaluated D-NeRF/Plenoptic scenes primarily contain objects with one appearance interval. The resolution mismatch in Table 1 is real, but Table 2 in the same paper reports matched-resolution numbers (4DGS at 400×400: 35.05 vs Ours 36.13), so the attentive reader already sees the smaller true advantage. The CDF error is different: it is an internal inconsistency in a core equation that determines the entire per-Gaussian optimization schedule. The formula in Eq. 21/27 is not a valid approximation to the standard normal CDF, and it produces pathological I_i values for dynamic primitives whose temporal position is near the start or end of the sequence. Since the ablation (Table 4) attributes 0.69 dB to Adaptive Optimization, a reader cannot trust the described mechanism until this is fixed. The likely correct formula is well-known (Page 1977), so this is probably a typographical error in the camera-ready text; nevertheless, the paper as written is not reproducible. A numerical check of Eq. 21 at t=0 and a Monte Carlo/quadrature comparison of I_i for a boundary-τ Gaussian would settle whether the implementation deviates from the text.","tokens_in":21066,"tokens_out":17881,"duration_ms":154524,"concrete_test":"Numerically evaluate Eq. 21 at t=0 and t=−1 and compare with Φ(0)=0.5 and Φ(−1)≈0.1587; if Q(0)≠0.5, the formula is wrong. Then for τ=0.9, σ=0.05, k=4, compute I_i via Eq. 20 using Eq. 21 and compare with direct quadrature of ∫_0^1 exp(−4((t−0.9)/0.05)^2) dt. If the values differ by more than an order of magnitude, the adaptive schedule in Eq. 23 is not computing the claimed temporal integral. Additionally, inspect the released code (if available) at the I_i computation: if it implements 1/(1+exp(α1 t^3+α2 t)) instead of the printed e^{1+...}, the manuscript needs a correction rather than a method change.","verdict_should_be":"UNCHANGED","load_bearing_attack":"SaRO-GS's Adaptive Optimization (Sec. 4.3) rests on the integral I_i = F(t_end)−F(t_start) of the visibility function γ_i(t)=exp(−k((t−τ_i)/σ_i)^2) (Eqs. 19–20). To compute this cheaply, Eq. 21 approximates the standard normal CDF as Q(t)=1−1/e^{1+α1 t^3+α2 t} with α1=0.070565992, α2=1.5976. This is not the Page approximation: at t=0 it yields 1−1/e≈0.632 instead of 0.5, and for t=−1 it gives ≈−0.95, so Q is not a CDF and can be negative/unbounded for the negative arguments that arise when t−τ_i is negative. The appendix repeats the same formula (Eq. 27). Because Eq. 23 sets the per-Gaussian learning rate lr_i=lr_base·I_max/I_i and densification threshold κ_i=κ_base·I_i/I_max, a Gaussian with τ_i near the start of the observation window (e.g., τ=0.9, σ=0.05, k=4) has u_start=√(2k)(0−0.9)/0.05≈−50.9, making Q(u_start) hugely negative and I_i effectively infinite; such a dynamic primitive would be treated as static (low learning rate, high densification threshold), the opposite of the intended schedule. The correct Page form is 1−1/(1+e^{α1 t^3+α2 t}) for t≥0 with symmetry for t<0. Since Table 4 shows removing Adaptive Optimization costs 0.69 dB (36.13→35.44), the reported contribution is not reproducible from the equations as written. The most plausible resolution is a typo in the manuscript and a correct implementation, but the paper does not say so.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SaRO-GS, a dynamic scene representation based on 4D Gaussian primitives with a scale-aware residual field and a per-primitive adaptive optimization schedule. Each 4D Gaussian carries a temporal position and a lifespan; at query time a Gaussian-like state function and an MLP-decoded residual feature project the primitive into 3D space for rendering with 3D Gaussian Splatting. The scale-aware residual field uses hexplanes with a MipMap stack for spatial-only planes so that feature lookup accounts for the ellipsoidal footprint of each Gaussian. The adaptive schedule scales learning rate and densification threshold by the temporal integral of the visibility function. Experiments on D-NeRF and Plenoptic Video report state-of-the-art PSNR with real-time rendering, and an ablation study attributes gains to each proposed component.","tokens_in":21527,"tokens_out":5045,"duration_ms":51280,"significance":"If the reported results hold, the method is a practically interesting contribution: it combines real-time rendering with explicit handling of object appearance and disappearance, and it provides dynamic-static segmentation as a byproduct. The ablations in Table 4 support the value of the scale-aware field, temporal properties, adaptive optimization, and residual regularization. The paper is also clearly written and the method is plausible. However, two issues presently weaken the central claims: the CDF approximation used in the adaptive optimization is not a valid CDF as written, and the main tables compare methods at different resolutions and over different scene subsets. Both are fixable in revision, but they must be corrected before the state-of-the-art and ablation claims can be accepted as stated.","major_comments":[{"comment":"The approximation Q(t) = 1 - 1/e^{1+alpha1*t^3+alpha2*t} is not the standard normal CDF and is not the Page approximation cited as [35]. At t=0 it gives 1-1/e = 0.632 instead of 0.5, and for negative t it can become negative (for t=-1 it is approximately -4.3). Since Eq. (23) sets lr_i = lr_base * I_max / I_i and kappa_i = kappa_base * I_i / I_max, a Gaussian with tau_i near the observation boundary can yield a negative or hugely negative Q(u_start), making I_i effectively infinite and reversing the intended optimization schedule: such a dynamic primitive would be treated as static. The 0.69 dB gain attributed to adaptive optimization in Table 4 is therefore not reproducible from the equations as written. The correct form of the Page approximation is Q(t) = 1 - 1/(1 + e^{alpha1*t^3 + alpha2*t}) for t >= 0 with the symmetry relation Q(-t) = 1 - Q(t); the authors should correct Eqs. (21) and (27) accordingly and verify that the ablation numbers remain unchanged.","section":"Sec. 4.3, Eq. (21); Appendix B, Eq. (27)"},{"comment":"The evaluation protocol is not consistent across methods. In Table 1, 4DGS is evaluated at 800x800 while Ours and the remaining methods are evaluated at 400x400; Table 2 then shows that 4DGS at 400x400 reaches 35.05 dB, reducing the reported PSNR gap from 2.08 dB to 1.08 dB. In Table 3, the HexPlane average excludes the Coffee Martini scene while the Ours average includes it, as confirmed by the per-scene Table D2, so the two averages are not computed over the same set of scenes. The central state-of-the-art claim should be based on identical resolution and identical scene subsets for all methods, with any differently-configured numbers clearly separated. The additional numbers already present in Table 2 and Table D2 should be used to present a fully consistent comparison.","section":"Tables 1, 2, 3 and D2"}],"minor_comments":[{"comment":"The sentence 'Then, based on Eq. 34, we employ 3DGS to render 3D Gaussians' refers to a nonexistent equation; the rendering equation is Eq. (4) in Sec. 3.1.","section":"Sec. 4.1"},{"comment":"The table footnotes are hard to parse: '1: excludes the Coffee Martini scene' and '2: Only report SSIM instead of MS-SSIM like others' should specify exactly which per-scene values are included in each average and which metric variant was used for each method.","section":"Table 3 and Sec. 6.2"},{"comment":"The caption of Fig. 1 contains garbled characters and should be re-typeset; also, the figure's speed-quality plot would benefit from a legend entry for '*' clarifying that 4DGS was re-measured at 400x400.","section":"Fig. 1 and Sec. 6.1"},{"comment":"In the definition of the scale-aware feature, the subscript in pi_{x,y} should be pi_{i,j} to match the summation over C_so = {(x,y),(x,z),(y,z)}; the current notation is inconsistent.","section":"Eq. (18)"}],"recommendation":"major_revision","confidential_remarks":"The manuscript appears to be the camera-ready version of an ACM MM 2024 paper. If it is being submitted as a journal version, the overlap with the conference publication should be disclosed and the incremental contribution clarified. My recommendation is based on the technical issues above, not on venue fit. The CDF error and the evaluation inconsistencies are load-bearing for the paper's main claims, but both are correctable within a revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is a credible extension of 4DGS. What is genuinely new: a scale-aware residual field that encodes the footprint of each Gaussian in the spatial-only planes via a mipmap stack, and a per-Gaussian adaptive optimization schedule based on the temporal visibility integral. Temporal position and lifespan for 4D Gaussians are handled more cleanly than in earlier deformation-based work. The ablations in Table 4 show each component earns its keep, and the dynamic/static segmentation byproduct is a nice bonus. For someone working on real-time dynamic scene rendering, this is a useful piece of work.\n\nThe soft spots are real but not fatal. First, the evaluation. Table 1 compares 4DGS at 800x800 while evaluating the proposed method at 400x400; when the resolution is matched in Table 2, the D-NeRF gap drops from 2.08 dB to 1.08 dB. Still a lead, but the SOTA claim is overstated. The Plenoptic Video numbers are more straightforward, with a smaller but consistent improvement over Spacetime-GS.\n\nSecond, the stress-test on the CDF approximation is correct. Eq. 21 (and Eq. 27 in the appendix) is not the Page approximation and is not a valid CDF: Q(0) = 1 - 1/e ≈ 0.632 instead of 0.5, and Q goes negative for t below about -0.6. Since the adaptive optimization schedule in Eq. 23 scales learning rate and densification threshold directly with I_i from this approximated integral, the equations as written cannot reproduce the reported 0.69 dB contribution of that component. The most likely explanation is a typo, and the correct Page form is easy to restore, but the manuscript does not acknowledge this, so the contribution is not reproducible from the text. This needs fixing before anyone builds on it.\n\nMinor points: the dynamic/static segmentation is evaluated only qualitatively, and the single symmetric Gaussian state function assumes each primitive appears and disappears around one temporal center, which limits representation of objects reappearing over disjoint intervals. These are limitations, not errors.\n\nWho is this for? Researchers in dynamic scene reconstruction and Gaussian splatting; it is a fair step beyond 4DGS and Spacetime-GS, and the scale-aware encoding is worth knowing. It deserves a serious referee, but the verdict should be conditional: matched-resolution comparisons and a corrected CDF approximation are required. I would not cite it in its current form, and my reading group would enjoy debating the evaluation inconsistency and the CDF fix.","headline":"Solid incremental 4DGS extension with real-time gains, but the headline numbers come from an unfair comparison and the adaptive optimization equation is wrong as written.","tokens_in":22084,"tokens_out":3980,"would_cite":false,"duration_ms":35121,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"SaRO-GS reconstructs temporally complex dynamic scenes in real time by giving each 4D Gaussian its own lifespan and a scale-aware residual field.","keywords":["4D Gaussian Splatting","dynamic scene reconstruction","real-time rendering","scale-aware residual field","adaptive optimization","temporal complexity","neural radiance fields","view synthesis"],"falsifier":"Render a synthetic scene where an object disappears and later reappears in the same place with the same appearance. If the single-peak state function is the limiting factor, the reappearance should show ghosting or blur, and the per-scene PSNR should drop well below the paper's reported D-NeRF average unless the optimizer happens to split the object into multiple primitives with different temporal centers.","tokens_in":20864,"feed_emoji":"⚡","tokens_out":5808,"duration_ms":52837,"temperature":0.7,"pith_summary":"The paper claims that temporally complex dynamic scenes—objects moving boldly, appearing, and disappearing—can be reconstructed in real time with quality that surpasses prior methods by representing the scene as a set of 4D Gaussian primitives, each carrying its own temporal position and lifespan. To keep the representation accurate as primitives split and shrink, it adds a Scale-aware Residual Field that encodes each Gaussian's spatial extent, not just its center. An Adaptive Optimization Schedule tunes the learning rate and densification threshold per primitive according to how long it is visible, so short-lived dynamic primitives are not starved of gradient signal. On the D-NeRF monocular benchmark the method reports 36.13 dB PSNR at 182 FPS (400x400), and on the Plenoptic Video multi-view benchmark 32.15 dB PSNR at 40 FPS, surpassing NeRF-based and prior 4D Gaussian baselines in both quality and speed.","feed_headline":"Dynamic scenes rendered at 182 FPS with top quality","feed_subtitle":"Scale-aware residuals and per-Gaussian schedules beat NeRF and prior 4DGS baselines on both quality and speed.","key_machinery":"The machinery has four interlocking parts. Each 4D Gaussian owns a 4D position \\((x,y,z,\\tau_i)\\), a lifespan \\(\\sigma_i\\), and a state function \\(\\gamma_i(t) = \\exp\\left(-4\\left(\\frac{t-\\tau_i}{\\sigma_i}\\right)^2\\right)\\), which drives the projected 3D opacity and doubles as the primitive's temporal sampling probability. The Scale-aware Residual Field is a hexplane field whose spatial-only planes are MipMap stacks: a Gaussian's projected ellipse selects the level \\(l = \\min\\left(\\log_2(s_x/\\hat{s}^0_x), \\log_2(s_y/\\hat{s}^0_y)\\right)\\) for trilinear interpolation, aligning residual features with the primitive's actual extent and its self-splitting behavior. A small MLP decodes the residual feature into per-time residuals of position, covariance, and color, plus the lifespan. The Adaptive Optimization Schedule approximates each primitive's time-domain integral \\(I_i\\) with the logistic CDF and sets the densification threshold \\(\\kappa_i = \\kappa_{\\text{base}} \\cdot I_i / I_{\\max}\\) and learning rate \\(\\text{lr}_i = \\text{lr}_{\\text{base}} \\cdot I_{\\max} / I_i\\), so dynamic primitives with short lifespans are densified more easily and optimized more aggressively.","core_discovery":"In the paper's own terms, the central discovery is that giving each 4D Gaussian an explicit temporal center \\(\\tau_i\\) and lifespan \\(\\sigma_i\\), and making its visibility a symmetric Gaussian function of time, is enough to represent object appearance and disappearance without any deformation field. The Scale-aware Residual Field makes the encoding scale-aware by sampling the spatial-only hexplanes at a MipMap level matched to each Gaussian's projected footprint, so split children inherit features similar to their parent. Finally, the paper shows that optimization can be balanced per primitive: integrating each Gaussian's visibility over the observed time range yields a sampling probability that scales its densification threshold and learning rate. Together these mechanisms let the method render temporally complex scenes in real time while improving reconstruction quality over prior work.","pith_inferences":["The per-primitive lifespan could be reused as a handle for temporal editing, such as deleting an object from the whole sequence or shifting when it appears, without re-training the model.","The single-peak Gaussian visibility is the main constraint: an object that disappears and reappears at the same location would need several Gaussians or a learned multi-peak state function to be captured cleanly.","The MipMap-level trick for sampling grid features by primitive footprint is general and could be transferred to other grid-based fields whose samples are regions rather than points.","A direct stress test would be a scene with periodic or repeated appearances (for example, a rotating fan blade or a blinking light): quality should degrade relative to scenes with a single appearance event, tracing exactly to the single-peak assumption."],"forward_implications":["Temporal appearance and disappearance no longer require a deformation field or a canonical frame; each primitive simply switches on and off around its own learned lifespan.","Because lifespans are learned per primitive, the model can segment dynamic from static parts of a scene with no external supervision.","The method works for both monocular and multi-view inputs while preserving real-time rendering in both settings.","Reported frame rates are roughly two orders of magnitude faster than NeRF-based methods on the same benchmarks while improving PSNR."],"supporting_citations":[{"why":"Supplies the fast differentiable rasterizer, densification control, and opacity reset that the 4D Gaussian representation renders and optimizes with.","marker":"[22]"},{"why":"Contributes the hexplane decomposition into spatial-only and spatiotemporal planes that the Scale-aware Residual Field builds on.","marker":"[7]"},{"why":"The 4D-deformation Gaussian baseline whose handling of appearance and disappearance the paper targets and outperforms; its per-scene numbers are the main comparison.","marker":"[61]"},{"why":"Provides the Gaussian-like state function for primitive visibility over time and serves as a real-time multi-view baseline.","marker":"[27]"},{"why":"Defines the monocular synthetic D-NeRF benchmark used to evaluate temporally complex dynamic scene reconstruction.","marker":"[42]"},{"why":"Defines the multi-view real-world Plenoptic Video benchmark used to evaluate rendering quality and speed in complex scenes.","marker":"[26]"}],"fun_headline_variants":["SaRO-GS: 4D splats render complex motion at 182 FPS","Scale-aware residuals and adaptive schedules hit 182 FPS","4D Gaussian splatting without deformation fields, at 182 FPS","Per-primitive optimization speeds 4DGS to real-time dynamic rendering"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Every Gaussian's visibility over time is a single symmetric bump with a fixed shape, so one primitive cannot represent objects that appear abruptly, vanish, or show up in disjoint time intervals.","fun_headline_variants_meta":{"raw":{"variants":["SaRO-GS: 4D splats render complex motion at 182 FPS","Scale-aware residuals and adaptive schedules hit 182 FPS","4D Gaussian splatting without deformation fields, at 182 FPS","Per-primitive optimization speeds 4DGS to real-time dynamic rendering"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000243,"raw_usage":{"total_tokens":1517,"prompt_tokens":925,"completion_tokens":592,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":541,"completion_tokens_details":{"reasoning_tokens":510}},"tokens_in":541,"tokens_out":592,"duration_ms":6276,"temperature":1.0,"reasoning_tokens":510,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T19:47:47.765077+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Render a synthetic scene where an object disappears and later reappears in the same place with the same appearance. If the single-peak state function is the limiting factor, the reappearance should show ghosting or blur, and the per-scene PSNR should drop well below the paper's reported D-NeRF average unless the optimizer happens to split the object into multiple primitives with different temporal centers.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the multi-view real-world Plenoptic Video benchmark used to evaluate rendering quality and speed in complex scenes."}],"review_version":1}