{"id":"ea27642c-9546-417d-b04e-f10dd590fef3","arxiv_id":"2412.04457","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A comprehensive benchmark shows monocular dynamic Gaussian splatting methods are fast and brittle, with scene complexity dominating method differences.","lead":"This paper benchmarks many Gaussian splatting methods for turning monocular videos into 3D scenes, and finds they are fast but fragile, with real-world scene complexity swamping differences between methods. It also introduces a controlled synthetic dataset to isolate how camera motion and object motion affect reconstruction quality.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reimplementation parity for the Gaussian methods is not demonstrated; if EffGS/STG/4DGS/RTGS/DeformableGS are unfaithful ports, all rank-order findings could invert.","rationale":"The reader's weakest_assumption correctly identifies reimplementation fidelity as the main risk, and I agree with that identification. The paper's own text gives concrete reasons to worry: STG was not built for monocular input and required a substantial modification (App C.3); the authors removed STG's color decoder to make comparison fairer (Sec 3.3), and then evaluate it on datasets it was never designed for and that its original paper never reported. Removing the decoder is a deliberate ablation of the original method, which is legitimate only if its effect is quantified either by a parity check including the decoder or by an ablation. No such quantification is given. Similarly, the paper says EffGS originally uses an optical-flow-based loss for multi-view datasets that is omitted here (App C.3); that omission could systematically disadvantage EffGS on datasets with valid flow supervision. The paper's own statement that 'our implementations approximately match the results of the original works' (Sec 4) is the only evidence for parity, and the term 'approximately' is never defined. Because the whole paper is a ranking study, an undefined approximation is the load-bearing joint of the argument. The concrete test proposed is straightforward and standard: run original code on an overlapping configuration and report deltas. If all deltas are small, the findings stand and the paper is a strong contribution. If any delta is large, the corresponding rank-order claim must be re-examined or the method dropped from the headline comparisons. This is consistent with the reader's CONDITIONAL verdict; I do not see grounds to move to REJECT because the concern is addressable and the descriptive findings are valuable.","tokens_in":24601,"tokens_out":1846,"duration_ms":17335,"concrete_test":"For each reimplemented method, run the original authors' code on at least one dataset and one metric configuration that the original paper reported (e.g., DeformableGS on D-NeRF, 4DGS on D-NeRF, EffGS on D-NeRF, RTGS on D-NeRF). Report the PSNR/LPIPS delta between the original code and the reimplementation with identical data split and metric code. If any method shows a delta exceeding ~1 dB (or equivalent LPIPS delta), publish the comparison and explain the discrepancy; if parity holds for all five methods, the concern is resolved.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central argument is that dynamic Gaussian methods are 'fast and brittle, and scene complexity rules' (Table 2, Fig 5). The strongest support is the full-slate comparison; the weakest assumption is that the authors' reimplementations in a shared codebase 'approximately match the results of the original works' (Sec 4, first paragraph). This is asserted without any parity table. STG is particularly exposed: the paper states STG was originally designed for multi-view input only and that the authors had to modify it for monocular input by copying points 10x along the time axis (Sec 3.3, App C.3). That adaptation could easily disadvantage STG relative to methods that were natively monocular. The claimed effect is large (e.g., Table 2: STG 19.51 PSNR vs DeformableGS 24.07; TiNeuVox 24.54), so even a 2-3 dB implementation gap could change the qualitative story. The paper's own statements that most dynamic Gaussian methods do not report on all datasets, that STG was never evaluated on any of these datasets, and that 'the comparisons made possible by this full-slate evaluation are of special interest' (Sec 4) all raise the stakes: a novel comparison must carry the burden of proving the methods are being compared as their authors intended, and no such evidence is provided.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript benchmarks monocular dynamic view synthesis under Gaussian splatting. The authors collect 50 real-world scenes across D-NeRF, Nerfies, HyperNeRF, NeRF-DS, and iPhone, add a new synthetic dataset that varies camera baseline and object motion, and integrate five Gaussian methods (EffGS, STG, DeformableGS, 4DGS, RTGS) together with static 3DGS and TiNeuVox baselines into a shared codebase. They report three optimization runs per condition, standard deviations, and masked dynamic-region metrics. The paper's central thesis is that monocular dynamic Gaussian splatting is fast but brittle, that scene complexity and dataset factors overwhelm motion-representation differences, and that hybrid voxel/neural-field methods such as TiNeuVox remain competitive in quality. Eight findings summarize the evidence.","tokens_in":24901,"tokens_out":10598,"duration_ms":96685,"significance":"If the findings are correct, this is a useful and timely empirical contribution. Its strongest assets are the full-slate comparison on datasets where most original papers reported only subsets (Table 8), the controlled synthetic dataset with baseline/motion ablations, the use of three seeds and standard deviations, the masked static/dynamic metrics, and the public release of code and data. The paper is also appropriately candid about its limitations, including the lack of a method-independent scene-difficulty metric and the omission of concurrent works. The main risk to significance is reimplementation fidelity: the rank-order conclusions are only as trustworthy as the authors' integrated implementations, and that is not currently demonstrated.","major_comments":[{"comment":"The central assumption stated in Section 4, first paragraph, is that the integrated implementations 'approximately match the results of the original works', but the paper provides no parity table or quantitative evidence for this claim. This is especially consequential for STG: Section 3.3 and Appendix C.3 explain that STG was originally designed for multi-view input and had to be adapted for monocular input by copying the point cloud 10x along the time axis, and Table 2 reports a 4.56 dB gap between STG and DeformableGS (19.51 vs. 24.07). Because the qualitative conclusions are rank-order claims, even a 2-3 dB implementation gap on one method could change the story (compare TiNeuVox 24.54 and DeformableGS 24.07 in the same table). Please add a per-method, per-dataset parity check against the original papers' reported numbers on the scenes and splits they used, and justify or clearly separate the STG modifications and the STG/STG-decoder distinction.","section":"Sec. 4, first paragraph; Sec. 3.3; App. C.3; Table 2"},{"comment":"Section 4.4 states that collapsed optimization runs are excluded from the evaluation statistics, but the number, frequency, and location of these failures are never reported. This is load-bearing because Table 2 and Figure 5 are the empirical basis for both the quality rankings and the 'brittleness' claim. If some methods collapse more often, their reported means are optimistic and the comparison is not apples-to-apples. Please report failure counts per method and dataset, and provide a sensitivity analysis that either includes failed runs with a worst-case score (as already done for STG in Figure 14) or analyzes successful and failed runs separately.","section":"Sec. 4.4; Table 2; Fig. 14"},{"comment":"The evaluation protocol for HyperNeRF is inconsistent between the main text and the appendix. Section 3.1 says original train/test splits are used, but Appendix B.2 reports that HyperNeRF has known bad camera poses, that the authors reran COLMAP with dynamic and specular regions masked, and that poses were improved in 7 of 17 scenes; Appendix C.4 says the Dynamic 3D Gaussian implementation uses the improved poses. Table 9 shows that pose correction improves some methods but degrades TiNeuVox, DeformableGS, and 4DGS. The main results in Table 2 and Figure 5 should state explicitly which pose set is used for HyperNeRF, and the rank-order analysis should be shown under both original and corrected poses so that cross-dataset findings do not mix protocols.","section":"Sec. 3.1; App. B.2; App. C.4; Table 9"},{"comment":"Table 2 is the basis for Finding 2, but its aggregation rule is not specified. The appendix tables show that the five datasets have different sizes (D-NeRF 8 scenes, Nerfies 4, HyperNeRF 17, NeRF-DS 7, iPhone 14) and different winning methods across datasets. If Table 2 is an unweighted mean of dataset means, small datasets receive equal weight; if it is a scene-level mean, iPhone dominates. Since per-dataset winners differ (Figure 5), the aggregate rank order can be an artifact of the weighting choice. Please report the aggregation procedure and show whether Finding 2's ordering (field-based better than per-Gaussian, 4D worst) is stable under both conventions and across metrics.","section":"Table 2; Tables 3-7; Finding 2"},{"comment":"Finding 2's conclusion that 'going to 4D makes things worse' is based on a single representative of the 4D representation class, RTGS. Table 1 lists Rotor-Based 4DGS under the same '4D' type, and it is not evaluated. With only one 4D method, and one that the authors modified to fit the monocular setup, the claim about 4D representations generally is too strong. Please either include a second 4D method or restrict the conclusion to RTGS.","section":"Sec. 4.2; Table 1; Table 2"}],"minor_comments":[{"comment":"There are several typos: 'Liuten et al.' should be 'Luiten et al.' (Section 2.2), 'camaera' should be 'camera' (Section 3.1), 'substract' should be 'subtract' (Section 4.4), and 'outpeforms' should be 'outperforms' (Section 4.7).","section":"Secs. 2.2, 3.1, 4.4, 4.7"},{"comment":"The captions for the synthetic-dataset rankings describe 'solid bar' and 'textured bar', but the definitions are easy to confuse. Define them directly in the caption (e.g., all baselines vs. wide baselines) and apply the same convention consistently in the related appendix figures.","section":"Fig. 14; Figs. 30, 32, 34"},{"comment":"The caption says 'averaged across all five datasets' but does not state whether this is the mean over dataset-level averages or over scenes. State the weighting explicitly and include standard deviations for the aggregate values, not only for per-dataset tables.","section":"Table 2"},{"comment":"Section 3.3 introduces 'STG' and 'STG-decoder' but Table 2 reports both. Please clarify which version corresponds to the intended SpaceTimeGaussians method and why the decoder-removed variant is part of the headline comparison.","section":"Sec. 3.3; Table 2"},{"comment":"Appendix C.1 lists differences from default 3DGS hyperparameters (batch size, warm-up, opacity reset, initialization). The phrase 'shared hyperparameters' in Section 3.3 should be qualified, since these differences are part of the comparison protocol and are not identical across all methods.","section":"App. C.1; Sec. 3.3"},{"comment":"The limitations paragraph is honest about the lack of a method-independent scene-difficulty metric, but the claim that DyCheck's omega does not correlate is based on scatterplot inspection in Figure 16. Reporting a quantitative correlation coefficient would make the point precise.","section":"Sec. 6; Fig. 16"}],"recommendation":"major_revision","confidential_remarks":"To the editor: the reimplementation-parity concern raised in the reader's report is real and is the main reason for major revision. The authors should have the opportunity to add a parity table and failure-count sensitivity analysis; if those checks confirm the reported comparisons, the paper is a solid benchmark contribution. I would not reject on novelty or scope grounds; the paper fits the journal well. One editorial point: the HyperNeRF pose-correction protocol should be clearly disclosed in the main text, since it affects comparability with the original methods."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Bottom line: this is the apples-to-apples benchmark that monocular dynamic Gaussian splatting needed, and its central story—fast but brittle, scene complexity rules—is a fair reading of what they measured. The one caveat that matters is the unverified reimplementation parity. If you read the paper as a rank-order contest, read it with that grain of salt.\n\nWhat is genuinely new: the first full-slate comparison across five datasets and 50 scenes using the original train/test splits, including the first published iPhone results for dynamic Gaussian methods. The new synthetic dataset (SlidingCube/RotatingCube) is well designed for controlled ablations of camera baseline and object motion, and the masked metrics with SAM-Track are a real improvement over whole-image numbers that static backgrounds can swamp. Three runs per condition with standard deviations, the pose-error sensitivity study, and the explicit statement that adaptive density control causes optimization failures all show the kind of rigor this fast-moving field has been missing. I also give them credit for the taxonomy of motion representations; it is simple but useful.\n\nThe soft spots are real but not fatal. The claim in Section 4 that their implementations \"approximately match\" the original works is asserted without a single parity table. That matters most for STG, which was originally multi-view only and had to be adapted to monocular by copying point clouds 10x along the time axis (Section C.3); its poor performance could easily be an artifact of that adaptation. A 2–3 dB implementation gap on a method like DeformableGS would change the ordering in Table 2. Also, the headline statistics exclude collapsed optimization runs (Section 4.4), which is defensible but should be reported alongside failure-inclusive numbers; they do show failure-inclusive results on the synthetic dataset.\n\nHaving said that, the most robust findings survive even if some ranks shift: no method wins across all real datasets, TiNeuVox is a stronger quality baseline than most Gaussian papers admit, and adaptive densification is a genuine source of instability and overfitting. The code release helps: readers can verify the ports, and the authors should be pushed to provide parity evidence in any revision.\n\nWho this is for: practitioners choosing a method, researchers proposing new dynamic Gaussian models, and reviewers who need a baseline for future claims. I would bring it to a reading group and I would cite it. The reimplementation caveat is enough to make me want verification, but not enough to dismiss the benchmark. A serious editor should send this to peer review, and a serious referee should engage with the implementation details rather than desk-reject it.","headline":"A genuinely useful reality-check benchmark for monocular dynamic Gaussian splatting, with one real caveat: the fairness of the cross-method comparison rests on an asserted but unshown reimplementation parity.","tokens_in":25371,"tokens_out":1560,"would_cite":true,"duration_ms":18431,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This benchmark of monocular dynamic Gaussian-splatting methods argues that the field's speed claims are real—20 to 200 frames per second—but that image quality is consistently beaten by a slower voxel-based method, TiNeuVox, and that…","keywords":["monocular dynamic view synthesis","3D Gaussian splatting","benchmarking","motion representation","adaptive density control","scene complexity","specular surfaces","image quality metrics"],"falsifier":"Run the paper's five methods in their original released implementations with their original per-scene hyperparameters on the iPhone dataset and the narrow-baseline synthetic scenes, then compare masked LPIPS on dynamic regions; if a per-Gaussian method (EffGS or STG) or the 4D method RTGS beats DeformableGS and TiNeuVox, the paper's claims that field-based representations help and that scene complexity dominates would be contradicted.","tokens_in":24353,"feed_emoji":"📊","tokens_out":10124,"duration_ms":92840,"temperature":0.7,"pith_summary":"This paper attempts to settle a crowded field: many monocular dynamic Gaussian splatting (GS) methods were published simultaneously, each claiming to be best, and the authors argue that cannot all be true. They integrate five representative GS methods—per-Gaussian polynomial/Fourier motion (EffGS), per-Gaussian polynomial/RBF motion (STG), shared MLP deformation field (DeformableGS), shared HexPlane field (4DGS), and direct 4D Gaussians (RTGS)—into one codebase and compare them on 50 real scenes across five datasets plus a purpose-built synthetic dataset. They find that all Gaussian methods render fast (20–200 FPS) but are consistently beaten in image quality by TiNeuVox, a slower voxel-based neural field, and that no Gaussian method achieves a stable rank order across real datasets. The paper argues that scene complexity—camera baseline, motion magnitude, pose accuracy, specular surfaces, and the fraction of static background—overwhelms the differences between motion representations, and that adaptive density control makes optimization brittle. The contribution is an apples-to-apples comparison and a set of findings that reframe what should be measured when progress is claimed.","feed_headline":"Benchmark: no dynamic Gaussian method wins across datasets","feed_subtitle":"On 50 monocular scenes, a slower voxel method beats every Gaussian splatting method on image quality.","key_machinery":"The central mechanism is a unified evaluation harness: five motion-representation families implemented in one codebase with shared hyperparameters, plus static 3DGS and the voxel-based TiNeuVox as baselines, run on a total of 50 scenes from five datasets and on a new instructive synthetic dataset. The synthetic dataset varies camera baseline B and object-motion distance D independently, isolating how each factor hurts reconstruction, while tracking-derived dynamic masks force separate measurement of quality on moving regions rather than letting static background dominate the metrics. This harness is what lets the paper attribute performance differences to motion representation rather than to dataset split or implementation details, and it is also what exposes the brittleness of adaptive density control.","core_discovery":"On the paper's own terms, the central discovery is that monocular dynamic Gaussian splatting is currently fast but brittle, and that scene difficulty rules. Averaged over five real-world datasets totaling 50 scenes, the non-Gaussian voxel method TiNeuVox reaches 24.54 PSNR while the best Gaussian method (DeformableGS) reaches 24.07, and TiNeuVox trains in comparable time and converges reliably; Gaussian methods render at 20–200 FPS versus TiNeuVox's 0.3 FPS. Among Gaussian methods, field-based motion representations (MLP or HexPlane) give better LPIPS than per-Gaussian low-order motion models, and the most expressive representation—direct 4D Gaussians—performs worst in both quality and efficiency. Yet across datasets the winning method changes, so no clear rank order emerges; on strictly-monocular iPhone data all Gaussian methods lag TiNeuVox by a wide margin. A controlled synthetic dataset shows reconstruction degrades as camera baseline shrinks or object motion grows, that static 3DGS can look competitive by reconstructing the background while inserting pseudo-dynamic Gaussians along a motion path, and that adaptive density control causes overfitting, varying efficiency, and occasional catastrophic optimization failures.","pith_inferences":["Editorial: The paper's measurements could be converted into a scalar scene-difficulty score combining baseline, motion magnitude, pose noise, and frequency content; if such a score predicted each method's error across the 50 scenes, benchmark findings would become transferable to new videos.","Editorial: If scene complexity dominates method choice, an adaptive model selector that estimates baseline and motion speed from a video and then picks the motion representation could outperform any fixed choice; the paper's released data could train such a selector.","Editorial: The paper's negative results for per-Gaussian and 4D methods should be read as \"under uniform optimization,\" not \"never works,\" because the shared-hyperparameter design is deliberate and a practitioner with per-scene tuning might obtain different rankings.","Editorial: The brittleness of adaptive density control points to densification scheduling and regularization, rather than new motion representations, as likely sources of near-term progress in monocular dynamic Gaussian splatting."],"forward_implications":["Published claims that a new motion representation is universally best should be treated as unestablished unless the method is tested across datasets like D-NeRF, Nerfies, HyperNeRF, NeRF-DS, and iPhone with masked dynamic-region metrics.","Field-based motion (shared MLP or HexPlane) is a safer default than per-Gaussian polynomial/Fourier motion or direct 4D Gaussians when scenes are monocular with narrow baselines.","A static 3DGS baseline can look competitive on full-image metrics even when it fails on moving objects, so masked metrics on dynamic regions are necessary for meaningful comparison.","Efficiency claims are not stable across scenes because adaptive density control produces very different Gaussian counts, so training and rendering times need per-scene reporting.","TiNeuVox's quality advantage suggests that hybrid voxel-field representations remain a viable alternative to Gaussian splatting where rendering speed is not the priority."],"supporting_citations":[{"why":"Supplies the static 3DGS representation and adaptive density control that all the compared dynamic methods build on.","marker":"Kerbl et al. (2023)"},{"why":"Supplies TiNeuVox, the voxel-based neural field baseline that beats all Gaussian methods on image quality.","marker":"Fang et al. (2022)"},{"why":"Supplies the strictly-monocular iPhone dataset and the benchmarking framing that this paper extends.","marker":"Gao et al. (2022)"},{"why":"Supplies the HyperNeRF real-world dataset with its known camera pose problems that the paper analyzes and partly corrects.","marker":"Park et al. (2021b)"},{"why":"Supplies the NeRF-DS specular-object dataset used to test how reflective surfaces challenge all methods.","marker":"Yan et al. (2023)"},{"why":"Supplies EffGS, the per-Gaussian Fourier-plus-polynomial motion method compared in the benchmark.","marker":"Katsumata et al. (2023)"},{"why":"Supplies STG, the per-Gaussian polynomial-plus-RBF motion method compared in the benchmark.","marker":"Li et al. (2023a)"},{"why":"Supplies DeformableGS, the shared-MLP deformation field method that ranks best among Gaussian methods.","marker":"Yang et al. (2023)"},{"why":"Supplies 4DGS, the shared-HexPlane field method compared in the benchmark.","marker":"Wu et al. (2023)"},{"why":"Supplies RTGS, the direct 4D Gaussian method that performs worst in quality and efficiency.","marker":"Yang et al. (2024)"}],"fun_headline_variants":["No dynamic Gaussian method wins on all monocular scenes","Gaussian splatting is fast but brittle for monocular dynamics","Scene complexity rules: Gaussian methods inconsistent","TiNeuVox beats Gaussian splatting on monocular dynamic quality","Monocular dynamic Gaussians: fast but fragile across datasets"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The apples-to-apples comparison assumes that the integrated single-codebase reimplementations faithfully reproduce each original method, so the measured rank order reflects motion representation rather than implementation or hyperparameter differences.","fun_headline_variants_meta":{"raw":{"variants":["No dynamic Gaussian method wins on all monocular scenes","Gaussian splatting is fast but brittle for monocular dynamics","Scene complexity rules: Gaussian methods inconsistent","TiNeuVox beats Gaussian splatting on monocular dynamic quality","Monocular dynamic Gaussians: fast but fragile across datasets"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000579,"raw_usage":{"total_tokens":2757,"prompt_tokens":999,"completion_tokens":1758,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":615,"completion_tokens_details":{"reasoning_tokens":1677}},"tokens_in":615,"tokens_out":1758,"duration_ms":13129,"temperature":1.0,"reasoning_tokens":1677,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T21:23:44.904759+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the paper's five methods in their original released implementations with their original per-scene hyperparameters on the iPhone dataset and the narrow-baseline synthetic scenes, then compare masked LPIPS on dynamic regions; if a per-Gaussian method (EffGS or STG) or the 4D method RTGS beats DeformableGS and TiNeuVox, the paper's claims that field-based representations help and that scene complexity dominates would be contradicted.","supporting_citations":[{"cited_title":"Fast dynamic radiance fields with time-aware neural voxels","cited_arxiv_id":null,"evidence_quote":"Supplies TiNeuVox, the voxel-based neural field baseline that beats all Gaussian methods on image quality."},{"cited_title":"Dynamic novel-view synthesis: A reality check","cited_arxiv_id":null,"evidence_quote":"Supplies the strictly-monocular iPhone dataset and the benchmarking framing that this paper extends."}],"review_version":1}