{"id":"eba4ef68-595f-4c9b-a9f9-d6a444666b66","arxiv_id":"2505.05356","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A monocular continuous-wave time-of-flight reconstruction method based on 3D Gaussian splatting achieves depth accuracy comparable to neural radiance field baselines while being about 100x faster to train and render.","lead":"This paper adapts fast 3D Gaussian splatting to reconstruct moving 3D scenes from a single camera that measures time-of-flight light, using raw sensor frames instead of derived depth maps. It reports accuracy similar to slower neural methods while being about 100 times cheaper to optimize, thanks to two training heuristics that stabilize the optimization.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The accuracy claim is bounded by Eq. (7)'s piecewise-linear motion model: the method's clearest losses to F-ToRF occur on nonlinear-motion scenes, and Sec. 6 concedes the failure mode.","rationale":"The reader's conditional verdict and weakest-assumption match my reading. The central empirical claim, depth accuracy comparable to or better than ToRF and F-ToRF, is only as strong as the motion model used to render the asynchronous raw quads. Equation (7) is the explicit place where a piecewise-linear assumption enters; it is not a side detail because the entire raw-quad loss is computed at fractional times inside each quartet. The paper's own Table 1 shows the method's largest errors relative to F-ToRF exactly on scenes whose motion is rotational or strongly accelerated, and Sec. 6 concedes nonlinear motion and thin fast-moving objects as limitations. A second-order interpolation or per-timestamp MLP query would directly test whether the assumption is the cause. Other concerns, such as the absence of code, lack of error bars, and heuristics tuned on the benchmark, are real but secondary: they affect confidence in the numbers rather than identifying a failure mechanism in the argument. The 100x speedup and the qualitative success on several real scenes stand on their own, so the paper remains a useful contribution even if the accuracy claim is scoped to locally-linear motion. Therefore, no change to the conditional verdict is needed.","tokens_in":14146,"tokens_out":12959,"duration_ms":149606,"concrete_test":"Retrain on Arcing Cube and Fan with a quadratic (or higher-order) interpolation replacing Eq. (7), or with the deformation MLP evaluated at every raw-frame timestamp instead of only integer steps, keeping all hyperparameters fixed; then report d and dToF depth MSE against F-ToRF on those scenes. If the error drops to or below F-ToRF, the piecewise-linear model in Eq. (7) is the bottleneck and the accuracy claim must be scoped to locally-linear motion. If the error stays similar, the gap on nonlinear scenes comes from another part of the pipeline, and this concern does not land.","verdict_should_be":"UNCHANGED","load_bearing_attack":"To support \"similar or better accuracy than neural volumetric approaches\", the method must reconstruct depth for the evaluated dynamic scenes. Eq. (7) forces Gaussian positions at fractional raw-frame times to be linear interpolations between integer-time MLP outputs, which is only valid for piecewise-linear motion. For rotational or strongly accelerated motion, this assumption is violated, and Table 1 shows exactly where the method loses to F-ToRF: Arcing Cube dToF error is 1.060 vs 0.470, 3 Chairs ST d is 1.023 vs 0.855 and dToF is 0.639 vs 0.324, with Ortho ST also degraded by a reported local optimum. The authors state in Sec. 6 that \"our model can struggle when the motion is strongly nonlinear (as in Fan and to a smaller extent in Arcing Cube)\" and list thin fast-moving objects as hard. Because the central claim's motivating example includes \"fast motions like swinging baseball bats\", which are predominantly rotational, the claim overreaches for exactly the class of motion the paper highlights. This is a correctness risk, not a disagreement with consensus: the failure is visible in the paper's own numbers and acknowledged in its limitations.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes a dynamic 3D Gaussian splatting method for monocular continuous-wave time-of-flight (C-ToF) cameras, operating directly on raw asynchronous sensor quads rather than derived depth. The scene is represented by canonical Gaussians deformed by an MLP, with intermediate raw-frame motion assumed piecewise linear, and the optimization is stabilized by two heuristics: an occupancy bias implemented as a 10x lower reflectivity learning rate, and a low initial reflectivity bias. The method is evaluated on the T\"oRF real-world dataset, the F-T\"oRF real-world dataset, and the F-T\"oRF synthetic dataset, reporting depth errors competitive with or better than the neural volumetric baselines while being roughly two orders of magnitude faster to optimize and render in real time.","tokens_in":14432,"tokens_out":3361,"duration_ms":38105,"significance":"If the claims are substantiated, this is a practically important result: it would make single-camera dynamic C-ToF reconstruction feasible at near-interactive optimization times and real-time rendering, which no prior method achieves. The paper also gives a clear and honest analysis of why Gaussian-splatting optimization is brittle when depth is only indirectly supervised, and it identifies two simple, reproducible biases that substantially improve stability. The ablations in the supplement are valuable, documenting that the method without the heuristics fails dramatically and that the depth distortion loss is unstable in this setting. However, the central accuracy claim currently rests on a single run per scene, and the two heuristics are explicitly acknowledged to have no universal guarantees and appear to have been selected largely on the basis of the same benchmark scenes used for evaluation. These issues bound the strength of the claims as currently stated.","major_comments":[{"comment":"All quantitative depth errors are reported for a single optimization run per scene, with no error bars or repeated-seed statistics. This is load-bearing because the paper's own motivation is that the C-ToF optimization is under-constrained and brittle, and because the discussion invokes scene-specific optimization failures, e.g., the local optimum on one cube in Orthogonal Speed Test. A single run cannot distinguish a robust method from one that happens to converge on these seven synthetic scenes. I ask the authors to report means and standard deviations over at least three seeds, and to include the per-seed spread for the scenes where the claim of 'similar or better accuracy' is made.","section":"Section 5.1, Table 1"},{"comment":"The deformation model forces Gaussian positions at fractional raw-frame times to be linear interpolations between integer-time positions. The paper's own results show that this assumption is violated on exactly the scenes with nonlinear motion: in Table 1, our method is worse than F-T\"oRF in dToF on Arcing Cube (1.060 vs 0.470) and 3 Chairs Speed Test (0.639 vs 0.324), and the limitations section states that strongly nonlinear motion, as in Fan and partly Arcing Cube, causes the model to struggle. Yet the abstract and introduction advertise accurate reconstruction for fast motions like swinging baseball bats, which are predominantly rotational. This is not a disagreement with current consensus but a mismatch between the method's modeling assumption and the scope of the central claim. The claim should either be restricted to approximately piecewise-linear motions, or the paper should include an experiment or discussion quantifying when the piecewise-linear assumption is adequate (e.g., the real Baseball sequence appears to work well despite the bat's rotation, so the paper should explain why the synthetic rotational scenes fail while the real bat scene does not).","section":"Section 4, Eq. (7); Section 1; Section 6"},{"comment":"The two heuristics are introduced as general remedies for the under-constrained C-ToF optimization, but the paper explicitly acknowledges that they are based on empirical observations and may not be suitable for different scenes. Because the heuristics and hyperparameters (initial reflectivity 0.1, 10x reflectivity learning rate reduction, random background range [-1,1], loss weights alpha and beta) were selected using the same benchmark suites that are used for the final evaluation, the quantitative result overstates the expected accuracy on new scenes. This is a form of evaluation-set adaptation. I would like to see at least one held-out generalization check, e.g., a new capture or a synthetic scene with different object materials and motion types, along with a small sensitivity analysis over the two key heuristic parameters. Without this, the claim that the method 'produces accurate reconstructions under constrained C-ToF sensing conditions' remains specific to the tested datasets rather than to the general setting claimed in the title and abstract.","section":"Section 3.2, Heuristic 1 and Heuristic 2; Section 6, No universal guarantees"}],"minor_comments":[{"comment":"The symbol x is used both for a 3D world position in MLP(x, i) and for a pixel coordinate in G_k^{2D}(x), which makes Eq. (8) unnecessarily hard to read; please use distinct notations, e.g., p for 3D points and u for pixel positions.","section":"Section 4, Eq. (8)"},{"comment":"The denominator in the arctan formula appears as Q_{\\pi/2}-Q_{3\\pi/2} without subscripts being clearly visually separated in the typeset equation; a minor formatting fix would improve readability.","section":"Section 2, Eq. (1)"},{"comment":"Randomizing the background quad uniformly in [-1,1] at every iteration is an unusual and aggressive regularizer, and the paper does not discuss whether it biases the recovered background phase or amplitude; a short justification or ablation would help.","section":"Section 4, Random background"},{"comment":"For the real-world datasets, only qualitative results are shown; a quantitative real-world evaluation would strengthen the paper, though I understand that ground-truth dynamic depth is not available for these captures.","section":"Section 5.1, Metrics"},{"comment":"The explanation of why Jensen's inequality does not lead to equality is phrased loosely ('sine and cosine functions oscillate between convex and concave regions'); consider replacing this with a one-sentence algebraic statement that W(d_k) is not affine in d_k, so the finite sum of W(d_k) does not equal W of the mean depth.","section":"Section 3.1, Eq. (6)"}],"recommendation":"major_revision","confidential_remarks":"The paper is honest about its limitations, and the core idea is timely and potentially valuable. My main concern is that the evaluation does not yet support the breadth of the accuracy claim: single runs, heuristics tuned on the same benchmarks, and the piecewise-linear motion assumption all limit the generality of the central result. I am recommending major revision rather than rejection because these issues are addressable with additional experiments and a sharper statement of the paper's scope. I would also encourage the editor to consider whether a small amount of novelty overlap with the authors' prior F-T\"oRF work needs to be clarified in the presentation, since the image formation model and the benchmarks are largely inherited from that paper."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things about this one. First, the speedup is real: 40–60 minutes versus 60–72 hours for ToRF/F-ToRF, and real-time rendering, all while working directly on raw C-ToF quads. That is a practical advance for single-camera dynamic 3D capture. Second, the accuracy claim is more bounded than the abstract lets on. The method wins on five of seven synthetic scenes, but the two clearest losses are Arcing Cube and 3 Chairs Speed Test, both of which involve nonlinear motion—exactly the failure mode the authors admit in Sec. 6. The abstract highlights \"fast motions like swinging baseball bats,\" but the Fan scene, a spinning fan, remains unsolved. The paper's own numbers, not a reviewer's imagination, draw this boundary.\n\nWhat is genuinely new: adapting Gaussian splatting to raw C-ToF quads with asynchronous capture, and the two optimization heuristics—occupancy bias (slower reflectivity learning rate) and low-reflectivity initialization. These are simple but non-obvious, and the ablation table in the supplement shows they are load-bearing: without them, the mean depth error blows up to catastrophic levels. The depth-distortion loss comparison also earns credit; the authors show it produces oversmoothed depth and correctly motivates their alternative.\n\nThe soft spots are the usual ones. There is a single run per scene, no error bars, and no code artifact, so the reported margins are hard to assess. The heuristics were empirically tuned on these same seven synthetic scenes, and the baselines are the authors' own prior works. That is not disqualifying—ToRF and F-ToRF are the correct baselines—but it does mean the paper essentially says \"we beat our own previous methods on our own benchmarks,\" which is weak unless the release code or additional scenes.\n\nThe stress-test concern about Eq. (7) is legitimate, but not fatal. The paper acknowledges it in Sec. 6, and the actual results show the method is still competitive on most scenes. The bigger issue is that the abstract and intro overstate the scope: they promise \"similar or better accuracy\" for fast motions, then deliver it on a subset and openly struggle with rotational motion. A revision should temper that claim, add error bars, release the code, and perhaps test on a scene that is not part of the heuristic-tuning loop.\n\nOverall, this deserves a serious referee. It is a well-scoped engineering contribution with a clear speed benefit, honest limitations, and reproducible experiments aside from the missing code. I would accept it for review and push for revisions, not a desk reject.","headline":"Solid engineering with a real 100x speedup and honest limitations, but the accuracy claim is softer than the abstract's 'similar or better' on exactly the fast, nonlinear motions the paper highlights.","tokens_in":14967,"tokens_out":2224,"would_cite":true,"duration_ms":25309,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A single continuous-wave time-of-flight camera can reconstruct dynamic scenes about 100 times faster than neural volumetric baselines, with similar or better depth accuracy.","keywords":["continuous-wave time-of-flight","Gaussian splatting","dynamic scene reconstruction","monocular 4D reconstruction","raw sensor quads","radiance fields","depth from phase","optical-flow weak supervision"],"falsifier":"Render a synthetic scene with ground-truth depth and a known nonlinear trajectory—say a fan blade rotating with constant angular acceleration—through the raw-quad model, optimize with this method, and compare mean rendered depth to ground truth. If depth error remains at the level the paper reports for the curved and spinning test scenes and does not shrink with more iterations or smaller time steps, the piecewise-linear motion assumption is the binding limitation of the claimed accuracy.","tokens_in":13937,"feed_emoji":"📷","tokens_out":6601,"duration_ms":66200,"temperature":0.7,"pith_summary":"This paper tries to make monocular dynamic 3D reconstruction from a continuous-wave time-of-flight (C-ToF) camera practical. C-ToF cameras return four raw intensity samples per pixel, and depth must be inferred from their phase, so the property the optimization cares about—depth—is only an indirect byproduct of fitting those samples. The authors show that replacing the slow neural volumes of prior work with 3D Gaussian splatting makes this indirect optimization brittle, and they add two simple optimization biases—slowing the reflectivity learning rate and initializing reflectivity low—that keep Gaussians placed on true surfaces. On synthetic and real-world scenes, the method matches or improves depth accuracy relative to the neural baselines while cutting optimization and rendering time by about 100x, including on fast motion like a swinging baseball bat. If correct, it turns a single commodity C-ToF camera into a near-real-time dynamic 3D capture system.","feed_headline":"Gaussian splatting makes single-camera 4D capture 100x faster","feed_subtitle":"Matches neural time-of-flight radiance fields on depth while optimizing in under an hour.","key_machinery":"The load-bearing object is a differentiable C-ToF Gaussian rasterizer that turns each Gaussian's position, opacity, and spherical-harmonic reflectivity into the four raw sensor samples, rather than into a color image. The key identity is the raw-quad formation model $q(x) = q_{bg}T_N + \\sum_k \\frac{s r_k}{d_k^2}\\varphi(d_k) o_k G^{2D}_k(x) T_k^2$, where $\\varphi(d_k)$ is the sine/cosine phase vector and $T_k$ is squared transmittance; fitting this quantity is what makes depth indirect. A deformation MLP maps canonical Gaussian positions to per-timestep offsets, and between integer timesteps positions are linearly interpolated, enforcing the local-linearity that matches the asynchronous capture of the four quads. Two heuristics carry the optimization: a tenfold-lower reflectivity learning rate (occupancy bias) and a low reflectivity initialization, which together replace a depth-distortion loss and keep Gaussians concentrated at the true surface.","core_discovery":"The paper's central claim is that raw C-ToF samples—not recovered depth maps—are the right supervision signal for a Gaussian splatting reconstruction, provided the optimization is conditioned so that Gaussians behave like an opaque surface. The sensor model renders each Gaussian's returned-light contribution, including the inverse-square falloff and the sinusoidal phase modulation, into the four raw quad images; depth is then read out as the mean Gaussian depth along each ray. Fitting these quads is shown to be under-constrained: many multi-peak density arrangements reproduce the measurements while producing wrong depth, and Gaussian splatting lacks the implicit low-entropy bias of MLP volumetric fields. The paper's fix is two-fold: lower the reflectivity learning rate tenfold so the optimizer prefers moving, adding, or removing Gaussians over changing their reflectivity (occupancy bias), and initialize reflectivity near 0.1 so low-reflectivity surfaces are not pushed to incorrect far depths. With these changes, plus a piecewise-linear deformation model and optical-flow weak supervision, the method reports depth errors below both neural volumetric baselines on five of seven synthetic scenes, competitive results on the remaining two, and 40–60 minute optimizations rather than 60–72 hours.","pith_inferences":["Editorial inference: The same 'optimize the raw measurement, bias the optimizer toward occupancy' recipe could transfer to other indirect-imaging modalities—structured light, sonar, X-ray, or snapshot lidar—where the desired property is a byproduct of fitting the sensor signal; the paper does not claim this transfer.","Editorial inference: Replacing the piecewise-linear interpolation with a higher-order or per-Gaussian trajectory model, possibly regularized by rigidity, is the most direct route to the nonlinear-motion failures the paper reports; this is a testable extension, not a result of the paper.","Editorial inference: If the speed and accuracy hold at scale, a single C-ToF camera becomes a viable low-cost dynamic capture input for telepresence and measurement, but practical deployment would need to address the depth-wrapping ambiguity caused by a static camera."],"forward_implications":["A static monocular C-ToF camera can reconstruct dynamic scenes—including fast motion such as a swinging baseball bat—that previously required either a moving camera, extra RGB constraints, or tens of hours of neural volumetric optimization.","Optimization time drops from 60–72 hours to 40–60 minutes per sequence, and rendering runs in real time above 100 Hz, making iterative capture-and-analysis loops practical.","Fitting raw C-ToF quads directly, without a depth-distortion loss, yields depth estimates that are better on five of seven synthetic scenes and competitive on the remaining two.","The two heuristics are necessary: ablations with neither heuristic fail catastrophically, and adding the depth-distortion loss on top of the heuristics reintroduces oversmoothed depth and instability.","Depth wrapping in static-camera C-ToF backgrounds remains an ill-posed ambiguity that no tested method resolves."],"supporting_citations":[{"why":"Supplies the base Gaussian primitive representation, rasterizer, and optimization loop that the method adapts.","marker":"[13]"},{"why":"Establishes the asynchronous raw-quad C-ToF image formation model and dataset; central baseline and source of the model adapted here.","marker":"[26]"},{"why":"Defines the phasor-based neural C-ToF radiance field and the moving-camera dataset; primary accuracy baseline.","marker":"[1]"},{"why":"Supplies the deformation-MLP dynamic Gaussian framework and comparison baseline.","marker":"[36]"},{"why":"Provides the optical flow estimates used for weak supervision of 3D motion offsets.","marker":"[31]"},{"why":"Introduces the depth-distortion loss the paper considers and rejects in favor of its two heuristics.","marker":"[10]"},{"why":"Supports the premise that monocular dynamic Gaussian splatting is fast but brittle, motivating the optimization heuristics.","marker":"[20]"}],"fun_headline_variants":["Raw ToF samples speed Gaussian splatting 100x for dynamic scenes","Fitting raw ToF quads makes 4D Gaussian splatting 100x faster","Gaussian splatting on raw ToF: 100x speedup, accurate dynamic depth","Indirect ToF depth via splatting: 100x faster single-camera 4D"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The scene's motion is approximately piecewise linear between integer timesteps, with Gaussian positions at fractional times obtained by linear interpolation; strongly curved or rotational motion, which the curved and spinning test scenes exhibit, breaks this assumption and degrades reconstruction.","fun_headline_variants_meta":{"raw":{"variants":["Raw ToF samples speed Gaussian splatting 100x for dynamic scenes","Fitting raw ToF quads makes 4D Gaussian splatting 100x faster","Gaussian splatting on raw ToF: 100x speedup, accurate dynamic depth","Indirect ToF depth via splatting: 100x faster single-camera 4D"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000649,"raw_usage":{"total_tokens":2990,"prompt_tokens":969,"completion_tokens":2021,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":585,"completion_tokens_details":{"reasoning_tokens":1925}},"tokens_in":585,"tokens_out":2021,"duration_ms":13781,"temperature":1.0,"reasoning_tokens":1925,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T23:05:52.447177+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Render a synthetic scene with ground-truth depth and a known nonlinear trajectory—say a fan blade rotating with constant angular acceleration—through the raw-quad model, optimize with this method, and compare mean rendered depth to ground truth. If depth error remains at the level the paper reports for the curved and spinning test scenes and does not shrink with more iterations or smaller time steps, the piecewise-linear motion assumption is the binding limitation of the claimed accuracy.","supporting_citations":[{"cited_title":"3D Gaussian splatting for real-time radiance field rendering","cited_arxiv_id":null,"evidence_quote":"Supplies the base Gaussian primitive representation, rasterizer, and optimization loop that the method adapts."},{"cited_title":"Flowed time of flight radiance fields","cited_arxiv_id":null,"evidence_quote":"Establishes the asynchronous raw-quad C-ToF image formation model and dataset; central baseline and source of the model adapted here."},{"cited_title":"T¨oRF: Time-of-flight radiance fields for dynamic scene view synthesis","cited_arxiv_id":null,"evidence_quote":"Defines the phasor-based neural C-ToF radiance field and the moving-camera dataset; primary accuracy baseline."},{"cited_title":"Deformable 3D Gaussians for high-fidelity monocular dynamic scene reconstruction","cited_arxiv_id":null,"evidence_quote":"Supplies the deformation-MLP dynamic Gaussian framework and comparison baseline."},{"cited_title":"RAFT: Recurrent all-pairs field transforms for optical flow","cited_arxiv_id":null,"evidence_quote":"Provides the optical flow estimates used for weak supervision of 3D motion offsets."},{"cited_title":"2D Gaussian splatting for geometrically accurate radiance fields","cited_arxiv_id":null,"evidence_quote":"Introduces the depth-distortion loss the paper considers and rejects in favor of its two heuristics."}],"review_version":1}