{"id":"d473d588-1efb-44ad-a717-bc1afe0d756b","arxiv_id":"2506.05348","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A dynamic-scene representation where Gaussian primitives live freely in 4D space-time with linear motion and Gaussian time windows achieves state-of-the-art novel-view quality on complex-motion benchmarks.","lead":"This paper introduces a new way to reconstruct moving 3D scenes from multi-view video, using many small Gaussian particles that can appear at any time and place and glide with a constant velocity. It reports sharper renderings than prior state-of-the-art methods on several benchmarks, while running fast enough for real-time use.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'anytime' claim is only tested at training timestamps; free short-lived primitives may fit observed frames rather than generalize across time, so the large-margin result is not yet established for novel times.","rationale":"The reader's weakest assumption was that complex motion may not be well approximated by many short-lived constant-velocity Gaussians; I agree that this is untested, but I would sharpen it into a temporal-generalization concern. The paper's headline capability is rendering at arbitrary time, yet all reported evaluations hold out views while keeping timestamps in the training set. Since FreeTimeGS deliberately frees primitive birth times, durations, positions, and velocities, the training objective can be satisfied by frame-specific primitives without producing a temporally coherent model. This is not an internal inconsistency in the equations, but it is a load-bearing gap between the central claim and the evidence. A temporal holdout experiment would distinguish per-frame memorization from genuine reconstruction of fast motion, and a synthetic acceleration/rotation stress test would quantify how the piecewise-linear representation scales. I would not change the reader's conditional verdict on this basis alone, because the missing evidence is obtainable and the method may well pass the test, but the condition should explicitly include temporal generalization. Reproducibility and baseline tuning concerns are real but secondary; they do not target the representation's core claim as directly as the temporal holdout does.","tokens_in":17952,"tokens_out":8100,"duration_ms":110213,"concrete_test":"Create a temporal holdout split of SelfCap (e.g., train on even-indexed frames and evaluate on odd-indexed frames at the same cameras) and run a synthetic scene with a rigid object undergoing constant acceleration or rotation with known ground truth. Report PSNR, LPIPS, and the number of primitives active per frame versus acceleration or angular speed. If the holdout PSNR gap to 4DGS and STGS shrinks sharply, or if primitive count grows steeply with acceleration beyond what the method can render in real time, the anytime/anywhere claim and the large-margin generalization are not established.","verdict_should_be":"UNCHANGED","load_bearing_attack":"FreeTimeGS's central claim is that Gaussian primitives can be placed \"at anytime and anywhere,\" with Eq. 1 giving each primitive constant-velocity straight-line motion and Eq. 4 a unimodal temporal opacity window. The paper never tests rendering at unobserved times: on Neural3DV, ENeRF-Outdoor, and SelfCap, evaluation is done on frames that were used in training, with only camera views held out. Because position, time, duration, and velocity are all free per primitive, the optimizer can fit the observed frames with short-lived Gaussians that appear only near training timestamps; the reported 2.4 dB and 1.4 dB SelfCap gains over 4DGS and STGS could therefore reflect per-frame fitting rather than a generalizable model of fast motion. The local-linearity assumption in Eq. 1 is a special case of this: for acceleration or rotation, the representation can only compensate by using more short-duration primitives, and no experiment measures how the required primitive count or reconstruction error grows with motion complexity. The paper's stated limitations address training time and relighting, but not temporal generalization, so the missing experiment is not covered by an acknowledged limitation.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes FreeTimeGS, a dynamic scene representation in which each 3D Gaussian primitive is assigned a birth time, duration, constant-velocity motion, and a unimodal temporal opacity function. The method avoids canonical-space deformation fields, instead allowing primitives to appear at arbitrary positions and times, and it introduces an opacity regularization loss, periodic relocation of low-opacity primitives, and ROMA-based 4D initialization. Experiments on Neural3DV, ENeRF-Outdoor, and a self-collected SelfCap dataset report state-of-the-art PSNR/DSSIM/LPIPS values and real-time rendering at 450+ FPS.","tokens_in":18273,"tokens_out":3756,"duration_ms":47305,"significance":"If the claims hold, FreeTimeGS provides a simple and efficient alternative to deformation-based dynamic Gaussian methods, with particularly strong results on fast and complex motions. The supplementary per-scene breakdowns and the primitive-count ablations are concrete and useful evidence for the representation's flexibility and rendering-speed trade-off. The opacity regularization and periodic relocation are clearly motivated by the reported local-minimum problem. However, the central 'anytime' component of the claim is not validated because all evaluations are performed at training timestamps, the largest improvements are reported on a self-collected dataset, and one public benchmark table uses different scene subsets for different metrics. The representation is plausible, but the temporal-generalization claim needs explicit experimental support before the headline conclusions can be accepted.","major_comments":[{"comment":"The central 'anytime' claim is never tested at timestamps that were not used in training. All experiments hold out camera views only; every test frame is also a training frame. Because each primitive has free position, time, duration, and velocity, and because temporal opacity is modeled as a unimodal Gaussian, the optimizer can place short-lived primitives near observed timestamps, so the reported large margins (e.g., 2.4 dB and 1.4 dB on SelfCap) may reflect per-frame fitting rather than generalization across time. I request an experiment that holds out entire timestamps, such as training on a subset of frames and evaluating on interleaved or withheld frames, with PSNR/DSSIM/LPIPS reported on novel times. In addition, to justify the constant-velocity local-linearity assumption, please report how the required primitive count or reconstruction error grows with motion complexity (e.g., acceleration or rotation), since Eq. (1) can otherwise only compensate by using more short-duration primitives.","section":"§4, Eq. (1), Eq. (4)"},{"comment":"The Neural3DV comparison uses different scene subsets for different metrics: DSSIM1 is reported only on Flame Salmon, while DSSIM2 excludes Coffee Martini. As a result, the claim of outperforming all baselines in all metrics is not supported by a consistent evaluation. For example, DSSIM1 for Ours equals STGS at 0.026, and the DSSIM2 advantage is computed on a subset that excludes a scene where several baselines perform poorly. Please report all metrics on the same set of scenes for all methods, or state explicitly that no consistent full-scene comparison is available. If practical, add multiple seeds and error bars to the main tables.","section":"Table 1"},{"comment":"The SelfCap dataset is the primary evidence for the large-margin improvement on fast motion, but it is self-collected and not released, and the evaluation protocol for the two strongest baselines includes traversing camera near-plane settings during testing to maximize floater removal. This protocol may advantage a method with fewer floaters and makes the comparison less standardized. Please report results for 4DGS and STGS under their default near-plane settings as well, describe exactly how the dynamic-region masks are applied when computing PSNR on the cropped masked images, and provide per-run variance or at least clarify how many random seeds were used.","section":"Table 3, §4.1"}],"minor_comments":[{"comment":"The text contains grammatical and typographical errors: 'an motion' in the abstract, 'Experiments results' in §1, and a duplicated sentence in §3.1 that begins 'In addition, our approach assigns a motion function...' twice. The Figure 2 caption also contains 'to to model'.","section":"§1, §3.1"},{"comment":"The caption 'entire/fastest).' has an unmatched parenthesis and should be rewritten as 'entire/fastest' or 'entire and fastest subsequence'.","section":"Table 4, Table 5"},{"comment":"The notation in Eq. (6) uses both sigma and sigma(t) without a clear distinction between the base opacity parameter and the temporal opacity function; please define them consistently and state explicitly that the stop-gradient prevents the regularization from shrinking the temporal opacity.","section":"Eq. (6)"},{"comment":"The statement that each Gaussian primitive consists of 'eight learnable parameters' is imprecise because scale, orientation, and spherical harmonics coefficients are multi-dimensional; rephrase as 'eight groups of learnable parameters'.","section":"§3.1"},{"comment":"The limitations paragraph acknowledges training time and relighting but does not mention the lack of validation on novel timestamps; given the paper's 'anytime' claim, this should be explicitly discussed as a limitation.","section":"§5"}],"recommendation":"major_revision","confidential_remarks":"The paper is a reasonable fit for the journal and the self-citations to EasyVolcap, 4K4D, and ENeRF are used as tooling and baselines rather than to support the central claim. The main risk is that the 'anytime' claim is not empirically supported; the suggested held-out-time experiment is essential and should be requested. If the authors can provide consistent benchmark tables and novel-time evaluations, the paper would likely be acceptable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The one thing to know: this is not another canonical-space deformation method. FreeTimeGS lets Gaussian primitives exist anywhere in space-time, each with its own temporal opacity window and a constant-velocity motion line. That is a real departure from 4DGS and STGS, and it is the reason the method can handle fast, discontinuous motion without building long-range correspondences. The results back it up: on the SelfCap dataset it beats 4DGS and STGS by 1.4-2.4 dB PSNR on the full image and more in dynamic regions, and it renders at 450+ FPS. The ablations are consistent with the story - removing the motion function, the opacity regularization, the periodic relocation, or the ROMA-based initialization each hurts, and the effect is largest on the fastest subsequence. The storage cost is also reasonable (53-125 MB depending on primitive count). Credit where due: the representation is new, the experiments are internally coherent, and the self-citations (EasyVolcap, 4K4D, ENeRF) are for tooling and baselines, not for the central claim. That is honest. The soft spots are real but not fatal. The biggest one is that the 'anytime' promise is never tested. All evaluation is on training timestamps with only camera views held out, so a greedy per-frame fitting story is not ruled out. The stress-test note is right: because position, time, duration, and velocity are all free, the optimizer could be placing short-lived Gaussians near observed frames rather than learning anything that interpolates in time. A simple test - train on every other frame and evaluate in between - would settle this, and it is absent. The paper's stated limitations (training time, relighting) do not cover this gap. Relatedly, the local-linearity assumption in Eq. 1 means acceleration and rotation are handled by throwing more short-duration primitives at the problem, and there is no analysis of how primitive count or error scale with motion complexity. That is a moderate concern, not a fatal one, but it would be good to see the scaling curve. Two smaller issues: there are no error bars or repeated-run variance anywhere, which matters for claims of 'large margin' improvement. And the Neural3DV main table uses selective metric conventions (DSSIM1 only on Flame Salmon, DSSIM2 excluding Coffee Martini); the per-scene breakdown in the supplementary does show full numbers, so it is not hiding data, but the main-table footnotes are odd and need justification. Who is this for? Anyone working on dynamic view synthesis with Gaussian splatting. The paper deserves a serious referee. If I were the editor, I would send it to review and ask for code and data release, a temporal interpolation experiment, and variance reporting on at least the SelfCap numbers. The core idea is sound and the gains are plausible; it just needs the 'anytime' claim backed by evidence.","headline":"FreeTimeGS is a genuinely different 4D Gaussian representation with strong results on fast-motion scenes, but the 'anytime' claim is not actually tested at unobserved times and the paper would benefit from code, data, and variance reporting.","tokens_in":845,"tokens_out":869,"would_cite":true,"duration_ms":28365,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"FreeTimeGS claims that freeing Gaussian primitives from a canonical time and giving each a straight-line motion and a single-peaked temporal window makes fast, complex dynamic scenes easier to reconstruct, and reports the best rendering…","keywords":["dynamic scene reconstruction","4D Gaussian primitives","novel view synthesis","space-time Gaussian representation","temporal opacity","opacity regularization","real-time rendering","complex motion"],"falsifier":"Reconstruct a turntable scene with a rapidly rotating textured object using FreeTimeGS and a deformation-based baseline under identical cameras. If, as rotation speed increases, FreeTimeGS's PSNR advantage shrinks or reverses and its primitive count grows much faster than the baseline's, then the constant-velocity straight-line motion assumption is the limiting factor, and the reported gains on fast motion would not generalize to rotation-dominated motion.","tokens_in":17784,"feed_emoji":"🎥","tokens_out":8132,"duration_ms":91097,"temperature":0.7,"pith_summary":"FreeTimeGS is a 4D scene representation for reconstructing dynamic scenes from multi-view video. Instead of anchoring Gaussian primitives in a canonical space and deforming them to each frame, it lets every Gaussian primitive be born at any position and time, move with a simple constant-velocity straight-line motion, and fade in and out through a single-peaked temporal opacity window. The paper argues this removes the need for long-range canonical-to-observation correspondences, which is the step that deformation-based methods struggle with on fast and complex motion, and that the extra flexibility pays off in quality and speed. On its own collected SelfCap dataset of dancing, pets, and bike repairs, the method reports PSNR gains of 2.4 dB and 1.4 dB over 4DGS and STGS, and 4.1 dB and 2.6 dB in dynamic regions, while rendering at 467 FPS. The practical stake is that live free-viewpoint replay of fast scenes, for film, games, or VR, becomes possible without a long offline deformation optimization.","feed_headline":"Beat fast-motion baselines by 2.4 dB with free-time Gaussians","feed_subtitle":"A 4D Gaussian representation that drops canonical deformation renders fast scenes sharply at over 450 FPS.","key_machinery":"The machine is the free Gaussian primitive: a 3D Gaussian that exists only inside a temporal window. Equation (1), $\\mu_x(t)=\\mu_x+v\\cdot(t-\\mu_t)$, is a constant-velocity motion function that moves the primitive to neighboring regions; Equation (4), $\\sigma(t)=\\exp(-\\frac{1}{2}((t-\\mu_t)/s)^2)$, is a single-peaked temporal opacity that lets the primitive's birth time and duration be optimized by gradient descent and controls how much it contributes at each time. Together they turn dynamic reconstruction into a search over short-lived primitives instead of a search over long-range deformations. The argument is carried by the combination of this representation with a regularization loss $L_{\\rm reg}=\\frac{1}{N}\\sum_i \\sigma_i\\cdot\\mathrm{sg}[\\sigma_i(t)]$ that suppresses high opacity early in training, a periodic relocation of underused primitives to high-score areas, and a four-dimensional initialization that seeds positions, times, and velocities from tracked 3D points.","core_discovery":"The central claim is that a 4D Gaussian representation in which primitives are not tied to a canonical time is enough, by itself, to model dynamic scenes with strong motion, and that the remaining optimization difficulties are solved by three accessories. Each primitive carries position $\\mu_x$, birth time $\\mu_t$, duration $s$, velocity $v$, scale, orientation, opacity, and color; at query time it is moved to $\\mu_x + v(t-\\mu_t)$ and its opacity is multiplied by $\\exp(-\\frac{1}{2}((t-\\mu_t)/s)^2)$. Because every primitive only needs to explain a short temporal neighborhood, the paper replaces deformation fields with a linear motion function, and claims this makes optimization better-posed. On top of this, a time-weighted opacity regularization term penalizes primitives whose opacity is near 1 during early training, preventing them from blocking gradients; a periodic relocation step moves low-opacity primitives toward regions with high sampling score; and positions, times, and velocities are initialized from ROMA-based multi-view matches and nearest-neighbor correspondences. The paper reports best results on Neural3DV, ENeRF-Outdoor, and SelfCap, including 33.19 PSNR on Neural3DV and 27.41/29.38 PSNR on SelfCap for entire/dynamic regions at 467 FPS.","pith_inferences":["An implication the paper does not test: curved motion such as rotation or acceleration must be approximated by many short straight-line segments, so the required primitive count should grow with angular speed and curvature; a scene with pure rotation would reveal whether the reported 96 MB model size holds.","A testable extension the paper leaves open: replacing ROMA-based multi-view initialization with monocular or sparse correspondences would separate how much of the gain comes from the representation itself and how much from a good starting point.","The opacity-blocking-gradient mechanism is stated for dynamic scenes, but it is a general optimization claim: if correct, the same early-training opacity penalty should stabilize static 3D Gaussian Splatting as well.","The paper notes its representation is orthogonal to long-video hierarchies; a natural next step is coupling FreeTimeGS with a temporal hierarchy to stream arbitrarily long sequences, which the paper does not attempt."],"forward_implications":["If the central claim is right, scenes with fast and complex motion no longer require learning long-range deformation correspondences; reconstruction reduces to optimizing short-lived primitives, and the failure modes of canonical-space deformation on large motions are bypassed.","The reported 467 FPS at 1080p on a single RTX 4090 means photorealistic free-viewpoint replay of dynamic scenes is real-time, so interactive applications such as VR, live sports replay, and virtual production can use the output directly.","Storage can be traded against quality smoothly: capping primitives at 500k or 70k lowers PSNR by only about 0.2–0.8 dB on Neural3DV while cutting model size from 125 MB to 41 MB or 8.3 MB, pointing to a compact streaming format.","Removing any of the four proposed components, the motion function, the 4D regularization, periodic relocation, or velocity initialization, degrades fast-motion PSNR by up to 3.8 dB in the fastest frames, so the recipe is load-bearing and not just a minor tweak.","On the SelfCap fast-motion dataset the gain is largest in dynamic regions, 4.1 dB and 2.6 dB over 4DGS and STGS, indicating the representation helps precisely where prior 4D Gaussian methods overfit or stall."],"supporting_citations":[{"why":"Supplies ROMA dense multi-view 2D matches used to triangulate 3D points for initializing position, time, and velocity.","marker":"[9]"},{"why":"Provides the 3D Gaussian Splatting rasterizer, loss settings, and densification machinery that the rendering and optimization loop builds on.","marker":"[14]"},{"why":"Provides the Neural3DV multi-view dynamic dataset used for the main comparisons.","marker":"[19]"},{"why":"STGS, the space-time Gaussian baseline whose motion representation is compared and surpassed on all datasets.","marker":"[21]"},{"why":"ENeRF-Outdoor, the outdoor dynamic dataset used for evaluation and comparison.","marker":"[23]"},{"why":"Background Matting V2, used to extract dynamic-region masks for the SelfCap dynamic-only metrics.","marker":"[24]"},{"why":"Deformable-3DGS, the canonical-space deformation baseline whose failure on long-range motion the method is designed to avoid and that is compared in the SelfCap experiments.","marker":"[44]"},{"why":"4DGS, the 4D Gaussian baseline whose entangled geometry-velocity representation is replaced in the 'w/o our motion' ablation and compared across all benchmarks.","marker":"[49]"}],"fun_headline_variants":["Free-time Gaussians render fast scenes sharply at 467 FPS","No canonical deformation: Free-time Gaussians beat baselines by 2.4 dB","4D Gaussians with birth times and velocities capture complex motion","Dynamic 3D without deformation fields: FreeTimeGS"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that fast and complex motion can be represented by a large enough set of short-lived Gaussian blobs, each moving in a straight line at constant speed and appearing and disappearing through one smooth rise-and-fall window.","fun_headline_variants_meta":{"raw":{"variants":["Free-time Gaussians render fast scenes sharply at 467 FPS","No canonical deformation: Free-time Gaussians beat baselines by 2.4 dB","4D Gaussians with birth times and velocities capture complex motion","Dynamic 3D without deformation fields: FreeTimeGS"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001122,"raw_usage":{"total_tokens":4698,"prompt_tokens":1003,"completion_tokens":3695,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":619,"completion_tokens_details":{"reasoning_tokens":3619}},"tokens_in":619,"tokens_out":3695,"duration_ms":29887,"temperature":1.0,"reasoning_tokens":3619,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T10:20:40.374145+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Reconstruct a turntable scene with a rapidly rotating textured object using FreeTimeGS and a deformation-based baseline under identical cameras. If, as rotation speed increases, FreeTimeGS's PSNR advantage shrinks or reverses and its primitive count grows much faster than the baseline's, then the constant-velocity straight-line motion assumption is the limiting factor, and the reported gains on fast motion would not generalize to rotation-dominated motion.","supporting_citations":[{"cited_title":"RoMa: Robust Dense Feature Matching.IEEE Conference on Computer Vision and Pattern Recognition, 2024","cited_arxiv_id":null,"evidence_quote":"Supplies ROMA dense multi-view 2D matches used to triangulate 3D points for initializing position, time, and velocity."},{"cited_title":"3d gaussian splatting for real-time radiance field rendering.ACM Transactions on Graphics (TOG), 42(4):1–14, 2023","cited_arxiv_id":null,"evidence_quote":"Provides the 3D Gaussian Splatting rasterizer, loss settings, and densification machinery that the rendering and optimization loop builds on."},{"cited_title":"Neural 3d video synthesis from multi-view video","cited_arxiv_id":null,"evidence_quote":"Provides the Neural3DV multi-view dynamic dataset used for the main comparisons."},{"cited_title":"Space- time gaussian feature splatting for real-time dynamic view synthesis.2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8508–8520, 2023","cited_arxiv_id":null,"evidence_quote":"STGS, the space-time Gaussian baseline whose motion representation is compared and surpassed on all datasets."},{"cited_title":"Efficient neural radiance fields for interactive free-viewpoint video","cited_arxiv_id":null,"evidence_quote":"ENeRF-Outdoor, the outdoor dynamic dataset used for evaluation and comparison."},{"cited_title":"Seitz, and Ira Kemelmacher- Shlizerman","cited_arxiv_id":null,"evidence_quote":"Background Matting V2, used to extract dynamic-region masks for the SelfCap dynamic-only metrics."}],"review_version":1}