{"id":"18099041-220b-4643-9736-d86f995edc09","arxiv_id":"2507.12137","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A self-supervised Gaussian splatting method for driving scenes models object motion with learnable B-spline and quaternion B-spline curves plus bidirectional temporal visibility masks, achieving state-of-the-art rendering among annotation-free baselines and competitive results with…","lead":"AD-GS reconstructs moving cars, pedestrians, and background from a single driving log without manual 3D box annotations, then renders the scene from new viewpoints and moments. It combines B-spline curves with trigonometric functions to model object motion and uses a simple two-class image segmentation to separate moving objects from static surroundings.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The one-time object/background split from SAM pseudo-masks is permanent (Sec. 3.3 and Sec. A): any dynamic object misclassified at initialization stays static forever, and no component can recover it, leaving the claimed gains conditional on an untested initialization assumption.","rationale":"The reader's weakest assumption pointed to pseudo-label correctness and LiDAR timestamps; my concern converges on the permanent nature of the object/background split derived from those pseudo-labels. This is more specific than generic label noise: the architecture has no feedback path to revise a Gaussian's category after initialization, so even one missed or mislabeled object is irrecoverable. The paper does provide useful ablations (Lobj, B-spline, t-mask) and internal consistency checks, and the reported improvements are plausible; I am not arguing for rejection. However, because the central comparison against PVG/EmerNeRF is advertised as annotation-free and robust, the lack of any test that perturbs or replaces the initial segmentation leaves the central claim conditional. The proposed ground-truth-mask experiment would settle whether this weakness actually limits performance: if perfect masks do not change the numbers, the pseudo-segmentation assumption is not the bottleneck; if they do, the reported margin is contingent on the quality of one external segmentation model. The reader's conditional verdict is therefore appropriate, and my read does not change it.","tokens_in":15858,"tokens_out":7044,"duration_ms":92618,"concrete_test":"Retrain AD-GS on KITTI-75% with ground-truth object masks (project KITTI 3D bounding boxes into the two cameras) while keeping every loss, hyperparameter, and initialization identical, and compare PSNR/SSIM/LPIPS and the PSNR restricted to moving-object regions against the SAM-pseudo-mask run. If the ground-truth-mask version does not materially improve (say, under 0.3 dB), the hand-prompted SAM split is not the load-bearing factor; if it improves substantially, the fixed initial partition is sensitive and the reported performance is conditional on pseudo-segmentation quality.","verdict_should_be":"UNCHANGED","load_bearing_attack":"AD-GS's object-awareness rests on a one-time partition. In Sec. 3.3, LiDAR points are projected onto the SAM/Grounded-SAM binary mask and permanently assigned to either object Gaussians (deformed by the B-spline/trigonometric model) or background Gaussians (never deformed). Section A (Attribute Inheritance) confirms that densification preserves this assignment, and the Lobj term in Eq. 9 only adjusts opacities; it cannot move a Gaussian from background to object. Thus any dynamic object or object part that is not caught by the hand-written prompt list ('car.bus.truck.van.human', plus 'bike' on nuScenes) or that is missed by segmentation/projection at initialization is permanently static, regardless of how well the B-spline motion model could have fit it. The paper's own Section D admits that low pseudo-label quality produces artifacts and Figure 10 shows failure cases, but no experiment quantifies how sensitive the reported margin over PVG/EmerNeRF is to this fixed initial split. Since the central claim is that the object-aware decomposition plus B-spline motion yields large gains over annotation-free baselines, the unchanging validity of this one-shot partition is the least secured load-bearing condition.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes AD-GS, a Gaussian-splatting framework for dynamic urban scene rendering without manual 3D annotations. LiDAR-initialized Gaussians are partitioned once into object and background components using Grounded-SAM binary masks; object Gaussians are deformed by a combination of B-spline curves, B-spline quaternion curves, and trigonometric functions, and are gated by bidirectional temporal visibility masks; background Gaussians remain geometrically static. Training is self-supervised with image reconstruction, rendered object-mask supervision, optical-flow supervision, inverse-depth supervision, and regularization losses. Experiments on KITTI, Waymo, and nuScenes compare AD-GS with self-supervised baselines (SUDS, EmerNeRF, PVG, Grid4D) and annotation-assisted methods, reporting consistent improvements over the self-supervised baselines and near-parity with supervised StreetGS/4DGF on Waymo.","tokens_in":16144,"tokens_out":5929,"duration_ms":73502,"significance":"If the results hold, AD-GS is a meaningful step toward annotation-free dynamic driving-scene rendering: on KITTI-75% it reports PSNR 29.16 / SSIM 0.920 / LPIPS 0.033 versus PVG's 27.13 / 0.895 / 0.049, and on Waymo it reaches PSNR 33.91, close to StreetGS's 33.97, while remaining real-time. The paper is strong in evaluating on three public benchmarks with held-out frames, including ablations of each loss term and module, and it reports rendering speed. However, the central decomposition into object and background relies on a one-shot pseudo-mask partition whose robustness is not quantified, baseline hyperparameters are not fully controlled on nuScenes, and no error bars or per-sequence results are provided. These points do not invalidate the method but limit the strength of the current claims.","major_comments":[{"comment":"The object/background decomposition is decided once at initialization from Grounded-SAM binary masks and is never revised: the attribute-inheritance rule in Sec. A ensures that densified Gaussians keep their parent's label, and Lobj in Eq. (9) only adjusts opacity. Consequently, any dynamic object missed by the hand-written prompt list ('car.bus.truck.van.human', plus 'bike' on nuScenes) or by the mask/projection step is permanently static and is never deformed by Eq. (7). This is load-bearing because the motion model applies only to object Gaussians and the flow supervision in Sec. 3.4 is restricted to the object mask. The paper's own Sec. D acknowledges that low pseudo-label quality produces artifacts (Fig. 10), but no experiment measures how sensitive the reported margins over PVG/EmerNeRF are to this fixed split. Please add a quantitative sensitivity analysis, e.g., train with ground-truth object masks as an oracle, drop individual prompt classes, or corrupt/mask part of the segmentation, and report the resulting PSNR/SSIM/LPIPS on KITTI or Waymo.","section":"Sec. 3.3 and Sec. A (Supplementary)"},{"comment":"PVG is evaluated on nuScenes with hyperparameters designed for Waymo, and EmerNeRF's hyperparameters are not described as tuned for nuScenes. Dynamic-scene Gaussian methods are sensitive to sampling densities, deformation network capacity, and opacity thresholds, so the nuScenes comparison in Table 3 is not fully controlled. Since the abstract claims significant improvement over state-of-the-art annotation-free methods, please retune both baselines on nuScenes, or supply a sensitivity analysis showing that the reported numbers are robust to hyperparameter choices, and report the settings used for every dataset.","section":"Supplementary Sec. B (Baseline details)"},{"comment":"The main tables report averages over sequences (3 KITTI logs, 8 Waymo logs, 6 nuScenes logs) without standard deviations or per-sequence numbers. This makes it impossible to judge whether the reported margins, e.g., PSNR 29.16 vs 27.13 on KITTI-75%, are consistent across logs or dominated by a single sequence. Please report per-sequence results and standard deviations for the main comparison tables so that the word 'significantly' in the abstract is supported statistically.","section":"Tables 1-3"}],"minor_comments":[{"comment":"The sentence 'Lobj makes the two Gaussian subsets stayed in their respective areas' is grammatically unclear; please rephrase to say that the loss encourages the subsets to remain in their assigned regions.","section":"Sec. 3.3, Eq. (9)"},{"comment":"The abstract says the scene is 'automatically segmented', but the segmentation is driven by hand-written text prompts listed in Sec. A; this dependence should be acknowledged in the main text.","section":"Abstract and Sec. 3.3"},{"comment":"The limitations section is relegated to the supplementary material; given its importance for the one-shot segmentation assumption, consider moving a condensed version into the main paper.","section":"Conclusion / Supp. Sec. D"},{"comment":"The paper states background Gaussians 'remain stationary' while Eq. (3) deforms color features of all Gaussians; clarify that 'stationary' refers to position and rotation, not appearance.","section":"Sec. 3.1 and Supp. Sec. A"},{"comment":"No code is released; please include a statement about code availability or provide a reproducibility appendix with the exact training schedules and per-sequence settings.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The core idea is promising and the experimental scope is broad, but the one-shot pseudo-mask partition is a structural assumption whose failure modes are acknowledged yet unquantified, and the baseline tuning on nuScenes is not controlled. These are fixable within revision: a sensitivity analysis with oracle masks or prompt-class ablations and properly tuned baselines would substantially strengthen the paper. I see no circularity concern, as all metrics are on held-out frames. If the authors provide the requested analyses, the paper would be suitable for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear [Colleague],\n\nThis paper deserves a serious referee, but the reader should know that the headline gains sit on top of an untested initialization assumption. The genuinely new piece is the motion model: learnable B-spline curves (including quaternion B-splines for rotation) combined with trigonometric terms, plus a bidirectional temporal visibility mask. That combination is not in the cited literature, and the ablations show it earns its keep. The gains over PVG and EmerNeRF are consistent across KITTI, Waymo, and nuScenes, and the numbers approach annotation-based methods like 4DGF and StreetGS on Waymo. The experimental section is more thorough than most in this area: loss ablations, module ablations, control-point and order sweeps, and an honest limitations subsection with failure figures.\n\nThe soft spots are real but not fatal. The one that matters is the one-time object/background split. As the stress-test says, the split is fixed at initialization from SAM/Grounded-SAM masks and the design confirms it: attribute inheritance preserves the assignment, and Lobj only adjusts opacities. A dynamic object missed by the hand-written prompt list or by the mask is permanently static, no matter how well the B-spline motion model could have fit it. The paper's Section D admits artifacts from low pseudo-label quality, but it never quantifies how much of the reported margin depends on getting that split right. That is a genuine gap and should be fixed before publication: vary the prompts, corrupt the masks, or count how many dynamic objects are misclassified and show the PSNR delta.\n\nAlso missing: code, seed variance, and some baseline tuning is not fully controlled (PVG on nuScenes runs on Waymo hyperparameters). Those are standard for this line of work and easy to address.\n\nThe central claim holds. The evaluation is on held-out frames, the ablations isolate the mechanism, and the method does what it says. The stress-test concern is a limitation, not a refutation.\n\nWho this is for: anyone doing self-supervised driving-scene rendering or motion bases for dynamic Gaussian splatting. I'd bring it to the reading group. A serious editor should send it to review; the appropriate outcome is major revision with a robustness analysis, not rejection.","headline":"A solid self-supervised driving-scene renderer with a genuinely novel motion basis, but the reported gains rest on a one-time pseudo-mask split that the paper leaves untested.","tokens_in":16693,"tokens_out":3411,"would_cite":true,"duration_ms":32086,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"AD-GS claims that an annotation-free model can render dynamic driving scenes as well as tracklet-supervised methods, reporting 29.16 PSNR on KITTI-75% and 33.91 PSNR on Waymo with a B-spline-plus-trigonometric motion model.","keywords":["autonomous driving scene rendering","3D Gaussian splatting","self-supervised learning","B-spline curves","dynamic scene decomposition","temporal visibility","free-viewpoint rendering","LiDAR-guided supervision"],"falsifier":"A decisive check is to run the released model on a KITTI sequence after changing the object text prompt to a class that excludes the actual moving vehicles; if the pipeline still delivers near-29 PSNR on KITTI-75%, the object/background decomposition is not the source of the reported gains, whereas a sharp drop would confirm that the simplified pseudo segmentation is load-bearing.","tokens_in":15617,"feed_emoji":"🚗","tokens_out":12010,"duration_ms":129703,"temperature":0.7,"pith_summary":"AD-GS is a self-supervised system for free-viewpoint rendering of urban driving logs. It claims that manual 3D object tracklets are unnecessary: a two-class split of the scene into moving objects and static background, driven by text-prompted pseudo segmentation, is enough to reach rendering quality that is competitive with supervised methods. The key move is replacing pure trigonometric or network motion models with learnable B-spline curves, plus quaternion B-splines for rotation, combined with global trigonometric terms, so that each Gaussian's trajectory can fit both global motion and local detail. On KITTI-75% it reports PSNR 29.16 versus 27.13 for the best self-supervised baseline, and on Waymo it lands within about 0.7 dB of the strongest annotation-dependent method. If correct, this would remove a major annotation bottleneck for driving simulation and closed-loop testing.","feed_headline":"No manual tracklets: driving scenes render near supervised quality","feed_subtitle":"On KITTI it reaches 29.16 PSNR, two points ahead of the best self-supervised baseline.","key_machinery":"The load-bearing object is the learnable motion model for each dynamic Gaussian: a uniform B-spline curve for position, defined by learnable control points and evaluated through a precomputed matrix formulation, together with a B-spline quaternion curve for rotation, both added to trigonometric terms. A B-spline is a piecewise polynomial curve whose value at any time depends only on nearby control points, so optimizing a frame's motion updates only a few control points and gives local fitting, while the trigonometric terms provide global fitting that resists noise from pseudo labels. The second mechanism is the bidirectional temporal visibility mask, which multiplies an object Gaussian's opacity by a Gaussian in time centered at the LiDAR acquisition timestamp, with separate learnable widths for before and after, so that suddenly appearing or disappearing objects do not receive misleading gradients from frames where they are invisible.","core_discovery":"The paper's central discovery is that the two failure modes of self-supervised driving-scene rendering, noisy scene decomposition and inaccurate local motion, can be addressed together without any manual 3D supervision. AD-GS partitions all Gaussians into objects and background using a simplified pseudo 2D segmentation, keeps the background stationary, and deforms object Gaussians over time using learnable B-spline curves combined with trigonometric functions. Each object Gaussian also carries a bidirectional temporal visibility mask centered at the LiDAR acquisition timestamp, which lets the model ignore frames where the object is not actually visible. Combined with flow, inverse-depth, visibility, and rigidity supervision, the model reports state-of-the-art self-supervised results and competitive results against annotation-based systems on KITTI, Waymo, and nuScenes.","pith_inferences":["A testable extension beyond the paper: the same two-class, text-prompted decomposition should transfer to other dynamic scene logs with small known sets of moving classes, but the paper only demonstrates urban driving.","The use of the LiDAR acquisition timestamp as a fixed prompt for visibility turns sensor timing into a free supervisory signal; one could extend it to cameras with rolling shutters, where visibility time varies per row.","Because the loss ablation shows segmentation guidance is the largest single lever, improving open-vocabulary segmentation quality should push the same architecture closer to or past annotation-dependent models without any change to the motion model.","The local-fitting argument implies a specific prediction: on objects visible for only a handful of frames, AD-GS should degrade more gracefully than trigonometric-only models, which can be tested directly on very short-lived objects."],"forward_implications":["A single driving log can be turned into a free-viewpoint, free-time rendering without any manual 3D tracklets, at speeds around 47 FPS on KITTI.","Two-class decomposition into objects and background is enough to isolate motion while avoiding the noise of fine-grained instance segmentation.","The combined B-spline and trigonometric motion model captures both global trajectory and local detail, improving PSNR over trigonometric-only rendering by about 2 dB on KITTI-75%.","Bidirectional temporal visibility masks reduce artifacts when objects appear or disappear, preventing invisible frames from corrupting the motion fit.","Even with only 25% of KITTI frames for training, the method still improves over self-supervised baselines, suggesting resilience to sparse supervision."],"supporting_citations":[{"why":"Supplies the differentiable 3D Gaussian splatting renderer and optimization machinery on which AD-GS builds.","marker":"[9]"},{"why":"Defines the trigonometric-only motion baseline whose global-fitting weakness AD-GS targets.","marker":"[2]"},{"why":"Provides the flow-supervision formulation and the KITTI evaluation protocol that AD-GS adopts.","marker":"[32]"},{"why":"Supplies the annotation-dependent baseline and the SfM-plus-LiDAR initialization and dynamic-mask evaluation setup AD-GS follows.","marker":"[35]"},{"why":"Provides the strongest annotation-assisted comparison on KITTI and Waymo against which the annotation-free claim is measured.","marker":"[5]"},{"why":"Generates the simplified pseudo 2D segmentation from text prompts that drives scene decomposition.","marker":"[26]"},{"why":"Generates the binary object and sky masks used for the two-class decomposition.","marker":"[12]"},{"why":"Supplies pseudo optical-flow labels that supervise object trajectories.","marker":"[8]"},{"why":"Supplies monocular inverse-depth pseudo ground truth for the depth loss.","marker":"[37]"},{"why":"Defines the quaternion B-spline interpolation used to deform Gaussian rotations with local control.","marker":"[11]"}],"fun_headline_variants":["Self-supervised driving scenes rival supervised rendering without tracklets","Object-aware splatting replaces manual labels for dynamic driving scene rendering","B-spline motion model makes self-supervised driving rendering precise","Pseudo 2D segmentation replaces tracklets in driving scene Gaussian splatting"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the simplified pseudo 2D segmentation and the LiDAR acquisition timestamp correctly identify which Gaussians are dynamic and when each object is visible, so that the motion model and temporal visibility mask receive a correct separation of moving and static content.","fun_headline_variants_meta":{"raw":{"variants":["Self-supervised driving scenes rival supervised rendering without tracklets","Object-aware splatting replaces manual labels for dynamic driving scene rendering","B-spline motion model makes self-supervised driving rendering precise","Pseudo 2D segmentation replaces tracklets in driving scene Gaussian splatting"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000584,"raw_usage":{"total_tokens":2709,"prompt_tokens":871,"completion_tokens":1838,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":487,"completion_tokens_details":{"reasoning_tokens":1763}},"tokens_in":487,"tokens_out":1838,"duration_ms":15754,"temperature":1.0,"reasoning_tokens":1763,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T16:53:07.524189+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A decisive check is to run the released model on a KITTI sequence after changing the object text prompt to a class that excludes the actual moving vehicles; if the pipeline still delivers near-29 PSNR on KITTI-75%, the object/background decomposition is not the source of the reported gains, whereas a sharp drop would confirm that the simplified pseudo segmentation is load-bearing.","supporting_citations":[{"cited_title":"3D Gaussian splatting for real-time radiance field rendering","cited_arxiv_id":null,"evidence_quote":"Supplies the differentiable 3D Gaussian splatting renderer and optimization machinery on which AD-GS builds."},{"cited_title":"SUDS: Scalable urban dynamic scenes","cited_arxiv_id":null,"evidence_quote":"Provides the flow-supervision formulation and the KITTI evaluation protocol that AD-GS adopts."},{"cited_title":"Grid4D: 4D decomposed hash encoding for high-fidelity dynamic gaus- sian splatting","cited_arxiv_id":null,"evidence_quote":"Supplies the annotation-dependent baseline and the SfM-plus-LiDAR initialization and dynamic-mask evaluation setup AD-GS follows."},{"cited_title":"Segment any- thing","cited_arxiv_id":null,"evidence_quote":"Generates the binary object and sky masks used for the two-class decomposition."},{"cited_title":"A general construction scheme for unit quaternion curves with simple high order derivatives","cited_arxiv_id":null,"evidence_quote":"Defines the quaternion B-spline interpolation used to deform Gaussian rotations with local control."}],"review_version":1}