{"id":"7409a4bd-183d-4a8a-9121-d31971ab1bcd","arxiv_id":"2412.03844","paper_version":4,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"HybridGS separates transient objects from static scenes by using 2D Gaussians per image for transients and 3D Gaussians for the static background, improving novel-view synthesis quality.","lead":"This paper proposes a way to render 3D scenes that contain moving objects by representing the moving objects separately in each photo and the stable background as a shared 3D model. The method produces sharper novel-view images than previous approaches on scenes with pedestrians and other distractors.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The planar-transient assumption is untested for multi-view-consistent transients: a stationary distractor is baked into 3DGS during warm-up and the co-visible frustum in Alg. 1 treats it as static.","rationale":"The strongest claim is that HybridGS sets a new SOTA for novel view synthesis in scenes with transients. For that claim to be true, the hybrid representation must actually separate image-specific transient content from multi-view static content. The mechanism by which it does so is the assumption that transients are single-view planar objects (Sec. 1, Sec. 4.2). This is not a hyperparameter or an engineering detail; it is the core inductive bias. The paper's benchmarks (NeRF On-the-go, RobustNeRF) do contain moving distractors, which is exactly the regime where the assumption holds, and the reported gains there are consistent. But the assumption has a concrete failure mode: a semantically transient object that is geometrically consistent during training. The warm-up stage (Sec. 4.3) trains 3DGS on the full images without masks, so such an object is assigned to the 3D branch. Later stages cannot reliably move it because the 2D mask is trained on residuals after the 3D branch already explains the object, and the masked loss (Eq. 11) removes gradient from masked pixels. The second failure mode is complementary: if a static region is poorly reconstructed by 3DGS, the per-image 2D Gaussians can absorb it and remove it from the static scene; nothing in the losses penalizes this because 2D Gaussians have no cross-view consistency. These are not speculative edge cases; paused pedestrians and parked cars are common in casually captured tourist-style footage, which is precisely the intended application. The paper provides no experiment that isolates this regime, and Algorithm 1's co-visible frustum is not instrumented to measure how often it contains transient content. The reader's conditional verdict already names the planar-transient assumption; my reading agrees and would keep the verdict conditional. The proposed controlled experiment (same scene, distractor stationary vs. moving across training frames) is a minimal, decisive test. I also note the paper honestly discloses the illumination limitation (Sec. 5.4), but that limitation is separate from the concern here.","tokens_in":16201,"tokens_out":12405,"duration_ms":129741,"concrete_test":"Construct a controlled RobustNeRF-style scene in which one distractor object remains at a fixed 3D location for half of the training frames but is absent from the test views; train HybridGS and compare PSNR/SSIM on the distractor-free test set against the identical scene with the distractor moved every frame. If the stationary-transient variant degrades by more than roughly 0.5 dB or renders ghost geometry at the distractor location, the planar single-view premise is breached; if performance is unchanged, the concern is refuted.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires that transient objects are single-view planar (Sec. 1, Sec. 4.2), and this premise is load-bearing in two places. First, the warm-up stage (Sec. 4.3) trains 3DGS on full images before any 2D Gaussians exist; a semantically transient object that is geometrically consistent across training views (parked car, person standing still, slow-moving distractor within the K=4-view batch) is reconstructed as 3D Gaussians and is unlikely to be removed later, because the masked 3D loss (Eq. 11) has no gradient where the 2D mask is low, and the 2D mask is learned from residuals after the 3D model already explains the object. Second, Algorithm 1 restricts 3DGS updates to the intersection of K sampled frustums and treats it as static; if a distractor occupies that common frustum, the multi-view gradient is consistent and reinforces the ghost. The converse failure is symmetric: per-image 2D Gaussians have no cross-view constraint, so any static detail missed during warm-up can be absorbed into the 2D branch and deleted from the static scene. The reported benchmarks use mostly moving distractors and provide no stress test for stationary-but-absent transients, and no measurement of how often ground-truth transients fall inside the co-visible frustum. Until this is tested, the SOTA claim is conditional on the moving-transient regime.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes HybridGS, a hybrid scene representation for novel-view synthesis of scenes containing transient objects. The method uses per-image 2D Gaussians to model transient content and multi-view-consistent 3D Gaussians to model the static scene, together with a multi-view regulated supervision scheme (Algorithm 1) and a three-stage training procedure (warm-up, iterative training, joint fine-tuning). Experiments on NeRF On-the-go and RobustNeRF report consistent improvements over RobustNeRF, NeRF On-the-go, 3DGS, and SLS-mlp, with average gains of 1.10 dB PSNR and 5.27% SSIM on NeRF On-the-go, plus ablations and efficiency measurements.","tokens_in":16522,"tokens_out":6477,"duration_ms":64593,"significance":"If the reported results are reliable, HybridGS is a practically attractive contribution: it removes the need for semantic features, decouples transients and statics explicitly, and is faster and more storage-efficient than vanilla 3DGS on the reported Corner scene. The benchmark coverage is broad, with two established datasets, qualitative decomposition results, and ablations over components and 2D Gaussian counts. The main limitation of the claimed significance is that the paper frames the decomposition as fundamental ('transients are planar because they lack multi-view consistency'), yet the evaluation only concerns moving distractors and does not probe multi-view-consistent transients. The empirical case is also weakened by the absence of variance statistics and code. With those gaps addressed, the SOTA claim would be credible.","major_comments":[{"comment":"The central premise that transients can be modeled as planar objects because they 'lack multi-view consistency and usually only appear in a single view' is load-bearing but untested for multi-view-consistent transients. A stationary or slowly moving distractor (e.g., a parked car visible in a contiguous block of frames) is multi-view consistent during those frames and will be captured by the 3DGS branch in the warm-up stage. Afterwards, Eq. (11) masks the 3D loss with (1-M_t), while M_t is learned from the residuals of a 3D model that already explains the distractor, so the 2D branch has no gradient signal to claim the region. Algorithm 1 further reinforces such a distractor when it falls in the intersection of the K sampled frustums. The paper should either restrict the claim to moving transients or provide an experiment that measures the method's behavior on stationary-but-transient objects and reports the fraction of transient pixels inside the co-visible frustum during training.","section":"Sec. 4.2, Sec. 4.3, Algorithm 1"},{"comment":"The training schedule is stated inconsistently. Section 5.1 says the process uses a 1k-step warm-up, then iterative training with 10k 2DGS steps and 1k 3DGS steps, and a 30k-step joint fine-tuning. Figures 8 and 10 instead state 'Warm-up: 0∼1,010, Iterative Training: 1,010∼40,400, Joint Training: 40,400∼60,600', which implies roughly 1k/39.4k/20.2k steps. These two accounts cannot both be right, and the discrepancy affects both the efficiency claim (0.18 GPU hours) and the convergence analysis in Figure 10. Please specify the exact number of iterations for each stage and for each branch, and align the figure captions.","section":"Sec. 5.1 and Figures 8/10"},{"comment":"The empirical evidence consists of single runs with no variance estimate or repeated-seed statistics, and no code is released. Given that several reported gaps over SLS-mlp are small (e.g., 0.10 dB on Android in Table 2), the 'sets a new standard' claim is stronger than the evidence supports. Please report mean/std over at least three runs or, at minimum, state that the numbers are from a single run and make the code available to allow reproduction.","section":"Tables 1-4"}],"minor_comments":[{"comment":"The sentence 'we introduce our hybrid representation of scenes using 3D Gaussians in Sec. 4.1 and 2D Gaussians in Sec. 8.1' refers to a supplementary section; the 2D Gaussian description belongs in the main paper or the reference should be to a main-paper section.","section":"Sec. 4, first paragraph"},{"comment":"The word 'Guassians' should be 'Gaussians' in the sentence '10k 2D Guassians achieve...'.","section":"Sec. 5.3"},{"comment":"The 2D Gaussian rendering is written as a plain sum of alpha-weighted colors without transmittance terms; please clarify whether this is intentional and describe any normalization or saturation handling, since it differs from the alpha-blending formula in Eq. (4).","section":"Eq. (6)"},{"comment":"The update step is written as a generic parameter update without specifying the loss; state explicitly that the loss is Eq. (9) during warm-up and Eq. (11) during iterative training, and when the binarization threshold ε=0.1 is applied.","section":"Algorithm 1"},{"comment":"The caption explains that Crab (1) and Crab (2) differ in the test set, but the main text never defines them; please move that explanation into Section 5.2.2.","section":"Table 2"},{"comment":"The Photo Tourism results are only qualitative; a sentence clarifying that this dataset is not evaluated quantitatively would help readers interpret Figure 11.","section":"Sec. 5.4 and Figure 11"}],"recommendation":"major_revision","confidential_remarks":"The manuscript's citation practice is acceptable; the self-citation to VastGaussian is relevant and not excessive. The main editorial concern is that the paper's empirical claims are presented as definitive ('sets a new standard') without code or repeated runs. If the authors can address the stationary-transient stress test and provide code and standard deviations, the paper would be a good fit for the journal."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a good paper in its stated regime. Modeling transients as per-image 2D Gaussians and statics as 3D Gaussians is a simple idea that I have not seen elsewhere in the cited literature, and the paper works it through honestly. The empirical work is solid: consistent gains over SLS-mlp and others on RobustNeRF and the six NeRF On-the-go scenes they report, plus ablations showing each component earns its keep. The efficiency numbers (0.18 GPU hours, lower storage than vanilla 3DGS) are a real plus, and the writing is refreshingly clear about the central assumption: transients are treated as planar because they lack multi-view consistency and usually appear in a single view.\n\nThe soft spot is exactly that assumption, and it is load-bearing. The paper never tests what happens when a distractor is visible across multiple views—a parked car, a person standing still, a slow-moving object. In the warm-up stage, 3DGS trains on full images before any 2D masks exist, so a multi-view-consistent distractor gets baked into the 3D model. Later, the masked 3D loss suppresses gradients where the 2D mask says transient, but if the 3D model already explains the object, residuals are small and the mask may never learn to cover it. Algorithm 1's co-visible frustum (K=4 random views) has the same failure mode: if the distractor sits in all four views, the multi-view gradient reinforces it. This is not fatal for the moving-distractor benchmark regime, where the method clearly works, but the paper's claim of a 'new standard for novel view synthesis in scenes with transients' is too broad without stress-testing stationary-but-transient objects.\n\nOther issues are minor but real. The training-step counts are inconsistent: Sec. 5.1 says 1k warm-up, 10k+1k iterative, 30k joint, while the supplementary figures for Corner (101 images) show 1,010 warm-up, 1,010–40,400 iterative, and 40,400–60,600 joint. That needs fixing or explaining. No code is released, and there are no variance or repeat-run statistics. The 2D-Gaussian-count ablation is on a single scene.\n\nBottom line: this deserves a serious referee. The idea is clean, the experiments are mostly convincing for moving transients, and the limitation is stated clearly even if not tested. I would suggest the authors add a stationary-distractor experiment, reconcile the step counts, and release code. If I worked in this subfield, I would want it in the literature.","headline":"A clean empirical contribution for removing moving distractors in 3DGS, with a clearly stated but untested assumption about single-view transients.","tokens_in":17089,"tokens_out":2973,"would_cite":true,"duration_ms":29780,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Hybrid 2D and 3D Gaussians separate transient objects from static scenes in casually captured photos.","keywords":["3D Gaussian Splatting","Novel view synthesis","Transient object decomposition","2D Gaussians","Multi-view consistency","Casually captured images","NeRF On-the-go","RobustNeRF"],"falsifier":"Render a scene where the same moving distractor (for example, a person or vehicle) is visible from multiple overlapping viewpoints at the same time and position: if the method renders it inconsistently across those views or folds it into the static 3D model, the single-view planar assumption is violated. A synthetic scene with a known transient object present in all K co-visible sampled views should cause the multi-view supervision to treat it as static, which can be directly checked against a ground-truth segmentation.","tokens_in":15989,"feed_emoji":"📸","tokens_out":4212,"duration_ms":36854,"temperature":0.7,"pith_summary":"The paper argues that transient objects in casually captured photos can be modeled as flat, image-specific 2D Gaussians while the static scene lives in a shared set of 3D Gaussians. This split follows from a simple observation: transients appear in only one view and therefore violate multi-view consistency, whereas the static background satisfies it. On the NeRF On-the-go and RobustNeRF benchmarks, the method reports a PSNR gain of about 1.1 dB over the previous best and better SSIM on all scenes, plus faster training and lower memory than plain 3DGS. The paper also introduces multi-view regulated supervision and a three-stage training schedule that together keep the 2D/3D decomposition stable.","feed_headline":"2D+3D Gaussians split transients from statics in casual photos","feed_subtitle":"Per-image 2D Gaussians absorb moving objects; 3D Gaussians keep the static scene clean, beating previous best by about 1.1 dB PSNR.","key_machinery":"The load-bearing object is the hybrid Gaussian representation: per-image 2D Gaussians, whose centroid, scale, rotation, color, and opacity live in image space, and 3D Gaussians, whose parameters live in world space and are initialized from COLMAP point clouds. The 2D Gaussians render a transient image and a mask; the 3D Gaussians render the static image; the final render is a mask-weighted blend. To keep the 3D branch static-only, the paper supervises it only on the intersection of the frustums of K randomly sampled views, and restricts optimization to Gaussians whose centers fall in that co-visible region. The training alternates between freezing one branch and refining the other, then joint fine-tuning, so the 2D Gaussians learn residuals that the 3D branch cannot explain.","core_discovery":"The central discovery is that the static/transient decomposition can be achieved without semantic segmentation or per-image uncertainty networks by choosing the representation to match the geometric consistency of each part. Transient content is rendered by a fixed number of 2D Gaussians per image, which act as a per-view planar layer that also produces a soft transient mask via accumulated opacity. The static scene is rendered by 3D Gaussians supervised only in co-visible frustum regions across K sampled views, which prevents transient pixels from being baked into the 3D model. The two layers are combined with alpha blending and trained in three stages: warm-up, alternating refinement, and joint fine-tuning. The paper reports state-of-the-art results on both benchmark datasets and shows that the learned masks capture not just pedestrians and vehicles but also shadows and motion blur.","pith_inferences":["The same 2D/3D split could be extended to model appearance and illumination changes in unconstrained web photo collections, since the paper notes that 2D Gaussians also capture photometric differences between images.","The per-image planar assumption implies a testable limit: a scene where the same transient object is visible from multiple viewpoints at the same time (for example, a stationary car photographed from several angles) will force the 3D branch to absorb it, blurring the decomposition.","A video variant could reuse 2D Gaussians across nearby frames with a small motion prior instead of allocating a full set per frame, reducing per-image storage and training cost."],"forward_implications":["Clean static novel views can be rendered from casually captured phone photos without semantic labels or pretrained feature networks.","Explicit transient masks are produced for free from the 2D Gaussian opacity, useful for editing or filtering.","The method needs fewer 3D Gaussians because transients are not baked in, cutting storage and enabling faster training and rendering than plain 3DGS.","The decomposition naturally absorbs non-semantic per-image effects like shadows and motion blur, not just recognized objects.","The multi-view supervision stabilizes training and reduces overfitting to training views."],"supporting_citations":[{"why":"Supplies the 3D Gaussian Splatting representation and rasterization pipeline that forms the static branch.","marker":"[11]"},{"why":"Introduces the NeRF On-the-go benchmark dataset with casual captures and transients used for evaluation.","marker":"[24]"},{"why":"Provides the RobustNeRF dataset and a robust-loss baseline that the method must outperform.","marker":"[26]"},{"why":"SLS-mlp is the prior state-of-the-art 3DGS transient-handling method that HybridGS compares against.","marker":"[27]"},{"why":"Shows 2D Gaussians as an efficient image representation, the basis for the transient branch.","marker":"[48]"},{"why":"Taming 3DGS is the implementation base the method builds on.","marker":"[28]"},{"why":"gsplat provides the open-source rendering library with custom CUDA kernels used for both branches.","marker":"[43]"}],"fun_headline_variants":["HybridGS: 2D Gaussians handle transients, 3D keeps statics","Per-image 2D planes + 3D Gaussians split transients from statics","HybridGS: decompose scenes by consistency, not semantics","2D splats for transients, 3D splats for statics, no masks needed"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Transient objects can always be treated as flat single-view objects because they never appear consistently across multiple views; if a transient is visible from several angles at the same location, the per-image 2D Gaussian representation cannot model it and the static/transient split would blur.","fun_headline_variants_meta":{"raw":{"variants":["HybridGS: 2D Gaussians handle transients, 3D keeps statics","Per-image 2D planes + 3D Gaussians split transients from statics","HybridGS: decompose scenes by consistency, not semantics","2D splats for transients, 3D splats for statics, no masks needed"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000624,"raw_usage":{"total_tokens":2887,"prompt_tokens":940,"completion_tokens":1947,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":556,"completion_tokens_details":{"reasoning_tokens":1856}},"tokens_in":556,"tokens_out":1947,"duration_ms":14739,"temperature":1.0,"reasoning_tokens":1856,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T22:01:58.018466+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Render a scene where the same moving distractor (for example, a person or vehicle) is visible from multiple overlapping viewpoints at the same time and position: if the method renders it inconsistently across those views or folds it into the static 3D model, the single-view planar assumption is violated. A synthetic scene with a known transient object present in all K co-visible sampled views should cause the multi-view supervision to treat it as static, which can be directly checked against a ground-truth segmentation.","supporting_citations":[{"cited_title":"3d gaussian splatting for real-time radiance field rendering","cited_arxiv_id":null,"evidence_quote":"Supplies the 3D Gaussian Splatting representation and rasterization pipeline that forms the static branch."},{"cited_title":"Nerf on-the-go: Exploiting uncertainty for distractor-free nerfs in the wild","cited_arxiv_id":null,"evidence_quote":"Introduces the NeRF On-the-go benchmark dataset with casual captures and transients used for evaluation."},{"cited_title":"Fleet, and Andrea Tagliasacchi","cited_arxiv_id":null,"evidence_quote":"Provides the RobustNeRF dataset and a robust-loss baseline that the method must outperform."},{"cited_title":"Gaussianimage: 1000 fps image representation and compres- sion by 2d gaussian splatting","cited_arxiv_id":null,"evidence_quote":"Shows 2D Gaussians as an efficient image representation, the basis for the transient branch."},{"cited_title":"Taming 3dgs: High-quality radiance fields with limited resources","cited_arxiv_id":null,"evidence_quote":"Taming 3DGS is the implementation base the method builds on."}],"review_version":1}