{"id":"314ed299-9a81-4fc0-8094-02d5f454f2ec","arxiv_id":"2606.28656","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":6.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"SemDynReg constructs per-object ID maps from SAM and image features to regularize position, scale, and rotation of top-k Gaussians per object in dynamic 3DGS.","lead":"This paper adds object-level consistency constraints to deformation fields in dynamic 3D Gaussian Splatting by building semantic object-ID maps from SAM masks and feature matching. A smart generalist might read it to see whether semantic cues can reduce unwanted inter-object coupling in real-time dynamic scene rendering.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"Object-ID map accuracy from feature relevance matching is the load-bearing assumption for correct regularization.","rationale":"The reader's weakest assumption directly identifies the same dependency on correct object-ID map construction. Because the review is abstract-only, the concern remains untested; full-text experiments might contain supporting ablations, but the load-bearing risk is unchanged.","tokens_in":1744,"tokens_out":281,"duration_ms":24070,"concrete_test":"On a held-out subset of frames, compute mean IoU between the generated object-ID map and SAM-derived (or manual) object labels; if average IoU < 0.85, re-train/evaluate the deformation model using oracle object IDs and measure whether the PSNR/SSIM advantage over baselines persists.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim rests on semantics-guided regularization improving object-level deformation consistency. This requires the object-ID map (built by feature relevance matching against a predefined dictionary, then used to select pixel-wise top-k Gaussians) to assign labels without errors. Label errors would cause regularization on position/scale/rotation to be applied to incorrect groups, breaking intra-object consistency and potentially invalidating the reported PSNR/SSIM/LPIPS gains. The abstract states the construction but supplies no accuracy metric, ablation, or failure-case analysis for the matching step.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The manuscript introduces SemDynReg, a semantics-guided framework for dynamic 3D Gaussian Splatting. It extracts segmentation masks with SAM, derives semantic features, builds an object-ID map through feature relevance matching against a predefined object dictionary, selects pixel-wise top-k contributing Gaussians per object, and applies consistency regularization on their position, scale, and rotation parameters. The central claim is that this approach improves object-level deformation consistency and outperforms baselines in rendering quality metrics (PSNR, SSIM, LPIPS).","tokens_in":1854,"tokens_out":383,"duration_ms":23774,"significance":"If the experimental claims hold, the work addresses a key limitation in deformation field-based 3DGS methods by introducing explicit object-level semantic constraints, which could lead to more consistent deformations in dynamic scenes. This is a targeted improvement over prior methods that lack such modeling. The use of SAM and feature matching provides a concrete mechanism for incorporating semantics.","major_comments":[{"comment":"The abstract asserts improvements in PSNR, SSIM, and LPIPS without providing any quantitative tables, ablation studies, error bars, or details on how the regularization loss is balanced against the rendering loss. This makes the central claim unverifiable from the provided information.","section":"Abstract"},{"comment":"The object-ID map is built via feature relevance matching with a predefined dictionary and used to select top-k Gaussians for regularization. However, no accuracy metric, ablation study, or analysis of potential label errors is presented, despite this being load-bearing for ensuring the regularization is applied to the correct object groups without misdirection.","section":"Method (object-ID map construction)"}],"minor_comments":[{"comment":"The project page URL is provided, which is helpful for reproducibility.","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive comments. We address each major point below and indicate planned revisions to strengthen the manuscript.","responses":[{"response":"We agree that the abstract, as a concise summary, does not contain tables, ablations, or loss-weighting details. The full quantitative results (including PSNR/SSIM/LPIPS tables, ablations, and the regularization loss formulation with its balancing hyperparameter) appear in Sections 4 and 3.2 of the manuscript. To improve verifiability at a glance, we will revise the abstract to include specific average metric improvements reported in the experiments while respecting length limits.","revision_made":"yes","referee_comment":"[Abstract] The abstract asserts improvements in PSNR, SSIM, and LPIPS without providing any quantitative tables, ablation studies, error bars, or details on how the regularization loss is balanced against the rendering loss. This makes the central claim unverifiable from the provided information."},{"response":"We acknowledge that the reliability of the object-ID map is central to the method. The current manuscript does not report accuracy metrics or error analysis for this component. In the revised version we will add an ablation study quantifying object-ID map accuracy (using available ground-truth masks where possible), sensitivity to label errors, and the downstream effect on deformation consistency and rendering metrics.","revision_made":"yes","referee_comment":"[Method (object-ID map construction)] The object-ID map is built via feature relevance matching with a predefined dictionary and used to select top-k Gaussians for regularization. However, no accuracy metric, ablation study, or analysis of potential label errors is presented, despite this being load-bearing for ensuring the regularization is applied to the correct object groups without misdirection."}],"tokens_in":1375,"tokens_out":388,"duration_ms":22596,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The main takeaway is that this paper adds an object-level regularization step to dynamic 3D Gaussian Splatting. They run SAM to get masks, pull semantic features from the images, match those features against a fixed object dictionary to make an object-ID map, then pick the pixel-wise top-k Gaussians for each object and add a consistency term on their position, scale, and rotation. The goal is to stop Gaussians inside one object from drifting apart or getting pulled by neighboring objects.\n\nThat combination of semantic guidance and per-object top-k selection is not in the prior dynamic-3DGS papers they cite, so the technique itself is new. It directly targets the stated weakness of plain deformation fields.\n\nThe evidence side is thin. The abstract says the method raises PSNR and SSIM while lowering LPIPS, yet it shows no tables, no ablations, no loss-weighting details, and no check on whether the object-ID map is accurate. The stress-test note is right: if the feature matching assigns the wrong labels, the regularization gets applied to the wrong Gaussians and any reported gains become unreliable. Without those checks the central claim stays unverified.\n\nThis is for people already running dynamic 3DGS who care about object consistency in rendered output. A reader in that narrow area could pick up the regularization trick if the experiments later prove solid.\n\nI would send it to peer review. The problem is real, the proposed fix is straightforward, and a referee can ask for the missing numbers and validation steps.","headline":"The new piece is using SAM masks plus feature matching to build an object-ID map and then regularize deformation only on the top-k Gaussians per object; the abstract gives no numbers or ablations so the quality gains cannot be checked.","tokens_in":2352,"tokens_out":404,"would_cite":false,"duration_ms":28733,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Semantics-guided regularization on top-k Gaussians per object enforces consistent deformation in dynamic 3D Gaussian Splatting.","keywords":["dynamic 3D Gaussian splatting","deformation regularization","semantic guidance","object-level consistency","rendering quality","Segment Anything Model","object-ID map"],"falsifier":"Direct measurement of intra-object deformation variance (for example, standard deviation of position or rotation updates among Gaussians sharing the same object label) on a held-out dynamic sequence; if the variance remains as high as in baselines, the regularization has not produced the claimed consistency.","tokens_in":2636,"feed_emoji":"🔄","tokens_out":673,"duration_ms":22237,"temperature":0.7,"pith_summary":"The paper presents a framework that adds object-level semantic constraints to the deformation process in 3D Gaussian Splatting for dynamic scenes. It first obtains segmentation masks and semantic features, then builds an object-ID map through relevance matching against a fixed dictionary. Guided by this map, the method locates the top-k Gaussians that contribute to each object's pixels and applies regularization to keep their position, scale, and rotation changes consistent within the object. A reader would care because existing deformation-field methods allow Gaussians inside one object to move independently or pull neighboring objects along, producing visible artifacts; the new constraints aim to eliminate that.","feed_headline":"Object-ID map drives consistent deformation in dynamic 3DGS","feed_subtitle":"Semantics assign top-k Gaussians per object and regularize their position, scale, and rotation, raising PSNR and SSIM.","key_machinery":"Object-level consistency regularization applied to position, scale, and rotation of top-k Gaussians identified via the object-ID map.","core_discovery":"By constructing an object-ID map from semantic feature matching and then imposing consistency regularization on the deformation parameters of the pixel-wise top-k contributing Gaussians for each object, the approach achieves spatially consistent object-wise deformation that prior methods without explicit object-level constraints cannot guarantee.","pith_inferences":["The same object-ID map could be reused at inference time to support per-object editing or freezing of motion without retraining the deformation field.","Scenes containing many small or partially occluded objects may require a larger top-k value or a more robust dictionary to maintain label accuracy.","Extending the regularization to also penalize velocity or acceleration differences among same-object Gaussians could further stabilize long-term dynamics."],"forward_implications":["Object-level deformation consistency improves because Gaussians assigned to the same object are forced to share similar deformation updates.","Rendering quality rises, with higher PSNR and SSIM and lower LPIPS, because inconsistent motion and inter-object coupling are reduced.","Unwanted coupling between distinct objects decreases because regularization is applied separately per object-ID.","The method remains compatible with existing deformable 3DGS pipelines since the regularization term is added on top of the original deformation field."],"fun_headline_variants":["Semantic map regularizes top-k Gaussians for object consistency","Object-ID map enforces deformation consistency in dynamic 3DGS","Semantics guide per-object regularization of Gaussian parameters","Feature matching yields consistent dynamic 3D Gaussian deformations","Object-level cues regularize position scale and rotation in 3DGS"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"The object-ID map produced by feature relevance matching against a predefined dictionary correctly labels the top-k Gaussians belonging to each object without introducing errors that misdirect the regularization.","fun_headline_variants_meta":{"raw":{"variants":["Semantic map regularizes top-k Gaussians for object consistency","Object-ID map enforces deformation consistency in dynamic 3DGS","Semantics guide per-object regularization of Gaussian parameters","Feature matching yields consistent dynamic 3D Gaussian deformations","Object-level cues regularize position scale and rotation in 3DGS"]},"model":"grok-4.3","cost_usd":0.003419,"raw_usage":{"total_tokens":1727,"prompt_tokens":665,"num_sources_used":0,"completion_tokens":79,"cost_in_usd_ticks":34190500,"prompt_tokens_details":{"text_tokens":665,"audio_tokens":0,"image_tokens":0,"cached_tokens":64},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":983,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":665,"tokens_out":79,"duration_ms":13358,"temperature":1.0,"reasoning_tokens":983,"cache_read_input_tokens":64,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-30T10:17:43.777424+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"Direct measurement of intra-object deformation variance (for example, standard deviation of position or rotation updates among Gaussians sharing the same object label) on a held-out dynamic sequence; if the variance remains as high as in baselines, the regularization has not produced the claimed consistency.","supporting_citations":[],"review_version":1}