{"id":"451f750c-6876-4afe-99fe-237f4b8edd9e","arxiv_id":"2411.10033","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A text-driven 3D editing framework that tags 3D Gaussian points via cross-attention and uses SDS plus pseudo-GT guidance to edit only the target region.","lead":"GSEditPro offers text-only editing of 3D scenes by labeling each 3D Gaussian point with a semantic tag obtained from cross-attention maps of a fine-tuned text-to-image model. It locates the object to edit without manual masks and preserves the rest of the scene, which could simplify 3D content editing for games, film, and VR.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"GSEditPro does not specify how binary labels propagate to Gaussians created by cloning/splitting during SDS optimization, so the central claim of precise 3D localization is under-specified.","rationale":"The reader correctly identifies localization as the load-bearing component, but locates the risk in cross-attention map quality and SAM refinement. My concern is more specific and more decisive: even granting perfect initial attention masks, the paper does not explain how the 3D labels survive the densification process that the optimization explicitly invokes. The text says gradients are confined to labeled Gaussians (Sec. 3.2) and also says SDS guides cloning and splitting (Sec. 3.3), but no label-inheritance rule is given. This is not a matter of tuning or consensus; it is an internal specification gap. If labels are static, geometry edits requiring new Gaussians outside the initial mask should fail; if labels are inherited, the paper omits a mechanism that is central to its claimed progressive localization. A conditional acceptance is appropriate, but the condition should explicitly require either a stated label-propagation procedure or released code demonstrating one. The reader's attention-map concern remains valid as an empirical risk, which is why my agreement is partial rather than full.","tokens_in":14762,"tokens_out":5026,"duration_ms":58921,"concrete_test":"Instrument the 3D-GS optimization loop with per-Gaussian label logging during densification on one insertion task (e.g., 'a * man wearing a birthday hat'). After each clone/split operation, record whether the new Gaussian inherits a label and whether it receives SDS gradient. Then check two facts: (1) all Gaussians receiving non-zero SDS gradient are labeled; (2) the final edited region contains Gaussians that were not labeled at initialization. If both hold, label propagation must be happening through an unspecified mechanism; if the edited region contains only originally labeled Gaussians, the claimed geometry expansion needs re-explanation. This single experiment distinguishes a fixable implementation detail from a genuine gap in the central localization claim.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The method's central claim is that gradients reach only the labeled Gaussians: Section 3.2 states 'we only allow gradients to propagate within the labeled Gaussians whose weights of back-projection bigger than the threshold.' But Section 3.3 says the SDS loss 'guide[s] the cloning and splitting of the Gaussians,' and Equation (7) includes anchor losses over positions, scales, rotations, opacities, and colors. The paper never states what happens to a Gaussian's binary label when it is cloned or split during optimization. If newly created Gaussians are unlabeled, they receive no SDS gradient, so the edited geometry cannot grow beyond the initially labeled set, which contradicts the shown beard, hat, and object-insertion results. If they inherit the parent label, then a label-propagation or tracking rule is required, analogous to GaussianEditor's semantic tracking, and none is described. This is an internal gap, independent of whether the cross-attention masks are initially accurate, and it makes the claimed 'progressive localization' unverifiable without code or an explicit mechanism.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes GSEditPro, a text-only 3D scene editing method for 3D Gaussian Splatting. The method first fine-tunes Stable Diffusion with DreamBooth and collects cross-attention maps for the target token during image-to-image generation; these maps are thresholded, clustered with DBSCAN, optionally refined with SAM point prompts, and backprojected into 3D to assign binary labels to Gaussians. Only labeled Gaussians receive gradients from an SDS loss (Eq. 3), while a pseudo-GT image (Eq. 5) combined with L1 and D-SSIM losses (Eq. 6) and anchor losses (Eq. 4) constrain the optimization. The paper reports qualitative results on object replacement and insertion, ablations for localization and pixel guidance, CLIP directional similarity scores, and user studies against Instruct-NeRF2NeRF, GaussianEditor, and DreamEditor.","tokens_in":14999,"tokens_out":4887,"duration_ms":46484,"significance":"If the method works as described, it is a practical contribution: it removes manual mask and depth input, supports object insertion, and introduces a semantic labeling scheme for Gaussians. The paper is honest in reporting a failure case and includes ablations that support the usefulness of localization and pixel-level guidance. However, the central mechanism (attention-derived 3D labels) is only qualitatively validated, and two technical details—label inheritance during adaptive density control and the actual role of the pseudo-GT loss—need to be pinned down before the claims can be fully assessed. The reported gains in CLIP dir and user preference are consistent with the qualitative results but are not accompanied by statistical analysis.","major_comments":[{"comment":"Section 3.3 states that LSDS \"guide[s] the cloning and splitting of the Gaussians\" (Eq. 3), while Section 3.2 restricts gradient propagation to \"labeled Gaussians whose weights of back-projection bigger than the threshold.\" The paper never states whether the binary label is inherited by Gaussians created during cloning or splitting. If the new Gaussians are unlabeled, they receive no SDS gradient and the edited geometry cannot grow beyond the initially labeled set, which contradicts the beard, hat, and object-insertion results. If they inherit the parent label, a tracking rule equivalent to GaussianEditor's semantic tracking is needed, and none is described. Please specify the label-propagation rule and the back-projection weight threshold used in the experiments.","section":"Section 3.2/3.3"},{"comment":"In Eq. (5), Ipgt = M \\odot Iedit + (1 - M) \\odot Iori, so inside the mask Ipgt equals Iedit. Consequently the L1 term of Eq. (6) vanishes in the edited region, and only the masked-out region (with some boundary influence via D-SSIM) is constrained by the preservation loss. The paper's third contribution claims pixel-level guidance \"guide[s] 3D Gaussian rendering for more detailed results,\" but Eqs. (5)-(6) do not directly supervise the inside of the mask; the edited region is driven only by SDS. Please clarify what \"detailed results\" means here and state whether any term in Eq. (7) supervises the inner masked region beyond LSDS.","section":"Section 3.4, Eqs. (5)-(6)"},{"comment":"The localization module is the load-bearing component of the method, but no experiment quantifies its accuracy. The 2D masks come from thresholded cross-attention maps, DBSCAN filtering, and SAM refinement, and are backprojected into 3D labels; Section 4.4 only shows an end-to-end ablation (with vs. without localization) and qualitative figures. Please add a quantitative evaluation of the masks or labels—for example, IoU against hand-annotated masks on sampled views, or precision/recall of the 3D labels—so the \"precise localization\" claim can be separated from the optimization quality.","section":"Section 3.2 / Section 4.4"}],"minor_comments":[{"comment":"The term \"Img2ImgPipe\" is used without definition; please specify the exact inference settings (number of steps, guidance scale, resolution, prompt format) used to obtain cross-attention maps.","section":"Section 3.2"},{"comment":"Equations (4) and (7) both use lambda with subscripts, but lambda_i in Eq. (4) indexes generation count while lambda_P in Eq. (7) indexes parameter type; please reconcile the notation.","section":"Equations (4) and (7)"},{"comment":"The Figure 3 caption refers to a \"guessed sketch\" while the text calls it a \"static mask\"; please unify the terminology.","section":"Figure 3"},{"comment":"The paper explains that DreamEditor is evaluated on a subset of its released preprocessed datasets; please state explicitly how many tasks and scenes are shared with the comparisons in Table 1, since the two tables are otherwise not directly comparable.","section":"Section 4.1"},{"comment":"Reference [CLV24] lacks publication venue and complete bibliographic information; please complete the entry.","section":"References"},{"comment":"Equation (1) uses N both as the total number of Gaussians and as the running index; please use distinct symbols for clarity.","section":"Eq. (1)"}],"recommendation":"major_revision","confidential_remarks":"The paper fits the scope of a graphics venue, and the core idea is timely. The main technical risk is the missing label-propagation rule during adaptive density control; this is fixable in revision by specifying the rule or providing code. I would also like to see a quantitative localization evaluation before endorsing the central claim. I do not see grounds for rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Read GSEditPro. The contribution is a specific composition: use cross-attention maps from a DreamBooth-fine-tuned text-to-image model to label 3D Gaussians, progressively refine those labels during optimization, and preserve the background with a pseudo-GT image. That combination is not in the cited prior work. The writing is clear, the figures are effective, and the ablation shows localization matters. The method appears to work on the shown scenes.\n\nThe soft spots are in the specification. The biggest one: Section 3.2 says gradients only flow through labeled Gaussians, yet Section 3.3 says the SDS loss guides cloning and splitting. The paper never explains what happens to a Gaussian's binary label when it is cloned or split. If new Gaussians are unlabeled, they get no SDS gradient, so geometry cannot grow beyond the initially labeled set—contradicting the beard and hat results. If they inherit the parent label, there's an implicit tracking rule that is never stated. This is an internal gap, not just a missing detail, and it makes the 'precise localization' claim unverifiable without code or an explicit mechanism.\n\nSecond, the pseudo-GT loss in Eq. 5 is mostly a background-preservation loss: inside the mask it equals I_edit, so it contributes no gradient for the edited region. That's fine for its stated purpose, but the paper's language about 'detailed results' from pixel-level guidance overstates it.\n\nThird, the evaluation is small: 14 tasks for the GaussianEditor/I-N2N comparison, 7 for DreamEditor, no error bars, and no code. That's typical for a graphics paper but it does limit confidence.\n\nThe stress-test note about label propagation is correct and hits the central claim. None of this is fatal—the method is plausible and the composition is new—but the paper needs a revision that specifies the label-propagation rule and ideally releases code.\n\nMy take: this deserves a serious referee. It's a subfield-level contribution, not a breakthrough, but it's honest work with a clear mechanism and useful ablations. I'd send it to review with a request to address the label-propagation gap. I wouldn't cite it until that's clarified.","headline":"GSEditPro is a competent composition of known ideas with a genuine new twist, but a missing label-propagation rule for cloned Gaussians leaves the central localization claim under-specified.","tokens_in":15496,"tokens_out":2617,"would_cite":false,"duration_ms":24756,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"GSEditPro claims text-only prompts are enough for precise 3D-GS editing when cross-attention maps label which Gaussians change.","keywords":["3D Gaussian Splatting","text-driven 3D editing","cross-attention localization","Score Distillation Sampling","DreamBooth fine-tuning","Segment Anything masks","pseudo ground truth","object insertion"],"falsifier":"Run the localization module on a scene where the target object's true 3D segmentation is known, backproject the resulting masks, and compute the intersection-over-union between labeled and ground-truth Gaussians across several camera views; a low overlap for objects that are small, occluded, or weakly attended would show that the attention map cannot carry the localization precision the method claims.","tokens_in":14586,"feed_emoji":"🎨","tokens_out":8116,"duration_ms":75409,"temperature":0.7,"pith_summary":"This paper argues that a 3D scene stored as Gaussian splats can be edited with a text prompt alone, if the editor first decides which Gaussians should change. Its localization module reads cross-attention maps from a DreamBooth-fine-tuned text-to-image model, turns those maps into clean 2D masks with thresholding, DBSCAN, and SAM, and backprojects the masks onto the 3D Gaussians so that only the labeled set receives gradient updates. The optimization then uses Score Distillation Sampling within the labeled region, plus a pseudo-ground-truth image that pins unchanged areas to the original render. A reader should care because prior NeRF-based editing struggles to locate objects inside implicit fields, while earlier Gaussian editors often require manual masks or depth input; the paper claims text-only precision is achievable and demonstrates it on object changing and object insertion tasks.","feed_headline":"Attention maps tell a 3D editor which Gaussians to change","feed_subtitle":"Text-only editing labels target splats, then SDS plus pseudo-ground truth keeps the rest of the scene intact.","key_machinery":"The load-bearing object is the attention-based progressive localization module. It works by taking the cross-attention maps of the edited keyword from a DreamBooth-fine-tuned text-to-image model, thresholding them, cleaning the result with DBSCAN, and using the highest-attention pixels as positive point prompts and the lowest as negative prompts for SAM, which returns a precise per-view mask. Those 2D masks are backprojected through the differentiable splatting renderer to attach a binary label to every 3D Gaussian, and only labeled Gaussians are allowed to receive SDS gradients. The same module reappears during optimization to generate a dynamic mask for the pseudo-ground-truth image, so the localization is progressive: coarse and static at first, then refreshed as the edited shape emerges.","core_discovery":"The central claim is that attention-based progressive localization converts the explicit Gaussian representation into a semantic segmentation that makes text-only editing precise. After DreamBooth fine-tunes the text-to-image model around the edited subject token, an image-to-image pass on sampled views yields cross-attention maps for the target word; the module thresholds the maps at 0.5, filters with DBSCAN, and feeds the strongest points to SAM as prompts to obtain clean per-view masks. These masks are backprojected during differentiable splatting, and Gaussians whose backprojection weight exceeds a threshold are labeled as editable, so SDS gradients only reach the intended region. During optimization the method reuses the localization module in a second stage to build a dynamic mask and constructs a pseudo-ground-truth image $I_{pgt} = M \\odot I_{edit} + (1 - M) \\odot I_{ori}$, combining edited content with preserved original pixels and supervising the render with L1 and D-SSIM losses. The paper concludes from qualitative comparisons, CLIP directional similarity, and user studies that this pipeline outperforms Instruct-NeRF2NeRF, GaussianEditor, and DreamEditor in localization accuracy and visual fidelity.","pith_inferences":["Beyond the paper: the per-Gaussian labels produced by this module could be reused as a mask for other operations, such as deletion, style transfer, or property control, without retraining the localization.","Beyond the paper: because DreamBooth fine-tuning is per subject, the cost of the method grows with the number of distinct edits; a zero-shot localizer from an off-the-shelf attention model would be the natural next test of the same idea.","Beyond the paper: a stress test worth running is occlusion and small targets; if attention maps lose the object when it occupies only a few pixels, the labels will inherit that failure, and the claimed precision would degrade exactly where manual masks used to be needed."],"forward_implications":["Text-only editing becomes possible for both changing existing objects and inserting new ones, without user-supplied masks or depth maps.","Gradients restricted to labeled Gaussians should keep non-edited geometry and appearance intact across views.","The pseudo-ground-truth pixel guidance should suppress the background bleeding that pure SDS optimization causes, because unchanged pixels are explicitly supervised toward the original render.","Because localization is refreshed during optimization, the editing region can track the evolving Gaussian distribution instead of being frozen at an initial 2D mask.","If the comparison studies are representative, users prefer the results over I-N2N, GaussianEditor, and DreamEditor in the tested scenes."],"supporting_citations":[{"why":"DreamBooth: fine-tunes the diffusion model around the subject token, supplying both the generation prior and the attention maps used for localization.","marker":"[RLJ∗23]"},{"why":"DreamFusion: introduces the Score Distillation Sampling loss that guides 3D optimization under text guidance.","marker":"[PJBM22]"},{"why":"3D Gaussian Splatting: the explicit representation whose per-Gaussian labels make the localization and gradient gating possible.","marker":"[KKLD23]"},{"why":"Prompt-to-Prompt: shows cross-attention layers link words to image regions, the premise behind using attention maps for localization.","marker":"[HMT∗22]"},{"why":"Segment Anything: refines the coarse attention-derived masks into precise per-view segmentations via point prompts.","marker":"[KMR∗23]"},{"why":"GaussianEditor: contributes the backprojection scheme and anchor loss reused here, and serves as the main 3D-GS baseline.","marker":"[CCZ∗23]"},{"why":"Instruct-NeRF2NeRF: the NeRF-based baseline whose whole-image Instruct-Pix2Pix updates motivate the need for explicit localization.","marker":"[HTE∗23]"},{"why":"DreamEditor: the mesh-based DreamBooth baseline used to compare geometry and shape quality of edits.","marker":"[ZWL∗23]"},{"why":"Stable Diffusion: the base text-to-image model whose cross-attention layers the localization reads after DreamBooth fine-tuning.","marker":"[RBL∗22]"},{"why":"Instruct-Pix2Pix: the underlying editor of the two main baselines, standing as the alternative to the DreamBooth plus SDS optimization route.","marker":"[BHE23]"}],"fun_headline_variants":["Attention maps pinpoint Gaussians for text-only 3D edits","Progressive attention localizes 3D edits with just text","Text prompts guide precise Gaussian splatting edits","Semantic labels from attention make 3D editing precise"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the cross-attention maps, after thresholding, DBSCAN clustering, and SAM refinement, are spatially aligned with the target object in every rendered view, so that backprojection labels the correct Gaussians and the edit stays inside them.","fun_headline_variants_meta":{"raw":{"variants":["Attention maps pinpoint Gaussians for text-only 3D edits","Progressive attention localizes 3D edits with just text","Text prompts guide precise Gaussian splatting edits","Semantic labels from attention make 3D editing precise"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000504,"raw_usage":{"total_tokens":2498,"prompt_tokens":1022,"completion_tokens":1476,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":638,"completion_tokens_details":{"reasoning_tokens":1409}},"tokens_in":638,"tokens_out":1476,"duration_ms":11269,"temperature":1.0,"reasoning_tokens":1409,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T20:02:19.530022+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the localization module on a scene where the target object's true 3D segmentation is known, backproject the resulting masks, and compute the intersection-over-union between labeled and ground-truth Gaussians across several camera views; a low overlap for objects that are small, occluded, or weakly attended would show that the attention map cannot carry the localization precision the method claims.","supporting_citations":[],"review_version":1}