Pith. sign in

REVIEW 3 major objections 6 minor 48 references

GSEditPro: 3D Gaussian Splatting Editing with Attention-based Progressive Localization

T0 review · 3 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read GSEditPro claims text-only prompts are enough for precise 3D-GS editing when cross-attention maps label which Gaussians change.

desk verdict GSEditPro is a competent composition of known ideas with a genuine new twist, but a missing label-propagation rule for cloned Gaussians leaves the central localization claim under-specified. read the letter →

arxiv 2411.10033 v1 pith:R32GORSH submitted 2024-11-15 cs.CV

Yanhao Sun , RunZe Tian , Xiao Han , XinYao Liu , Yan Zhang , Kai Xu This is my paper · ORCID
classification cs.CV
keywords 3DGaussianSplattingtext-driveneditingcross-attentionlocalizationScoreDistillationSamplingDreamBoothfine-tuningSegmentAnythingmaskspseudogroundtruthobjectinsertion
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that a 3D scene stored as Gaussian splats can be edited with a text prompt alone, if the editor first decides which Gaussians should change. Its localization module reads cross-attention maps from a DreamBooth-fine-tuned text-to-image model, turns those maps into clean 2D masks with thresholding, DBSCAN, and SAM, and backprojects the masks onto the 3D Gaussians so that only the labeled set receives gradient updates. The optimization then uses Score Distillation Sampling within the labeled region, plus a pseudo-ground-truth image that pins unchanged areas to the original render. A reader should care because prior NeRF-based editing struggles to locate objects inside implicit fields, while earlier Gaussian editors often require manual masks or depth input; the paper claims text-only precision is achievable and demonstrates it on object changing and object insertion tasks.

What carries the argument

The load-bearing object is the attention-based progressive localization module. It works by taking the cross-attention maps of the edited keyword from a DreamBooth-fine-tuned text-to-image model, thresholding them, cleaning the result with DBSCAN, and using the highest-attention pixels as positive point prompts and the lowest as negative prompts for SAM, which returns a precise per-view mask. Those 2D masks are backprojected through the differentiable splatting renderer to attach a binary label to every 3D Gaussian, and only labeled Gaussians are allowed to receive SDS gradients. The same module reappears during optimization to generate a dynamic mask for the pseudo-ground-truth image, so the localization is progressive: coarse and static at first, then refreshed as the edited shape emerges.

What would settle it

Run the localization module on a scene where the target object's true 3D segmentation is known, backproject the resulting masks, and compute the intersection-over-union between labeled and ground-truth Gaussians across several camera views; a low overlap for objects that are small, occluded, or weakly attended would show that the attention map cannot carry the localization precision the method claims.

Watch

Extended reading notes

Core claim

The central claim is that attention-based progressive localization converts the explicit Gaussian representation into a semantic segmentation that makes text-only editing precise. After DreamBooth fine-tunes the text-to-image model around the edited subject token, an image-to-image pass on sampled views yields cross-attention maps for the target word; the module thresholds the maps at 0.5, filters with DBSCAN, and feeds the strongest points to SAM as prompts to obtain clean per-view masks. These masks are backprojected during differentiable splatting, and Gaussians whose backprojection weight exceeds a threshold are labeled as editable, so SDS gradients only reach the intended region. During optimization the method reuses the localization module in a second stage to build a dynamic mask and constructs a pseudo-ground-truth image $I_{pgt} = M \odot I_{edit} + (1 - M) \odot I_{ori}$, combining edited content with preserved original pixels and supervising the render with L1 and D-SSIM losses. The paper concludes from qualitative comparisons, CLIP directional similarity, and user studies that this pipeline outperforms Instruct-NeRF2NeRF, GaussianEditor, and DreamEditor in localization accuracy and visual fidelity.

Load-bearing premise

The load-bearing premise is that the cross-attention maps, after thresholding, DBSCAN clustering, and SAM refinement, are spatially aligned with the target object in every rendered view, so that backprojection labels the correct Gaussians and the edit stays inside them.

Editorial extensions

If this is right

  • Text-only editing becomes possible for both changing existing objects and inserting new ones, without user-supplied masks or depth maps.
  • Gradients restricted to labeled Gaussians should keep non-edited geometry and appearance intact across views.
  • The pseudo-ground-truth pixel guidance should suppress the background bleeding that pure SDS optimization causes, because unchanged pixels are explicitly supervised toward the original render.
  • Because localization is refreshed during optimization, the editing region can track the evolving Gaussian distribution instead of being frozen at an initial 2D mask.
  • If the comparison studies are representative, users prefer the results over I-N2N, GaussianEditor, and DreamEditor in the tested scenes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the per-Gaussian labels produced by this module could be reused as a mask for other operations, such as deletion, style transfer, or property control, without retraining the localization.
  • Beyond the paper: because DreamBooth fine-tuning is per subject, the cost of the method grows with the number of distinct edits; a zero-shot localizer from an off-the-shelf attention model would be the natural next test of the same idea.
  • Beyond the paper: a stress test worth running is occlusion and small targets; if attention maps lose the object when it occupies only a few pixels, the labels will inherit that failure, and the claimed precision would degrade exactly where manual masks used to be needed.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes GSEditPro, a text-only 3D scene editing method for 3D Gaussian Splatting. The method first fine-tunes Stable Diffusion with DreamBooth and collects cross-attention maps for the target token during image-to-image generation; these maps are thresholded, clustered with DBSCAN, optionally refined with SAM point prompts, and backprojected into 3D to assign binary labels to Gaussians. Only labeled Gaussians receive gradients from an SDS loss (Eq. 3), while a pseudo-GT image (Eq. 5) combined with L1 and D-SSIM losses (Eq. 6) and anchor losses (Eq. 4) constrain the optimization. The paper reports qualitative results on object replacement and insertion, ablations for localization and pixel guidance, CLIP directional similarity scores, and user studies against Instruct-NeRF2NeRF, GaussianEditor, and DreamEditor.

Significance. If the method works as described, it is a practical contribution: it removes manual mask and depth input, supports object insertion, and introduces a semantic labeling scheme for Gaussians. The paper is honest in reporting a failure case and includes ablations that support the usefulness of localization and pixel-level guidance. However, the central mechanism (attention-derived 3D labels) is only qualitatively validated, and two technical details—label inheritance during adaptive density control and the actual role of the pseudo-GT loss—need to be pinned down before the claims can be fully assessed. The reported gains in CLIP dir and user preference are consistent with the qualitative results but are not accompanied by statistical analysis.

major comments (3)
  1. [Section 3.2/3.3] Section 3.3 states that LSDS "guide[s] the cloning and splitting of the Gaussians" (Eq. 3), while Section 3.2 restricts gradient propagation to "labeled Gaussians whose weights of back-projection bigger than the threshold." The paper never states whether the binary label is inherited by Gaussians created during cloning or splitting. If the new Gaussians are unlabeled, they receive no SDS gradient and the edited geometry cannot grow beyond the initially labeled set, which contradicts the beard, hat, and object-insertion results. If they inherit the parent label, a tracking rule equivalent to GaussianEditor's semantic tracking is needed, and none is described. Please specify the label-propagation rule and the back-projection weight threshold used in the experiments.
  2. [Section 3.4, Eqs. (5)-(6)] In Eq. (5), Ipgt = M \odot Iedit + (1 - M) \odot Iori, so inside the mask Ipgt equals Iedit. Consequently the L1 term of Eq. (6) vanishes in the edited region, and only the masked-out region (with some boundary influence via D-SSIM) is constrained by the preservation loss. The paper's third contribution claims pixel-level guidance "guide[s] 3D Gaussian rendering for more detailed results," but Eqs. (5)-(6) do not directly supervise the inside of the mask; the edited region is driven only by SDS. Please clarify what "detailed results" means here and state whether any term in Eq. (7) supervises the inner masked region beyond LSDS.
  3. [Section 3.2 / Section 4.4] The localization module is the load-bearing component of the method, but no experiment quantifies its accuracy. The 2D masks come from thresholded cross-attention maps, DBSCAN filtering, and SAM refinement, and are backprojected into 3D labels; Section 4.4 only shows an end-to-end ablation (with vs. without localization) and qualitative figures. Please add a quantitative evaluation of the masks or labels—for example, IoU against hand-annotated masks on sampled views, or precision/recall of the 3D labels—so the "precise localization" claim can be separated from the optimization quality.
minor comments (6)
  1. [Section 3.2] The term "Img2ImgPipe" is used without definition; please specify the exact inference settings (number of steps, guidance scale, resolution, prompt format) used to obtain cross-attention maps.
  2. [Equations (4) and (7)] Equations (4) and (7) both use lambda with subscripts, but lambda_i in Eq. (4) indexes generation count while lambda_P in Eq. (7) indexes parameter type; please reconcile the notation.
  3. [Figure 3] The Figure 3 caption refers to a "guessed sketch" while the text calls it a "static mask"; please unify the terminology.
  4. [Section 4.1] The paper explains that DreamEditor is evaluated on a subset of its released preprocessed datasets; please state explicitly how many tasks and scenes are shared with the comparisons in Table 1, since the two tables are otherwise not directly comparable.
  5. [References] Reference [CLV24] lacks publication venue and complete bibliographic information; please complete the entry.
  6. [Eq. (1)] Equation (1) uses N both as the total number of Gaussians and as the running index; please use distinct symbols for clarity.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation found; the method couples attention-based localization and SDS optimization through a shared diffusion prior but not by construction.

full rationale

GSEditPro's pipeline is a masked-SDS editing system, not a self-deriving result. The localization labels are obtained from cross-attention maps of a DreamBooth-fine-tuned T2I model, thresholded, DBSCAN-filtered, SAM-refined, and back-projected into 3D-GS. The SDS loss then updates Gaussians under the same text prompt; this is a shared generative prior, not a fitted parameter renamed as a prediction. The pseudo-GT construction Ipgt = M ⊙ Iedit + (1 − M) ⊙ Iori in Eq. (5) is a deliberate masked blend: by construction the preservation loss in Eq. (6) acts only outside M, so it cannot be accused of secretly defining the edit target inside M; the paper uses it as a background consistency term, while the actual edit is driven by LSDS. No numerical constant is fitted to the evaluation metric, no external unicity theorem is imported from the authors' prior work, and the cited baselines and priors (GaussianEditor, DreamBooth, DreamFusion) are outside the author list. The only self-referential aspect is that the same diffusion model supplies both the spatial prior (attention maps) and the score supervision; that is a design coupling, not an equation-level identity. Therefore no circular step meets the quoted-reduction bar.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the reliability of cross-attention maps, SAM segmentation, and 3D-GS backprojection. Many hyperparameters are hand-chosen and not reported numerically, which increases the burden on the reader to trust the qualitative and quantitative results. No new physical or conceptual entities are introduced.

free parameters (6)
  • attention map threshold = 0.5
    Used to binarize attention maps into preliminary editing masks in Section 3.2; no sensitivity analysis is provided.
  • SAM point prompt counts = top 5 positive, bottom 3 negative
    Section 3.2: point prompts for SAM are chosen as the top 5 and lowest 3 attention values; hand-set without ablation.
  • DBSCAN parameters = not specified
    Used to filter outlier attention regions in Section 3.2; exact eps and min_samples are not reported.
  • static mask duration = 2000 iterations
    Section 3.4: the first 2000 training iterations use the static mask before switching to the dynamic mask.
  • loss weights lambda_sds, lambda_l1, lambda_ssim, lambda_P = not specified
    Equation 7 defines the total loss but gives no numeric values for the weighting coefficients.
  • anchor loss schedule = lambda_i grows with generation index
    Equation 4: the anchor loss weights increase with the number of terms, but the exact schedule is not specified.
assumptions (5)
  • domain assumption Cross-attention maps of the fine-tuned diffusion model spatially indicate the location of the target word in the rendered image.
    Section 3.2, based on Prompt-to-Prompt; load-bearing for localization.
  • domain assumption SAM can produce precise segmentation masks from point prompts derived from attention maps.
    Section 3.2; relies on SAM's point-prompting capability.
  • domain assumption Backprojection of 2D masks through 3D-GS rendering is accurate enough to label the correct Gaussians.
    Section 3.2; similar to GaussianEditor's backprojection approach.
  • domain assumption COLMAP camera poses and depths are reliable for multi-view consistency.
    Section 4.1 dataset setup; used for training the original Gaussians and for sampled render views.
  • domain assumption DreamBooth fine-tuning preserves the T2I model's editing ability and improves attention map accuracy.
    Section 3.2; the method depends on the fine-tuned model for both localization and editing guidance.

how reviews work

0 comments
Cite this review

Pith. "Pith review of GSEditPro: 3D Gaussian Splatting Editing with Attention-based Progressive Localization." pith.science (2026). https://pith.science/paper/R32GORSH

@misc{pith2026241110033,
  author       = {Pith},
  title        = {Pith review of: GSEditPro: 3D Gaussian Splatting Editing with Attention-based Progressive Localization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/R32GORSH}},
  note         = {Machine review of arXiv:2411.10033}
}
read the original abstract

With the emergence of large-scale Text-to-Image(T2I) models and implicit 3D representations like Neural Radiance Fields (NeRF), many text-driven generative editing methods based on NeRF have appeared. However, the implicit encoding of geometric and textural information poses challenges in accurately locating and controlling objects during editing. Recently, significant advancements have been made in the editing methods of 3D Gaussian Splatting, a real-time rendering technology that relies on explicit representation. However, these methods still suffer from issues including inaccurate localization and limited manipulation over editing. To tackle these challenges, we propose GSEditPro, a novel 3D scene editing framework which allows users to perform various creative and precise editing using text prompts only. Leveraging the explicit nature of the 3D Gaussian distribution, we introduce an attention-based progressive localization module to add semantic labels to each Gaussian during rendering. This enables precise localization on editing areas by classifying Gaussians based on their relevance to the editing prompts derived from cross-attention layers of the T2I model. Furthermore, we present an innovative editing optimization method based on 3D Gaussian Splatting, obtaining stable and refined editing results through the guidance of Score Distillation Sampling and pseudo ground truth. We prove the efficacy of our method through extensive experiments.

Figures

Figures reproduced from arXiv: 2411.10033 by the authors.

Figure 1
Figure 1. Results of GSEditPro. GSEditPro enables users to conduct high-quality editing in various scenes using text prompts only. Abstract With the emergence of large-scale Text-to-Image(T2I) models and implicit 3D representations like Neural Radiance Fields (NeRF), many text-driven generative editing methods based on NeRF have appeared. However, the implicit encoding of geometric and textural information poses challenges in… view at source ↗
Figure 2
Figure 2. Method Overview. GSEditPro edits scenes represented by 3D Gaussian Splatting using text prompt only. The key process of our method contains three parts: 1) Locating Gaussian editing regions via cross-attention, which assigns semantic labels to each Gaussian and determines whether the gradients can be propagated between them (Section 3.2); 2) Optimizing editing regions using DreamBooth, which uses LSDS as the guidanc… view at source ↗
Figure 3
Figure 3. The visual difference of the Mask between the two stages. The static Mask is a guessed sketch from the DreamBooth, and the dynamic mask is located in real-time after 3D-GS capable of rendering decent edited results. there may still be changes outside the editing region in rendered re￾sults. We create a pseudo-GT image for each rendered image dur￾ing training to ensure consistency between the editing results and the … view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: More multi-view results of our method in different scenes. The left column is the original view and the other three columns are the multi-view editing results. Our method is capable of conducting various kinds of editing in different scenes. Different from most previou…
Figure 5
Figure 5. Figure 5: Comparisons on adding objects to the given scene. Results of the GaussianEditor shown above are generated without manually adjusting the estimated depth during training. Our method gives attention to proper locations and generates satisfactory results. In contrast, Gau…
Figure 6
Figure 6. Figure 6: Comparisons with GaussianEditor [CCZ∗ 23] and I-N2N [HTE∗ 23]. We show superior ability in making successful edits and controlling editing regions, without leaking noisy Gaussians to backgrounds and irrelative regions. As presented in [PITH_FULL_IMAGE:figures/full_fig…
Figure 7
Figure 7. Figure 7: Comparisons with DreamEditor [ZWL∗ 23]. Our method adds a better-shaped mustache on the man, and a more detailed red top hat on the plush toy. For the dog, we make a more precise editing. Please be aware that the fragmented clothing in the first scene and the rough edg…
Figure 8
Figure 8. Figure 8: Ablation study on attention-based localization of edit￾ing areas in 3D. Prompt: "a * stump on fire". Utilizing attention￾based localization in 3D allows for better preservation of the entire scene’s details. Without our 3D localization, plants near the stump lose their…
Figure 10
Figure 10. Figure 10: A failed case. Prompt: "a plush * toy wearing shoes". The middle column shows a bad generation of diffusion models, leading to the failure of editing(the right column). References [BHE23] BROOKS T., HOLYNSKI A., EFROS A. A.: Instructpix2pix: Learning to follow image e…
Figure 9
Figure 9. Figure 9: Ablation study on Pixel-Level Guidance using pseudo￾GT images. Prompt: "a * clown". Due to the strong fluidity of Gaussians, the background remains cluttered with 3D static local￾ization when pixel-level guidance is removed. However, with the guidance of pseudo-GT imag…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

48 extracted references · 30 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION fin.entry add.period write newline FUNCTION new.block output.state before.all = 'skip after.block 'output.state := if FUNCTION new.sentence output.state after.block = 'skip output.state before.all = 'skip after.sentence 'output.state := if if FUNCTION not #0 #1 if FUNCTION and 'skip pop #0 if FUNCTIO...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION fin.entry.original add.period write newline FUNCTION new.block output.state before.all = 'skip after.block 'output.state := if FUNCTION new.sentence output.state after.block = 'skip output.state before.all = 'skip after.sentence 'output.state := if if FUNCTION not #0 #1 if FUNCTION and 'skip pop #0 i...

  3. [3]

    Brooks T., Holynski A., Efros A. A. : Instructpix2pix: Learning to follow image editing instructions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2023), pp. 18392--18402

    work page 2023

  4. [4]

    T., Mildenhall B., Verbin D., Srinivasan P

    Barron J. T., Mildenhall B., Verbin D., Srinivasan P. P., Hedman P. : Mip-nerf 360: Unbounded anti-aliased neural radiance fields. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2022), pp. 5470--5479

    work page 2022

  5. [5]

    R., M \"u ller N., Porzi L., Kontschieder P., Wang Y.-X

    Chen J.-K., Bul \`o S. R., M \"u ller N., Porzi L., Kontschieder P., Wang Y.-X. : Consistdreamer: 3d-consistent 2d diffusion for high-fidelity scene editing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2024), pp. 21071--21080

    work page 2024

  6. [6]

    : Gaussianeditor: Swift and controllable 3d editing with gaussian splatting

    Chen Y., Chen Z., Zhang C., Wang F., Yang X., Wang Y., Cai Z., Yang L., Liu H., Lin G. : Gaussianeditor: Swift and controllable 3d editing with gaussian splatting. arXiv preprint arXiv:2311.14521 (2023)

    arXiv 2023

  7. [7]

    : Dge: Direct gaussian 3d editing by consistent multi-view editing

    Chen M., Laina I., Vedaldi A. : Dge: Direct gaussian 3d editing by consistent multi-view editing

    work page

  8. [8]

    : Segment anything in 3d with nerfs

    Cen J., Zhou Z., Fang J., Shen W., Xie L., Jiang D., Zhang X., Tian Q., et al. : Segment anything in 3d with nerfs. Advances in Neural Information Processing Systems 36 (2023), 25971--25990

    work page 2023

Show all 48 references
  1. [9]

    : Hyper-3dg: Text-to-3d gaussian generation via hypergraph

    Di D., Yang J., Luo C., Xue Z., Chen W., Yang X., Gao Y. : Hyper-3dg: Text-to-3d gaussian generation via hypergraph. arXiv preprint arXiv:2403.09236 (2024)

    2024 arXiv

  2. [10]

    : Density-based spatial clustering of applications with noise

    Ester M., Kriegel H.-P., Sander J., Xu X. : Density-based spatial clustering of applications with noise. In Int. Conf. knowledge discovery and data mining (1996), vol. 240

    1996

  3. [11]

    : Gaussianeditor: Editing 3d gaussians delicately with text instructions

    Fang J., Wang J., Zhang X., Xie L., Tian Q. : Gaussianeditor: Editing 3d gaussians delicately with text instructions. arXiv preprint arXiv:2311.16037 (2023)

    2023 arXiv

  4. [12]

    : Prompt-to-prompt image editing with cross attention control

    Hertz A., Mokady R., Tenenbaum J., Aberman K., Pritch Y., Cohen-Or D. : Prompt-to-prompt image editing with cross attention control. arXiv preprint arXiv:2208.01626 (2022)

    2022 arXiv

  5. [13]

    J., Norouzi M., Salimans T

    Ho J., Saharia C., Chan W., Fleet D. J., Norouzi M., Salimans T. : Cascaded diffusion models for high fidelity image generation. Journal of Machine Learning Research 23, 47 (2022), 1--33

    2022

  6. [14]

    A., Holynski A., Kanazawa A

    Haque A., Tancik M., Efros A. A., Holynski A., Kanazawa A. : Instruct-nerf2nerf: Editing 3d scenes with instructions. In Proceedings of the IEEE/CVF International Conference on Computer Vision (2023), pp. 19740--19750

    2023

  7. [15]

    T., Abbeel P., Poole B

    Jain A., Mildenhall B., Barron J. T., Abbeel P., Poole B. : Zero-shot text-guided object generation with dream fields. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (2022), pp. 867--876

    2022

  8. [16]

    : 3d gaussian splatting for real-time radiance field rendering

    Kerbl B., Kopanas G., Leimk \"u hler T., Drettakis G. : 3d gaussian splatting for real-time radiance field rendering. ACM Transactions on Graphics 42, 4 (2023), 1--14

    2023

  9. [17]

    C., Lo W.-Y., et al

    Kirillov A., Mintun E., Ravi N., Mao H., Rolland C., Gustafson L., Xiao T., Whitehead S., Berg A. C., Lo W.-Y., et al. : Segment anything. In Proceedings of the IEEE/CVF International Conference on Computer Vision (2023), pp. 4015--4026

    2023

  10. [18]

    : Decomposing nerf for editing via feature field distillation

    Kobayashi S., Matsumoto E., Sitzmann V. : Decomposing nerf for editing via feature field distillation. Advances in Neural Information Processing Systems 35 (2022), 23311--23330

    2022

  11. [19]

    : Magic3d: High-resolution text-to-3d content creation

    Lin C.-H., Gao J., Tang L., Takikawa T., Zeng X., Huang X., Kreis K., Fidler S., Liu M.-Y., Lin T.-Y. : Magic3d: High-resolution text-to-3d content creation. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2022), 300--309

    2022

  12. [20]

    : Climatenerf: Extreme weather synthesis in neural radiance field

    Li Y., Lin Z.-H., Forsyth D., Huang J.-B., Wang S. : Climatenerf: Extreme weather synthesis in neural radiance field. In Proceedings of the IEEE/CVF International Conference on Computer Vision (2023), pp. 3227--3238

    2023

  13. [21]

    : BLIP-2: bootstrapping language-image pre-training with frozen image encoders and large language models

    Li J., Li D., Savarese S., Hoi S. : BLIP-2: bootstrapping language-image pre-training with frozen image encoders and large language models. In ICML (2023)

    2023

  14. [22]

    : Dreamscene: 3d gaussian-based text-to-3d scene generation via formation pattern sampling

    Li H., Shi H., Zhang W., Wu W., Liao Y., Wang L., Lee L.-h., Zhou P. : Dreamscene: 3d gaussian-based text-to-3d scene generation via formation pattern sampling. arXiv preprint arXiv:2404.03575 (2024)

    2024 arXiv

  15. [23]

    : Editing conditional radiance fields

    Liu S., Zhang X., Zhang Z., Zhang R., Zhu J.-Y., Russell B. : Editing conditional radiance fields. In Proceedings of the IEEE/CVF international conference on computer vision (2021), pp. 5773--5783

    2021

  16. [24]

    : Sked: Sketch-guided text-based 3d editing

    Mikaeili A., Perel O., Safaee M., Cohen-Or D., Mahdavi-Amiri A. : Sked: Sketch-guided text-based 3d editing. In Proceedings of the IEEE/CVF International Conference on Computer Vision (2023), pp. 14607--14619

    2023

  17. [25]

    : Latent-nerf for shape-guided generation of 3d shapes and textures

    Metzer G., Richardson E., Patashnik O., Giryes R., Cohen-Or D. : Latent-nerf for shape-guided generation of 3d shapes and textures. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2023), pp. 12663--12673

    2023

  18. [26]

    P., Tancik M., Barron J

    Mildenhall B., Srinivasan P. P., Tancik M., Barron J. T., Ramamoorthi R., Ng R. : Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM 65, 1 (2021), 99--106

    2021

  19. [27]

    T., Mildenhall B

    Poole B., Jain A., Barron J. T., Mildenhall B. : Dreamfusion: Text-to-3d using 2d diffusion. ArXiv abs/2209.14988 (2022)

    2022 arXiv

  20. [28]

    : High-resolution image synthesis with latent diffusion models

    Rombach R., Blattmann A., Lorenz D., Esser P., Ommer B. : High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (2022), pp. 10684--10695

    2022

  21. [29]

    : Hierarchical text-conditional image generation with clip latents

    Ramesh A., Dhariwal P., Nichol A., Chu C., Chen M. : Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125 1, 2 (2022), 3

    2022 arXiv

  22. [30]

    W., Hallacy C., Ramesh A., Goh G., Agarwal S., Sastry G., Askell A., Mishkin P., Clark J., et al

    Radford A., Kim J. W., Hallacy C., Ramesh A., Goh G., Agarwal S., Sastry G., Askell A., Mishkin P., Clark J., et al. : Learning transferable visual models from natural language supervision. In International conference on machine learning (2021), PMLR, pp. 8748--8763

    2021

  23. [31]

    : Dreambooth3d: Subject-driven text-to-3d generation

    Raj A., Kaza S., Poole B., Niemeyer M., Ruiz N., Mildenhall B., Zada S., Aberman K., Rubinstein M., Barron J., et al. : Dreambooth3d: Subject-driven text-to-3d generation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (2023), pp. 2349--2359

    2023

  24. [32]

    : Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation

    Ruiz N., Li Y., Jampani V., Pritch Y., Rubinstein M., Aberman K. : Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2023), pp. 22500--22510

    2023

  25. [33]

    L., Ghasemipour K., Gontijo Lopes R., Karagol Ayan B., Salimans T., et al

    Saharia C., Chan W., Saxena S., Li L., Whang J., Denton E. L., Ghasemipour K., Gontijo Lopes R., Karagol Ayan B., Salimans T., et al. : Photorealistic text-to-image diffusion models with deep language understanding. Advances in neural information processing systems 35 (2022), ...

    2022

  26. [34]

    L., Frahm J.-M

    Schonberger J. L., Frahm J.-M. : Structure-from-motion revisited. In Proceedings of the IEEE conference on computer vision and pattern recognition (2016), pp. 4104--4113

    2016

  27. [35]

    : Vox-e: Text-guided voxel editing of 3d objects

    Sella E., Fiebelman G., Hedman P., Averbuch-Elor H. : Vox-e: Text-guided voxel editing of 3d objects. In Proceedings of the IEEE/CVF International Conference on Computer Vision (2023), pp. 430--440

    2023

  28. [36]

    : Clip-nerf: Text-and-image driven manipulation of neural radiance fields

    Wang C., Chai M., He M., Chen D., Liao J. : Clip-nerf: Text-and-image driven manipulation of neural radiance fields. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2022), pp. 3835--3844

    2022

  29. [37]

    S., Girdhar R., Misra I

    Wang X., Darrell T., Rambhatla S. S., Girdhar R., Misra I. : Instancediffusion: Instance-level control for image generation. arXiv preprint arXiv:2402.03290 (2024)

    2024 arXiv

  30. [38]

    : Prolificdreamer: High-fidelity and diverse text-to-3d generation with variational score distillation

    Wang Z., Lu C., Wang Y., Bao F., Li C., Su H., Zhu J. : Prolificdreamer: High-fidelity and diverse text-to-3d generation with variational score distillation. Advances in Neural Information Processing Systems 36 (2024)

    2024

  31. [39]

    : View-consistent 3d editing with gaussian splatting

    Wang Y., Yi X., Wu Z., Zhao N., Chen L., Zhang H. : View-consistent 3d editing with gaussian splatting. arXiv preprint arXiv:2403.11868 (2024)

    2024 arXiv

  32. [40]

    : Tiger: Text-instructed 3d gaussian retrieval and coherent editing

    Xu T., Chen J., Chen P., Zhang Y., Yu J., Yang W. : Tiger: Text-instructed 3d gaussian retrieval and coherent editing. arXiv preprint arXiv:2405.14455 (2024)

    2024 arXiv

  33. [41]

    : Deforming radiance fields with cages

    Xu T., Harada T. : Deforming radiance fields with cages. In European Conference on Computer Vision (2022), Springer, pp. 159--175

    2022

  34. [42]

    : Gaussian grouping: Segment and edit anything in 3d scenes

    Ye M., Danelljan M., Yu F., Ke L. : Gaussian grouping: Segment and edit anything in 3d scenes. arXiv preprint arXiv:2312.00732 (2023)

    2023 arXiv

  35. [43]

    : Nerf-editing: geometry editing of neural radiance fields

    Yuan Y.-J., Sun Y.-T., Lai Y.-K., Ma Y., Jia R., Gao L. : Nerf-editing: geometry editing of neural radiance fields. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2022), pp. 18353--18364

    2022

  36. [44]

    Y., Luong T., Baid G., Wang Z., Vasudevan V., Ku A., Yang Y., Ayan B

    Yu J., Xu Y., Koh J. Y., Luong T., Baid G., Wang Z., Vasudevan V., Ku A., Yang Y., Ayan B. K., et al. : Scaling autoregressive models for content-rich text-to-image generation. arXiv preprint arXiv:2206.10789 2, 3 (2022), 5

    2022 arXiv

  37. [45]

    : Tip-editor: An accurate 3d editor following both text-prompts and image-prompts

    Zhuang J., Kang D., Cao Y.-P., Li G., Lin L., Shan Y. : Tip-editor: An accurate 3d editor following both text-prompts and image-prompts. arXiv preprint arXiv:2401.14828 (2024)

    2024 arXiv

  38. [46]

    : Adding conditional control to text-to-image diffusion models

    Zhang L., Rao A., Agrawala M. : Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision (2023), pp. 3836--3847

    2023

  39. [47]

    : Gala3d: Towards text-to-3d complex scene generation via layout-guided generative gaussian splatting

    Zhou X., Ran X., Xiong Y., He J., Lin Z., Wang Y., Sun D., Yang M.-H. : Gala3d: Towards text-to-3d complex scene generation via layout-guided generative gaussian splatting. arXiv preprint arXiv:2402.07207 (2024)

    2024 arXiv

  40. [48]

    : Dreameditor: Text-driven 3d scene editing with neural fields

    Zhuang J., Wang C., Lin L., Liu L., Li G. : Dreameditor: Text-driven 3d scene editing with neural fields. In SIGGRAPH Asia 2023 Conference Papers (2023), pp. 1--10

    2023

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.