{"id":"7c5acb01-5cbb-433e-9331-a523d62ece9c","arxiv_id":"2412.00518","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Text-guided 3D editing is reframed as multiview image inpainting, giving consistent edits in seconds instead of hours.","lead":"Instant3dit edits a 3D object by inpainting four rendered views at once with a text-guided diffusion model, then reconstructing the edited shape with an off-the-shelf 3D reconstructor in about three seconds. It bypasses slow score-distillation optimization, making localized text-guided editing practical for meshes, NeRFs, and Gaussian splats.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"3D editing quality is never directly measured: Table 1 evaluates 2D multiview grids, not reconstructed assets, so the central claim of high-quality 3D edits with preserved unedited regions rests on qualitative results.","rationale":"The reader's weakest assumption identified the risk that the 2x2-grid-plus-LRM reconstruction is lossy and may distort unedited regions. My concern is closely related but shifts the emphasis: the paper never quantitatively tests that assumption on the final 3D asset. The evaluation in Section 4 is an image-level proxy, and the 3D applications in Section 5 are qualitative plus one informal user study. This is a missing-support problem rather than a demonstrated flaw: the method could well work, but the central claim is stronger than the evidence. I did not find an internal inconsistency in the pipeline itself; the ablations support the importance of 3D-aware masks, and the speed advantage is clearly established. I therefore keep the reader's CONDITIONAL verdict unchanged, because the concern warrants conditional acceptance with a request for 3D-level evaluation, not rejection. I mark agreement as 'partial' because the reader centered on the method's assumption while I emphasize the evaluation gap that leaves that assumption untested.","tokens_in":14542,"tokens_out":5468,"duration_ms":64147,"concrete_test":"Run the full Instant3dit pipeline on a held-out set of at least 50 Objaverse meshes with user-style masks and prompts. For each edit, compute: (1) preservation: bidirectional Chamfer distance and normal consistency between the edited and original mesh restricted to the complement of the mask volume; (2) prompt fidelity: CLIP score and DreamSim between 8 novel-view renderings of the edited asset and the prompt/reference; (3) repeat the same protocol for NeRFiller and, if available, MVInpainter. Report means and 95% confidence intervals. If outside-mask Chamfer distance is near zero and novel-view CLIP is significantly above the baselines, the 2D-grid metrics in Table 1 transfer to 3D; otherwise the central 3D-editing claim is not established.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that Instant3dit performs high-quality 3D editing in seconds while preserving unedited regions. The only quantitative evaluation (Section 4, Table 1) measures multiview inpainting on 2x2 image grids, not the final reconstructed 3D assets. SSIM, LPIPS, and DreamSim are computed by reconstructing a NeRF from the inpainted grid and re-rendering to the same four views, which measures self-consistency rather than fidelity of the edited 3D shape. FID is computed on images. None of these metrics test whether the downstream reconstructor Phi (Section 3) produces a faithful edited shape: LRM is lossy, can hallucinate or smooth geometry, and may alter the unedited region because nothing in the diffusion objective enforces pixel-wise agreement with Ic(S, M) outside the mask. The mesh pipeline (Section 5) adds ROAR to preserve topology and UVs, but the NeRF and Gaussian-splat pipelines have no such guarantee and are only shown qualitatively. The informal user study against NeRFiller (86% preference on 208 pairs) is the only 3D-level comparison; it reports no error bars, no per-item breakdown, and no comparison to the concurrent MVInpainter. The paper's own Limitations section concedes thin-mask failures and white-background bias, which are exactly the situations where the reconstructed 3D edit would be distorted or empty. Thus the 'higher-quality results compared to previous works' claim for 3D editing is not yet supported by direct measurement of edited 3D geometry and novel-view appearance.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper introduces Instant3dit, a method for localized generative editing of 3D assets (meshes, NeRFs, Gaussian splats) by reformulating the task as multiview image inpainting. Given a 3D mask and text prompt, the method renders four canonical views of the masked object, inpaints them with a fine-tuned diffusion model, and reconstructs the edited 3D representation with an off-the-shelf large reconstruction model (LRM). The authors propose three types of 3D-consistent training masks (coarse, sculpting, surface) and a fine-tuning strategy starting from SDXL-inpainting. Experiments compare multiview inpainting quality against SDXL, SDXL-inpainting, and Instant3D baselines, and report an informal user study against NeRFiller. The method is claimed to run in about 3 seconds per edit.","tokens_in":14880,"tokens_out":5491,"duration_ms":46137,"significance":"If the core claim is confirmed, this is a practically valuable contribution: it replaces SDS-style optimization that takes tens of minutes to hours with a feedforward pipeline, and the design of 3D-consistent training masks is a useful reusable recipe. The paper also contributes a benchmark and ablations that map training-mask type to downstream performance. The speed advantage is structurally credible because the pipeline uses a single diffusion pass plus a fast LRM. However, the claim of 'higher-quality results' for 3D editing is currently supported mainly by 2D-grid metrics and an informal user study, not by direct quantitative evaluation of reconstructed assets, so the significance is somewhat conditional.","major_comments":[{"comment":"The quantitative evaluation is entirely at the level of 2x2 multiview image grids. The SSIM/LPIPS/DreamSim numbers are obtained by reconstructing a NeRF from the inpainted grid and re-rendering to the same four camera angles, which measures self-consistency between the inpainted images and their LRM reconstruction, not the fidelity of the edited 3D shape. The abstract's claim of 'higher-quality results compared to previous works' for 3D editing therefore lacks direct support; please add evaluation on reconstructed assets (e.g., novel-view renderings, geometric and color preservation outside the mask, or a formal user study on 3D outputs).","section":"Section 4, Table 1 and Evaluation Metrics"},{"comment":"For NeRF and Gaussian splat editing, the method does not enforce that the reconstructed asset matches the original outside the mask: the diffusion objective inpaints only masked pixels, but the LRM (NeRF-LRM or GS-LRM) can alter the unmasked region during reconstruction. Only the mesh pipeline (with ROAR, Section 5) guarantees preservation of topology, UVs, and triangulation outside the edit. Please provide quantitative evidence of preservation for NeRF/GS outputs, or explicitly restrict the preservation claim to the mesh pipeline.","section":"Section 5, NeRF/GS pipelines"},{"comment":"The only 3D-level comparison with a prior method is the informal user preference study against NeRFiller (15 users, 208 pairs, 86% preference), with no error bars, no per-item analysis, and no comparison to the concurrent MVInpainter. This is too weak to support the 'higher-quality' claim. Either report a formal user study with statistical significance, add a quantitative comparison on reconstructed assets, or soften the claim to 'comparable or better in informal evaluation.'","section":"Section 5, user study"}],"minor_comments":[{"comment":"The figure labels include '25 sec.' while the caption and abstract state '≈ 3 seconds per multiview edit'; clarify whether 25 seconds is the total mesh pipeline time (3s edit + 3s MeshLRM + 20s post-processing) or a typo.","section":"Figure 1"},{"comment":"The sentence 'Compared to prior work, our approach does not require an optimization to leverage the generative priorhttps://git.corp.adobe.com/pages/adobe-research/mvgenfill-client/' is broken: an internal URL is pasted into the sentence and the sentence is incomplete. Remove or fix.","section":"Section 2, Related Work"},{"comment":"The plane equation is written as '{x in R3 | x · p = p · n}' and the face-selection condition 'v1+v2+v3/3 · p >= p · n' mixes a point with a normal; presumably the intended condition is (midpoint - p) · n >= 0. Please fix the notation.","section":"Section 3.2, Type I mask"},{"comment":"'SDXL-inpaiting' is a typo for 'SDXL-inpainting' (also in Section 4 text).","section":"Table 1"},{"comment":"The 'User Generated' column in Table 2 reports ClipG on only 15 masks, and the text notes FID is not meaningful at that size; consider reporting standard errors or confidence intervals for the user-generated subset.","section":"Section 4, mask ablation"},{"comment":"The inset is described qualitatively ('scores remain consistent') without reporting the actual numbers; please include the quantitative values or a plot with axis labels.","section":"Section 4, generalization to novel camera angles"}],"recommendation":"major_revision","confidential_remarks":"The manuscript reads like an early preprint: it contains an internal Adobe URL in Section 2, inconsistent timing labels in Figure 1, and several typos. The core idea is sound and the ablation of mask design is the most solid part. The main gap is the mismatch between the strength of the central claim ('higher-quality results' for 3D editing) and the level of the evaluation. If the authors add a direct 3D-level evaluation or carefully scope the claim, the paper could become acceptable. I would not recommend rejection because the speed contribution and the mask dataset are useful."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague, quick take on Instant3dit.\n\nThe paper's substance is a pragmatic engineering result: fine-tune an off-the-shelf inpainting diffusion model (SDXL-inpainting) to produce multiview-consistent 2x2 grids under 3D-aware masks, then rely on LRMs to lift the grids back to NeRF, Gaussian splat, or mesh. The speed claim holds up: one diffusion pass (~3 s) plus reconstruction is genuinely orders of magnitude faster than SDS-based editing. The ablations are the best part. They show convincingly that 3D-aware multiview masks matter, that fine-tuning an inpainting backbone beats fine-tuning a multiview generator (Instant3D), and that the three mask types generalize reasonably to user masks. The mesh pipeline with ROAR and adaptive remeshing is a nice touch for preserving UVs and topology.\n\nThe soft spot is evaluation, and the stress-test note is right: Table 1 measures multiview inpainting on image grids, not the quality of the reconstructed 3D asset. The SSIM/LPIPS/DreamSim numbers are self-consistency checks (re-render the reconstructed NeRF to the same four views), not evidence that the edited shape is faithful or that unedited regions are preserved. The head-to-head with NeRFiller is an informal user study with no error bars and no comparison to the concurrent MVInpainter. The synthetic benchmark is also close to the training distribution. The paper's own Limitations section concedes thin-mask and white-background failures, which are exactly where the 3D reconstruction would suffer.\n\nThat said, the gaps are not fatal. The central contribution is the pipeline and training recipe, and those are supported. The missing piece is direct measurement of edited 3D geometry and novel-view appearance. A serious referee should ask for that, plus code/data release and a direct MVInpainter comparison. The broken internal link and minor typos are trivial.\n\nWho gets value: researchers working on fast 3D editing or multiview diffusion, and practitioners who want a feed-forward editing loop. It deserves a serious referee. My recommendation: send to peer review with a request for stronger 3D-level evaluation rather than desk-reject.","headline":"Solid engineering contribution for fast 3D editing, but the quality claim is only directly supported on 2D grids, not on reconstructed 3D assets.","tokens_in":15437,"tokens_out":2442,"would_cite":true,"duration_ms":94830,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"3D editing can be cast as multiview image inpainting, cutting edit time from hours to about three seconds.","keywords":["3D editing","multiview inpainting","diffusion models","large reconstruction models","score distillation sampling","mesh editing","NeRF editing","Gaussian splatting"],"falsifier":"Run the pipeline on a shape with a thin mask, such as an antenna or slender handle, with a non-white background behind the object; if the inpainted grid either drops the thin mask, turns the background white instead of generating the prompted content, or produces four views that cannot be reconstructed into one coherent shape, the central claim fails.","tokens_in":14354,"feed_emoji":"🖌️","tokens_out":6384,"duration_ms":55766,"temperature":0.7,"pith_summary":"The paper aims to make generative 3D editing as fast and simple as 2D inpainting. Instead of running a slow optimization loop that distills a 2D diffusion prior into a 3D representation, it renders the input object and a user-drawn 3D mask into a 2x2 grid of four canonical views, inpaints those views with a multiview-consistent diffusion model, and reconstructs the edited object with an off-the-shelf large reconstruction model. If this works as claimed, editing a mesh, NeRF, or Gaussian splat takes about three seconds of diffusion time plus a short reconstruction, down from hours, while unedited regions stay intact. This matters because SDS-based editing is both slow and prone to fuzzy, inconsistent results; replacing it with a feedforward image-generation pass is a structural change to how 3D editing is done.","feed_headline":"3D editing now takes seconds, done as multiview image inpainting","feed_subtitle":"One diffusion pass fills four canonical views; a reconstructor turns them into a mesh, NeRF, or Gaussian splat.","key_machinery":"The central object is a multiview inpainting diffusion model operating on a 2x2 grid. The grid holds four renderings of the object from canonical azimuths (0, 90, 180, 270 degrees) at a fixed elevation, with the user's 3D mask rendered into a binary mask grid. Conditioning on the color image, the mask, and a text prompt, the model inpaints the grid in one pass; a posed multiview reconstruction operator (large reconstruction model variants for NeRF, mesh, and Gaussian splat, plus an optional adaptive remeshing step) converts the inpainted grid into the edited 3D asset. The load-bearing training choice is the dataset of 3D masks: three procedural mask types (coarse convex-hull blobs, tight sculpting selections, local surface patches) mimic the edits users make, and rendering these masks along with the object keeps occlusion geometry consistent across views.","core_discovery":"The central claim is that 3D editing can be cast as multiview image inpainting: the edited shape is obtained by inpainting four rendered views of the object with a diffusion model trained to be simultaneously multiview-consistent and text-guided, then mapping the inpainted grid back to 3D with a large reconstruction model. The authors argue that the previous two problems of 3D inpainting—long runtimes and low quality—both stem from SDS-style optimization, and that removing the optimization removes both issues at once. The paper supports this with a benchmark of 500 held-out multiview images, ablations over diffusion backbones (fine-tuning from an inpainting-pretrained large text-to-image model works best), a mask ablation showing that 3D-consistent masks matter, generalization tests to novel camera angles, and a user study in which the method is preferred over the closest prior method in 86% of paired comparisons.","pith_inferences":["Editorial: Because the method treats 3D editing as 2D inpainting on a grid, it should compose with any improved multiview generator or reconstructor; a video-diffusion prior could raise consistency further, at some speed cost.","Editorial: The mask design may determine the method's ceiling: the reported white-background and thin-mask failures suggest that training on more diverse backgrounds and fine masks would widen the range of user edits that work reliably.","Editorial: A natural stress test is to measure geometric drift in the unmasked region after reconstruction; if the reconstruction is lossy, exact preservation requires the optimization-based post-processing rather than the plain feedforward path."],"forward_implications":["Edits complete in about three seconds for the image stage, with a few more seconds for reconstruction, removing the multi-minute-to-hours SDS optimization bottleneck.","The same pipeline edits meshes, NeRFs, and Gaussian splats by swapping the reconstruction operator; no per-representation training is needed.","Because the unedited regions stay untouched in the input renders, and with adaptive remeshing on meshes, original topology, UVs, and rigging survive the edit.","Training on three types of 3D masks (coarse region, sculpted volume, surface patch) transfers to arbitrary user-drawn masks, as measured by prompt adherence and consistency metrics.","The model generalizes to novel camera azimuths and field-of-view angles despite being fine-tuned at fixed canonical angles, due to strong conditioning from the masked image."],"supporting_citations":[{"why":"Supplies the 2x2 multiview diffusion formulation and the NeRF reconstruction operator used throughout.","marker":"[24]"},{"why":"Provides the pretrained text-to-image and inpainting backbone that is fine-tuned for multiview consistency.","marker":"[34]"},{"why":"Establishes the latent diffusion framework and the lower-capacity backbone variants tested in ablations.","marker":"[38]"},{"why":"Is the closest prior 3D inpainting method and the main comparison baseline in the user study.","marker":"[48]"},{"why":"Gives the large reconstruction model that turns inpainted views into a Gaussian splat.","marker":"[56]"},{"why":"Gives the large reconstruction model that turns inpainted views into a mesh.","marker":"[49]"},{"why":"Provides the adaptive remeshing step that preserves original mesh attributes in unedited regions.","marker":"[3]"},{"why":"Introduces the SDS loss whose slowness and fuzzy output motivate the feedforward alternative.","marker":"[35]"},{"why":"Shows that training mask design is critical for inpainting-model performance, motivating the three mask types.","marker":"[55]"},{"why":"Is a concurrent multiview inpainting work with different masking and consistency strategies, used for positioning.","marker":"[6]"}],"fun_headline_variants":["3D editing in seconds: cast as multiview inpainting","Replace SDS with multiview inpainting for quick 3D edits","Multiview inpainting makes 3D editing take seconds","Edit meshes, NeRFs, or splats in seconds via inpainting","One inpainting pass edits 3D shapes in ~3 seconds"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that four canonical white-background views of a masked object carry enough information for an off-the-shelf reconstruction model to produce a faithful edited shape, and that the fine-tuned diffusion model follows user-drawn masks as reliably as the three procedural mask types it was trained on.","fun_headline_variants_meta":{"raw":{"variants":["3D editing in seconds: cast as multiview inpainting","Replace SDS with multiview inpainting for quick 3D edits","Multiview inpainting makes 3D editing take seconds","Edit meshes, NeRFs, or splats in seconds via inpainting","One inpainting pass edits 3D shapes in ~3 seconds"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000529,"raw_usage":{"total_tokens":2525,"prompt_tokens":894,"completion_tokens":1631,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":510,"completion_tokens_details":{"reasoning_tokens":1534}},"tokens_in":510,"tokens_out":1631,"duration_ms":12579,"temperature":1.0,"reasoning_tokens":1534,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T05:16:29.782855+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the pipeline on a shape with a thin mask, such as an antenna or slender handle, with a non-white background behind the object; if the inpainted grid either drops the thin mask, turns the background white instead of generating the prompted content, or produces four views that cannot be reconstructed into one coherent shape, the central claim fails.","supporting_citations":[{"cited_title":"Instant3d: Fast text-to-3d with sparse-view generation and large reconstruction model","cited_arxiv_id":null,"evidence_quote":"Supplies the 2x2 multiview diffusion formulation and the NeRF reconstruction operator used throughout."},{"cited_title":"Sdxl: Improving latent diffusion models for high-resolution image synthesis","cited_arxiv_id":null,"evidence_quote":"Provides the pretrained text-to-image and inpainting backbone that is fine-tuned for multiview consistency."},{"cited_title":"High-resolution image syn- thesis with latent diffusion models","cited_arxiv_id":null,"evidence_quote":"Establishes the latent diffusion framework and the lower-capacity backbone variants tested in ablations."},{"cited_title":"Nerfiller: Completing scenes via generative 3d inpainting","cited_arxiv_id":null,"evidence_quote":"Is the closest prior 3D inpainting method and the main comparison baseline in the user study."},{"cited_title":"Gs-lrm: Large recon- struction model for 3d gaussian splatting","cited_arxiv_id":null,"evidence_quote":"Gives the large reconstruction model that turns inpainted views into a Gaussian splat."},{"cited_title":"ROAR: Robust Adaptive Reconstruction of Shapes Using Planar Projections","cited_arxiv_id":"2307.00690","evidence_quote":"Provides the adaptive remeshing step that preserves original mesh attributes in unedited regions."},{"cited_title":"Lin, Jimei Yang, Jianming Zhang, Eli Shechtman, and Huchuan Lu","cited_arxiv_id":null,"evidence_quote":"Shows that training mask design is critical for inpainting-model performance, motivating the three mask types."}],"review_version":1}