{"id":"58e0a0cf-336f-4294-bc59-e8de0ef9859e","arxiv_id":"2501.04374","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A text-conditioned diffusion adapter operating on the triplane latents of a frozen large reconstruction model enables natural-language editing of generated 3D objects.","lead":"Instructive3D adds a text-conditioned diffusion adapter to an existing single-image 3D reconstruction model, letting users edit the look of generated 3D objects with natural language prompts. The method avoids training or generating edited 3D objects by learning changes in the model's internal triplane representation instead.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim is only as strong as the benchmark, and the benchmark's ground truth is generated by the same InstructPix2Pix-plus-Real3D pipeline used to create the training targets, so the reported quantitative gains may measure self-reconstruction rather than genuine instruction-following 3D…","rationale":"The reader's weakest_assumption focused on whether InstructPix2Pix-edited triplane targets are valid and sufficient training signals, and the reader's rationale noted that the quantitative benchmark is not independent because ground truth comes from the same Real3D/InstructPix2Pix pipeline. My stress-test converges on the same circularity as the most load-bearing issue, but I frame it more precisely around the evaluation protocol in Sec. 4.2: the ground-truth edited objects used for metrics are generated by exactly the pipeline that creates the training targets, so the reported LPIPS/PSNR/SSIM/FID/KID numbers cannot distinguish genuine instruction-following from reproduction of the training distribution. I also emphasize that the 'geometrically consistent modifications' claim lacks any geometric metric, which is a separate, directly checkable gap. The reader's verdict of CONDITIONAL is appropriate because the method is plausible and the qualitative results may be genuine, but acceptance should require independent, non-circular evaluation with released artifacts. I therefore leave the verdict unchanged rather than escalating to rejection, since the central mechanism is not disproven; it is simply unvalidated by the current evidence.","tokens_in":15294,"tokens_out":2028,"duration_ms":21791,"concrete_test":"Re-run the evaluation on an independent ground-truth set: take 20-30 real 3D objects (e.g., from a different dataset like ShapeNet or manually modeled assets), apply controlled edits in a 3D editor (color change, texture/pattern addition) to create true edited meshes, render multi-view images of both original and edited objects, and feed the original renders plus the same text prompts to Instructive3D and to the baselines. Compute (i) LPIPS/CLIP against renders of the true edited meshes and (ii) mesh-level Chamfer distance between the input mesh and the output mesh for both Instructive3D and baselines. If the LPIPS/CLIP gaps of Table 1 shrink or invert, or if Instructive3D's output meshes show Chamfer distance comparable to or larger than baselines on color-only edits, the central claims of superior editing quality and geometric consistency are not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's headline claims are (a) that Instructive3D produces 'qualitatively superior 3D objects with the properties specified by the edit prompts' and (b) that edits are 'geometrically consistent.' Both claims rest on an evaluation that is circular with respect to the training data. In Sec. 3.2, training targets x_edit are produced by rendering an Objaverse object, editing the rendered image with InstructPix2Pix, and encoding that edited image with the frozen Real3D LRM. In Sec. 4.2, the reported quantitative comparison (Table 1) uses the same construction as ground truth: the 'after-edit object generated by Real3D' is exactly the Real3D decode of an InstructPix2Pix-edited render. The 45-object evaluation set is sampled from the same Objaverse LVIS distribution used for training. Consequently, low LPIPS, high SSIM/PSNR, and low FID/KID largely measure how well the adapter reproduces its own training distribution, not how well it performs edits that generalize to genuine 3D manipulations. The geometry-preservation claim is even less supported: no geometric metric (e.g., Chamfer distance, mesh IoU, normal consistency) is reported anywhere, so 'geometrically consistent modifications' is asserted rather than measured. The baselines (Text2Mesh, Paint3D, TEXTure) are not trained on the synthetic triplane distribution and operate on meshes directly, making the comparison asymmetric; the large metric gaps in Table 1 are therefore not interpretable as evidence of superior editing quality. The paper itself acknowledges the dependence on the LRM triplane quality in the Limitations, but does not address the circularity of its own benchmark. Qualitative figures may indeed show useful edits, but they cannot, by themselves, validate the quantitative superiority claim against an independent standard.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Instructive3D, an adapter for a frozen large reconstruction model (Real3D) that performs text-prompted edits in the triplane latent space. A per-plane 2D VAE compresses triplane features, and a conditional latent diffusion model is trained on pairs obtained by rendering Objaverse objects, editing the renders with InstructPix2Pix, and encoding both original and edited images with Real3D. At inference, an input image and edit prompt are mapped through the Real3D encoder, the diffusion adapter, and the Real3D decoder. The experiments compare against Text2Mesh, Paint3D, and TEXTure on Objaverse LVIS using LPIPS, PSNR, SSIM, FID, KID, and CLIP scores, together with an ablation of per-plane 2D VAEs versus a 3D VAE.","tokens_in":15610,"tokens_out":4195,"duration_ms":45219,"significance":"If the evaluation were convincing, the paper would offer a practical and compute-efficient recipe for adding text-instruction editing to existing LRMs without collecting edited 3D object pairs. The architectural choice of freezing the LRM and training only a triplane adapter is sensible, and the per-plane 2D VAE ablation is a useful design data point. The qualitative figures show plausible edits across several object categories. However, the central quantitative claim is undermined by the evaluation protocol, which uses the same InstructPix2Pix-plus-Real3D pipeline to generate both training targets and test ground truth, so the numerical gains largely measure self-reconstruction of the training distribution rather than demonstrated generalization to genuine 3D instruction-following.","major_comments":[{"comment":"The training targets x_edit in Section 3.2 are triplanes obtained by passing InstructPix2Pix-edited rendered images through Real3D, and the evaluation ground truth in Section 4.2 is described as 'the after-edit object generated by Real3D,' which is the same construction. Consequently, LPIPS, PSNR, SSIM, FID, and KID measure how well the adapter reproduces the exact pipeline that generated its training targets, not how well it generalizes to unseen edits or to genuinely novel 3D manipulations. The evaluation set is also sampled from the same Objaverse LVIS distribution used for training. The authors should add an evaluation that does not depend on this circular target: for example, a human perceptual study on edit fidelity and geometry preservation, held-out object categories and prompts, novel objects outside the LVIS distribution, and a direct baseline that applies InstructPix2Pix to the input render and reconstructs with Real3D. Without such controls, the numbers in Table 1 cannot support the abstract's claim of 'qualitatively superior 3D objects.'","section":"§3.2 and §4.2, Table 1"},{"comment":"The claim that edits are 'geometrically consistent' or that the method 'preserves the geometry' is asserted rather than measured. No geometric metric—such as Chamfer distance, mesh IoU, normal consistency, or Hausdorff distance—appears in Table 1 or anywhere in the experiments; all reported metrics are computed on rendered images, which conflate appearance changes with geometric changes. The paper should report a direct geometric comparison between the input and edited meshes or triplanes, and ideally compare this quantity against the baselines under the same mesh-extraction protocol, to support the geometry-preservation claim.","section":"Abstract and §5, Table 1"},{"comment":"The comparison with Text2Mesh, Paint3D, and TEXTure is asymmetric in a way that inflates the reported gap. Those baselines consume a mesh produced by Real3D, whereas Instructive3D's outputs are decoded by the same frozen Real3D that produced the ground-truth objects, so the LPIPS/PSNR/SSIM metrics favor the proposed method by construction. Additionally, the baselines are not trained on the synthetic triplane distribution and may fail for reasons unrelated to edit quality. The authors should add an image-space control baseline (e.g., InstructPix2Pix on the input render followed by Real3D reconstruction) and standardize the evaluation so that all methods are scored on the same rendered views and the same mesh-extraction pipeline.","section":"§4.1 and §4.2, Table 1"},{"comment":"The data-curation and training protocol is underspecified in ways that matter for generalization. The manual filtering of InstructPix2Pix pairs is not quantified (no number of generated versus retained pairs, no inter-annotator agreement, no criteria), no statistics on prompt types or per-prompt counts are given, and the split between training and evaluation objects is not described beyond '45 objects' for evaluation. The identity-preservation training with Didentity is also never evaluated quantitatively: there is no 'no-edit' test measuring how much the output changes when the prompt is empty or a period. Without these numbers, the reader cannot assess how much of the reported performance is due to filtering, prompt distribution, or identity shortcutting. The paper should report these statistics and include an identity-preservation experiment.","section":"§3.2, §3.6, and §4.3"}],"minor_comments":[{"comment":"The text says the method 'consistently outperforms prior works across all key metrics,' but in Table 1 the CLIP score for Instructive3D (21.94) is slightly lower than TEXTure (21.98); the claim should be corrected to 'all metrics except CLIP' or similar.","section":"§4.2, Table 1 discussion"},{"comment":"The VAE loss in Eq. (2) writes (x_k − x̃_k)^2 without an explicit sum over k or over spatial/channel dimensions; the notation should be clarified, for example by summing over the three planes and over all elements of each plane.","section":"§3.3, Eq. (2)"},{"comment":"The caption refers to 'The third column presents the meshes produced by the UNet model,' but the figure shows rows rather than columns; this should be corrected.","section":"Figure 3 caption"},{"comment":"There are several typographical errors: 'concatentate' (§3.3), 'boarder' (§1), 'randiance' (§2.1), 'turqoise' (Figure 17 caption), and inconsistent spacing in 'V AE.' A careful proofreading pass is needed.","section":"Throughout"},{"comment":"The conditioning mechanism in Eq. (4) is incompletely specified: it is not stated how the null prompt '.' is embedded, whether classifier-free guidance is used at inference, or how the concatenated latent [z_t, z] interacts with the cross-attention text conditioning. This information is needed to reproduce the method.","section":"§3.4, Eq. (4)"}],"recommendation":"major_revision","confidential_remarks":"The central methodological idea is plausible and the paper is clearly written in parts, but the quantitative evaluation as it stands is circular with respect to the training-data generation pipeline, and the geometry-preservation claim is unmeasured. I would be willing to reconsider after the authors add a non-circular evaluation (human study, held-out distributions, an InstructPix2Pix-plus-Real3D control baseline) and direct geometric metrics. The paper may also be better framed as a method for learning to replicate InstructPix2Pix edits in triplane space, with claims appropriately scoped to that setting."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The thing to know: this is a sensible, cheap adapter that adds text-conditioned editing to a frozen LRM, by training a per-plane triplane VAE and a latent diffusion model on synthetic pairs from InstructPix2Pix plus Real3D. That is a genuinely new combination in the cited literature, and it is compute-efficient: only the adapter is trained, on about 14.5K objects. The architecture is clearly explained, and the ablation of 2D vs 3D VAE is a useful detail.\n\nWhat the paper does not have is an independent evaluation. In Sec. 3.2, the training targets x_edit are Real3D triplanes of InstructPix2Pix-edited renders. In Sec. 4.2, the 'ground truth' for Table 1 is the same construction: Real3D decoding of an InstructPix2Pix-edited render. So low LPIPS, high SSIM/PSNR, and low FID/KID largely measure how well the adapter reproduces its own training distribution. The baselines (Text2Mesh, Paint3D, TEXTure) work on meshes and were not trained on this distribution, which makes the comparison asymmetric and the huge metric gaps uninterpretable. On top of that, the 'geometrically consistent' claim is asserted rather than measured: no Chamfer distance, mesh IoU, or normal consistency anywhere. The CLIP gains are marginal (21.86 vs 21.94) and reported without error bars. The eval is 45 objects with no confidence intervals, and no code or data is released.\n\nThe Limitations section honestly says the output depends on the LRM's triplane quality, but it does not address the circularity of the benchmark. That is the load-bearing flaw: the quantitative claims do not establish that the edits transfer to genuine 3D manipulation. The qualitative results in the appendix look plausible, but they cannot carry the 'superior' claim by themselves.\n\nSo what is the paper good for? It gives the community a clear recipe for adding text editing to LRMs without generating edited 3D pairs, and it would be a useful reference for anyone building on Real3D or similar models. With a user study or an evaluation against genuinely independent edited 3D assets, plus a geometry metric and released artifacts, it could be solid. As it stands, the method is worth engaging with but the quantitative comparison is not trustworthy.\n\nRecommendation: this deserves a serious referee, not a desk reject. The idea is timely, the writing is clear, and the flaws are fixable in revision — but the authors need to redo the evaluation and soften the claims.","headline":"A compute-efficient triplane-latent editing adapter for frozen LRMs, with a genuinely novel combination but an evaluation that is circular with its own training-data pipeline.","tokens_in":16301,"tokens_out":2825,"would_cite":false,"duration_ms":26161,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Instructive3D shows that a frozen large reconstruction model can follow text instructions to edit 3D objects by running a latent diffusion process in triplane space.","keywords":["3D editing","large reconstruction models","triplane latent space","latent diffusion","text-guided editing","InstructPix2Pix","Objaverse","geometric consistency"],"falsifier":"Render Instructive3D outputs for the same object and prompt (e.g., \"add a flame design to the shoe\") from all eight cube-corner viewpoints used in the paper's evaluation: if the flame design appears only in some views or the object's silhouette changes between views, the geometrically consistent triplane-editing claim is refuted. A sharper test compares the output mesh against a manually edited 3D ground truth rather than against a 2D-edited rendering.","tokens_in":15034,"feed_emoji":"🎨","tokens_out":5524,"duration_ms":50190,"temperature":0.7,"pith_summary":"Instructive3D claims that text-prompted editing of 3D objects can be folded into a Large Reconstruction Model without retraining the model or building edited 3D object pairs. The trick is to treat the LRM's triplane latent representation as an editable image-like space: a small adapter compresses each triplane plane, runs a text-conditioned diffusion process on the compressed latents, and decodes the result back through the frozen LRM. Training data comes from InstructPix2Pix, which edits single-view 2D renders, followed by the frozen LRM to turn those edits into triplane targets. If the claim holds, any LRM user can change color, texture, and surface patterns with natural-language prompts while the underlying geometry stays fixed, at a small fraction of the cost of full 3D-edit supervision.","feed_headline":"Text prompts now edit 3D objects in triplane space","feed_subtitle":"A small diffusion adapter gives frozen LRMs fine-grained control over color, texture, and patterns without 3D edit pairs.","key_machinery":"The load-bearing object is the triplane latent space of Real3D, a set of three 64x64x40 feature planes that jointly encode a 3D object's shape and appearance. The Tri-VAE compresses each plane independently to a 16x16x4 latent (about 68M parameters per plane encoder), and the LTriD diffusion UNet (about 870M parameters) denoises the stacked latents conditioned on a CLIP text embedding, with the original latent concatenated to the noisy latent to preserve the object's structure. Two training stages matter: Tri-VAE is trained first on 12,584 rendered objects, then LTriD is trained with null prompts for identity preservation and with InstructPix2Pix-generated edit pairs for prompt-conditioned edits. The whole scheme transfers edits into the triplane, so the frozen Real3D decoder renders them consistently from any viewpoint.","core_discovery":"The paper's central claim is that fine-grained, text-guided editing can live inside a Large Reconstruction Model's latent space rather than in a post-processing mesh-editing stage. Concretely, the authors add a Triplane Variational Autoencoder and a latent triplane diffusion model (LTriD) on top of the frozen Real3D LRM. During inference, an input image is encoded to a triplane, the adapter denoises a latent version of that triplane conditioned on a CLIP text embedding of the edit prompt, and the frozen decoder turns the edited triplane into a 3D object. Against baselines that first generate a mesh with Real3D and then edit with Text2Mesh, Paint3D, or TEXTure, the paper reports that Instructive3D gives lower LPIPS, FID, and KID and higher PSNR, SSIM, and CLIP scores, with edits that hold across viewpoints because the modification happens in the shared triplane representation.","pith_inferences":["Because the adapter only touches the triplane space, the same trained weights could in principle be plugged into any LRM that produces Real3D-style triplanes, such as TripoSR or other triplane-based reconstructions, without re-training on their own decoders.","The method inherits the edit distribution of InstructPix2Pix; prompts or edit types that InstructPix2Pix cannot render in 2D, such as precise geometric deformation, will not transfer to 3D. A testable extension is augmenting the paired dataset with procedural 3D material edits to broaden coverage.","Null-prompt identity training hints that the adapter could double as a geometry-preserving denoiser, cleaning reconstruction artifacts from an LRM's triplane even without any edit instruction.","The paper's consistency claim is bounded by Real3D's decoder; a direct comparison against manually edited 3D ground truths would separate triplane-edit fidelity from LRM reconstruction fidelity."],"forward_implications":["Users can edit generated 3D objects with natural-language prompts, changing color, texture, and surface patterns while preserving the underlying geometry.","Training only requires 2D edited images plus a frozen LRM encoder, so the expensive step of generating precisely edited 3D object pairs is avoided.","The same frozen LRM decoder turns edited triplanes into view-consistent meshes, so a text-driven edit persists across different viewpoints.","The adapter is trained on about 14.5K objects, making the approach compute- and data-efficient compared with fine-tuning a full LRM.","Because the edit is applied inside the triplane representation, the method does not require separate mesh-editing tools or per-instance optimization at inference time."],"supporting_citations":[{"why":"Supplies the edited image pairs: InstructPix2Pix edits a rendered view according to the text prompt, and these 2D edits become triplane training targets.","marker":"[2]"},{"why":"Frozen LRM whose encoder produces triplanes and whose decoder renders edited objects; the paper's quality ceiling depends on it.","marker":"[18]"},{"why":"Defines the triplane-based large reconstruction model architecture that Instructive3D extends with an editing adapter.","marker":"[16]"},{"why":"Latent diffusion framework that motivates training the diffusion model in a compressed triplane latent space rather than pixel or triplane space.","marker":"[39]"},{"why":"Objaverse provides the 3D objects rendered to build the triplane feature dataset and the LVIS edited-image pairs.","marker":"[9]"},{"why":"CLIP text embeddings condition the diffusion UNet's cross-attention layers, connecting natural-language prompts to triplane edits.","marker":"[36]"},{"why":"Baseline that edits meshes with text; Instructive3D compares against it for edit quality.","marker":"[30]"},{"why":"Baseline that paints textures on meshes; one of the mesh-editing approaches Instructive3D is compared against.","marker":"[58]"},{"why":"Baseline for text-guided texturing of 3D shapes; part of the comparison supporting the claimed superiority in edit quality.","marker":"[38]"}],"fun_headline_variants":["Text prompts edit 3D objects without 3D edit pairs","Instructive3D: fine-grained 3D editing via text and triplane diffusion","Frozen LRM gets text-driven 3D editing with a diffusion adapter","Triplane latent diffusion adds fine-grained text control to 3D models","Text-guided 3D edits that stay consistent across viewpoints"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper's edited 3D \"ground truth\" is actually a triplane computed from a 2D image edit, so the whole method stands on whether InstructPix2Pix edits are faithful 3D-consistent changes and whether the frozen LRM encodes them without loss.","fun_headline_variants_meta":{"raw":{"variants":["Text prompts edit 3D objects without 3D edit pairs","Instructive3D: fine-grained 3D editing via text and triplane diffusion","Frozen LRM gets text-driven 3D editing with a diffusion adapter","Triplane latent diffusion adds fine-grained text control to 3D models","Text-guided 3D edits that stay consistent across viewpoints"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000186,"raw_usage":{"total_tokens":1380,"prompt_tokens":1054,"completion_tokens":326,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":670,"completion_tokens_details":{"reasoning_tokens":229}},"tokens_in":670,"tokens_out":326,"duration_ms":3786,"temperature":1.0,"reasoning_tokens":229,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T21:35:14.025100+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Render Instructive3D outputs for the same object and prompt (e.g., \"add a flame design to the shoe\") from all eight cube-corner viewpoints used in the paper's evaluation: if the flame design appears only in some views or the object's silhouette changes between views, the geometrically consistent triplane-editing claim is refuted. A sharper test compares the output mesh against a manually edited 3D ground truth rather than against a 2D-edited rendering.","supporting_citations":[{"cited_title":"High-resolution image synthesis with latent diffusion models","cited_arxiv_id":null,"evidence_quote":"Latent diffusion framework that motivates training the diffusion model in a compressed triplane latent space rather than pixel or triplane space."},{"cited_title":"Objaverse: A universe of annotated 3d objects, 2022","cited_arxiv_id":null,"evidence_quote":"Objaverse provides the 3D objects rendered to build the triplane feature dataset and the LVIS edited-image pairs."},{"cited_title":"Learning transferable visual models from natural language supervi- sion","cited_arxiv_id":null,"evidence_quote":"CLIP text embeddings condition the diffusion UNet's cross-attention layers, connecting natural-language prompts to triplane edits."},{"cited_title":"Text2mesh: Text-driven neural stylization for meshes","cited_arxiv_id":null,"evidence_quote":"Baseline that edits meshes with text; Instructive3D compares against it for edit quality."},{"cited_title":"Texture: Text-guided texturing of 3d shapes, 2023","cited_arxiv_id":null,"evidence_quote":"Baseline for text-guided texturing of 3D shapes; part of the comparison supporting the claimed superiority in edit quality."}],"review_version":1}