{"id":"ebf834ae-7969-4420-b46c-e6128c6ff003","arxiv_id":"2506.05338","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A furniture-removal pipeline for indoor meshes and panoramas that uses a simplified defurnished mesh to guide ControlNet inpainting, producing cleaner results than NeRF or RGB-D baselines.","lead":"This paper builds a pipeline that removes furniture from both 3D indoor meshes and their 360-degree panoramas, using a simplified furniture-free mesh as a geometric guide for image inpainting. The method is aimed at real-estate staging and digital-twin cleanup, where empty-room visuals are needed.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The quantitative case for the SDM-control advantage is built on an in-distribution random-mask proxy and a synthetic geometry test of the SDM itself; neither establishes that the X-ray control improves output on real furniture masks, so the headline comparison to NeRFiller may not transfer.","rationale":"The reader identified the same core weakness: the random-mask proxy may not represent real furniture removal. My stress-test agrees but sharpens it in two directions. First, the evaluation is in-distribution with respect to the fine-tuning task, so Table 1 shows that the fine-tuned ControlNet learned the synthetic protocol, not that it generalizes to real furniture masks. Second, the 2.3 cm vs 24.1 cm geometry result in Section 4.2 is not a test of the full defurnishing pipeline: it compares the SDM, which is constructed by planar simplification and plane extension on the same ground-truth mesh, against a Poisson-extracted NeRFiller mesh. That tells us the SDM is a clean planar mesh, not that the CN inpainting guided by it produces better assets. The paper does have independent support: a working system, detailed ablations, honest failure cases, and qualitative real-scene comparisons that are visually plausible. Those are real evidence and prevent rejection. However, the quantitative pillars of the central claim are too weak to fully accept the headline advantage over radiance-field methods, so the conditional verdict stands. The concrete test I propose reuses the paper's own synthetic benchmark but evaluates final outputs with real furniture masks and post-processing, which would directly settle whether the advantage transfers.","tokens_in":15786,"tokens_out":7227,"duration_ms":89812,"concrete_test":"Re-run the Section 4.1/4.2 comparison on synthetic furnished-unfurnished pairs in which the masks are the actual segmentation masks of procedurally inserted Objaverse furniture with shadows, and evaluate the full pipeline outputs after super-resolution and blending against the ground-truth unfurnished panoramas, for SD, CN-thibaud, CN-ours, and NeRFiller at matched output resolution. If CN-ours's masked-PSNR and LPIPS advantage over baselines and NeRFiller disappears or shrinks once real furniture masks and final post-processing are used, the claimed benefit of the SDM control is proxy-specific rather than a property of real defurnishing.","verdict_should_be":"UNCHANGED","load_bearing_attack":"In Section 3.4, the ControlNet is fine-tuned on 50,000 unfurnished panoramas with masks built by superimposing random circles, and the evaluation in Section 4.1 uses 700 unfurnished panoramas with the same random-mask protocol and the same SDM-derived Canny controls; the paper also states that all Table 1 metrics are computed before super-resolution and blending. This is an in-distribution test of the fine-tuned model: it measures adaptation to equirectangular images and to the SDM edge style, not the ability to remove real furniture, whose masks are elongated and correlated (wardrobes, sofas, beds) and whose shadows and reflections are not included in the mask (Section 3.1). The synthetic geometry comparison in Section 4.2 (2.3 cm vs 24.1 cm RMSE) does not close this gap: it compares the SDM itself against a ground-truth unfurnished planar mesh, and because the SDM is produced by planar simplification and plane extension on the same mesh, this largely validates the mesh-processing construction on piecewise-planar scenes rather than the CN inpainting or the final textured asset. The only real-scene evidence against NeRFiller is qualitative, and the synthetic image metrics in Table 2 are secondary and computed before final post-processing. The central claim that the SDM 'X-ray' control produces higher-quality defurnished assets on real furnished scenes therefore rests on an unvalidated distributional assumption.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript presents \"Defurnishing with X-Ray Vision,\" a pipeline that jointly removes furniture from textured indoor meshes and their associated 360° panoramas. The method first segments furniture in panoramas with a semantic segmentation model, projects the masks onto the mesh, simplifies the mesh, removes furniture faces, and closes holes to obtain a simplified defurnished mesh (SDM). Depth and normal images rendered from the SDM are converted to Canny edge maps, which serve as control signals for ControlNet-based inpainting of the panoramas; the ControlNet is fine-tuned on 50,000 unfurnished panoramas with random composite masks. The inpainted panoramas are then super-resolved, blended, and used to texture the SDM. The paper reports ablations against base Stable Diffusion inpainting and an off-the-shelf Canny ControlNet, qualitative comparisons to NeRFiller and Wei et al., and a synthetic geometry experiment showing lower RMSE than NeRFiller.","tokens_in":16115,"tokens_out":4762,"duration_ms":56516,"significance":"If the empirical claims hold, the pipeline is practically valuable: it targets a real application, produces high-resolution defurnished panoramas and clean meshes at a modest runtime, and introduces a geometric prior that does not rely on semantic labels at inference time. The paper is also transparent about failure cases and includes a synthetic geometry test. The core idea of using SDM-rendered edges as global structure guidance for 2D inpainting is sensible and potentially useful. However, the quantitative validation currently centers on a proxy task that closely matches the training distribution, and the synthetic geometry test does not exercise the inpainting/control component; the significance of the X-ray control over standard inpainting is therefore not yet established.","major_comments":[{"comment":"The ablation in Section 4.1 is computed on 700 unfurnished panoramas with random masks and SDM-derived Canny controls, which is the same task used to fine-tune the ControlNet in Section 3.4 (50,000 unfurnished panoramas, composite random-circle masks). The comparison therefore measures in-distribution performance on the synthetic training task rather than defurnishing of real furnished scenes, where furniture masks are elongated and correlated and where shadows and reflections are not covered by the masks. The observed gains over base SD and the off-the-shelf Canny ControlNet may reflect adaptation to equirectangular image statistics and to the SDM edge style rather than a generalizable advantage on real furniture removal. Furthermore, the paper states that all metrics are calculated before super-resolution or blending are applied, so Table 1 does not measure the final output quality. Please add an evaluation on real furnished scenes, ideally with manually refined masks and metrics on the final post-processed results, before claiming that the X-ray control improves defurnished asset quality.","section":"Section 4.1, Table 1"},{"comment":"The synthetic geometry result, 2.3 cm RMSE for the SDM versus 24.1 cm for NeRFiller, compares the SDM itself against a ground-truth unfurnished mesh in a procedurally furnished scene. Because the SDM is obtained by planar simplification and plane extension from the same type of mesh representation, this largely validates the mesh-processing construction on piecewise-planar scenes; it does not measure the effect of the ControlNet inpainting or the final textured asset. A reader cannot infer from this experiment that the X-ray control improves the output on real data. Please report geometry error for the full pipeline on real furnished scenes, or state explicitly in the main text that this number is for the SDM only and not for the final textured output.","section":"Section 4.2 and Supplementary Section 8"},{"comment":"The segmentation masks explicitly exclude shadows and reflections cast by furniture, and Figure 7c lists 'spurious shadows' as a failure mode. Yet the comparison to NeRFiller in Figure 5 is qualitative and claims that our method is 'trained to be robust' to light reflections and shadows. No quantitative evidence on real scenes supports this robustness, and the synthetic image metrics in Table 2 use procedurally inserted furniture with approximate shadows rather than the true shadow and reflection statistics of real panoramas. Please provide a real-scene evaluation that isolates shadow and reflection handling, or temper the robustness claim to match the available evidence.","section":"Section 3.1, Figure 5, Figure 7c"}],"minor_comments":[{"comment":"The composite mask generation ranges (number of circles, radii, center locations) are described only as 'predefined ranges'; please specify the actual values for reproducibility.","section":"Section 3.4"},{"comment":"The word 'perspetive' should be 'perspective'.","section":"Supplementary Material, Figure 10 caption"},{"comment":"There are typos in the final paragraph: 'on avergae' should be 'on average' and 'magniture' should be 'magnitude'.","section":"Supplementary Material, Section 8"},{"comment":"The FFTMax loss divides by X_T(x), which can be zero for frequency components that are absent in the target image; please define a guard or clarify how this case is handled.","section":"Section 3.5, Eq. (1)"},{"comment":"The citation for ADE20K points to a paper on cooperative semantic segmentation; the standard ADE20K dataset reference would be more appropriate for the claim that the training data is 'similar to ADE20K'.","section":"Section 3.1, Reference [60]"}],"recommendation":"major_revision","confidential_remarks":"The paper comes from an industrial lab and includes comparisons to methods whose code is not always public; the authors should be asked to release code or provide more detailed real-scene results. The central idea is plausible, but the current quantitative support is too heavily based on an in-distribution proxy and a mesh-only synthetic test; additional real-scene evaluations or clearly scoped claims would make the contribution solid."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nQuick take: this is a solid applied pipeline from Matterport, and the novel piece is real: they build a simplified defurnished mesh (SDM), render depth/normal Canny edges from it, and feed those as ControlNet conditioning to inpaint furniture out of panoramas. That global geometric prior is new relative to prior work, and the FFTMax loss is a small but sensible addition. Qualitatively the results are convincing—the plane-extension hole filling avoids the wall-floor warping you get from Poisson remeshing, and the fine-tuned ControlNet clearly beats vanilla SD and off-the-shelf Canny on equirectangular inputs.\n\nThe soft spots are in the numbers. Table 1 is an in-distribution proxy: the ControlNet is fine-tuned on 50,000 unfurnished panoramas with random composite masks, and the 700-image ablation uses the same mask protocol and SDM-derived edges. It measures adaptation to equirectangular images and to the SDM edge style, not necessarily the ability to remove real furniture, whose masks are elongated and correlated and whose shadows/reflections are explicitly not covered. Metrics are also computed before super-resolution and blending, so they don't describe the final asset. The synthetic geometry comparison (2.3 cm vs 24.1 cm RMSE) largely validates the SDM construction on piecewise-planar scenes; it doesn't isolate the CN inpainting contribution. There are no error bars, FFTMax is not ablated, and training/test disjointness is unstated. The Wei et al. comparison is a single qualitative ScanNet scene.\n\nThat said, the paper is honest—Section 4.1 admits metrics are pre-post-processing, Section 3.4 describes the composite masks as mimicking arbitrary objects, and they show failure cases. The central idea is sound and the engineering is real. It deserves a serious referee, but the authors should be pushed to evaluate on real furniture masks (or masks from segmentation on furnished scenes) and to report metrics on the final output. I'd happily bring it to the reading group.","headline":"A genuinely useful, practical defurnishing pipeline built on an SDM-guided ControlNet, but the headline quantitative claims rest on an in-distribution random-mask proxy and need stronger real-scene validation.","tokens_in":16666,"tokens_out":1922,"would_cite":true,"duration_ms":22645,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a simplified defurnished mesh, rendered into depth and normal edge maps, is the right control signal for removing furniture from both panoramas and 3D meshes, producing sharper and more accurate empty rooms than…","keywords":["defurnishing","360-degree panorama inpainting","3D mesh editing","ControlNet","Stable Diffusion","geometric prior","digital twins","indoor scene understanding"],"falsifier":"Take one real furnished scene, physically remove the furniture and scan the empty room as ground truth, then run the pipeline on the furnished panoramas and measure masked-region error against the empty-room ground truth; if the advantage over vanilla Stable Diffusion shrinks or disappears, the random-mask fine-tuning is the weak link.","tokens_in":15582,"feed_emoji":"🧹","tokens_out":5902,"duration_ms":61777,"temperature":0.7,"pith_summary":"The paper tries to establish that furniture can be removed from scanned indoor spaces, in both the 3D mesh and the 360-degree panoramas, by first building a simplified defurnished mesh that acts as an X-ray of the room's structure. The mesh is rendered into depth and normal images, edges are extracted, and those edges guide a ControlNet-based Stable Diffusion inpainter to fill furniture regions with empty-room content. The inpainted panoramas then re-texture the mesh. The paper argues this beats radiance-field methods on sharpness and mesh accuracy, reporting a synthetic test where the simplified mesh error is 2.3 cm versus 24.1 cm for NeRFiller, and beats plain Stable Diffusion and off-the-shelf ControlNet on perceptual metrics. A reader should care because defurnishing is the enabling step for virtual staging and clean digital twins.","feed_headline":"A mesh X-ray guides diffusion to strip furniture from 3D scans","feed_subtitle":"Simplified empty-room mesh feeds Canny edges to ControlNet, beating radiance fields on sharpness and geometry.","key_machinery":"The simplified defurnished mesh (SDM) is the central object: a planar-approximated version of the original mesh with furniture faces removed and holes closed by extending neighboring wall and floor planes, representing the room as if empty. Rendered depth and normal images from the SDM, run through a Canny edge detector, become the control signal for a fine-tuned Canny ControlNet inpainter. The fine-tuning uses 50,000 unfurnished panoramas with random composite masks, and the paper adds an FFTMax loss during super-resolution that penalizes predicted high-frequency energy only when it exceeds the target, suppressing artefacts while restoring texture detail.","core_discovery":"The central claim is that the geometry of the unfurnished room, not the pixel content, is the right control signal for furniture removal. The method obtains a simplified defurnished mesh (SDM) by removing furniture faces and filling holes with plane extensions, renders depth and normal maps from the SDM, extracts Canny edges from those maps, and feeds the edges as ControlNet conditioning to a Stable Diffusion inpainter. The geometric prior carries information hidden by furniture in any single panorama, so the inpaint follows true wall, floor, and ceiling edges rather than hallucinated content. The paper reports that this yields sharper panoramas and cleaner meshes than radiance-field inpainting, and better masked-region perceptual metrics than vanilla Stable Diffusion and off-the-shelf Canny ControlNet.","pith_inferences":["An implication the authors leave implicit is that the random-composite-mask fine-tuning is a proxy for real furniture removal statistics; a testable next step is to collect real mask-shadow pairs and fine-tune on those instead.","Because the SDM control is purely geometric, the same pipeline could plausibly extend to other structure-guided edits, such as removing built-in clutter or altering room layout, without retraining the inpainter beyond the control channel.","A natural hybrid not tested here would feed the SDM edge maps into a multi-view transformer inpainter to enforce cross-view consistency, though current multi-view transformers operate at lower resolution.","The 2.3 cm versus 24.1 cm geometry comparison is on synthetic data with inserted Objaverse objects; reproducing the same protocol on real furnished scans would show whether the gap persists outside synthetic settings."],"forward_implications":["If the claim holds, defurnished digital twins can be produced at native panorama resolution with explicit clean meshes, avoiding the blur and volumetric blobs left by radiance-field methods.","For downstream tasks requiring metric accuracy, the SDM provides an order-of-magnitude better geometry than radiance-field inpainting on the synthetic test: 2.3 cm versus 24.1 cm root mean squared error.","Because the control signal is geometric rather than semantic, the same mesh-guided inpainting can be repurposed for related tasks such as clutter removal, as demonstrated on a ScanNet scene.","The pipeline runs in roughly 10 minutes for a 30-panorama scene on a single A10G GPU, making it practical for real-estate-scale processing.","The method remains vulnerable to hallucination, ignored control signals, spurious shadows, and view inconsistency in occluded regions, as the paper's failure cases show."],"supporting_citations":[{"why":"Supplies ControlNet conditioning, the mechanism that injects the Canny edge control into Stable Diffusion inpainting.","marker":"[65]"},{"why":"Stable Diffusion is the base inpainting model that the pipeline fine-tunes and runs.","marker":"[39]"},{"why":"Provides the fine-tuned Stable Diffusion defurnishing weights and the blending strategy the pipeline builds on.","marker":"[42]"},{"why":"NeRFiller is the radiance-field inpainting baseline that the paper compares against for image quality and mesh accuracy.","marker":"[56]"},{"why":"Canny edge detection converts the SDM-rendered depth and normal images into the control signal.","marker":"[4]"},{"why":"Objaverse objects are inserted into unfurnished models for the synthetic geometry test measuring 2.3 cm versus 24.1 cm error.","marker":"[13]"},{"why":"Matterport3D provides the real-world furnished scenes used in the main radiance-field comparisons.","marker":"[5]"},{"why":"Wei et al. is the RGB-D clutter removal baseline whose ScanNet mesh result the paper uses for the decluttering comparison.","marker":"[58]"}],"fun_headline_variants":["Geometry, not pixels, guides furniture removal from panoramas","X-ray mesh controls diffusion to clear furniture from 3D scans","Empty-room mesh edges beat radiance fields in defurnishing","Geometric prior from SDM improves furniture removal quality","SDM edges drive ControlNet inpainting for furniture removal"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that random composite masks painted on unfurnished panoramas reproduce the statistics of real furniture removal, including long wardrobes and the shadows and reflections that the segmentation masks do not cover; the quantitative evaluations are built on this proxy task.","fun_headline_variants_meta":{"raw":{"variants":["Geometry, not pixels, guides furniture removal from panoramas","X-ray mesh controls diffusion to clear furniture from 3D scans","Empty-room mesh edges beat radiance fields in defurnishing","Geometric prior from SDM improves furniture removal quality","SDM edges drive ControlNet inpainting for furniture removal"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00126,"raw_usage":{"total_tokens":5126,"prompt_tokens":877,"completion_tokens":4249,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":493,"completion_tokens_details":{"reasoning_tokens":4165}},"tokens_in":493,"tokens_out":4249,"duration_ms":37409,"temperature":1.0,"reasoning_tokens":4165,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T10:21:12.420952+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take one real furnished scene, physically remove the furniture and scan the empty room as ground truth, then run the pipeline on the furnished panoramas and measure masked-region error against the empty-room ground truth; if the advantage over vanilla Stable Diffusion shrinks or disappears, the random-mask fine-tuning is the weak link.","supporting_citations":[{"cited_title":"Adding Conditional Control to Text-to-Image Diffusion Models","cited_arxiv_id":null,"evidence_quote":"Supplies ControlNet conditioning, the mechanism that injects the Canny edge control into Stable Diffusion inpainting."},{"cited_title":"An Empty Room is All We Want: Au- tomatic Defurnishing of Indoor Panoramas","cited_arxiv_id":null,"evidence_quote":"Provides the fine-tuned Stable Diffusion defurnishing weights and the blending strategy the pipeline builds on."},{"cited_title":"NeRFiller: Completing Scenes via Generative 3D Inpainting","cited_arxiv_id":null,"evidence_quote":"NeRFiller is the radiance-field inpainting baseline that the paper compares against for image quality and mesh accuracy."},{"cited_title":"A computational approach to edge detection","cited_arxiv_id":null,"evidence_quote":"Canny edge detection converts the SDM-rendered depth and normal images into the control signal."},{"cited_title":"Objaverse: A Universe of Annotated 3D Objects, 2022","cited_arxiv_id":null,"evidence_quote":"Objaverse objects are inserted into unfurnished models for the synthetic geometry test measuring 2.3 cm versus 24.1 cm error."},{"cited_title":"Matterport3D: Learning from RGB- D Data in Indoor Environments.International Conference on 3D Vision (3DV), 2017","cited_arxiv_id":null,"evidence_quote":"Matterport3D provides the real-world furnished scenes used in the main radiance-field comparisons."},{"cited_title":"Clutter Detection and Removal in 3D Scenes with View-Consistent Inpainting","cited_arxiv_id":null,"evidence_quote":"Wei et al. is the RGB-D clutter removal baseline whose ScanNet mesh result the paper uses for the decluttering comparison."}],"review_version":1}