{"id":"1792ef75-173f-4582-9994-dc5a4e0dcd15","arxiv_id":"2411.17864","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A diffusion-based model decomposes an image into a clean background and a transparent foreground layer that retains shadows and reflections, enabling object removal and spatial edits.","lead":"LayerDecomp is an image-editing method that splits a photo into a clean background layer and a transparent foreground layer, keeping shadows and reflections attached to the object. It uses a large diffusion model trained on simulated and real paired photos, and its creators report large gains over prior methods in object removal and spatial editing.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Foreground layer is never directly evaluated against ground truth on images with visual effects; re-composition metrics are insensitive to how effects are split between layers, so the central claim about faithful visual-effect preservation lacks direct quantitative support.","rationale":"The paper's strongest claim has two components: (1) the predicted background is clean and (2) the predicted foreground transparently contains the object and its visual effects, so that editing the foreground moves the effect naturally. Component (1) is directly supervised and benchmarked. Component (2) is the differentiator of the method, and it is the least supported. On real camera-captured pairs, the foreground diffusion loss is masked out (Sec. 3.1), and only Lconsist (Eq. 3) supervises the foreground. Because alpha blending is many-to-one, Lconsist cannot identify the correct split of visual effects between layers. The paper's ablations use re-composition fidelity (which is exactly Lconsist) and background fidelity (which does not test foreground content), so they are consistent with a model that leaves part of a shadow in the background and compensates in the foreground. The random re-composition test in Table 5 is closer, but FID and CLIP-FID over the whole image cannot determine whether the effect moved with the object or was hallucinated elsewhere. User studies show preference over baselines, but those baselines are not designed for layered decomposition, so they do not validate the absolute correctness of the foreground layer. The missing piece is any direct evaluation of the foreground layer against ground truth on images containing shadows or reflections. This is a falsifiable gap: if the model actually allocates visual effects correctly, it should score well on a benchmark with ground-truth foreground layers. Until such an evaluation is reported, the central claim should be treated as conditional. The reader's weakest assumption identifies the same non-identifiability issue, and the conditional verdict remains appropriate; no stronger action is warranted because the concern is about missing evidence rather than a demonstrated failure.","tokens_in":95,"tokens_out":7829,"duration_ms":139288,"concrete_test":"Build a held-out evaluation set of real-background photographs composited with ground-truth transparent foreground layers that include shadows and reflections (e.g., using the authors' MetaShadow synthesis pipeline applied to real images, or by augmenting DESOBAv2 pairs with the corresponding object assets). For each image, compute the alpha-MSE and RGB-PSNR/LPIPS of the predicted foreground restricted to the shadow or reflection region, and compare LayerDecomp against the V2 variant (without Lconsist) and MULAN/LayerDiffusion baselines. If the consistency loss does not substantially improve these foreground-targeted metrics, the claim that Lconsist enforces faithful visual-effect preservation is falsified.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The weakest point is the supervision of the transparent foreground on real camera-captured data. In Sec. 3.1, when the ground-truth RGBA foreground is absent, the diffusion loss Ldm is masked out on foreground positions, so the only supervisor for the foreground layer is the consistency loss Lconsist (Eq. 3). Alpha blending is not injective: for a fixed composite image and a predicted background, the foreground (alpha, color) is unconstrained on transparent pixels and has one free degree per pixel elsewhere. Thus Lconsist alone cannot determine whether a soft shadow is placed in the foreground or left in the background; any split whose alpha blend reproduces the input achieves zero consistency loss. The background diffusion loss removes the most degenerate solutions (e.g., foreground fully transparent, background equal to input), but it does not pin down the foreground uniquely: the model could shift part of a shadow into the background with a compensating change in foreground and still satisfy both losses. The ablations in Table 1 evaluate the background against ground truth and the re-composite against the input, but not the foreground layer itself on any data with ground-truth visual effects. The re-composite metric is the training objective of Lconsist, so improvement on it is expected and does not demonstrate that the shadow or reflection is correctly allocated to the foreground. Supplementary Table 5 uses random re-composition but reports only FID and CLIP-FID, which are distribution-level and cannot localize whether effects moved with the object. Consequently, the central claim of faithfully preserved visual effects in the foreground layer is not directly quantitatively supported for real images; it rests on an untested assumption that the pretrained DiT prior and simulated-data transfer resolve the non-identifiability in favor of the intended decomposition.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"LayerDecomp is a generative framework that takes a composite image and a binary object mask as input and outputs two layers: a clean background and a transparent foreground (RGBA) that is intended to retain visual effects such as shadows and reflections. The method is built on a 5B-parameter Diffusion Transformer, uses an RGBA-VAE adopted from LayerDiffusion, and is trained on a hybrid dataset: large-scale simulated triplets (background, foreground, composite) plus a smaller set of camera-captured background/composite pairs. To supervise the foreground when ground-truth annotations are unavailable, the paper introduces a pixel-space consistency loss between the original composite and the alpha-blended re-composite of the two predicted layers. The method is evaluated on object removal (RORD, MULAN, DESOBAv2, Emu-Edit) and spatial editing (moving/resizing), with ablations and two user studies.","tokens_in":15079,"tokens_out":5450,"duration_ms":48366,"significance":"If the central claim holds, this is a practically valuable advance: it would enable training-free, layer-wise editing with shadows and reflections preserved in the foreground, going beyond prior inpainting-based removal and text-driven editing. The empirical effort is substantial: three public benchmarks, two user studies, ablation experiments, a proposed data pipeline for simulated triplets, and the use of a large pretrained DiT backbone. The paper is generally clear and the proposed dataset construction is a reasonable way to address the scarcity of multi-layer data. However, the key novelty claim — faithful preservation of visual effects in the foreground layer — is not directly supported for real camera-captured images, and the training objective leaves the layer split underdetermined. These gaps are load-bearing for the main claim and need to be addressed before the paper can be accepted.","major_comments":[{"comment":"The training objective for camera-captured data does not uniquely determine the decomposition. As stated in Sec. 3.1, when the ground-truth foreground is absent, the diffusion loss is masked out on the foreground, so the only supervision for the foreground is the consistency loss Lconsist in Eq. (3). Because alpha blending is not injective, for a fixed composite and a predicted background there are infinitely many (alpha, color) foregrounds that reproduce the composite; the background diffusion loss removes only the most degenerate splits (e.g., fully transparent foreground) but does not force a soft shadow or reflection to be assigned to the foreground rather than retained in the background. The paper neither states nor tests this assumption, although it is essential for the claim that visual effects are faithfully preserved in the foreground. A direct evaluation of the predicted foreground against ground-truth RGBA on simulated data with visual effects, or a controlled sensitivity test that shifts the ground-truth effect allocation, would substantiate the claim.","section":"Sec. 4.1, Table 1 and Supp. Table 5"},{"comment":"The only quantitative evaluation of the foreground layer is through re-composition: the Comp columns in Table 1 and the random re-composition FID/CLIP-FID in Supp. Table 5. These metrics are essentially the training objective of Lconsist, so it is expected that adding Lconsist improves them; they do not demonstrate that shadows or reflections are correctly allocated to the foreground. The paper should compare the predicted foreground (alpha and color) against ground-truth foreground layers on the simulated test set, or use a metric that penalizes wrong effect allocation (e.g., masked foreground PSNR/LPIPS computed on the effect region). Without such a direct evaluation, the central claim of faithful visual-effect preservation is not supported by the quantitative results.","section":"Sec. 4.1, Table 1 and Supp. Table 5"},{"comment":"No error bars, confidence intervals, or statistical significance tests are reported for any quantitative result. Several metrics are computed on relatively small sets (635 images for Table 1, 1,029 for the RORD subset, 1,000 for the MULAN subset, 750 for DESOBAv2, and 60 images for the Table 3 user study). FID and CLIP-FID are known to have high variance on such sample sizes, and the reported differences between methods are often only a few points (e.g., FID 21.77 vs. 39.48 in Table 2). The reader cannot determine whether these differences are meaningful. Please report standard deviations or bootstrap confidence intervals, and indicate whether user-study preferences are statistically significant (e.g., with a paired test).","section":"Tables 1–6"},{"comment":"The paper does not release code, data, or a link to either; the project page is mentioned but the manuscript contains no URL or repository. Given that the method's key advantage rests on a new training dataset and the consistency-loss formulation, independent verification is currently not possible. In addition, the shadow synthesis in the data pipeline relies on an under-review method (Ref. [40], 'MetaShadow'), whose details are not provided. The authors should at least state the data availability plan and describe the shadow synthesis pipeline in enough detail to be reproducible.","section":"Reproducibility"}],"minor_comments":[{"comment":"The notation for image variables is inconsistent: both IRGBcomp and IRGB_comp appear; please standardize the subscript/superscript placement.","section":"General"},{"comment":"In Table 2, the 'Ours' row reports both L and T results, but it is unclear whether these come from the same model with different mask inputs or from two variants of the method; please clarify.","section":"Table 2"},{"comment":"In Figure 3 and other figures, abbreviations such as 'CNI', 'SDI', 'PP' are used without definition in the captions; please define them or refer to the table where they are defined.","section":"Figure 3"},{"comment":"Equation (3) uses an L1 norm; please state explicitly whether it is a mean absolute error over pixels and channels, and specify how the alpha-blending A is defined with respect to the RGBA representation.","section":"Sec. 3.2, Eq. (3)"},{"comment":"Figure 2's caption contains a typo: 'refelctions' should be 'reflections'.","section":"Figure 2"},{"comment":"In the user-study tables, the reported percentages for 'Ours' are very high (e.g., 83.54% overall), but the tie option is not shown in the table; please report the tie percentages and the exact number of participants and trials per condition.","section":"Tables 3–4"}],"recommendation":"major_revision","confidential_remarks":"The paper is technically sound in its empirical breadth and the proposed data pipeline is a reasonable engineering contribution, but the central claim about faithful visual-effect preservation in the foreground is not yet supported by direct evidence. The underdetermined layer split and the circular re-composition metric are the main correctness risks. I would be willing to look at a revision that adds a direct foreground evaluation on simulated data with ground-truth RGBA and an explicit discussion of the split ambiguity. The lack of code/data release is also a notable concern for a methods paper."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First take: LayerDecomp is a real step forward in image layer decomposition—it tackles a task nobody else attacked in this form, and the engineering is substantial. The core idea is simple: given a composite image and object mask, fine-tune a DiT to denoise both a clean RGB background and an RGBA foreground that carries shadows and reflections; train on simulated triplets plus camera-captured pairs, and use a pixel-space consistency loss when foreground GT is missing. That combination is new relative to LayerDiffusion, MULAN, and ObjectDrop. The paper also does a lot empirically: three public benchmarks, two user studies, ablations, and comparisons against a wide set of baselines. The background removal results look strong, and the qualitative examples of shadows and reflections moving with the object are convincing.\n\nThe soft spots are real but not fatal. No code, data, or checkpoints are released, and no error bars appear on any table. The user-study description is thin—17 and 23 raters, but no recruitment details, no instruction text, no per-user variance. More substantively, the stress-test concern about the foreground supervision holds up. For real camera-captured data, foreground is supervised only through the consistency loss, and alpha blending is not injective: an infinite family of (BG, FG) pairs produces the same composite. The diffusion loss on the background removes the most degenerate solutions, but it does not pin down whether a soft shadow lands in the FG or stays in the BG. The re-composite metric in Table 1 is close to the training objective, so gains there are expected. Supplementary Table 5 uses random recomposition but only FID and CLIP-FID, which cannot localize the effect. So the central claim—faithful visual-effect preservation in the foreground—is not directly quantitatively supported on real images. It rests on an untested prior assumption. That said, I do not think this sinks the paper. The background-removal evidence is solid, the qualitative FG results are suggestive, and the non-identifiability is a limitation the authors should be asked to address, not a contradiction in their method.\n\nCitations look fair; the paper builds on LayerDiffusion, MULAN, and ObjectDrop and says so. No circularity issue. Who is this for: computer vision folks working on image editing, object removal, and layer representations. It deserves a serious referee, conditional on the authors releasing artifacts and adding a direct foreground evaluation (e.g., on simulated triplets with GT alpha and color, or a perceptual metric on the FG layer alone). My call: accept, pending those revisions.","headline":"A solid, useful layer-decomposition paper whose foreground-effect claims outrun the evaluation; worth refereeing, but the authors should be pushed to release data/code and directly evaluate the foreground layer.","tokens_in":15724,"tokens_out":2063,"would_cite":true,"duration_ms":19185,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a diffusion transformer trained with a consistency loss can decompose any masked image into a photorealistic clean background and a transparent foreground that keeps shadows and reflections, enabling direct object…","keywords":["layer decomposition","visual effects","transparent foreground","RGBA layers","consistency loss","object removal","spatial editing","diffusion transformer"],"falsifier":"Run the model on a set of real composite/background pairs and check whether the predicted foreground, placed on a neutral gray background, reproduces the shadow or reflection at the correct location and opacity; if the background layer retains any of the shadow or the foreground carries background texture, the consistency loss has not achieved a correct visual-effect split.","tokens_in":14600,"feed_emoji":"🖼️","tokens_out":4862,"duration_ms":42575,"temperature":0.7,"pith_summary":"This paper proposes LayerDecomp, a generative framework that takes an image and an object mask and splits it into two editable layers: a clean background without the object, and a transparent foreground that carries the object together with its visual effects such as shadows and reflections. The authors' central claim is that this decomposition can be learned from a mix of synthetic triplets and real camera-captured before/after pairs, provided the model is trained with a consistency loss that forces the two predicted layers to re-blend into the original input. If correct, common editing operations such as object removal, moving, and resizing become simple layer manipulations that do not require fine-tuning and do not lose the grounding effects that make composites look real. The practical payoff is that users can edit the foreground independently and still recompose a coherent image.","feed_headline":"A diffusion model splits images into editable layers, shadows and all","feed_subtitle":"Editing results keep shadows and reflections attached to the object, so moved objects still look grounded.","key_machinery":"The central mechanism is the consistency loss of Eq. (3): predicted background and foreground latents are reparameterized to clean estimates, decoded through RGB and RGBA decoders, alpha-blended, and compared pixel-wise to the input composite; this is what supervises transparent foregrounds when no foreground ground truth exists. The supporting machinery is a diffusion transformer with per-image type embeddings that denoises both layer latents jointly, an RGBA variational autoencoder adapted from the base autoencoder, and a synthetic data pipeline that builds object assets with synthesized shadows and blends them onto random backgrounds.","core_discovery":"The discovery is that layer decomposition with visual effects can be treated as joint diffusion denoising of two latents: a background latent in RGB and a foreground latent in RGBA, conditioned on the composite image and object mask, with alpha blending as the reconstruction interface. Because real-world foreground ground truth is unavailable, the method adds a pixel-space consistency loss: at each denoising step the predicted layers are decoded, alpha-blended, and compared with the input composite, so visual effects are pushed into the foreground layer rather than hallucinated or lost. The paper reports that adding an RGBA foreground output, visual-effect-aware synthetic training, and the consistency loss progressively improves background quality and recomposition fidelity, and that the resulting model outperforms existing inpainting and spatial-editing systems on object removal and spatial editing benchmarks and user studies.","pith_inferences":["Because alpha blending is not injective, the training objective cannot by itself pick a unique decomposition; the split quality must come from the pretrained diffusion prior, which makes the distribution of synthetic effects and the quality of the input mask load-bearing for generalization.","The same two-layer scheme could plausibly extend to other semi-transparent effects such as smoke, mist, or wet-floor reflections if reliable paired data or simulators for those effects become available.","A direct way to stress-test the claim is to decompose images where the shadow is visually ambiguous (for example, soft or detached shadows) and check whether the foreground layer still carries the effect rather than leaving it in the background."],"forward_implications":["Object removal no longer requires loose masks or inpainting; tight masks suffice because shadows and reflections are assigned to the foreground layer instead of being regenerated in place.","Spatial edits such as moving and resizing carry the visual effects along with the object, so recomposited scenes stay physically plausible without extra shadow modeling.","Multiple objects can be decomposed sequentially into separate layers, allowing each object to be edited independently and then recombined coherently.","Downstream composition is a simple alpha-blending operation, so no model fine-tuning is needed at editing time."],"supporting_citations":[{"why":"Supplies the RGBA variational autoencoder and latent transparency design used to encode and decode the transparent foreground layer.","marker":"[49]"},{"why":"Defines the diffusion transformer backbone that LayerDecomp is fine-tuned from.","marker":"[30]"},{"why":"Provides the counterfactual camera-captured pair idea used to collect real-world background/composite training data.","marker":"[41]"},{"why":"Supplies the shadow synthesis method that generates shadow intensity maps for the simulated object assets.","marker":"[40]"},{"why":"Provides a multi-layer RGBA dataset used both as a reference for training data construction and as an evaluation benchmark for decomposition quality.","marker":"[39]"},{"why":"Supplies an instruction-driven object removal benchmark and baselines used in the paper's user studies and quantitative comparisons.","marker":"[36]"}],"fun_headline_variants":["Diffusion splits images into editable layers with effects","Layer decomposition with visual effects via diffusion","Edit images by breaking into layers with shadows intact","Diffusion model separates foreground with shadows for editing","Diffusion separates layers, preserving shadows and reflections"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"For real camera-captured pairs there is no foreground ground truth, so the foreground is supervised only by re-blending to the input; because many different splits re-blend identically, the model must rely on its pretrained prior to decide that shadows and reflections belong to the foreground rather than the background.","fun_headline_variants_meta":{"raw":{"variants":["Diffusion splits images into editable layers with effects","Layer decomposition with visual effects via diffusion","Edit images by breaking into layers with shadows intact","Diffusion model separates foreground with shadows for editing","Diffusion separates layers, preserving shadows and reflections"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000614,"raw_usage":{"total_tokens":2840,"prompt_tokens":916,"completion_tokens":1924,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":532,"completion_tokens_details":{"reasoning_tokens":1855}},"tokens_in":532,"tokens_out":1924,"duration_ms":12657,"temperature":1.0,"reasoning_tokens":1855,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T11:45:07.827862+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the model on a set of real composite/background pairs and check whether the predicted foreground, placed on a neutral gray background, reproduces the shadow or reflection at the correct location and opacity; if the background layer retains any of the shadow or the foreground carries background texture, the consistency loss has not achieved a correct visual-effect split.","supporting_citations":[{"cited_title":"Metashadow: Object-centered shadow detec- tion, removal, and synthesis","cited_arxiv_id":null,"evidence_quote":"Supplies the shadow synthesis method that generates shadow intensity maps for the simulated object assets."},{"cited_title":"Mulan: A multi layer anno- tated dataset for controllable text-to-image generation","cited_arxiv_id":null,"evidence_quote":"Provides a multi-layer RGBA dataset used both as a reference for training data construction and as an evaluation benchmark for decomposition quality."},{"cited_title":"Emu edit: Precise image editing via recognition and gen- eration tasks","cited_arxiv_id":null,"evidence_quote":"Supplies an instruction-driven object removal benchmark and baselines used in the paper's user studies and quantitative comparisons."}],"review_version":1}