{"id":"214f5a4c-5701-4b53-be12-c0561392ffb2","arxiv_id":"2412.11972","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A single-step rectified-flow diffusion model trained on synthetic Blender renders generates controllable shadows for object images, with a new public benchmark.","lead":"This paper trains a diffusion model on synthetic 3D-rendered images to generate controllable shadows for 2D object photos, letting users adjust shadow direction, softness, and intensity in a single fast inference step. It could let product photographers and compositors add realistic, editable shadows to pictures in real time without building a 3D scene.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Real-image generalization claim is untested for non-eye-level camera viewpoints; training fixes the camera on the y-axis, so arbitrary real-world camera angles may break shadow geometry.","rationale":"The paper's technical contribution—single-step rectified-flow shadow generation with scalar light conditioning—is well supported on synthetic data: Table 2 and Fig. 8 show rectified flow achieves IoU around 0.73–0.77 in one step across all tracks, and Fig. 10 shows the scalar conditioning is on par with the blob representation. No internal inconsistency appears in the rectified flow objective or the conditioning mechanism. However, the central claim of generalization to real images is the weakest point. The reader's weakest assumption correctly identifies the synthetic-to-real distribution shift, but our review sharpens it to a specific omitted factor: the training camera is fixed at eye level (negative y-axis, only translated along y), so the model has never seen varied camera pitches. Real product photography often uses non-zero pitches, where the projection of a ground-plane shadow relative to the object silhouette differs qualitatively from eye-level views. The paper's real-image examples are consistent with the training viewpoint, so they do not demonstrate viewpoint robustness. Our proposed test—using a 3D scan of a real object to render ground-truth shadows for multiple camera pitches and comparing IoU—would settle whether this specific distribution shift actually breaks the model. This reinforces the reader's conditional verdict rather than changing it; the paper needs quantitative real-image evaluation, with camera-pitch variation as a concrete test case, before the generalization claim can be accepted. The lack of comparison with prior shadow-generation methods remains a secondary gap but does not affect our assessment of the internal validity.","tokens_in":14910,"tokens_out":11180,"duration_ms":105215,"concrete_test":"Take a set of real product photos of the same physical object captured from several camera pitches (e.g., 0°, 20°, 40° from horizontal) with a fixed azimuth. Create a 3D scan of the object and render, with the paper's Blender pipeline and a fixed area light, a ground-truth shadow map for each photo's viewpoint and the same light azimuth. Run the trained model on each real cutout with the corresponding light parameters and compute IoU between predicted and ground-truth shadow maps as a function of camera pitch. If IoU degrades significantly (e.g., drops by more than 0.2) for pitches beyond 0°, the real-image generalization claim is not valid for general camera angles.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim that the model 'generalizes well to real-world images' is supported only by qualitative results in Sec. 4.4, and the synthetic training pipeline in Sec. 3.1 restricts the camera to a fixed pitch: the camera is placed at the negative y-axis and moved only along that axis, producing eye-level views of objects on a ground plane. All synthetic evaluation (Sec. 4.2, Table 2) uses this same camera setup. For real-world images, especially product photos taken from a high or low angle, the geometric relationship between a 2D object cutout and a cast shadow on the ground plane is substantially different from the eye-level case the model was trained on. The model has never seen such viewpoints, so the predicted shadow map may be geometrically inconsistent (wrong shadow length, direction, or anchor point) despite looking plausible in the displayed eye-level examples. This is a specific, load-bearing distribution-shift failure mode that the paper does not address, and it undermines the abstract's strong generalization claim unless the method is shown to be robust to camera pitch variation.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a controllable shadow generation pipeline for 2D object cutouts. It builds a large synthetic dataset of 257,612 renders from 9,872 artist-created meshes using Blender Cycles with a square area light, rendering for each light-parameter triple (θ, φ, s) a color image, a binary mask, and a gray-scale shadow map. The authors train an SDXL-based latent diffusion model with a rectified flow objective, conditioning on the object image and mask via latent concatenation and on the light parameters via sinusoidal embeddings added to the timestep embedding. They conduct ablations over prediction types (ε, v, sample, rectified flow), number of sampling steps, training iterations, and conditioning mechanisms (scalar vs. blob map), reporting IoU, RMSE, s-RMSE, and ZNCC on three test tracks with held-out meshes and 10 seeds. The main quantitative finding is that rectified flow with one sampling step outperforms the other prediction types with 20 steps. Qualitative results on real images demonstrate softness, direction, and intensity control, and the authors release the synthetic benchmark.","tokens_in":15109,"tokens_out":6881,"duration_ms":58437,"significance":"The quantitative core of the paper is solid: the multi-seed, multi-metric ablations with held-out meshes provide credible evidence that the rectified flow objective is particularly well-suited to single-step shadow prediction in this setting. The release of a public benchmark with three control tracks is a useful community resource. If the real-image generalization holds in broader conditions, the method could enable practical real-time shadow control for image compositing. However, the paper's second central claim—generalization to real-world images—is currently supported only by a qualitative gallery from a restricted camera viewpoint, and the experiments do not compare against prior shadow generation methods. These gaps limit the current evidence for the full set of claims.","major_comments":[{"comment":"The claim that the model \"generalizes well to real-world images\" is load-bearing but is supported only by qualitative examples. The training pipeline (Sec. 3.1) fixes the camera at the negative y-axis and moves it only along that axis, yielding eye-level views; the synthetic test tracks (Table 1) use the same camera setup. Real-world photographs frequently use non-eye-level camera angles, and the mapping from a 2D object cutout to a ground-plane shadow changes with camera pitch. The qualitative results in Table 3 and SM Sec. 8 are limited to apparently eye-level views, so the model's behavior under pitch variation is untested. I request either (a) an experiment that varies camera pitch on synthetic scenes (e.g., rendering the same object and light settings from multiple camera elevations and measuring shadow-map accuracy), (b) a quantitative real-image evaluation with known ground-truth shadows or a user study, or (c) a tempered claim that restricts generalization to the eye-level setting.","section":"Sec. 4.4 and Abstract"},{"comment":"The paper does not compare the proposed method against any existing shadow generation approach. Related work discusses SSN [40], Pixel Height Maps [41], PixHt-Lab [42], ObjectDrop [51], and Desobav2 [23], but none of these are evaluated on the released benchmark or on an existing dataset. Without such a comparison, the contribution's practical benefit relative to the state of the art is not established. At minimum, I expect a comparison with SSN and one recent diffusion-based method (e.g., ObjectDrop or Desobav2) on the released test tracks, or an explanation of why a comparison is infeasible (e.g., conditioning differences). This is necessary to support the claim that the method produces \"high-quality results\" in the context of existing work.","section":"Sec. 4, Experiments"},{"comment":"The intensity conditioning ablation concludes that conditioning on S(θ, φ, s, I) is equivalent to post-hoc scaling by a scalar I, but the comparison is presented only as pairs of images. Because the paper uses quantitative metrics for other ablations, the same metrics (IoU, RMSE, ZNCC) should be reported for the two intensity-control variants; without them, the conclusion of equivalence is not established. This matters because the abstract and introduction list intensity control as one of the controllable shadow properties.","section":"Sec. 4.3, Fig. 11"}],"minor_comments":[{"comment":"The sampling procedure for light parameters is under-specified. The intervals for θ, ϕ, and s imply up to 46 × 361 × 7 = 116,242 possible combinations, but 257,612 training images were rendered; clarify whether combinations are randomly sampled per iteration, and how the counts in Table 1 were derived.","section":"Sec. 3.1, Table 1"},{"comment":"The sinusoidal embedding formula for ω_i^d is non-standard; please double-check the indexing, since the term i·(i−1) suggests a quadratic schedule while the standard formulation uses 10000^{-2i/d}. If the formula is intentional, clarify; if it is a typo, correct it.","section":"Eq. (7)"},{"comment":"The phrase \"replicating it twice and concatenating them with the original\" is ambiguous; it should state that the gray-scale channel is duplicated to form a 3-channel RGB input for the VAE.","section":"Sec. 3.2.3"},{"comment":"The x-axis for training iterations starts at 0 and then jumps to 10k; consider labeling all ticks consistently (0, 10k, 30k, ...) or removing the 0 tick to avoid implying a linear scale.","section":"Fig. 8 and Figs. 16-17"},{"comment":"The statement \"We use 50 models to create a test set and consider the remaining 9,872 models for the trainings\" is consistent with the total 9,922, but the test tracks use overlapping subsets (50, 15, 15 models); please state explicitly whether the 15-model tracks are disjoint from the 50-model track or drawn from the same held-out set.","section":"Sec. 4.1"},{"comment":"Please specify how the real foreground images were cut out (manual or automatic matting) and how the model inputs (object image and mask) were prepared for the real-image tests, since small cutout errors can affect shadow quality.","section":"Sec. 4.4"}],"recommendation":"major_revision","confidential_remarks":"The paper's central quantitative claim—that rectified flow enables single-step shadow prediction—is well supported by the ablations. The main reservations are the strength of the real-world generalization claim relative to the fixed-camera training setup, and the absence of any comparison with prior shadow generation methods. These are fixable within the manuscript's scope by adding experiments or tempering claims. The self-citation to the authors' Flash Diffusion paper is not problematic for the central result. The benchmark release is a valuable asset for the community."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing to know: this is a genuinely new task formulation—diffusion-based controllable shadow generation with explicit light parameters—and the central technical claim holds up. The paper trains a latent diffusion model (SDXL backbone) on a large Blender synthetic dataset with a rigid-body pre-processing step to ground the meshes, conditions on object image, mask, and spherical light coordinates (θ, φ) plus area-light size s, and shows that a rectified flow objective produces high-quality shadow maps in a single sampling step. The ablations are careful: four prediction types, multiple sampling steps and training iterations, ten seeds, four metrics, and held-out meshes. The result that rectified flow at one step beats ε, v, and sample prediction at twenty steps is credible and clearly reported. The scalar conditioning via sinusoidal embedding is simple and works about as well as a blob-based light representation. The three-track public benchmark is a useful resource.\n\nThe soft spots are in the generalization claim, not the core method. The paper does not compare quantitatively against any prior shadow generation method (SSN, PixHt-Lab, DesoBA, etc.), which makes the positioning in the literature incomplete. More importantly, the real-image evidence is purely qualitative, and the training pipeline fixes the camera at eye-level on the negative y-axis. The stress-test concern is valid: for real photos taken from high or low camera angles, the geometric relationship between the 2D cutout and its cast shadow is outside the training distribution, and the paper does not test or even acknowledge this. So the abstract's sentence that the model 'generalizes well to real-world images' is too strong; 'shows plausible qualitative generalization for eye-level product photos' would be accurate. This is a fixable weakness: add a quantitative real-image evaluation (even a small user study) or a camera-pitch robustness experiment, and soften the claim if it doesn't hold.\n\nMinor: the synthetic test set comes from the same rendering pipeline as training, so the numeric results are in-distribution; the sphere radius r is a fixed free parameter not explored. These are minor.\n\nWho gets value: people working on compositing, shadow synthesis, and synthetic-data pipelines. It is a practical paper, not a conceptual reorganisation, but the benchmark and the single-step result deserve referee time. I would send it to peer review and ask for the real-image evaluation and the literature comparison before acceptance. If I were working in this area, I would cite it for the task formulation and the rectified-flow finding.","headline":"A genuinely new task formulation and a well-ablated single-step rectified-flow result, but the real-image generalization claim overreaches because the training camera is fixed at eye level.","tokens_in":15604,"tokens_out":2707,"would_cite":true,"duration_ms":24552,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A diffusion model trained only on synthetic Blender renders generates realistic, controllable shadows for cut-out object photos in one sampling step.","keywords":["shadow generation","diffusion models","rectified flow","single-step inference","synthetic data","controllable image synthesis","latent diffusion","image compositing"],"falsifier":"Render or photograph a real object with known geometry and a known light position, run the model on the object cutout with matching ($\\theta$, phi, s), and compare the predicted shadow map against a ray-traced ground truth; if soft IoU under matched real conditions falls far below the synthetic-test range (roughly 0.73-0.77), the synthetic-to-real generalization claim is not supported.","tokens_in":1474,"feed_emoji":"💡","tokens_out":2106,"duration_ms":58489,"temperature":0.7,"pith_summary":"The paper claims that a diffusion model can produce a realistic shadow map for a cut-out object image in a single sampling step, with direct control over the shadow's direction, softness, and intensity. To get there, the authors train the model exclusively on 257,612 synthetic renders of 9,872 3D meshes lit by a single square area light whose position and size are given as spherical coordinates. The central empirical claim is that a rectified-flow training objective makes one-step inference competitive with, and at one step better than, twenty-step epsilon-, v-, and sample-prediction diffusion training. If correct, this makes controllable shadow generation fast enough for real-time compositing and removes the need for 3D scene reconstruction at inference time.","feed_headline":"Single-step diffusion model renders controllable shadows","feed_subtitle":"Trained on synthetic renders only, it predicts shadow direction, softness, and intensity in one step.","key_machinery":"The load-bearing object is the rectified-flow objective, $\\min_\\psi \\mathbb{E}\\left[\\|(x_1-x_0)-f_\\psi(x_t,t)\\|^2\\right]$ with $x_t = x_1 t + (1-t)x_0$, which trains the denoiser to follow straight-line paths from noise to data and thereby makes a single Euler step sufficient at inference. It is paired with a conditioning mechanism that encodes the light-source parameters (polar angle $\\theta$, azimuth $\\phi$, area-light size $s$) as sinusoidal embeddings added to the timestep embedding, so that control signals enter the network without extra latent channels or external light maps.","core_discovery":"The paper's central claim is that rectified flow, a diffusion training objective that forces the network to follow a straight interpolation path between noise and data, allows a latent diffusion model to predict high-quality shadow maps in a single sampling step. The model, built on an SDXL backbone with cross-attention removed, is conditioned on the object cutout, its binary mask, and scalar light parameters ($\\theta$, $\\phi$, $s$) encoded with sinusoidal embeddings injected into the timestep embedding; intensity is handled by scalar multiplication of the predicted map. On the authors' synthetic benchmark, the rectified-flow model reaches soft IoU around 0.73-0.77 in one step across three test tracks (softness, horizontal direction, vertical direction), outperforming $\\varepsilon$-, $v$-, and sample-prediction models even at twenty steps. The authors further claim that the model, trained purely on synthetic Blender renders, generalizes to real photographs, and they release the synthetic dataset and a three-track public benchmark for future evaluation.","pith_inferences":["The scalar-conditioning trick likely transfers to other physics parameters in image-conditioned diffusion (light intensity falloff, camera tilt, object scale), since it injects continuous control without architectural changes.","The near-deterministic nature of the image-plus-light to shadow-map mapping is probably why a single flow step suffices; tasks with more output variation (e.g., full relighting with reflections) may need more steps or distillation.","A quantitative real-world test is the natural next experiment: re-render scanned real objects under matched light parameters, or build a small real photo set with ray-traced reference shadows, to measure the synthetic-to-real gap.","Training data restricted to a single area light in a void means the model has never seen environment lighting or multiple lights; extending the dataset to environment maps would test whether controllability generalizes to everyday lighting."],"forward_implications":["One-step inference puts controllable shadow generation in real-time range for compositing, product photography, and video pipelines.","Because conditioning enters through scalar embeddings, shadow direction, softness, and intensity can be swept continuously without retraining or auxiliary control networks.","The approach is background-free: the same predicted shadow map can be blended into any target background.","Rectified flow, not only for generative synthesis, emerges as a strong default for dense-prediction diffusion tasks where output diversity is low.","The released three-track benchmark gives the community quantitative targets (IoU, RMSE, S-RMSE, ZNCC) for future shadow-generation methods."],"supporting_citations":[{"why":"Supplies the rectified-flow objective that makes single-step sampling work.","marker":"[26]"},{"why":"Provides the flow-matching framework (vector field ODE) that rectified flow instantiates.","marker":"[21]"},{"why":"The SDXL architecture used as backbone; source of the sinusoidal timestep embedding repurposed for light parameters.","marker":"[34]"},{"why":"The latent diffusion/VAE machinery that compresses RGB images and masks into the latent space where denoising happens.","marker":"[35]"},{"why":"The diffusion formulation, sinusoidal embeddings, and epsilon-prediction baseline the paper contrasts with rectified flow.","marker":"[10]"},{"why":"The SSN blob-light conditioning baseline used as the comparison for the proposed scalar conditioning.","marker":"[40]"}],"fun_headline_variants":["Single-step diffusion casts controllable shadows","Rectified flow enables one-step shadow control","Synthetic-only training yields real-world shadows","One-pass shadow maps from a diffusion model","Controllable shadows in a single diffusion step"],"cache_read_input_tokens":17920,"weakest_assumption_plain":"The model is trained and quantitatively evaluated only on synthetic renders from the same Blender pipeline, so the claim that it generalizes to real-world photos rests on the assumption that those renders capture how real object photos cast shadows; the real-image evidence is qualitative only.","fun_headline_variants_meta":{"raw":{"variants":["Single-step diffusion casts controllable shadows","Rectified flow enables one-step shadow control","Synthetic-only training yields real-world shadows","One-pass shadow maps from a diffusion model","Controllable shadows in a single diffusion step"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000289,"raw_usage":{"total_tokens":1683,"prompt_tokens":922,"completion_tokens":761,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":538,"completion_tokens_details":{"reasoning_tokens":710}},"tokens_in":538,"tokens_out":761,"duration_ms":6942,"temperature":1.0,"reasoning_tokens":710,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T14:24:49.857921+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Render or photograph a real object with known geometry and a known light position, run the model on the object cutout with matching ($\\theta$, phi, s), and compare the predicted shadow map against a ray-traced ground truth; if soft IoU under matched real conditions falls far below the synthetic-test range (roughly 0.73-0.77), the synthetic-to-real generalization claim is not supported.","supporting_citations":[{"cited_title":"Flow straight and fast: Learning to generate and transfer data with rectified flow","cited_arxiv_id":null,"evidence_quote":"Supplies the rectified-flow objective that makes single-step sampling work."},{"cited_title":"Flow matching for generative modeling","cited_arxiv_id":null,"evidence_quote":"Provides the flow-matching framework (vector field ODE) that rectified flow instantiates."},{"cited_title":"Sdxl: Improving latent diffusion models for high-resolution image synthesis, 2023","cited_arxiv_id":null,"evidence_quote":"The SDXL architecture used as backbone; source of the sinusoidal timestep embedding repurposed for light parameters."},{"cited_title":"High- resolution image synthesis with latent diffusion mod- els","cited_arxiv_id":null,"evidence_quote":"The latent diffusion/VAE machinery that compresses RGB images and masks into the latent space where denoising happens."},{"cited_title":"Denoising diffusion probabilistic models","cited_arxiv_id":null,"evidence_quote":"The diffusion formulation, sinusoidal embeddings, and epsilon-prediction baseline the paper contrasts with rectified flow."},{"cited_title":"Ssn: Soft shadow network for image compositing","cited_arxiv_id":null,"evidence_quote":"The SSN blob-light conditioning baseline used as the comparison for the proposed scalar conditioning."}],"review_version":1}