{"id":"88045c14-2329-4e00-ba46-00a25563174d","arxiv_id":"2411.17696","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A diffusion model relights indoor images from user scribbles while preserving the room's colors and textures.","lead":"ScribbleLight is an AI model that relights a single indoor photo from scribbles the user draws to brighten or darken parts of the room. It combines an albedo-preserving diffusion model with a ControlNet guided by surface normals, and the paper shows qualitative and quantitative gains over adapted baselines.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The training objective is self-reconstruction (Eq. 3) with scribbles generated from the input's own shading, so the model is never trained on mismatched scribble-image pairs; any relighting at test time is an emergent generalization, and the paper lacks a control test showing the scribbles are…","rationale":"The reader's weakest_assumption precisely identifies self-reconstruction under source-consistent scribbles as the fragile point, and my analysis reaches the same conclusion with additional specificity: the training distribution contains no examples where the scribble disagrees with the input lighting, so the conditional distribution learned at training time is p(x | scribble derived from x), not p(y | scribble specifying a new lighting). The BigTime evaluation, which uses target-derived scribbles, therefore tests an out-of-distribution condition, and the model's apparent relighting is an emergent property of the diffusion prior and ControlNet rather than a direct consequence of the loss. The proposed neutral-scribble control test would settle whether the scribbles are causally responsible for the output changes; if they are, the reader's conditional verdict remains appropriate, and if they are not, the verdict should move toward rejection. Since the test is not yet performed, the reader's CONDITIONAL verdict already reflects the appropriate level of confidence, so no change to the verdict is recommended.","tokens_in":13717,"tokens_out":8523,"duration_ms":78585,"concrete_test":"Run ScribbleLight inference on the 206 BigTime test pairs with the scribble map set to the neutral value 0.5 everywhere, keeping source images, albedo, normals, text prompts, and inference seeds identical to the reported setup. Compute RMSE/PSNR/LPIPS of these neutral-scribble outputs against both the source and target images. If the neutral outputs remain close to the source (e.g., LPIPS to source below 0.05 and clearly worse target metrics than the reported LPIPS of 0.394), then the scribbles are the effective control signal, mitigating the concern. If the neutral outputs are as close to the target as the reported scribble-conditioned outputs, the relighting is not causally driven by the user scribbles, and the central claim of scribble-based control fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires that the model changes lighting according to user scribbles. However, the training loss in Eq. 3 is self-reconstruction: the target is the input image I, and the scribbles M are automatically generated from the same image's shading Smono (Sec. 3.3). Thus the model is never trained on a single example where the scribble specifies a lighting different from the input. The reported BigTime evaluation uses scribbles generated from the target image's shading, which is an out-of-distribution condition relative to training. So the quantitative gains may reflect the model following bright/dark hints in the scribbles (local brightness adjustment) rather than true scene relighting. The paper provides no control experiment that isolates the causal contribution of the scribbles; for example, a neutral-scribble condition (all 0.5) or a mismatched-scribble condition is absent. Without such a control, the possibility remains that the diffusion prior and albedo conditioning alone produce the relighting effect, and the scribbles are not load-bearing for the claimed local fine-grained control.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces ScribbleLight, a diffusion-based method for relighting a single indoor image from sparse user scribbles that indicate brighten/darken regions. The method fine-tunes Stable Diffusion with an albedo condition (with noise injected into the albedo latent to tolerate imperfect IID estimates) and a ControlNet that ingests a latent encoding of scribbles and surface normals, regularized by a decoder that reconstructs shading and normals. Training is performed on unpaired LSUN Bedroom images, with the input image itself as the denoising target; scribbles are automatically generated from the same image's estimated shading. The paper reports quantitative improvements over two adapted baselines on the BigTime dataset and qualitative demonstrations of turning lights on/off, adding highlights, cast shadows, and progressive editing.","tokens_in":13945,"tokens_out":3784,"duration_ms":34841,"significance":"If the central claim is valid, ScribbleLight is a practically useful interactive relighting tool, and the design choices (albedo-conditioned diffusion with noise-injected albedo, and the control encoder–decoder) are interesting technical contributions that could inform follow-up work. The paper provides ablations (Tables 2 and 3) that support the value of albedo conditioning and the normal/decoder components, and it includes qualitative evidence of user-controllable effects. However, the significance of the central claim is currently weakened by the absence of evidence that the scribbles are causally responsible for the observed relighting behavior, as detailed in the major comments.","major_comments":[{"comment":"The training objective for the ControlNet is self-reconstruction: the denoising target in Eq. (3) is the input image I itself, while the scribble condition M is generated from the same image's estimated shading Smono (Sec. 3.3). The model is therefore never trained on a single example where the scribble specifies a lighting different from the input. Consequently, any relighting at test time is an emergent behavior, and the paper provides no control experiment (e.g., a neutral all-0.5 scribble or a scribble mismatched to the input) to show that the scribble content, rather than the diffusion prior and albedo conditioning alone, drives the results. This is load-bearing for the paper's central claim of scribble-based local control; please add such controls and, if possible, training on paired relighting data (e.g., synthetic multi-light renderings) to demonstrate that the model learns a genuine mapping from scribble changes to lighting changes.","section":"§3.3, Eq. (3)"},{"comment":"The auto-generated test scribbles are derived from the target image's shading (Sec. 4.1), so the model is given an oracle hint derived from the ground-truth relit image. Combined with the self-reconstruction training objective, the reported gains in Table 1 may partly reflect the model performing local brightness adjustment in response to bright/dark hints rather than physically consistent relighting. Please report additional quantitative results with scribbles derived only from the source image (no target information) and with neutral/randomized scribbles, and explicitly compare these against the current target-derived scribble results to establish that the scribbles are the causal factor.","section":"§4.1, §4.2, Table 1"},{"comment":"The scribble generation rule states \"M(x) = 1 when I(x) > µ + σ\" and similar for the other thresholds, but the surrounding text says the scribbles are generated from the shading Smono, and I is a color image while µ and σ are described as intensity statistics of the training data. This appears to be a technical typo (presumably Smono(x) should be used), and it must be corrected because the exact preprocessing is necessary for reproducibility.","section":"§3.3"}],"minor_comments":[{"comment":"Please specify the dimensions/channels of the lighting feature map f and the architecture of the control encoder EC more precisely, as these details are relevant to the claim that the latent contains sufficient shading and geometry information.","section":"§3.2"},{"comment":"The phrase \"Lower RMSE and higher PSNR indicate better per-pixel similarity to the reference\" is slightly redundant; it may be clearer to simply state that RMSE is lower-is-better and PSNR is higher-is-better.","section":"§4.1"},{"comment":"The paper reports mean and best performance over 5 seeds but does not report standard deviations or significance tests; adding these would strengthen the quantitative claims.","section":"§4.1, Table 1"},{"comment":"In Table 4, the description of the RGB↔X baseline says the irradiance field is derived from the target image; please clarify whether the same target shading is provided to all methods, since the comparison is meant to be under monochromatic shading input.","section":"Appendix B"}],"recommendation":"major_revision","confidential_remarks":"The central concern is the training/evaluation protocol: the model is trained to reconstruct the input conditioned on its own scribbles, and evaluated with target-derived scribbles, which may reward local brightness adjustment rather than true relighting. I would ask the authors to add control experiments that isolate the causal contribution of the scribbles, and to consider whether the method should be trained or fine-tuned on paired relighting data. The paper is otherwise well written and the technical components are interesting, but the current evidence does not fully support the headline claim of scribble-driven fine-grained lighting control."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing to know: this is the first scribble-driven indoor relighting method, and the qualitative results are genuinely plausible—turning lamps on and off, shifted window light, soft shadows. The technical core is simple but sensible: an albedo-conditioned Stable Diffusion where the albedo latent gets a fixed level of noise to keep color and texture while allowing lighting variation, plus a ControlNet whose latent is regularized by an encoder-decoder that predicts normals and shading. The ablations in Tables 2 and 3 show each piece earns its keep, and the appendix comparison using full shading maps (including IIDiffusion) is a fairer test that ScribbleLight wins cleanly.\n\nThe soft spot is the training objective. In Eq. 3 the target is the input image I, and the scribbles are auto-generated from the same image's shading estimate. So the model never sees a training example where the scribble says \"make this area darker than it is.\" It only learns to map the input's own bright and dark regions back to the same image. Test-time scribbles come from a different target lighting, so the quantitative gains on BigTime could be the model following simple brighten/darken hints rather than doing scene-level relighting. The paper has no control condition—e.g., neutral scribbles (all 0.5) or mismatched scribbles—to show the scribbles are load-bearing. That is a real gap, and it is the main thing a referee should push on.\n\nBaseline adaptation is also uneven: LightIt* gets retrained, RGB↔X does not. The appendix shading-map comparison partly repairs this (IIDiffusion is also used off-the-shelf, and ScribbleLight beats it), but the headline scribble table is still confounded. No code or weights are released, and the metrics report mean and best over five seeds without error bars, which makes the size of the gains hard to judge.\n\nWho should read this: anyone working on interactive image editing or single-image relighting. It is a solid system paper with a clear novelty claim. It deserves peer review, not a desk reject. A careful reviewer can ask for the scribble-causality control, code release, and error bars. I would hesitate to cite it until the code is out and the control experiment settles whether the scribbles do what the paper claims, but the idea itself is worth knowing.","headline":"First scribble-driven indoor relighting with plausible results, but self-reconstruction training leaves the scribbles' causal role unproven without a control.","tokens_in":14471,"tokens_out":3245,"would_cite":false,"duration_ms":29127,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A scribble-based generative model can relight a single indoor photo locally, turning lamps on or off and adding shadows, while preserving the original colors and textures.","keywords":["single image relighting","indoor scenes","scribble-based editing","albedo conditioning","diffusion models","ControlNet","intrinsic image decomposition","local lighting control"],"falsifier":"Take a photo with a visible lamp and draw a scribble that darkens a shadow side opposite the lamp; if the output only darkens the marked pixels and leaves the lamp's highlight and adjacent shading unchanged, the method is following scribbles as a filter, not relighting. A second check is to rerun the BigTime evaluation with scribbles derived from the target image's true shading and compare against brightness-only baselines; if the gap collapses, the reported improvement may be simple hint-following rather than physically grounded relighting.","tokens_in":13496,"feed_emoji":"🖍️","tokens_out":5090,"duration_ms":41339,"temperature":0.7,"pith_summary":"This paper claims that a single indoor photograph can be relit in a local, user-controlled way from scribble strokes alone. A user marks regions to brighten or darken, and the model can turn lamps on or off, add or move cast shadows, introduce highlights, or change window light, while keeping the room's original colors and material textures. The method, called ScribbleLight, is built by conditioning a Stable Diffusion model on the image's estimated albedo and steering the diffusion with an encoder-decoder ControlNet that takes scribbles and surface normals. The authors show quantitative and qualitative results against adapted baselines, and demonstrate iterative refinement where small scribble changes produce proportional lighting changes.","feed_headline":"ScribbleLight: scribbles relight indoor photos, keeping colors intact","feed_subtitle":"Draw brighten or darken strokes in a single photo to switch lamps, add shadows, and shift daylight.","key_machinery":"The load-bearing mechanism is an albedo-conditioned latent diffusion model paired with a scribble-and-normal ControlNet. The albedo image, produced by intrinsic image decomposition, is encoded to a latent and concatenated with the noisy image latent, with a fixed noise level added to the albedo latent during training so the model trusts the image prior and tolerates albedo errors. The ControlNet's encoder compresses the scribble map and normal map into a lighting feature latent, and a decoder is trained to recover shading and normals from that latent, forcing the latent to retain geometric and shading information. A binary scribble encodes brighten versus darken intent, and unmarked regions are left to the model to fill plausibly.","core_discovery":"On the paper's own terms, the central discovery is that scribbles are a sufficient control signal for single-image indoor relighting, provided the generative model is constrained to preserve the image's intrinsic albedo. The paper argues that a naive diffusion-based relighting model follows the scribble but drifts in color and texture, and that the drift is fixed by conditioning the diffusion denoiser on an albedo map with deliberately added noise, which makes the model robust to imperfect albedo estimates. It further argues that guiding the denoiser with a latent code jointly decoded to shading and normals keeps geometry intact and lets sparse scribbles produce physically plausible local effects such as soft highlights and layered shadows.","pith_inferences":["Implicit in the design is a testable extension to colored scribbles, which would let users control the color of the added light; the paper lists this as future work but does not implement it.","If the self-reconstruction training assumption is the main risk, one could stress-test it by evaluating on paired captures of the same room under controlled lighting changes and separating simple brighten-or-darken compliance from physically consistent shading transfer.","The noise-injected albedo conditioning suggests the same architecture could serve as a generic intrinsic-preserving image editor beyond relighting, for example local brightness adjustment that keeps material colors fixed, though the paper does not make that claim."],"forward_implications":["Users can iteratively relight one photograph: turn lights on or off, move window light, add highlights and shadows, and refine with new scribbles.","Because multiple diffusion seeds give multiple valid results from the same scribbles, the method can act as a proposal tool for interior design and virtual staging.","Local lighting control is decoupled from materials: the albedo conditioning keeps wall, floor, and furniture colors stable across edits.","The approach does not need paired images of the same room under different lighting, only single photos with automatically generated scribble labels, so it can train on large real-image collections."],"supporting_citations":[{"why":"Provides the pretrained Stable Diffusion v2 generator that ScribbleLight fine-tunes into an albedo-conditioned denoiser.","marker":"[55]"},{"why":"Supplies the ControlNet mechanism used to steer the denoiser with scribble and normal latents.","marker":"[76]"},{"why":"Computes the albedo maps used as the conditioning signal and is the source of albedo error that the noise injection is designed to tolerate.","marker":"[8]"},{"why":"Produces the monochromatic shading maps from which training scribbles are generated and which the control decoder learns to predict.","marker":"[7]"},{"why":"Estimates surface normals that guide geometry-preserving lighting effects.","marker":"[1]"},{"why":"Is the LSUN Bedrooms training set of unpaired real indoor images that the model learns relighting from.","marker":"[70]"},{"why":"Provides the BigTime time-lapse pairs used to evaluate relighting against reference images.","marker":"[38]"},{"why":"Is the LightIt baseline, retrained as LightIt*, that ScribbleLight must beat in albedo preservation and local control.","marker":"[32]"},{"why":"Is the RGB-X baseline that recomposes intrinsic components and is compared against on the same metrics.","marker":"[74]"}],"fun_headline_variants":["Turn lamps on with a scribble: single-image relighting","Scribble your light: indoor photos recover true colors","One stroke changes lighting, keeps texture intact","Scribble-driven relighting: geometry and albedo preserved","Draw highlights and shadows in any indoor photo"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The riskiest premise is that training on unpaired photos, where the target is the input image itself and the scribbles are generated from that same image's shading, teaches genuine relighting behavior rather than local brightening or darkening in response to the scribbles.","fun_headline_variants_meta":{"raw":{"variants":["Turn lamps on with a scribble: single-image relighting","Scribble your light: indoor photos recover true colors","One stroke changes lighting, keeps texture intact","Scribble-driven relighting: geometry and albedo preserved","Draw highlights and shadows in any indoor photo"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000513,"raw_usage":{"total_tokens":2457,"prompt_tokens":871,"completion_tokens":1586,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":487,"completion_tokens_details":{"reasoning_tokens":1509}},"tokens_in":487,"tokens_out":1586,"duration_ms":10098,"temperature":1.0,"reasoning_tokens":1509,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T11:49:27.714174+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a photo with a visible lamp and draw a scribble that darkens a shadow side opposite the lamp; if the output only darkens the marked pixels and leaves the lamp's highlight and adjacent shading unchanged, the method is following scribbles as a filter, not relighting. A second check is to rerun the BigTime evaluation with scribbles derived from the target image's true shading and compare against brightness-only baselines; if the gap collapses, the reported improvement may be simple hint-following rather than physically grounded relighting.","supporting_citations":[{"cited_title":"High-resolution image synthesis with latent diffusion models","cited_arxiv_id":null,"evidence_quote":"Provides the pretrained Stable Diffusion v2 generator that ScribbleLight fine-tunes into an albedo-conditioned denoiser."},{"cited_title":"Colorful diffuse intrinsic image decomposition in the wild","cited_arxiv_id":null,"evidence_quote":"Computes the albedo maps used as the conditioning signal and is the source of albedo error that the noise injection is designed to tolerate."},{"cited_title":"Intrinsic image decomposi- tion via ordinal shading","cited_arxiv_id":null,"evidence_quote":"Produces the monochromatic shading maps from which training scribbles are generated and which the control decoder learns to predict."},{"cited_title":"Learning intrinsic image de- composition from watching the world","cited_arxiv_id":null,"evidence_quote":"Provides the BigTime time-lapse pairs used to evaluate relighting against reference images."},{"cited_title":"Lightit: Illumination modeling and control for diffusion models, 2024","cited_arxiv_id":null,"evidence_quote":"Is the LightIt baseline, retrained as LightIt*, that ScribbleLight must beat in albedo preservation and local control."},{"cited_title":"Rgb ↔x: Image decomposition and synthesis using material- and lighting-aware diffusion models","cited_arxiv_id":null,"evidence_quote":"Is the RGB-X baseline that recomposes intrinsic components and is compared against on the same metrics."}],"review_version":1}