{"id":"d556bedb-ffb0-445d-a290-c838a085840e","arxiv_id":"2412.00177","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"LumiNet transfers lighting between indoor scenes from images alone by conditioning a diffusion model on latent intrinsics from the source and a lighting code from the target.","lead":"LumiNet is a diffusion-based system that relights an indoor scene from a source image using the lighting of a target image, keeping geometry and albedo intact. It combines latent intrinsic maps with a ControlNet-style architecture and synthetic training data from a StyleGAN relighter.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Cross-scene relighting rests on the frozen encoder's illumination invariance and lighting-code transferability, yet neither is measured; the central claim is plausible but unverified.","rationale":"I agree with the reader's weakest-assumption analysis. The architecture's only mechanism for separating 'what the scene is' from 'how it is lit' is the frozen fλ encoder, and the paper presents no direct evidence for that separation. This is not a claim of fraud or error: the qualitative results are suggestive, the same-scene MIIW numbers are competitive, and the authors deserve credit for reporting an evaluation-code bug and explicit limitations. But the abstract's strongest sentence is about cross-scene, image-only transfer, and that is exactly the regime in which an untested latent code could silently fail. A model trained on same-scene pairs can appear to relight in-the-wild images by learning to copy target color/contrast onto the source image while preserving source edges; the low-dimensional MLP injection makes such a shortcut especially easy. The proposed test exploits MIIW's controlled 25-condition captures to measure whether Ao and IL_t have the semantics the paper assigns them. If the test passes, the central claim is materially strengthened; if it fails, the paper's explanation for its own results is wrong and the cross-scene generalization claim would need to be substantially rewritten. I therefore keep the reader's CONDITIONAL verdict: the technical idea is viable and the experiments are promising, but the load-bearing assumption must be verified before acceptance. The absence of released artifacts is a separate reproducibility concern that also supports the conditional rating.","tokens_in":15623,"tokens_out":13062,"duration_ms":121743,"concrete_test":"Using the same fλ invocation LumiNet uses at inference, run fλ on the MIIW test set, which has 25 lighting conditions per scene. For each scene s and condition c, record the intrinsic map A_{s,c} and the lighting code ℓ_{s,c}. (1) Illumination invariance of Ao: compute the mean LPIPS/L2 distance between A_{s,c} and A_{s,c'} for the same scene under different lights, and compare with the mean distance between A_{s,c} and A_{s',c} for different scenes under the same condition label; if within-scene distance is not substantially smaller, Ao is contaminated by lighting. (2) Transferability of IL_t: compute a silhouette score on ℓ_{s,c} using scene labels versus condition labels (or train a linear classifier to predict each); if ℓ clusters by scene rather than by lighting condition, IL_t is scene-specific and Eq. (3)'s cross-scene conditioning is unjustified.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—generalizable cross-scene relighting—requires the frozen latent-intrinsic encoder fλ (Sec. 5.1) to have two properties that the paper never measures: (i) the source intrinsic map Ao is invariant to the source image's illumination, so it carries geometry and albedo only; and (ii) the 16-D target lighting code IL_t describes transferable lighting, independent of the target scene's layout and materials. Because LumiNet is trained only on same-scene pairs (Eq. 3 conditions on Ao and IL_t extracted from the same scene), the architecture could learn a same-scene shortcut: residual lighting in Ao, or a scene-specific meaning of IL_t. If Ao retains shadows or highlights, the diffusion model receives conflicting source and target lighting signals. If IL_t encodes luminaire positions or camera viewpoint rather than an illuminant, then applying it to a different scene in Fig. 1 is ill-posed, and the impressive lamp-switching behavior could come from the source image plus a global color/brightness code. No quantitative cross-scene evaluation exists: Table 1 reports same-scene MIIW metrics, and Table 2's in-the-wild results have no lighting ground truth. The paper's acknowledged limitations (small/occluded lamps, no intensity control, chromatic failures) reinforce that the lighting code's portability is the load-bearing, untested assumption.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"LumiNet addresses single-image indoor relighting by combining a pretrained latent-intrinsic encoder with a ControlNet-style latent diffusion model. Given a source image and a target lighting image, it extracts a latent intrinsic map A_o and a 16-dimensional lighting code I_L, concatenates the spatially expanded lighting code with A_o as ControlNet conditioning, and additionally injects an MLP-transformed lighting embedding into cross-attention. Training uses same-scene relit pairs from MIIW and BigTime plus synthetic pairs generated by a variational StyleGAN-based pipeline; a bypass decoder and seed-selection/rectified-flow post-processing are used at inference. The paper reports MIIW RMSE/SSIM, an in-the-wild surface-normal proxy and a user study, and ablations. The central claim is that, despite same-scene-only training, LumiNet generalizes to cross-scene lighting transfer in the wild, preserving geometry and albedo while synthesizing shadows, specularities, and interreflections.","tokens_in":15899,"tokens_out":8500,"duration_ms":71801,"significance":"Relighting indoor scenes from a single image with no 3D or multi-view input is an important problem, and LumiNet's architecture is a sensible combination of existing components: latent intrinsics provide a compact conditioning signal, and the diffusion prior supplies plausible light transport. The paper is transparent in places: it evaluates on an external MIIW test split, uses the official evaluation code, acknowledges the sensitivity of pixel metrics, and explicitly states in §6.1 that it does not surpass the MIIW-specialist methods Latent-Intrinsic and SA-AE on the benchmark. The qualitative examples, especially lamp switching and second-order effects such as reflections, are striking. However, the evidence for the headline claims is currently incomplete: the abstract overstates Table 1, the cross-scene generalization rests on an unvalidated invariance/transferability assumption about the frozen encoder, and the in-the-wild quantitative metrics rely on a proxy normal estimator and an unreported small user study. These are fixable with additional experiments and more careful claims.","major_comments":[{"comment":"The abstract and introduction claim LumiNet 'outperforms existing approaches,' but §6.1 and Table 1 show the opposite on the main benchmark: on the MIIW test set, Ours has RMSE 0.310 and SSIM 0.440 (color-corrected 0.240 and 0.527), while Latent-Intrinsic [75] achieves RMSE 0.297 and SSIM 0.473 (color-corrected 0.222 and 0.571), and SA-AE [25] is also better in most rows. The only MIIW methods LumiNet clearly beats are RGB↔X variants. The central quantitative claim should be revised to 'competitive with specialist methods and better than generalist diffusion baselines,' or supported by new cross-scene quantitative evidence.","section":"Abstract, §6.1, Table 1"},{"comment":"The method's cross-scene generalization rests on two properties of the frozen latent-intrinsic encoder f_λ that are never measured: (i) the source intrinsic map A_o is illumination-invariant, and (ii) the 16-dimensional lighting code I_L^t is transferable across scenes with different layouts and materials. Because training uses same-scene pairs only (Eq. (3) conditions on A_o and I_L^t extracted from the same scene), the model could learn a same-scene shortcut in which A_o retains residual lighting or I_L^t encodes scene-specific information such as luminaire positions or viewpoint. The paper should validate these properties directly, for example by measuring A_o stability across the 25 illuminations of the same MIIW scene, by testing whether I_L^t codes from one scene produce consistent relighting when applied to another scene's A_o, and by training or evaluating on explicitly cross-scene held-out pairs.","section":"§5.1, §5.3, Eq. (3)"},{"comment":"The surface-normal consistency evaluation uses RGB↔X to estimate normals for both the original and relit images and treats the original-image normals as ground truth. This is not ground-truth geometry, and since RGB↔X is itself one of the baselines in the table, the metric conflates relighting quality with one estimator's behavior on the relit images; it could favor outputs whose appearance leads that estimator to return normals similar to the original. At minimum, use an independent normal or depth estimator or multi-view geometric data if available, and report the full error distribution, not just the median, with confidence intervals.","section":"§6.2, Table 2"},{"comment":"The user study (31 participants) is the main evidence for open-world superiority, but the paper reports only mean rank scores with no number of test scenes, no variance, and no significance tests. Since I-PQ, L-PQ, and P-PQ are ordinal rankings, the authors should report per-scene distributions, inter-rater agreement, and pairwise Wilcoxon signed-rank tests (or equivalent) to show that the reported advantages, such as L-PQ 1.30 versus 2.52 for the nearest baseline, are not noise.","section":"§6.2, §6.3, Table 2"},{"comment":"There is no quantitative cross-scene evaluation anywhere in the paper: Table 1 is same-scene MIIW relighting with ground truth, and Table 2's in-the-wild metrics have no lighting ground truth. The paper should add a controlled cross-scene experiment, for example by taking a source image from one MIIW scene and a target lighting condition from another MIIW scene and reporting perceptual metrics or a forced-choice user study against same-scene-trained baselines. Without this, the generalizable-relighting claim is supported only by qualitative examples.","section":"§5.2, §5.3, §6.2"}],"minor_comments":[{"comment":"Report the number of lighting directions per scene, the CLIP similarity threshold, and the number of scenes kept after filtering; currently 'seven lighting variations' and '≈1K' are the only details, and no sensitivity analysis is given for these choices.","section":"§4.1"},{"comment":"The symbol ε is used both for the target noise and for the encoder in expressions such as ε(S_{L_t}); rename the encoder, for example E, and define all conditioning tensors explicitly.","section":"§5.3, Eq. (3)"},{"comment":"The row 'Ours (previous)' is not defined in the text; clarify what differs from 'Ours,' or remove the row if it refers to an earlier version of the method.","section":"Table 1"},{"comment":"The user study is described in both §6.2 and §6.3 with overlapping content; consolidate the descriptions and specify whether the 31-participant study is the same one reported in Table 2.","section":"§6.2, §6.3"},{"comment":"The notation S_o^{L_o}, S_t^{L_t}, A_o, and I_L should be defined in the caption, and the 'Loss' arrow and the VAE-omission note are difficult to read at the current resolution.","section":"Figure 2"}],"recommendation":"major_revision","confidential_remarks":"The transparency about the evaluation-code bug and the explicit admission in §6.1 that the method does not surpass MIIW-specialist baselines are commendable, but the abstract and contributions still overclaim. The missing validation of the frozen latent-intrinsic encoder's invariance and the proxy nature of the in-the-wild metrics are the main technical risks. If the authors add the cross-scene validation experiments and tighten the claims, the paper could become acceptable; without those changes, the central generalization claim remains unproven."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: solid engineering, genuinely new architecture, some very nice qualitative demos, but the abstract oversells the results and the core cross-scene claim rests on an assumption the paper never tests. Worth refereeing, but the authors need to fix the claim-evidence gap.\n\nWhat's new: the combination of latent-intrinsic conditioning (from Zhang et al.'s latent intrinsics) with a ControlNet-style diffusion model, plus the variational StyLitGAN to synthesize relit training pairs from real images. That is a real departure from prior work, which either relit within the same scene or used pixel-space conditions. The bypass decoder, nearest-neighbor seed selection, and rectified-flow cleanup are sensible engineering choices, and the ablations show each contributes.\n\nWhat it does well: the qualitative results are the best evidence. Specular highlights, cast shadows, inter-reflections, lamps switching on and off across scenes with different layouts—the figures are convincing. The user study (31 participants) strongly prefers LumiNet over baselines, and the surface-normal consistency check (median angular error 2.74°) is a reasonable proxy for geometry preservation. The paper is also honest about failure cases and even credits external testers for finding a bug in the evaluation code; that sort of transparency is welcome.\n\nSoft spots, in descending order of size. First, Table 1 does not support the abstract's 'outperforming existing approaches.' On MIIW, LumiNet trails Latent-Intrinsic and SA-AE, though it beats RGB↔X. The text admits this, but the abstract doesn't. That is a straightforward claim-evidence mismatch.\n\nSecond, the central cross-scene generalization claim rests on the assumption that the frozen latent-intrinsic encoder yields illumination-invariant features and transferable lighting codes. The paper never measures either, and since training uses only same-scene pairs, a shortcut is plausible: the intrinsic map could retain residual lighting, or the 16-D lighting code could encode scene-specific layout. Table 2 evaluates the relit images via user study and normal consistency, not whether the lighting actually matches the target. I would want to see a direct test—for example, applying one target lighting code to multiple source scenes and verifying the illumination is consistently reproduced—or a cross-scene MIIW experiment.\n\nSmaller issues: the user study has no error bars or significance tests, the normal ground truth comes from a competitor's estimator, and no code is released. Individually minor, but they compound.\n\nWho's it for: anyone working on image-based relighting, diffusion-based scene editing, or latent intrinsic models. The citation pattern is fair—they build directly on [75] and credit it properly. I'd send it to peer review, expecting the authors to tone down the abstract and add at least an indirect validation of the lighting code's portability.","headline":"LumiNet is a promising engineering contribution whose qualitative results are compelling, but the abstract overstates what the MIIW table shows and the cross-scene guarantee is an untested assumption; referee it, but expect revisions.","tokens_in":16411,"tokens_out":3762,"would_cite":true,"duration_ms":33780,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"LumiNet claims that a diffusion model conditioned on latent intrinsic and extrinsic codes can relight an indoor scene from a target image alone, preserving geometry and albedo while transferring shadows, specular highlights, and indirect…","keywords":["indoor scene relighting","latent intrinsics","diffusion models","ControlNet","lighting transfer","cross-attention","StyleGAN","image-based relighting"],"falsifier":"Render a single synthetic indoor scene twice under two known lighting conditions with a physically based renderer, extract the latent intrinsic features from both renders, and run LumiNet to relight the first render with the second render's lighting. If the extracted feature maps differ noticeably between the two lighting conditions, or if the relit output deviates substantially from the ground-truth second render, then the illumination-invariance premise that carries the method is false.","tokens_in":15427,"feed_emoji":"💡","tokens_out":8268,"duration_ms":62225,"temperature":0.7,"pith_summary":"LumiNet sets out to prove that indoor scene relighting can be done from two ordinary images alone: a source image whose geometry and materials are kept, and a target image whose lighting is imported. The paper argues that by conditioning a diffusion model on latent intrinsic features—an illumination-invariant scene map plus a 16-dimensional lighting code extracted from the target—the model can reproduce complex light behavior such as cast shadows, specular highlights, and inter-reflections, even across scenes with different layouts. It also claims that a variational StyleGAN-based data-generation pipeline supplies enough paired relighting examples to train this cross-scene transfer, and that the resulting method beats existing image-only relighters on indoor benchmarks and in-the-wild images. If true, LumiNet would make relighting a practical image-editing operation rather than a 3D-reconstruction problem.","feed_headline":"LumiNet swaps indoor lighting between scenes from images alone","feed_subtitle":"It preserves the source room's geometry and materials while importing the target image's shadows, gloss, and indirect light.","key_machinery":"The load-bearing mechanism is the latent intrinsic decomposition: a pretrained encoder $f_\\lambda$ that turns an image into a lighting-invariant feature map $A_o$ of shape $H \\times W \\times 128$ and a 16-dimensional lighting code $I_L$. This split is what lets LumiNet hold the source scene fixed while importing the target's lighting. On top of it, the Latent Intrinsic ControlNet concatenates $\\{A_o, I_{L_t}'\\}$ and passes the result through convolutions into the denoiser, and a learned MLP adaptor ($3072 \\to 4096 \\to 4096 \\to 4096 \\to 3072$) injects the lighting code into the cross-attention layers of a frozen latent diffusion model, so the lighting condition acts both as a spatial map and as a global embedding.","core_discovery":"The central discovery is that lighting transfer can be decomposed in latent space and re-injected through a diffusion model without explicit geometry, materials, or 3D reconstruction. Given a source image and a target image, a pretrained latent-intrinsic encoder maps each into a lighting-invariant feature map (geometry and albedo) and a low-dimensional extrinsic lighting code. LumiNet's Latent Intrinsic ControlNet concatenates the source's invariant map with the target's lighting code as the condition for a latent diffusion denoiser, while a learned MLP adaptor projects the lighting code into the cross-attention dimension of the pretrained text-to-image backbone. Training pairs are generated by a variational StyleGAN relighter that maps real bedroom images into a relightable latent space, together with multi-illumination real captures, and the model is trained only on same-scene pairs yet transfers lighting across different scenes at inference. The paper's claim is that this preserves geometry and albedo while synthesizing direct and indirect lighting effects, with a bypass decoder and post-processing steps (nearest-neighbor seed selection and flow-based inversion) added to clean up artifacts.","pith_inferences":["Extension: the same latent intrinsic split could drive other lighting-conditioned edits, such as harmonizing a composited object into a scene or transferring daylight direction outdoors, since the conditioning signal is already separated from geometry.","Extension: interpolating single dimensions of the 16-dimensional lighting code and inspecting the relit images would test whether the code dimensions are semantically interpretable as light direction, intensity, or color.","Extension: because the method's ceiling is set by the pretrained encoder's invariance, an obvious stress test is to feed the same scene under extreme lighting (near-dark and overexposed) and measure whether the extracted feature map stays stable; the paper's failure cases with small or occluded lamps and chromatic shifts suggest that invariance is only partial.","Extension: the training data comes largely from StyleGAN-generated bedroom scenes and multi-illumination captures, so relighting kitchens or offices with unusual luminaires would test whether lighting transfer generalizes beyond the furniture and light-fixture distribution seen in training."],"forward_implications":["Indoor relighting becomes a two-image editing operation: no 3D reconstruction, multi-view capture, or per-scene optimization is needed.","Because training uses same-scene pairs but inference works cross-scene, the latent intrinsic split appears to capture lighting as a separable, transferable quantity rather than a scene-specific texture.","The method can handle light sources that are not visible in the source image, inferring where lamps should turn on and casting their light plausibly.","Second-order effects such as specular highlights, reflections, and inter-reflections can be produced by a latent-conditioned diffusion model, not only by explicit light transport simulation.","Pixel-wise metrics understate the gain: a one-pixel shift in a highlight can dominate RMSE and SSIM, so perceptual and user-study evaluation is the more meaningful comparison."],"supporting_citations":[{"why":"Supplies the pretrained latent-intrinsic encoder that decomposes images into an illumination-invariant feature map and a 16-dimensional lighting code; the entire conditioning scheme rests on this decomposition.","marker":"[75]"},{"why":"Provides the StyleGAN-based relighting generator whose latent lighting codes drive the variational data-generation pipeline used to create training pairs.","marker":"[7]"},{"why":"Defines the ControlNet conditioning architecture that LumiNet modifies to operate on latent intrinsics and extrinsics instead of image-space maps.","marker":"[70]"},{"why":"The pretrained latent diffusion model whose frozen denoiser is conditioned through the latent ControlNet and the MLP adaptor; LumiNet builds on its generative prior.","marker":"[52]"},{"why":"The Multi-Illumination dataset supplies same-scene relight pairs for training and serves as the quantitative benchmark for comparison.","marker":"[45]"},{"why":"RGB↔X is the main diffusion-based baseline that LumiNet compares against; it relies on G-buffer and material-aware conditioning and is reported to struggle with cross-scene transfer.","marker":"[68]"},{"why":"IC-Light is the relighting baseline used for in-the-wild comparisons; LumiNet claims better full-scene relighting without text prompts.","marker":"[71]"},{"why":"The bypass-decoder idea, borrowed from low-light enhancement, is used to preserve source-image detail and identity when decoding from the latent space.","marker":"[59]"}],"fun_headline_variants":["Diffusion model relights indoor scenes using latent intrinsics","LumiNet transfers lighting without 3D geometry or material maps","Swap indoor lighting with LumiNet: just two images needed","Latent intrinsics let diffusion models re-light any room","LumiNet: relight a scene from any reference photo"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the pretrained latent-intrinsic encoder returns a genuinely illumination-invariant, scene-agnostic split—the feature map holds only geometry and albedo while the 16-dimensional lighting code transfers across scenes—so if residual lighting leaks into the map or scene-specific content leaks into the code, the cross-scene conditioning fails.","fun_headline_variants_meta":{"raw":{"variants":["Diffusion model relights indoor scenes using latent intrinsics","LumiNet transfers lighting without 3D geometry or material maps","Swap indoor lighting with LumiNet: just two images needed","Latent intrinsics let diffusion models re-light any room","LumiNet: relight a scene from any reference photo"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000626,"raw_usage":{"total_tokens":2908,"prompt_tokens":965,"completion_tokens":1943,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":581,"completion_tokens_details":{"reasoning_tokens":1858}},"tokens_in":581,"tokens_out":1943,"duration_ms":12632,"temperature":1.0,"reasoning_tokens":1858,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T05:39:40.334290+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Render a single synthetic indoor scene twice under two known lighting conditions with a physically based renderer, extract the latent intrinsic features from both renders, and run LumiNet to relight the first render with the second render's lighting. If the extracted feature maps differ noticeably between the two lighting conditions, or if the relit output deviates substantially from the ground-truth second render, then the illumination-invariance premise that carries the method is false.","supporting_citations":[{"cited_title":"Latent intrinsics emerge from training to relight","cited_arxiv_id":null,"evidence_quote":"Supplies the pretrained latent-intrinsic encoder that decomposes images into an illumination-invariant feature map and a 16-dimensional lighting code; the entire conditioning scheme rests on this decomposition."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the StyleGAN-based relighting generator whose latent lighting codes drive the variational data-generation pipeline used to create training pairs."},{"cited_title":"Adding conditional control to text-to-image diffusion models","cited_arxiv_id":null,"evidence_quote":"Defines the ControlNet conditioning architecture that LumiNet modifies to operate on latent intrinsics and extrinsics instead of image-space maps."},{"cited_title":"High-resolution image syn- thesis with latent diffusion models","cited_arxiv_id":null,"evidence_quote":"The pretrained latent diffusion model whose frozen denoiser is conditioned through the latent ControlNet and the MLP adaptor; LumiNet builds on its generative prior."},{"cited_title":"A multi-illumination dataset of indoor object ap- pearance","cited_arxiv_id":null,"evidence_quote":"The Multi-Illumination dataset supplies same-scene relight pairs for training and serves as the quantitative benchmark for comparison."},{"cited_title":"Rgb-x: Image decomposition and synthesis using material-and lighting-aware diffusion models","cited_arxiv_id":null,"evidence_quote":"RGB↔X is the main diffusion-based baseline that LumiNet compares against; it relies on G-buffer and material-aware conditioning and is reported to struggle with cross-scene transfer."},{"cited_title":"Scaling in-the-wild training for diffusion-based illumination harmo- nization and editing by imposing consistent light transport","cited_arxiv_id":null,"evidence_quote":"IC-Light is the relighting baseline used for in-the-wild comparisons; LumiNet claims better full-scene relighting without text prompts."},{"cited_title":"Zero-reference low-light enhancement via physical quadru- ple priors","cited_arxiv_id":null,"evidence_quote":"The bypass-decoder idea, borrowed from low-light enhancement, is used to preserve source-image detail and identity when decoding from the latent space."}],"review_version":1}