{"id":"96155e34-1570-4405-8154-1d67c95007da","arxiv_id":"2509.03680","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A video diffusion transformer fine-tuned on synthetic and real data predicts HDR environment maps from images/videos, cutting peak light-direction error by roughly 45% on sunny outdoor scenes versus DiffusionLight.","lead":"LuxDiT fine-tunes a video diffusion transformer to generate HDR environment maps from a single image or video, using synthetic training data plus low-rank adaptation on real HDR panoramas. It reports lower lighting direction error and better temporal consistency than prior diffusion-based estimators, with applications in virtual object insertion and relighting.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper's own Supp. Table 13 shows LoRA fine-tuning degrades shading-based lighting estimates, yet all main results use LoRA scale 1.0, leaving the real-world generalization claim under-supported.","rationale":"I read the paper in good faith and find the architecture plausible: a DiT conditioned on input tokens, dual tonemapped HDR targets, and a two-stage training scheme are reasonable design choices. The main quantitative tables, if accurate, would support a strong result. However, the most load-bearing latent assumption is that Stage II LoRA improves semantic alignment across all real inputs without destroying the physical shading cues learned in Stage I. The paper contains direct evidence against this assumption in Supp. Table 13: LoRA scale monotonically degrades angular error on synthetic foreground objects. Since all main results use scale 1.0, and since the LoRA training data are not representative of object-centric photographs, the claim of broad real-world generalization is under-supported. The Table 7 identical numbers are a serious reporting concern, but they do not directly threaten the headline comparison with baselines; the LoRA tradeoff does. A targeted LoRA-scale sweep on real object-centric data plus a sun-visibility split would settle whether the concern lands. This does not overturn the reader's conditional verdict; it sharpens the condition under which the central claim should be accepted.","tokens_in":18703,"tokens_out":6725,"duration_ms":75183,"concrete_test":"Run the full three-sphere and peak-angular-error evaluation at LoRA scales 0.0, 0.25, 0.5, 0.75, 1.0 on (a) the Laval Outdoor sunny subset, (b) held-out Poly Haven, and (c) a real object-centric benchmark such as Objects with Lighting or a new set of object photos with known HDR ground truth. Also split the Laval Outdoor sunny subset by whether the sun is visible in the input crop. If the optimal LoRA scale differs across benchmarks, or if the peak-angular-error improvement disappears when the sun is outside the field of view, the paper's generalization claim needs qualification and the fixed LoRA scale of 1.0 is not justified.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that LuxDiT generalizes from synthetic data to real scenes and outperforms prior work depends on Stage II LoRA adaptation improving semantic alignment without destroying the physically grounded shading cues learned in Stage I. The paper's own Supp. Table 13 directly contradicts this for synthetic foreground objects: as LoRA scale increases from 0.0 to 1.0, diffuse angular error grows from 1.59 to 3.94, matte from 2.07 to 5.89, and mirror from 3.41 to 7.40. Yet all main results (Tables 1-3, 8) are reported at LoRA scale 1.0. The LoRA training distribution is perspective crops of HDR panoramas and LDR 360 videos, which are quite different from typical in-the-wild object-centric photographs. The paper never reports a LoRA-scale sweep on a held-out real object-centric benchmark, so it is unclear whether the benchmark wins reflect a genuine general improvement or an artifact of matching the panorama-crop domain. If LoRA scale must be tuned per content type, the single-model 'outperforms SOTA' claim is not established. The reporting anomaly in Table 7 (identical channel-concat numbers across Laval Indoor and Poly Haven) further weakens confidence in the experimental reporting, but the LoRA tradeoff is the more load-bearing logical threat.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces LuxDiT, a video diffusion transformer (built on CogVideoX) that estimates HDR environment maps from a single LDR image or video. The method encodes two tone-mapped HDR representations (Reinhard and log) with a pretrained VAE, concatenates their latents, and denoises them jointly while conditioning on input tokens through separate adaptive layer normalization and a directional embedding. Training is two-stage: first on roughly 190,000 synthetic scenes rendered from Objaverse objects and MatSynth materials, then LoRA fine-tuning on 2,386 real HDR panoramas and LDR panoramic videos. Evaluations on Laval Indoor/Outdoor and Poly Haven report competitive or better metrics than DiffusionLight and other baselines, a roughly 46% reduction in mean peak angular error on Laval Outdoor sunny scenes, improved temporal consistency on video benchmarks, and competitive virtual-object-insertion results. The supplement includes additional ablations, including a LoRA-scale study on synthetic foreground objects.","tokens_in":19019,"tokens_out":7614,"duration_ms":72704,"significance":"If correct, LuxDiT is a practically significant contribution: a single feed-forward generative model can estimate full HDR environment maps with high-frequency detail and temporal coherence, avoiding DiffusionLight's test-time multi-exposure ensembling. The two-stage recipe (physics-based synthetic pretraining followed by lightweight LoRA semantic alignment) is a sensible, reusable design. Strengths include the dual-tonemapping HDR representation with learned fusion, the token-based conditioning for non-aligned outputs, the large-scale synthetic data generation, and the transparent evaluation on standard benchmarks. The paper also includes an unusually honest supplement that reports a LoRA failure mode on synthetic foreground objects. However, the central generalization claim is not fully established because the optimal LoRA scale appears content-dependent, and one key ablation table contains a clear reporting error.","major_comments":[{"comment":"The channel-concatenation row reports identical angular errors (7.09, 10.04, 11.07) for both Laval Indoor and Poly Haven. Since the other rows and Table 1 show different error distributions for these datasets, identical values are implausible and indicate a copy/reporting error. This row is the primary quantitative evidence for the claim that token conditioning is necessary, so as printed the ablation cannot be used to support that claim. Please correct the table with the actual per-dataset numbers, or explicitly state which dataset was used for that variant.","section":"Table 7"},{"comment":"The LoRA interpolation scale is a free parameter, and the main results all use scale 1.0. Table 4 shows that scale 1.0 is best on Poly Haven, but Supp. Table 13 shows the opposite on synthetic foreground objects: diffuse angular error degrades from 1.59 to 3.94, matte from 2.07 to 5.89, and mirror from 3.41 to 7.40 as the scale goes from 0.0 to 1.0. The paper does not report a LoRA-scale sweep on a held-out real object-centric benchmark, so it is not established that scale 1.0 generalizes beyond panorama-crop-like inputs. Because the introduction and abstract claim generalization to 'real-world scenes' and 'casually captured images', this is load-bearing. Please add a sweep on a real object-centric set (e.g., Objects-with-Lighting) or clearly scope the claims to panorama-crop-like imagery and state the domain for which scale 1.0 is intended.","section":"Section 5.5 / Supp. Table 13"},{"comment":"The abstract states that the method is 'outperforming existing state-of-the-art techniques', but the table-level support is mixed. On Laval Indoor, DiffusionLight has lower angular error on diffuse and matte spheres (2.500 vs 2.555 and 3.421 vs 3.526), though Ours wins on mirror and on si-RMSE/n-RMSE. On Laval Outdoor, DiffusionLight has lower diffuse angular error (1.936 vs 2.018). The strongest and consistent gains are on Poly Haven and on si-RMSE/n-RMSE. Please qualify the claim (e.g., 'competitive or better on standard metrics, with consistent gains in RMSE and directional accuracy') and, if feasible, report significance or confidence intervals for the headline differences.","section":"Table 1 / Abstract"}],"minor_comments":[{"comment":"Typography: 'Angular error (AE) on is used to evaluate WEB360 LDR videos' should read 'Angular error (AE) is used...'.","section":"Table 3 caption"},{"comment":"The caption of Table 4 does not specify the dataset. The text says the sweep is on Poly Haven; please state in the caption that this is the held-out Poly Haven set and clarify whether any of those HDRIs overlap with the Stage II LoRA training set.","section":"Section 5.5 / Table 4 caption"},{"comment":"The LoRA-scale values shown in Supp. Figure 13 (0.1, 0.2, 0.4, 0.6, 0.7, 0.8) do not match the values used in Table 4 (0.25, 0.5, 0.75). Please make the visual exploration consistent with the quantitative ablation.","section":"Supp. Figure 13 vs Table 4"},{"comment":"The text says 114 WEB360 videos are held out for evaluation, but Table 3 uses 'randomly select 12 LDR panoramic videos' from WEB360. Please clarify whether the 12 sampled videos are a subset of the 114 held-out videos and how they were selected.","section":"Section 5.3 / Supp. B.2"},{"comment":"It is unclear which baseline numbers were recomputed by the authors under the same protocol and which were copied from prior papers. For reproducibility, please mark copied numbers and, where feasible, run all baselines under the same three-sphere pipeline.","section":"Section 5.1 / Baselines"}],"recommendation":"major_revision","confidential_remarks":"The identical channel-concatenation row in Table 7 is a data-integrity issue that must be resolved before I would trust the architecture ablation; please ask the authors for the raw per-dataset numbers. The LoRA-scale concern can be addressed with one additional experiment on a held-out real object-centric benchmark, or by explicitly narrowing the claims. If the authors can provide that evidence or scope the claims appropriately, the paper would be close to acceptable; the core idea and evaluation are otherwise solid."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know before you read it. The core move is new for this subfield: take a video diffusion transformer (CogVideoX), fine-tune it to denoise HDR environment maps directly with token-based visual conditioning and a dual Reinhard/log tonemap output. That is mechanistically different from DiffusionLight's chrome-ball inpainting and from StyleLight's GAN inversion, and it yields HDR maps with high-frequency directional detail, which is what lighting estimation for object insertion actually needs. Second thing: the paper is more honest about its own weak point than most. Supp. Table 13 shows the LoRA semantic-alignment stage actively degrades lighting accuracy on synthetic foreground objects as the scale goes 0 to 1 (diffuse AE 1.59 to 3.94), and all main results sit at scale 1.0. That tension is real, but the catastrophic reading — no real-object evidence — is wrong: Table 11 evaluates on the Objects with Lighting dataset at scale 1.0 and gets 1.3 to 2.0 degrees angular error. So the LoRA issue is an open question about the right scale per content type, not a refutation.\n\nWhat the paper does well. Standard benchmarks (Laval Indoor/Outdoor, Poly Haven), standard metrics, sensible baselines, and genuinely informative ablations (channel concat vs token conditioning, with/without the synthetic stage, LoRA scale, FOV, elevation). The 45% peak-angular-error gain on sunny Laval Outdoor (23.7 vs 44.4 degrees) is a real, large improvement for shadow-critical applications. The video results also show a large stability gain over frame-by-frame inference (PAE std 1.95 vs 10.31).\n\nSoft spots, in proportion. (1) Table 7 has a clear copy-paste error — identical channel-concat numbers on Laval Indoor and Poly Haven — which damages confidence in the eval harness even though the qualitative conclusion (channel concat underperforms) is credible. (2) The abstract's global \"outperforming\" claim is wider than the evidence: on Laval Indoor the model is about on par with DiffusionLight (better si-RMSE, slightly worse angular error on two of three materials); the clear wins are outdoor and on peak direction. (3) The LoRA scale was selected by ablating on Poly Haven, which is also a reported test benchmark — that is light test-set tuning on one dataset; Laval and the video sets are clean. (4) No error bars on stochastic sampling, and no code or data released. All addressable.\n\nWho this is for: anyone working on lighting estimation, relighting, or virtual object insertion. Send it to a serious referee — conditional accept, with the eval harness and a LoRA-scale sweep on real objects as the main asks.","headline":"A genuinely new mechanism for HDR lighting estimation that mostly delivers on its claims, but the paper's own LoRA-scale supplement, a Table 7 copy-paste error, and an abstract broader than the Laval Indoor numbers all say: referee it carefully, don't desk-reject it.","tokens_in":19532,"tokens_out":5547,"would_cite":true,"duration_ms":54625,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"LuxDiT, a video diffusion transformer fine-tuned on synthetic renders and adapted with LoRA on real HDR panoramas, generates HDR environment maps from a single image or video and cuts sun-direction error on sunny outdoor scenes by roughly 4","keywords":["lighting estimation","HDR environment maps","video diffusion transformer","LoRA fine-tuning","synthetic data pretraining","temporal consistency","virtual object insertion","conditional generation"],"falsifier":"Render HDR panoramas of real outdoor scenes whose sun position is known from GPS and time, mask out any visible sky, and run LuxDiT versus DiffusionLight: if mean peak angular error does not stay near the 23.7-degree level, or at least substantially below 44.4 degrees, across varied weather, season, and geography, the synthetic-to-real transfer claim is weakened. A second check is to run LuxDiT on a held-out set of synthetic foreground objects with ground-truth HDR and confirm whether LoRA scale 1.0 reproduces the degradation shown in the supplement's Table 13.","tokens_in":18596,"feed_emoji":"💡","tokens_out":5781,"duration_ms":57022,"temperature":0.7,"pith_summary":"LuxDiT treats lighting estimation not as regression but as conditional denoising: given a casually captured image or video, a video diffusion transformer generates a full 360-degree HDR environment map. The paper argues that training on about 190,000 synthetic object scenes teaches the model physically grounded cues for light direction and intensity, while a LoRA fine-tune on roughly 2,386 real HDR panoramas fixes semantic mismatches without destroying those cues. On sunny outdoor scenes, mean peak sun-direction error drops to 23.7 degrees versus 44.4 degrees for DiffusionLight, and video-conditioned inference reduces temporal flicker. If the claim holds, a feed-forward generative model can compete with or exceed prior lighting estimators while producing high-frequency directional detail useful for virtual object insertion.","feed_headline":"Video diffusion model cuts sunlight-direction error nearly in half","feed_subtitle":"LuxDiT generates HDR environment maps from a single image or video, beating prior estimators and reducing flicker.","key_machinery":"Three components carry the argument: (1) a dual-tonemapped HDR representation, where the same panorama is encoded as a Reinhard-mapped LDR image and a log-intensity LDR image, allowing a latent diffusion model trained on LDR data to generate HDR content; (2) token-based conditioning, in which input video latents are flattened into tokens and processed jointly with denoising tokens under separate adaptive layer normalization, enabling global reasoning over non-local shading cues; and (3) a two-stage training scheme: synthetic supervised pretraining for physically grounded priors, followed by LoRA adaptation on real HDR panoramas for semantic alignment.","core_discovery":"The paper's central claim is that a video diffusion transformer, conditioned on visual input through token-level attention rather than channel concatenation, can directly synthesize HDR environment maps whose direction, intensity, and temporal stability beat existing approaches. The HDR target is represented as two complementary tone-mapped latent streams, Reinhard and log, decoded and fused by a lightweight MLP. Stage I training on randomized synthetic scenes supplies physically grounded shading cues; Stage II applies LoRA on real HDR panoramas to align predicted lighting with the input scene's semantics. The reported result is a roughly 45% reduction in mean peak angular error on Laval Out","pith_inferences":["A testable extension is to treat the LoRA weight as a calibrated slider: the paper's supplement shows higher LoRA scale improves real-scene accuracy but degrades synthetic foreground objects, suggesting users could trade semantic alignment against physics-based accuracy per application.","The dual-tonemapped latent formulation is a reusable trick for other inverse-rendering tasks that need HDR outputs from VAEs trained only on LDR content, such as albedo or material estimation.","Video-level diffusion could serve as a temporal prior for optimization-based inverse rendering, supplying a consistent lighting initialization that reduces flicker and speeds convergence."],"forward_implications":["A single feed-forward generative pass replaces expensive test-time ensembles such as DiffusionLight's multi-exposure chrome-ball inpainting.","Video-conditioned inference is more temporally stable than frame-by-frame estimation, reducing flicker in relighting and virtual object insertion.","Large-scale synthetic rendering is the key to out-of-domain generalization: removing it degrades performance on Laval Indoor, as shown in the paper's ablation.","LoRA adaptation on real panoramas improves semantic alignment, lowering angular error on Poly Haven while preserving the pretrained model's lighting priors.","Accurate peak sun direction enables plausible shadow casting; in the paper's user study, LuxDiT was preferred over StyleLight and DiffusionLight in about 60% of object-insertion comparisons."],"supporting_citations":[{"why":"The multi-exposure chrome-ball diffusion baseline whose peak angular error and temporal stability LuxDiT is compared against and outperforms.","marker":"[44]"},{"why":"The pretrained video diffusion transformer and VAE that LuxDiT fine-tunes for conditional HDR panorama synthesis.","marker":"[62]"},{"why":"The low-rank adaptation technique used to fine-tune on real HDR panoramas without full-model overfitting.","marker":"[23]"},{"why":"Supplies the 3D objects used to render roughly 190,000 synthetic scenes for Stage I training.","marker":"[10]"},{"why":"Supplies the PBR material textures on planes and primitives that create the shading cues in synthetic scenes.","marker":"[53]"},{"why":"Provides the Laval Outdoor sunny benchmark and the peak angular error metric used for sun-direction evaluation.","marker":"[22]"},{"why":"Establishes the three-sphere evaluation protocol and serves as a baseline for angular and RMSE metrics.","marker":"[54]"},{"why":"Introduces the dual-tonemapped HDR representation that LuxDiT adapts to the latent diffusion setting.","marker":"[26]"},{"why":"Provides the Laval Indoor benchmark used to test out-of-domain generalization without training on it.","marker":"[15]"}],"fun_headline_variants":["Diffusion model cuts lighting direction error by 45%","Video diffusion halves HDR lighting prediction error","AI predicts scene lighting from a single image","Lighting estimation error halved with video diffusion","Diffusion-based HDR lighting from visual cues"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The load-bearing premise is that physically grounded shading cues rendered in synthetic object scenes transfer to real-world photographs, and that the LoRA fine-tune on roughly 2,386 real HDR panoramas corrects semantic mismatches without eroding those cues; the paper itself shows that higher LoRA weights degrade lighting accuracy on synthetic foreground objects.","fun_headline_variants_meta":{"raw":{"variants":["Diffusion model cuts lighting direction error by 45%","Video diffusion halves HDR lighting prediction error","AI predicts scene lighting from a single image","Lighting estimation error halved with video diffusion","Diffusion-based HDR lighting from visual cues"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000223,"raw_usage":{"total_tokens":1272,"prompt_tokens":699,"completion_tokens":573,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":443,"completion_tokens_details":{"reasoning_tokens":502}},"tokens_in":443,"tokens_out":573,"duration_ms":6953,"temperature":1.0,"reasoning_tokens":502,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T10:45:04.405949+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Render HDR panoramas of real outdoor scenes whose sun position is known from GPS and time, mask out any visible sky, and run LuxDiT versus DiffusionLight: if mean peak angular error does not stay near the 23.7-degree level, or at least substantially below 44.4 degrees, across varied weather, season, and geography, the synthetic-to-real transfer claim is weakened. A second check is to run LuxDiT on a held-out set of synthetic foreground objects with ground-truth HDR and confirm whether LoRA scale 1.0 reproduces the degradation shown in the supplement's Table 13.","supporting_citations":[{"cited_title":"DiffusionLight: light probes for free by painting a chrome ball","cited_arxiv_id":null,"evidence_quote":"The multi-exposure chrome-ball diffusion baseline whose peak angular error and temporal stability LuxDiT is compared against and outperforms."},{"cited_title":"Objaverse: A universe of annotated 3D objects","cited_arxiv_id":null,"evidence_quote":"Supplies the 3D objects used to render roughly 190,000 synthetic scenes for Stage I training."},{"cited_title":"Matsynth: A modern pbr materials dataset","cited_arxiv_id":null,"evidence_quote":"Supplies the PBR material textures on planes and primitives that create the shading cues in synthetic scenes."},{"cited_title":"Deep sky modeling for single image outdoor lighting estimation","cited_arxiv_id":null,"evidence_quote":"Provides the Laval Outdoor sunny benchmark and the peak angular error metric used for sun-direction evaluation."},{"cited_title":"Stylelight: Hdr panorama generation for lighting estimation and editing","cited_arxiv_id":null,"evidence_quote":"Establishes the three-sphere evaluation protocol and serves as a baseline for angular and RMSE metrics."},{"cited_title":"Neural gaffer: Relighting any object via diffusion","cited_arxiv_id":null,"evidence_quote":"Introduces the dual-tonemapped HDR representation that LuxDiT adapts to the latent diffusion setting."}],"review_version":1}