{"id":"28f7677d-a105-46e1-adc5-3dc747f96e26","arxiv_id":"2501.11515","paper_version":4,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"Modeling multi-exposure fusion as diffusion-based guided inpainting enables merging of 9-stop exposure pairs, robust to misalignment and lighting changes.","lead":"UltraFusion fuses a normal and an under-exposed photo into a high-quality HDR image, even when the two exposures differ by up to 9 stops. It treats fusion as guided inpainting with a diffusion model, which could extend practical HDR range in cameras and comes with a new benchmark.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 9-stop claim rests on RAFT pre-alignment that the authors admit can fail, and the UltraFusion benchmark has no ground truth, so a failed alignment that yields hallucinated highlights would be invisible in the reported non-reference metrics.","rationale":"Reader's weakest_assumption is the right one. Eq. 1 assumes RAFT yields correct bidirectional flow and occlusion masks after intensity mapping at up to 9 stops. The authors' own conclusion states this can fail and that restoration without under-exposed information is unreliable. This is not an external objection; it is the paper's stated limitation.\n\nWhat makes it load-bearing rather than merely a known caveat is that the 9-stop experiments cannot reveal the failure. The UltraFusion benchmark is unannotated: no HDR ground truth, no alignment ground truth, no highlight-region mask. The reported metrics are all no-reference perceptual quality scores, which can reward a plausible generated sun even if it is not the sun from the under-exposed frame. The user study compares preference, not fidelity. RealHDRV has ground truth but the supplement says after augmentation the inputs have 4 or 6 stops, so it does not exercise the 9-stop regime. Consequently, the strongest claim is supported only by aesthetics and plausibility, not by correctness of merge.\n\nA concrete GT-based check in the 9-stop regime, with and without oracle alignment, would settle whether the failure mode is the pre-alignment stage or the inpainting stage. Conditional acceptance remains appropriate because the method may work; the missing evidence is the blocker. I keep the reader's CONDITIONAL verdict.","tokens_in":18190,"tokens_out":7079,"duration_ms":78204,"concrete_test":"Take 20 static scenes from the UltraFusion benchmark (or capture new tripod scenes) and, for each, capture a dense exposure stack to produce ground-truth HDR. Tone-map the GT with the same display pipeline and compute PSNR/SSIM inside the over-exposed highlight mask between UltraFusion's output and the tone-mapped GT. Then rerun UltraFusion with RAFT pre-alignment replaced by ground-truth alignment computed from the static scene. If highlight-region error remains large even with oracle alignment, the guided-inpainting stage is not faithfully using the under-exposed information; if oracle alignment changes results substantially, the pre-alignment stage is the bottleneck. Either outcome directly tests the 9-stop claim.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing link in the central claim is pre-alignment (Sec. 3.1, Eq. 1): Iue->oe is produced by RAFT flow on intensity-mapped inputs plus a forward-backward consistency mask. If RAFT fails at 9 stops, the under-exposed guidance is wrong or masked away, and the guided-inpainting stage degrades to unguided diffusion inpainting. The authors explicitly concede this in the Conclusion: 'occlusion mask estimation may introduce errors, causing the restoration of certain highlight regions degrades to single image HDR. ... restoration without under-exposed information remains unreliable.' That concession marks the exact component that distinguishes UltraFusion from generic generative inpainting.\n\nThe evaluation cannot detect this failure on the claimed 9-stop setting. The new UltraFusion benchmark contains only under/over-exposed pairs, no ground truth; all scores on it are non-reference aesthetic metrics (MUSIQ, DeQA-Score, PAQ2PIQ, HyperIQA). Those metrics reward plausible, vivid output, so hallucinated highlights from failed alignment can still score well. RealHDRV provides ground truth, but the supplement reports only 4-6 stops after their augmentation, not 9. Thus the 9-stop claim is currently not verified for alignment correctness or reconstruction fidelity.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes UltraFusion, a two-image exposure fusion method formulated as guided inpainting. The over-exposed image serves as the reference canvas, while the under-exposed image is coarsely pre-aligned via RAFT optical flow and forward-backward consistency masking, then injected as soft guidance into a diffusion-based inpainting model. A decompose-and-fuse control branch and a fidelity control branch are introduced to better modulate the diffusion prior. Training pairs are synthesized by combining static multi-exposure images (SICE) with pseudo occlusion masks derived from video frames (Vimeo-90K). The method is evaluated on MEFB, RealHDRV, and a newly captured UltraFusion benchmark containing 100 under/over-exposed pairs with exposure differences up to 9 stops; comparisons are made against HDR reconstruction and multi-exposure fusion baselines using non-reference metrics, MEF-SSIM, TMQI, and a user study.","tokens_in":18435,"tokens_out":6018,"duration_ms":60201,"significance":"If the central claim holds, UltraFusion would substantially extend the practical exposure range of two-image fusion—from the typical 3–4 stops to about 9 stops—and would offer a novel reformulation of exposure fusion as guided inpainting, leveraging generative priors for natural tone mapping. The paper is clearly written, the guided inpainting formulation is specific, and the ablations (Table 3, Figs. 11–13) isolate the contribution of pre-alignment, the decompose-and-fuse branch, and the fidelity branch. The release of code and a new real-world benchmark is a further strength, though the benchmark currently lacks ground truth, which limits its quantitative value. The main weakness is that the headline 9-stop capability is not supported by ground-truth fidelity evaluation at that exposure range; the RealHDRV experiment covers only 4–6 stops after augmentation, and the UltraFusion benchmark is scored solely with non-reference aesthetic metrics.","major_comments":[{"comment":"The central claim that UltraFusion “can merge inputs with 9 stops differences” is not supported by ground-truth evaluation at 9 stops. The RealHDRV evaluation uses only 4–6 stops after the authors’ augmentation (Supplement C), and the new UltraFusion benchmark contains no ground truth, relying entirely on non-reference metrics (MUSIQ, DeQA-Score, PAQ2PIQ, HyperIQA) and a user study. Non-reference aesthetic metrics reward visually plausible outputs and cannot detect hallucinated or incorrectly reconstructed highlights. The paper’s own Conclusion (Section 5) concedes that when occlusion-mask estimation fails, highlight restoration degrades to “single image HDR” and “remains unreliable.” A controlled 9-stop evaluation with ground truth (e.g., static HDR scenes with known exposures, or synthetic 9-stop pairs rendered from HDR ground truth, reporting PSNR/SSIM/TMQI and MEF-SSIM) is needed before the headline claim is supportable.","section":"§4.1, §4.2, Supplement C"},{"comment":"Pre-alignment accuracy at 9-stop exposure differences is unmeasured. Equation (1) defines I_ue→oe as RAFT flow on intensity-mapped inputs with a forward-backward consistency mask, but the intensity-mapping function is not specified in enough detail to reproduce, and there is no quantitative evaluation of flow error or occlusion-mask correctness on 9-stop inputs. The paper’s Conclusion explicitly states that “a more exposure-robust optical flow algorithm” is needed. Because a failed alignment can cause the under-exposed guidance to be masked away, reducing the method to unguided inpainting, the authors should report a failure-rate analysis of the pre-alignment stage (e.g., using synthetic warps or a static scene with known ground-truth motion) and characterize how fusion quality degrades when alignment is wrong.","section":"§3.1, Eq. (1)"},{"comment":"Tables 1 and 2 report no error bars, no significance tests, and no multiple-run variance. Several reported improvements over strong baselines are on the order of 1–2 metric points for some metrics, and the HDR reconstruction baselines (HDR-Transformer, SCTNet, SAFNet) are re-implemented for two-exposure input with Photomatix tone mapping. Without variance estimates or audited baseline code, it is uncertain whether the observed gaps are robust. Please report mean ± std over multiple runs (or at least paired significance tests) and release the adapted baseline implementations or a detailed reproducibility description.","section":"Tables 1 and 2"},{"comment":"The training-data synthesis pipeline combines static SICE under-exposed patches with pseudo occlusion masks from Vimeo-90K first/last frames, but the paper does not characterize the effective exposure differences and motion magnitudes in the synthesized pairs. If the SICE brackets used are limited to a narrow EV range, the model may never see 9-stop dynamic examples during training. In that case, the claimed 9-stop capability would rest on extrapolation by the diffusion prior rather than on learned exposure fusion behavior. Please report the EV distribution and motion-magnitude distribution of the synthesized training data, or explicitly modify the pipeline to synthesize 9-stop dynamic pairs and verify that the model is trained on them.","section":"§3.3"}],"minor_comments":[{"comment":"The phrase “the first exposure fusion technique that can merge inputs with 9 stops differences” is a strong novelty claim; consider softening it or providing a more exhaustive comparison with prior extreme-exposure fusion methods (e.g., DeepFuse [38]) that may already operate at large exposure ranges with lower quality.","section":"Abstract, §1"},{"comment":"Equation (2) defines S_ue using Y_ue in “YUV space,” but it is not clear whether global or local mean/std are used and how the UV chroma channels are normalized; please clarify the exact computation.","section":"§3.2, Eq. (2)"},{"comment":"The statement “we re-implement them by following their default settings” should specify which official implementations were used and what changes were made to adapt them to two-exposure input and to apply Photomatix tone mapping.","section":"§4.1"},{"comment":"The EV-difference histogram in Fig. 6(a) is hard to read in the printed version; please report the exact number of pairs in each EV bin (e.g., 4–5, 6–7, 8–9) in the caption or in a small table.","section":"Fig. 6(a)"},{"comment":"The reference to “previous work [23]” (DiffBIR) for injecting z_t as a condition is not precise enough for reproduction; please state exactly where the diffusion latent is concatenated or added in the control branch.","section":"§3.2"},{"comment":"There are several typos and minor wording issues: “newtwork” (Section 3), “devided” (Section 2.1), “Evluation” (Supplement C), “Compre” (Supplement I), “Addtional” (Supplement J). These should be corrected in a final pass.","section":"Various"}],"recommendation":"major_revision","confidential_remarks":"The paper’s own concluding limitation—that failed occlusion masking degrades highlight restoration to unreliable single-image HDR—undercuts the headline 9-stop claim as currently evidenced. The authors should be asked to provide 9-stop ground-truth fidelity results or to temper the claim. The new UltraFusion benchmark, as released without ground truth, will be of limited value for future quantitative comparison unless it is accompanied by raw captures or derived HDR ground truth. The “first” claim in the abstract should also be verified against the full prior-art landscape, including practical extreme-exposure fusion systems."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea here is good and the paper is worth reading: they reframe exposure fusion as guided inpainting, using the under-exposed image as soft guidance rather than a hard constraint. That is a real conceptual shift, and it explains why the method tolerates misalignment and lighting changes better than previous multi-exposure fusion and HDR reconstruction. The decompose-and-fuse control branch, the fidelity branch, and the synthetic training pipeline are all sensible and well-ablated. The authors also deserve credit for being frank in the conclusion about the failure mode where the under-exposed guidance is masked away and restoration degrades to unreliable single-image HDR.\n\nThe soft spots are real but proportionate. The headline claim of merging 9-stop input pairs is not actually verified on 9-stop data with any fidelity metric. Their new UltraFusion benchmark has no ground truth, so all scores on it are non-reference aesthetic metrics, which can reward plausible hallucination. The only ground-truth evaluation is on RealHDRV, but the supplement says that after their augmentation the test inputs have 4 or 6 stops, not 9. So the central quantitative evidence for the 9-stop capability is missing. The pre-alignment stage relies on RAFT on intensity-mapped inputs, and the authors admit in the conclusion that flow and occlusion mask estimation can fail at extreme exposures; that is exactly the component that distinguishes their method from generic inpainting. If the flow goes wrong, the guide is wrong, and the diffusion prior may just draw something nice that is not the scene.\n\nWhat is not a flaw: they do evaluate on external benchmarks (MEFB, RealHDRV) with other methods, so the result is not forced by construction. The absence of error bars is annoying but common in this area. Code and data are not yet released, which matters here because baseline HDR reconstruction methods were re-implemented without audited code and the new benchmark needs to be checked.\n\nFor a reader in computational photography or image fusion, this is a useful paper and a serious candidate for publication. My recommendation: send it to peer review, but insist on either ground-truth or correspondence-based validation at the claimed 9-stop regime, and require code/data release as a condition. The method is interesting enough to deserve referees; let them push on the evidence.","headline":"A genuinely novel guided-inpainting approach to extreme exposure fusion, but the 9-stop claim is not yet verified by the evidence in the paper.","tokens_in":655,"tokens_out":847,"would_cite":true,"duration_ms":21919,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"UltraFusion claims exposure fusion can merge images nine stops apart by treating the under-exposed shot as guidance for inpainting burned-out highlights.","keywords":["exposure fusion","HDR imaging","guided inpainting","diffusion model","optical flow alignment","multi-exposure fusion","tone mapping","dynamic range"],"falsifier":"Take a scene with a fast-moving bright object at a 9-stop gap and compare the inpainted highlight region against the under-exposed frame: if the forward-backward consistency mask mislabels the moving highlight as occluded, the generated highlight will drift in shape or color from the actual source. A controlled version would corrupt the under-exposed guide in a known highlight region and check whether the output changes accordingly.","tokens_in":17965,"feed_emoji":"📷","tokens_out":7249,"duration_ms":64324,"temperature":0.7,"pith_summary":"UltraFusion claims that exposure fusion can be pushed from the usual 3-4 stop difference to 9 stops by reformulating fusion as guided inpainting. The idea: keep the over-exposed image (where highlights are clipped) as the base, and treat the under-exposed image as a soft guide that tells the model what the bright regions actually looked like. Because the guide is soft rather than a hard alignment, the method tolerates misregistration and lighting changes that break earlier fusion algorithms. The generative image prior also supplies natural tone mapping, so the output needs no separate tone-mapping step. If right, a two-shot capture on an ordinary camera covers roughly five additional stops of dynamic range.","feed_headline":"Nine-stop exposure fusion made practical by guided inpainting","feed_subtitle":"Under-exposed shot guides a diffusion model to fill burned highlights without ghosting or ugly tone mapping.","key_machinery":"The load-bearing object is the guided-inpainting formulation itself: the over-exposed image $I_{oe}$ is the base, the under-exposed image $I_{ue}$ becomes a soft guidance signal, and a pretrained diffusion U-Net generates the highlight content. The control path is a decompose-and-fuse branch that splits $I_{ue}$ into a luminance-normalized structure map $S_{ue}=(Y_{ue}-\\mu(Y_{ue}))/\\sigma(Y_{ue})$ and the chroma channels $C_{ue}$, so guidance survives extreme darkness, and merges them with $I_{oe}$ features through multi-scale cross-attention. A second fidelity control branch feeds the same structure and color into the VAE decoder to suppress texture drift. The whole pipeline is trained on synthetic pairs built from static SICE exposure brackets and Vimeo-90K video frames to simulate motion and occlusion.","core_discovery":"On the paper's own terms, the discovery is that severe multi-exposure fusion can be solved by guided inpainting in a pretrained latent diffusion model. The over-exposed frame is the image to be inpainted; the under-exposed frame, after brightness mapping, RAFT-based bidirectional flow warping, and occlusion masking, is the guidance. The under-exposed information is decomposed into luminance-normalized structure and chroma so the diffusion control branch can use it even when the input is very dark, then fused with the over-exposed features by multi-scale cross-attention; a fidelity control branch injects the same information into the VAE decoder to keep texture and color faithful. The authors report higher non-reference quality scores than previous fusion and HDR-reconstruction methods on MEFB and RealHDRV and on their own 100-pair UltraFusion benchmark with exposure gaps up to 9 stops, and they show that a plain ControlNet-style injection fails because the model does not know which frame is the reference.","pith_inferences":["An implicit consequence is that the failure mode concentrates on pre-alignment: replacing RAFT with an exposure-robust flow estimator or moving alignment inside the control branch should directly improve the hard cases the authors flag in the conclusion.","The 3-exposure extension in the supplement suggests the same guided-inpainting recipe applies to longer brackets; a natural next test is using a fused second guide instead of a single under-exposed frame.","The UltraFusion benchmark, with its 100 pairs up to 9 stops apart, could become a standard stress test for exposure fusion, provided its user-study and non-reference scores are treated as indicative rather than definitive.","A subtle risk the paper acknowledges: when the under-exposed guide is masked or too dark, the diffusion prior may generate plausible but unverified highlight content, so applications needing photometric accuracy should check the inpainted regions against the guide."],"forward_implications":["Two-image capture becomes sufficient for scenes that previously needed three or more brackets, cutting capture time and motion risk.","Because tone mapping emerges from diffusion priors rather than a separate operator, fusion and tone mapping no longer suffer cascaded errors.","Soft guidance tolerates imperfect alignment, lighting inconsistency, and even unrelated images as guidance, opening the door to image harmonization.","The synthetic training pipeline removes the need for a large paired dynamic-HDR ground-truth dataset, so the approach can scale to new cameras and lenses without new capture campaigns."],"supporting_citations":[{"why":"Supplies the pretrained RAFT optical flow network used for bidirectional flow estimation and occlusion masking in the pre-alignment stage.","marker":"[45]"},{"why":"ControlNet provides the reference control-branch architecture that UltraFusion adapts into its decompose-and-fuse branch.","marker":"[68]"},{"why":"Stable Diffusion's latent diffusion prior is the generative backbone that performs the actual inpainting and tone mapping.","marker":"[39]"},{"why":"SICE supplies high-quality static multi-exposure image pairs used to synthesize training data.","marker":"[1]"},{"why":"Vimeo-90K provides video frames with large motion that are paired with SICE patches to simulate dynamic occlusions in training.","marker":"[58]"},{"why":"HDR-Transformer is the main HDR-reconstruction baseline that UltraFusion compares against and outperforms on the benchmarks.","marker":"[27]"},{"why":"RealHDRV supplies the dynamic HDR test set with ground truths used for TMQI and other quantitative evaluation.","marker":"[42]"}],"fun_headline_variants":["Fusing 9-stop exposures with guided inpainting","Diffusion-based fusion for 9-stop HDR gaps","Under-exposed frame guides highlight inpainting","Beats HDR-Transformer on 9-stop fusion"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The pre-alignment stage assumes that RAFT, after intensity mapping, estimates correct flow and a correct occlusion mask between images up to 9 stops apart; if flow is wrong, the warped under-exposed guide is wrong and the inpainting cannot faithfully restore highlights.","fun_headline_variants_meta":{"raw":{"variants":["Fusing 9-stop exposures with guided inpainting","Diffusion-based fusion for 9-stop HDR gaps","Under-exposed frame guides highlight inpainting","Beats HDR-Transformer on 9-stop fusion"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000205,"raw_usage":{"total_tokens":1434,"prompt_tokens":1030,"completion_tokens":404,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":646,"completion_tokens_details":{"reasoning_tokens":339}},"tokens_in":646,"tokens_out":404,"duration_ms":4623,"temperature":1.0,"reasoning_tokens":339,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T18:10:00.683748+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a scene with a fast-moving bright object at a 9-stop gap and compare the inpainted highlight region against the under-exposed frame: if the forward-backward consistency mask mislabels the moving highlight as occluded, the generated highlight will drift in shape or color from the actual source. A controlled version would corrupt the under-exposed guide in a known highlight region and check whether the output changes accordingly.","supporting_citations":[{"cited_title":"Adding conditional control to text-to-image diffusion models","cited_arxiv_id":null,"evidence_quote":"ControlNet provides the reference control-branch architecture that UltraFusion adapts into its decompose-and-fuse branch."},{"cited_title":"High-resolution image synthesis with latent diffusion models","cited_arxiv_id":null,"evidence_quote":"Stable Diffusion's latent diffusion prior is the generative backbone that performs the actual inpainting and tone mapping."},{"cited_title":"Ghost-free high dynamic range imaging with context-aware transformer","cited_arxiv_id":null,"evidence_quote":"HDR-Transformer is the main HDR-reconstruction baseline that UltraFusion compares against and outperforms on the benchmarks."},{"cited_title":"Towards real-world HDR video reconstruc- tion: A large-scale benchmark dataset and a two-stage align- ment network","cited_arxiv_id":null,"evidence_quote":"RealHDRV supplies the dynamic HDR test set with ground truths used for TMQI and other quantitative evaluation."}],"review_version":1}