{"id":"48c87577-d478-43bc-9f64-d1e1cbe43944","arxiv_id":"2509.09427","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"FS-Diff is a diffusion model that jointly fuses and super-resolves low-resolution multimodal image pairs using clarity-aware CLIP semantics and a bidirectional Mamba feature extractor.","lead":"This paper presents FS-Diff, a diffusion-based method that simultaneously fuses and super-resolves low-resolution multimodal image pairs (infrared/visible, medical, multifocus). It introduces clarity-aware semantic guidance from a CLIP-style model and a new 859-pair aerial benchmark, reporting gains on several perceptual metrics over step-by-step baseline pipelines.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Bicubic-only degradation model may not transfer to real long-range shooting; central claim of practical applicability is unverified.","rationale":"The reader's weakest_assumption correctly identifies that the entire pipeline relies on bicubic downsampling as a proxy for real long-range degradation. This is the most load-bearing concern because the paper's central claim explicitly promises applicability to military reconnaissance and long-range detection, where real degradations are far more complex. The zero-shot foggy experiment is a step toward robustness, but it is still synthetic and does not include real sensor/atmospheric effects. The internal inconsistencies noted by the reader (e.g., VIF 0.486 vs 0.737 for the same MSRS x8 condition) are serious reporting issues, but they are secondary in the sense that they affect the trustworthiness of the reported numbers, whereas the degradation assumption affects the fundamental validity of the claimed real-world contribution. Agreeing with the reader's weakest assumption does not change the overall verdict: the paper remains conditionally acceptable, pending both a resolution of the reporting inconsistencies and an evaluation on realistic degradations. The proposed test directly targets the degradation assumption and would settle whether the concern lands.","tokens_in":37007,"tokens_out":7518,"duration_ms":81723,"concrete_test":"Assemble a set of real VI-IR aerial pairs with natural blur/noise (e.g., captured at different altitudes or under haze), downscale them using a realistic PSF-based degradation model instead of bicubic, and run the released FS-Diff model without fine-tuning. Compare VIF, SSIM, LPIPS, and detection mAP against the bicubic-degraded test set. If metrics drop by more than ~10% or qualitative inspection shows artifacts, the bicubic assumption is load-bearing and the practical claim is weakened.","verdict_should_be":"UNCHANGED","load_bearing_attack":"FS-Diff's training and evaluation both synthesize LR inputs by bicubic interpolation (Sec. 5.1: 'LR source image was downsampled from the HR source image via a bicubic approach'; Sec. 4: AVMS pairs 'downsampled via bicubic interpolation'). The paper's motivating applications are long-range UAV imaging where degradation arises from motion blur, atmospheric scattering, and sensor noise, not from bicubic resampling. The only robustness test (Table 6, 'foggy & LR') still uses synthetic fog and does not include a real-capture comparison. Consequently, even if all reported numbers are correct, they demonstrate performance on bicubic-degraded inputs, not on the real clarity-mismatched, long-distance inputs the paper claims to address. The conclusion that FS-Diff 'can recover richer details and semantics' in real scenes is unsupported until evaluated on real degraded pairs or a realistic degradation model. The paper's own limitation section only mentions model complexity, omitting this gap.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"FS-Diff proposes a unified conditional diffusion framework for simultaneous multimodal image fusion and super-resolution. The method initializes the fused high-resolution output as Gaussian noise and iteratively denoises using a modified U-Net conditioned on the low-resolution visible/infrared pair, a bidirectional feature Mamba (BFM) global representation, and semantic embeddings from a clarity-aware CLIP variant (CA-CLIP). A clarity sensing mechanism (CLSE) decides whether to use the clear image's embedding or the maximum cross-image embedding when both inputs are blurred. The paper also introduces an aerial-view multimodal benchmark (AVMS) with 859 aligned pairs, reports fusion and super-resolution results at 2x/4x/8x on public and new datasets, and evaluates detection and segmentation on the fused outputs. The central claim is that FS-Diff outperforms state-of-the-art methods at multiple magnifications while recovering richer details and semantics.","tokens_in":37254,"tokens_out":3622,"duration_ms":41419,"significance":"If the central claim is correct, FS-Diff is a meaningful step toward joint fusion and super-resolution in a single generative model, with a plausible mechanism for clarity-adaptive semantic conditioning. The AVMS dataset is a useful contribution for aerial multimodal fusion, and the code release is a strength. However, the quantitative evidence in the paper does not currently support the claimed superiority, and several metric/table inconsistencies make the evaluation unreliable. The core DDPM formulation is standard; the novelty lies in the BFM and CLSE conditioning. The benchmark dataset and code availability are valuable regardless of the metric issues, but the paper's main performance claims need to be substantiated with corrected and consistent evaluations.","major_comments":[{"comment":"The abstract and conclusion claim that FS-Diff 'outperforms the state-of-the-art methods at multiple magnifications.' The tables do not support a blanket claim. For example, Table 1 (MSRS, both blurred, x8): FS-Diff PSNR is 15.858 vs. DDFM+SR3's 17.407, and MSE is 2632.360 vs. 1770.990; FS-Diff also loses VIF on LLVIP IR-blurred x2 (0.607 vs. CDD+SR3's 0.650, Table 2). In Table 3 (MRI-PET), FS-Diff's MSE is 3194.768, worse than U2Fusion's 1874.329. FS-Diff wins some perceptual metrics (VIF, Q_AB/F, LPIPS) in many rows, but the paper's stated conclusion of general superiority is contradicted by the reported PSNR/MSE results. The text should be revised to report strengths and weaknesses precisely rather than claiming overall superiority.","section":"Abstract; Tables 1, 2, 3, 9"},{"comment":"Reported SSIM values exceed the theoretical maximum of 1.0 in many rows: e.g., Table 1 lists SSIM 1.139, 1.165, 1.277, 1.344, 1.377; Table 3 lists 1.377 and 1.476; Table 9 lists 1.797 and 1.941. Since SSIM is defined as a similarity index bounded above by 1, these values indicate either a nonstandard metric implementation, a computation error, or a mismatch between the metric and the images being compared. This casts doubt on the reliability of every SSIM comparison in the paper and requires correction or a clear explanation.","section":"Tables 1, 2, 3, 9"},{"comment":"The ablation table and the main results table report inconsistent numbers for the same configuration. Table 4 gives FS-Diff on MSRS (both blurred, x8) VIF = 0.737, Q_AB/F = 0.669, PSNR = 16.170; Table 1 for the identical setting gives VIF = 0.486, Q_AB/F = 0.481, PSNR = 15.858. These are materially different. If the ablation was run on a different subset or with a different protocol, that must be stated; otherwise the quantitative results are internally inconsistent and the reader cannot trust either table.","section":"Table 4 vs. Table 1"},{"comment":"The motivating applications are long-distance UAV imaging with motion blur, atmospheric scattering, and sensor noise, but all training and test low-resolution inputs are synthesized by bicubic downsampling (Section 5.1, 'downsampled from the HR source image via a bicubic approach'; Section 4, AVMS pairs 'downsampled via bicubic interpolation'). The only robustness test, Table 6, uses synthetic fog on AVMS. No real-captured degraded pairs or a realistic degradation model are evaluated. The paper's claim that FS-Diff addresses real long-distance imagery is therefore unverified; the limitation paragraph in Section 7 mentions only model complexity and does not acknowledge this gap. A real-capture or realistic-degradation experiment is needed before the practical-applicability claim can be sustained.","section":"Sections 4 and 5.1; Table 6"},{"comment":"The CLSE semantic selection rule in Eq. (12) is validated post hoc in Section 5.4.2 on the A VMS test set that is also used for the final comparisons. Table 5 compares several semantic selections on the same benchmark and then selects 'max semantics' for the both-blurred case. This is benchmark fitting: the rule is chosen based on test-set performance rather than on a held-out validation set. Since CLSE is a core contribution, the design choice should be justified by training/validation evidence or an independent test set, not by the final evaluation set used to report the method's performance.","section":"Section 5.4.2; Eq. (12)"}],"minor_comments":[{"comment":"The abstract says the AVMS benchmark covers '600 pairs of images,' while the full text and contributions state 859 pairs. The number should be harmonized.","section":"Abstract"},{"comment":"The notation for image dimensions is confusing: x ∈ R^{3HW}, y ∈ R^{HW}, F0 ∈ R^{3HW}. This appears to use a nonstandard flattened size rather than H×W×C; please use conventional tensor notation such as R^{H×W×3}.","section":"Section 3.2"},{"comment":"The header includes a typo 'PSRN' and the column order is inconsistent with the row values (some rows show seven numeric entries while the header lists six columns). Please reformat and verify all entries.","section":"Table 5"},{"comment":"Algorithm 2 uses z for the inference noise, while Eq. (18) uses ε_t; it should be stated explicitly that these are the same random variable, or the notation should be unified.","section":"Algorithm 2 and Eq. (18)"},{"comment":"The caption mentions '×8 magnification (162→128²)'; the '162' appears to be a typo for '16²' or should be omitted.","section":"Section 5.3 caption"},{"comment":"The computational complexity table lists 'Diff-IF+SR3', but Diff-IF is not among the methods described in Section 5.1's comparison list. Clarify whether this method was used in the main comparisons and why it appears only here.","section":"Table 10"},{"comment":"The complexity table reports FLOPs for SR3 as 91681.6 G, which is about three orders of magnitude larger than typical SR3 values; this may be a unit error (G vs. M) and should be checked.","section":"Section 6"}],"recommendation":"major_revision","confidential_remarks":"The manuscript has a useful dataset and a coherent framework, but the current quantitative evaluation is not internally consistent (e.g., SSIM > 1, Table 4 vs. Table 1). The performance claims in the abstract and conclusion are not supported by the full tables under this review. I recommend that the editor require the authors to correct the metric computations, reconcile the tables, and either temper the practical claims or add a real-degradation evaluation. These issues are fixable within the scope of the manuscript, so I do not recommend rejection at this stage."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"What you should know first: this paper integrates diffusion with CLIP-style semantic guidance and a bidirectional Mamba to do simultaneous multimodal fusion and super-resolution at 2x/4x/8x, and it introduces a new annotated aerial benchmark (AVMS). The integration is plausible and the benchmark is a useful resource. But the numbers as reported do not hold together, and the evaluation paradigm is narrower than the claims.\n\nThe genuinely new part is the package: clarity-aware semantic selection (CLSE) with CA-CLIP, the BFM block, and a diffusion head trained for joint fusion+SR at high magnification. Individually those ingredients exist elsewhere, but the combination and the 8x joint task are not in the cited prior work. The AVMS benchmark—859 registered VI/IR pairs with 3821 annotations—looks like a real contribution, and the code being public strengthens the work. Visual results in Figures 6 and 11-12 do show generally cleaner outputs than stepwise SR3-then-fuse baselines.\n\nThe soft spots are serious. The same MSRS both-blurred x8 setting reports VIF 0.486 in Table 1 and 0.737 in Table 4—that is a direct internal contradiction. SSIM values above 1.0 (e.g., 1.139, 1.377, 1.476) appear throughout, which is only possible with a nonstandard SSIM variant the paper never discloses. The abstract says the AVMS benchmark covers 600 pairs while the body says 859. These issues matter because the paper's case rests almost entirely on these tables. The CLSE semantic selection rule (Eq. 12) was also chosen after testing variants on the same AVMS test set used for final reporting, so part of the advantage is fitted to the benchmark.\n\nThe stress-test note about degradation is correct. All training and testing synthesize low-resolution inputs via bicubic downsampling (Sec. 4 and 5.1). The motivating applications are long-range UAV and reconnaissance shots, where real blur, atmospheric scattering, and sensor noise are the actual degradation. The foggy robustness test (Table 6) is still synthetic. As the paper stands, it demonstrates fusion+SR on bicubic-downsampled images, not on the real degraded inputs it claims to address. The limitation section only mentions model complexity, not this transfer gap.\n\nWho is this for? Researchers working on joint fusion and super-resolution, and anyone looking for an aerial VI/IR benchmark with detection and segmentation labels. It deserves a serious referee because the architecture and dataset are worth engaging with, but a referee should treat the reported numbers as unverified until the inconsistencies are resolved and the degradation assumption is tested on real or realistically degraded data. My recommendation: send it to peer review, with a clear request to rerun metrics with a documented SSIM implementation, reconcile the tables, and add an evaluation on real or at least more realistic degradations before acceptance.","headline":"A serious architecture for joint fusion+SR, but the empirical claims are currently undermined by inconsistent metric reporting and a degradation setup that doesn't match the paper's own motivating scenario.","tokens_in":37769,"tokens_out":2307,"would_cite":true,"duration_ms":25308,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"FS-Diff unifies multimodal image fusion and super-resolution into a single conditional diffusion process that recovers high-resolution fused images from low-resolution, unevenly blurred source pairs at 2x–8x magnification.","keywords":["image fusion","super-resolution","diffusion model","semantic guidance","clarity-aware CLIP","bidirectional Mamba","multimodal benchmark","aerial imaging"],"falsifier":"Take the trained FS-Diff and run it on real aerial visible-infrared pairs captured from a drone at 5–20 m altitude with natural motion blur and haze, without bicubic preprocessing. If the fusion quality (VIF, LPIPS, and detection mAP on the AVMS annotations) drops much more than on bicubic-degraded AVMS images, the central practical claim fails. Equivalently, if replacing the bicubic training data with a blind degradation model changes the rankings among methods, the reported advantage may be an artifact of the degradation assumption.","tokens_in":36906,"feed_emoji":"🖼️","tokens_out":3361,"duration_ms":35690,"temperature":0.7,"pith_summary":"The paper claims that simultaneous fusion and super-resolution of multimodal images can be done in one generative pass, rather than in separate steps. FS-Diff starts from pure Gaussian noise and iteratively denoises conditioned on the low-resolution source images and semantic embeddings extracted by a clarity-aware CLIP. The key is that when one input is blurred, the model uses the clear image's semantics; when both are blurred, it takes the maximum of both semantics. Experiments on infrared-visible, medical, and multifocus datasets at 2x, 4x, and 8x report better fidelity and downstream detection/segmentation than stepwise super-resolution-then-fusion baselines. The paper also introduces an aerial-view multimodal benchmark (AVMS) with 859 aligned image pairs and 3,821 annotated targets.","feed_headline":"One diffusion pass fuses and upscales blurry multimodal images","feed_subtitle":"FS-Diff's clarity-aware semantics guide denoising, beating stepwise fusion+SR at 2x-8x.","key_machinery":"The load-bearing component is the conditional denoising diffusion model. The target is a high-resolution fused image F0, produced at training time by CDD; during inference the model iterates from Gaussian noise to the fused output using a modified U-Net that predicts the noise at each timestep. Conditions are injected in two ways: a bidirectional feature Mamba (BFM) concatenates the two source images and the noisy image into a global joint representation, and a clarity-aware CLIP (CA-CLIP) provides semantic embeddings via cross-attention. The clarity-sensing selection rule (use the clear image's semantics when one is blurred, take the element-wise maximum when both are blurred) is what adapt","core_discovery":"FS-Diff treats joint fusion and super-resolution as a conditional generation problem. The forward process adds Gaussian noise to a high-resolution ground-truth fused image; the reverse process learns to remove that noise conditioned on the low-resolution source images and on semantic features. The semantic features come from a proposed clarity-sensing CLIP (CA-CLIP) that judges which input is clearer and selects the most informative semantics. A bidirectional feature Mamba block builds a global joint representation of the multimodal inputs. The paper reports that this one-stage approach outperforms two-stage methods (super-resolving then fusing) on VIF, Qabf, SSIM, and LPIPS metrics across m","pith_inferences":["The clarity-sensing selection rule is a form of adaptive fusion that could be generalized: a learned soft weighting of semantics might handle partial blur in both images more gracefully than the hard maximum.","Because the model is trained on bicubic downsampling, its performance under real atmospheric degradation is untested; a natural extension is to train with a degradation pool (motion blur, haze, noise) or use a blind-degradation model.","The diffusion process is computationally heavy (about 74 seconds per 128x128 image) relative to stepwise baselines; distillation or latent-space diffusion would be the natural next step to make this practical.","The AVMS dataset, with per-object annotations, could serve as a testbed for evaluating fusion methods by their effect on detection rather than only pixel-level metrics."],"forward_implications":["A single trained model can replace the standard pipeline of separate super-resolution and fusion, avoiding error accumulation between stages.","Fused outputs with recovered semantics should improve downstream tasks such as object detection and semantic segmentation on low-resolution aerial imagery.","The method extends beyond infrared-visible fusion to medical (MRI-PET, MRI-SPECT) and multifocus fusion, suggesting a general recipe for modality-agnostic joint fusion and super-resolution.","The new AVMS benchmark provides aligned visible-infrared aerial pairs with varying clarity and object annotations, enabling training and evaluation of long-range fusion systems."],"fun_headline_variants":["FS-Diff: one diffusion pass for fusion and super-resolution","Clarity-aware diffusion jointly fuses and upscales multimodal images","Semantic-guided diffusion for joint fusion and super-resolution","Diffusion model fuses and super-resolves images in one go","One diffusion pass for clarity-aware multimodal fusion and super-resolution"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The low-resolution inputs are simulated by bicubic downsampling of sharp high-resolution images, so the model's 'clarity' judgement and denoising may not transfer to real long-range aerial images with motion blur, atmospheric scattering, and sensor noise.","fun_headline_variants_meta":{"raw":{"variants":["FS-Diff: one diffusion pass for fusion and super-resolution","Clarity-aware diffusion jointly fuses and upscales multimodal images","Semantic-guided diffusion for joint fusion and super-resolution","Diffusion model fuses and super-resolves images in one go","One diffusion pass for clarity-aware multimodal fusion and super-resolution"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001601,"raw_usage":{"total_tokens":6253,"prompt_tokens":822,"completion_tokens":5431,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":566,"completion_tokens_details":{"reasoning_tokens":5356}},"tokens_in":566,"tokens_out":5431,"duration_ms":46593,"temperature":1.0,"reasoning_tokens":5356,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T19:05:49.896415+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the trained FS-Diff and run it on real aerial visible-infrared pairs captured from a drone at 5–20 m altitude with natural motion blur and haze, without bicubic preprocessing. If the fusion quality (VIF, LPIPS, and detection mAP on the AVMS annotations) drops much more than on bicubic-degraded AVMS images, the central practical claim fails. Equivalently, if replacing the bicubic training data with a blind degradation model changes the rankings among methods, the reported advantage may be an artifact of the degradation assumption.","supporting_citations":[],"review_version":1}