{"id":"198d35ff-90e3-4796-8ccc-f63264d1082e","arxiv_id":"2506.16730","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"TeSG fuses infrared and visible images by injecting mask-level and text-level semantics extracted from BLIP captions into cross-attention and gated fusion modules, improving downstream detection and segmentation.","lead":"This paper proposes TeSG, an infrared and visible image fusion method that uses text descriptions from a vision-language model to generate a semantic mask and text features that guide fusion. The authors report improved fusion quality and better object detection and segmentation results on standard benchmarks compared with nine existing methods.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The mask-semantics pathway in Eqs. (2)-(3) is underspecified and unvalidated; without an object-localization check of the generated masks, the central claim that textual semantics drives the reported gains is not established.","rationale":"The reader's weakest assumption identifies the same load-bearing point: the DDIM-difference mask generation in Eqs. (2)-(3) is both underspecified and unvalidated. I agree with that assessment. The paper's central contribution depends on the masks being semantically meaningful foreground/background partitions, yet the paper does not state how V* is chosen, what threshold is used for binarization, what diffusion timestep is used, or how the infrared image is converted for the RGB DDIM model. Without those details, the method cannot be reproduced, and without mask-to-ground-truth validation, the claim that textual semantics drive the gains is unsupported. The ablation study also fails to provide strong evidence: the differences in Table 4 are marginal, and two metrics (VIF and Qabf) slightly improve when the MGCA module is removed, so the module's contribution is not convincingly demonstrated. A secondary concern is that the closest text-guided baselines discussed in Sec. 2.2 (TeRF and the VLM-based method of Zhao et al.) are omitted from Tables 1-3, which weakens the 'outperforms SOTA' claim. However, even if those baselines were added, the mask-generation issue would remain the primary load-bearing concern because it undercuts the proposed mechanism itself. My recommendation is therefore to keep the reader's CONDITIONAL verdict: the paper is potentially useful but needs a concrete mask-localization check and reproducible details before the central claim can be accepted.","tokens_in":14412,"tokens_out":5648,"duration_ms":73181,"concrete_test":"Compute the SIG masks on the MSRS test set using the exact procedure from the released code (or settings supplied by the authors), and measure IoU/Dice between the generated mask M and the ground-truth pixel region for the keyword V* (e.g., person or car) using the MSRS segmentation labels. Separately, sweep the binarization threshold and DDIM timestep and re-run the fusion metrics and downstream mAP/mIoU. If the mask IoU is low or the metrics move by more than the ablation gaps in Table 4, the mask-guidance claim is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that mask-level and text-level semantics extracted from VLM descriptions guide fusion and improve downstream tasks. The load-bearing step is the SIG mask construction in Sec. 3.2: Mvis = Delta(Dtheta(Ivis, T) - Dtheta(Ivis, T~)) and M = Mvis union Mir. This assumes (i) a keyword V* can be selected from the BLIP caption and removed without altering other semantics; (ii) the difference between DDIM noise predictions, after normalization and binarization, localizes the region of V*; and (iii) a binarization threshold exists such that the union of visible and infrared masks is a reliable foreground/background partition. None of these is specified: there is no V* selection rule, no threshold value, no DDIM timestep or noise schedule, and no description of how the 1-channel infrared image is fed to an RGB diffusion model. The paper also provides no validation of M against ground-truth object regions, even though MSRS has segmentation labels. Because the Mask-Guided Cross-Attention module consumes M directly, the reported fusion and downstream gains could be driven by an arbitrary spatial weighting rather than by semantic grounding. The ablation evidence in Table 4 is too weak to rule this out: removing MGCA slightly improves VIF and Qabf, and all ablation differences are within a few hundredths. This is a reproducibility and validity gap rather than a demonstrated falsehood, so a conditional verdict is appropriate pending a mask-localization check.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes TeSG, a text-guided infrared and visible image fusion method that injects textual semantics at two levels: a mask level and a text level. A frozen BLIP model produces a caption, from which a keyword V* is removed; a frozen DDIM denoiser is then used to derive a semantic mask by thresholding the difference between noise predictions with and without V*, and the mask is combined with a similarly derived infrared mask. The mask guides a Mask-Guided Cross-Attention module, and the text embedding guides a Text-Driven Attentional Fusion module. Experiments compare TeSG with nine fusion methods on MSRS, RoadScene, and LLVIP, reporting fusion metrics and downstream object detection and semantic segmentation results, plus ablations on LLVIP.","tokens_in":14774,"tokens_out":3403,"duration_ms":36081,"significance":"If the mask-generation mechanism is validated, this is a meaningful contribution to text-guided image fusion: it combines a recent diffusion-based localization idea (DDIM noise difference) with a two-level semantic guidance scheme, and the paper provides a broad experimental comparison including downstream tasks. The authors are careful to use frozen BLIP and DDIM components, and the experimental protocol is extensive. However, the central mechanism that is supposed to make the method semantic rather than a generic spatial weighting is not specified in enough detail to reproduce or verify, and the ablation evidence is too weak to establish that the textual semantics are what drive the reported improvements. The paper would be substantially strengthened by a mask-localization evaluation and by ablations that control for the spatial mask itself.","major_comments":[{"comment":"The mask generation pipeline is underspecified. The paper does not state how the keyword V* is selected from the BLIP caption, what threshold or normalization procedure is used in Δ(·), how many DDIM steps or which noise schedule is used, or how the single-channel infrared image I_ir is fed into a diffusion model that was presumably trained on RGB images. Without these details, Eq. (2) and Eq. (3) are not reproducible, and the reported gains cannot be independently checked.","section":"Sec. 3.2, Eq. (2)-(3)"},{"comment":"There is no validation that the generated mask M actually localizes the semantic objects named in the caption. Since the MSRS dataset has segmentation labels, the authors could compute IoU or Dice between M and the ground-truth foreground regions. This is load-bearing because the Mask-Guided Cross-Attention module consumes M directly; if M is an arbitrary heatmap, the downstream gains could come from a generic spatial reweighting rather than from semantic grounding.","section":"Sec. 3.2 / Sec. 4.4"},{"comment":"The ablation evidence does not support the claim that the mask and text semantics drive the improvements. Removing the MGCA module (row a) improves VIF by 0.006 and Q_AB/F by 0.002 relative to the full model, and all ablation differences are within a few hundredths on most metrics. No error bars or multiple-run statistics are reported. The paper should add ablations with a random mask, a constant mask, and a mask-free variant that still performs spatial attention, so that the specific contribution of semantic grounding can be disentangled from the presence of any spatial weighting.","section":"Table 4, Sec. 4.4"},{"comment":"The downstream detection claims are stronger than the table supports. The text says TeSG is superior under mAP@0.75 and mAP@0.5:0.95, but in Table 2 the CDDFuse row reports 0.881 on the last column while the Ours row reports 0.879, and the mAP lead over CDDFuse is only 0.003. The column alignment also appears inconsistent with the header, which lists six metric columns while each row contains five numeric entries. Please clarify the exact metrics, correct the table, and temper the claims accordingly.","section":"Sec. 4.3, Table 2"}],"minor_comments":[{"comment":"The figure caption refers to \"TeGF (Our Method)\", which does not match the method name TeSG used elsewhere in the paper.","section":"Fig. 1 caption"},{"comment":"The notation for foreground and background features is confusingly typeset: the text uses F^m_v and F^m_v for what should be F^m_v and F^{\\bar m}_v, and similarly for the other terms. Please use distinct superscripts throughout.","section":"Sec. 3.3"},{"comment":"The figure caption mentions \"Reference Text: T\" and \"Query Text: T_Q\", but T_Q is never defined in the text; presumably it denotes the modified caption \\tilde{T} without V*. Please define it.","section":"Fig. 3"},{"comment":"References [1]-[4] appear to be placeholder entries with \"FirstName LastName\" and fixed page ranges; these must be replaced with the actual cited works.","section":"References"},{"comment":"The table header lists six metric columns (mAP, Precision, Recall, @0.50, @0.75, @0.50:0.95) but each row only has five numerical values; please align the table body with the header.","section":"Table 2"},{"comment":"The ablation section reports numerical differences of about 0.001 in some metrics without any measure of variance; at least note whether the differences are stable across training runs or report standard deviations.","section":"Sec. 4.4"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: TeSG is a plausible new combination of mask-level and text-level semantic guidance for infrared-visible image fusion, backed by extensive experiments, but the mask-generation pipeline is underspecified and the ablation evidence is too thin to pin the gains on the text semantics.\n\nWhat's actually new: the dual-level guidance—a mask derived from DDIM noise differences plus text-embedding gated attention—is genuinely new for this subfield, going beyond TextIF, TextFusion, TeRF, and Zhao et al. The paper is also honest that the text and mask are computed from the visible image, so the circularity burden is low, and the evaluation uses independent fusion metrics and downstream detection/segmentation. The experimental scope is a real asset: three datasets, five fusion metrics, YOLOv8 detection, DeepLabV3+ segmentation, and ablations.\n\nThe soft spots are real but addressable. Spots: the mask generation in Sec. 3.2 is load-bearing and underspecified. There is no rule for selecting the keyword V*, no binarization threshold, no DDIM timestep or noise schedule, and no description of how the 1-channel infrared image is fed to an RGB diffusion model. More importantly, the paper never validates that the generated masks actually localize the described objects, even though MSRS has segmentation labels. Without that check, the reported fusion gains could come from arbitrary spatial weighting rather than semantic grounding. The ablations don't help: removing the mask-guided module slightly improves VIF and Qabf, and all ablation differences are within a few hundredths. That is marginal, and the text says the opposite of what the numbers show in one case. Also, the reference list contains obvious placeholders (Alpher, Fotheringham-Smythe, etc.), which suggests the paper was submitted before a final proofread; not a scientific flaw, but it does not inspire confidence.\n\nWho this is for: researchers working specifically on text-guided multi-modal image fusion. The architecture is coherent and the reported numbers are strong if they hold up. It deserves a serious referee rather than a desk reject, but the referee should ask for full mask-generation details and a localization experiment against ground-truth regions. Send it to review with major revision.\n\nRecommendation: engage with it, but say no to accepting the current version as-is.","headline":"New combination of mask-level and text-level guidance for infrared-visible fusion, with solid experiments, but the load-bearing mask-generation step is underspecified and unvalidated.","tokens_in":15238,"tokens_out":2120,"would_cite":false,"duration_ms":23863,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"TeSG shows that dual-level textual semantics—mask and text—guide infrared-visible fusion to better support object detection and segmentation.","keywords":["infrared and visible image fusion","text-guided fusion","vision-language model","semantic mask","cross-attention","object detection","semantic segmentation"],"falsifier":"Computing the intersection-over-union between TeSG's semantic masks and ground-truth object or segment annotations on MSRS would settle the claim: if the masks align poorly (e.g., mIoU below 0.5) while the fusion metrics remain high, then the mask-level guidance is not the source of the reported improvement.","tokens_in":1184,"feed_emoji":"🖼️","tokens_out":3787,"duration_ms":63746,"temperature":0.7,"pith_summary":"This paper proposes TeSG, a fusion method that uses automatically extracted textual descriptions of a scene to guide how infrared and visible images are combined. The claim is that steering the fusion at two levels, a mask level that marks key targets and a text level that filters visual features, produces fused images that better support downstream tasks such as object detection and semantic segmentation. Across three public benchmarks the method reports state-of-the-art or competitive scores on standard fusion metrics, and best results on detection and segmentation. This matters because text guidance is flexible, requires no manual input, and offers a route from low-level pixel fusion toward semantics-aware fusion tuned for high-level vision tasks.","feed_headline":"Text descriptions steer IR-visible fusion to better detection","feed_subtitle":"TeSG fuses IR and visible images with mask and text semantics from a frozen VLM, beating nine methods on MSRS and LLVIP.","key_machinery":"The central mechanism is the Semantic Information Generator (SIG) combined with two attention-based modules. SIG runs a frozen BLIP encoder on the visible image to produce a textual description, removes a keyword, then compares the noise predicted by a frozen DDIM model conditioned on the full text versus the text without the keyword; the normalized and binarized difference defines a semantic mask, and the text encoder supplies text embeddings. The Mask-Guided Cross-Attention (MGCA) module splits both modalities into foreground and background using that mask, then reconstructs each modality's features from the other's via cross-attention on each region. The Text-Driven Attentional Fusion (TDAF) module concatenates the reconstructed features, applies cross-attention with the text embeddings, and uses a sigmoid gate plus learned spatial attention to weight visible versus infrared contributions pixel-wise.","core_discovery":"The central discovery claimed is that textual semantic guidance, applied both as a mask and as a text embedding, yields fused infrared-visible images with richer information, clearer edges, and improved performance on object detection and semantic segmentation compared with nine recent fusion methods. The mask guidance is derived from a frozen diffusion model's noise-difference map that isolates the region associated with a keyword in the BLIP-generated description, while the text guidance uses the full description's embedding to modulate cross-attention and gated fusion. The paper attributes the gains to jointly applying these two levels of guidance rather than to either alone, and its ablations show metric drops when each module is removed.","pith_inferences":["A natural next test is whether the DDIM-noise-difference masks actually align with human-annotated object boundaries; the paper does not report mask IoU, so that validation remains open.","Because text descriptions come from BLIP on the visible image, the guidance inherits visible-light biases; in very dark scenes the description may miss thermal-only targets, a limitation the current experiments may underweight.","The gated fusion formula resembles learnable per-pixel alpha blending conditioned on text; one could measure how often the spatial attention weight deviates from 0.5 to see when text substantially changes fusion decisions."],"forward_implications":["Fusion no longer needs manual text input: BLIP automatically supplies the descriptions, so the pipeline runs end to end on any paired input.","Downstream detection and segmentation improve without retraining the fusion network for a specific task, because guidance is generic text rather than a dedicated task head.","The same dual-level guidance could transfer to other fusion settings—such as medical or remote sensing imagery—where text can name salient structures of interest.","Ablations indicate that both mask and text guidance contribute, and removing either reduces information-based metrics, so future designs should keep both levels active."],"supporting_citations":[{"why":"Supplies the frozen BLIP encoder that generates the textual descriptions feeding both mask and text semantics.","marker":"[26]"},{"why":"Supplies the frozen DDIM diffusion model whose noise predictions, with and without a keyword, define the semantic mask.","marker":"[41]"},{"why":"MSRS is the training dataset and the main evaluation benchmark for fusion metrics and downstream tasks.","marker":"[43]"},{"why":"LLVIP provides low-light paired images used for generalization evaluation and for the ablation study.","marker":"[16]"},{"why":"RoadScene provides another cross-scene evaluation set for assessing generalization of the fusion model.","marker":"[48]"},{"why":"Text-IF is a key text-guided baselines that TeSG compares against and aims to improve on.","marker":"[51]"},{"why":"TextFusion is another text-guided baseline, providing the comparison point for controllable text-driven fusion.","marker":"[8]"},{"why":"CDDFuse is a strong non-text baseline whose reported metrics helps establish the state-of-the-art level TeSG claims to surpass.","marker":"[57]"}],"fun_headline_variants":["Text semantics guide IR-visible fusion for sharper detection","Two-level text guidance improves IR-VIS fusion tasks","VLM semantics steer fusion for detection and segmentation","Mask and text semantics boost IR-visible fusion","Text-guided fusion sharpens downstream vision tasks"],"cache_read_input_tokens":17408,"weakest_assumption_plain":"The method assumes that comparing a frozen diffusion model's noise predictions with and without a keyword cleanly isolates that object's region, and that the union of the two masks reliably separates foreground from background.","fun_headline_variants_meta":{"raw":{"variants":["Text semantics guide IR-visible fusion for sharper detection","Two-level text guidance improves IR-VIS fusion tasks","VLM semantics steer fusion for detection and segmentation","Mask and text semantics boost IR-visible fusion","Text-guided fusion sharpens downstream vision tasks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000541,"raw_usage":{"total_tokens":2580,"prompt_tokens":918,"completion_tokens":1662,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":534,"completion_tokens_details":{"reasoning_tokens":1590}},"tokens_in":534,"tokens_out":1662,"duration_ms":11863,"temperature":1.0,"reasoning_tokens":1590,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T23:37:26.330813+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Computing the intersection-over-union between TeSG's semantic masks and ground-truth object or segment annotations on MSRS would settle the claim: if the masks align poorly (e.g., mIoU below 0.5) while the fusion metrics remain high, then the mask-level guidance is not the source of the reported improvement.","supporting_citations":[{"cited_title":"Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation","cited_arxiv_id":null,"evidence_quote":"Supplies the frozen BLIP encoder that generates the textual descriptions feeding both mask and text semantics."},{"cited_title":"Piafusion: A progressive infrared and visible im- age fusion network based on illumination aware.Information Fusion, 83:79–92, 2022","cited_arxiv_id":null,"evidence_quote":"MSRS is the training dataset and the main evaluation benchmark for fusion metrics and downstream tasks."},{"cited_title":"Llvip: A visible-infrared paired dataset for low-light vision","cited_arxiv_id":null,"evidence_quote":"LLVIP provides low-light paired images used for generalization evaluation and for the ablation study."},{"cited_title":"U2fusion: A unified unsupervised image fusion net- work.IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(1):502–518, 2020","cited_arxiv_id":null,"evidence_quote":"RoadScene provides another cross-scene evaluation set for assessing generalization of the fusion model."},{"cited_title":"Text-if: Leveraging semantic text guidance for degradation-aware and interactive image fusion","cited_arxiv_id":null,"evidence_quote":"Text-IF is a key text-guided baselines that TeSG compares against and aims to improve on."},{"cited_title":"Textfusion: Unveiling the power of textual semantics for controllable image fusion","cited_arxiv_id":null,"evidence_quote":"TextFusion is another text-guided baseline, providing the comparison point for controllable text-driven fusion."},{"cited_title":"Cddfuse: Correlation-driven dual-branch feature decompo- sition for multi-modality image fusion","cited_arxiv_id":null,"evidence_quote":"CDDFuse is a strong non-text baseline whose reported metrics helps establish the state-of-the-art level TeSG claims to surpass."}],"review_version":1}