Pith. sign in

REVIEW 5 cited by

Image Fusion via Vision-Language Model

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2402.02235 v2 pith:2HV3INVJ submitted 2024-02-03 cs.CV

Zixiang Zhao , Lilun Deng , Haowen Bai , Yukun Cui , Zhipeng Zhang , Yulun Zhang , Haotong Qin , Dongdong Chen
show 3 more authors
Jiangshe Zhang Peng Wang Luc Van Gool
This is my paper
classification cs.CV
keywords fusionimageinformationsemantictextualvision-languagedescriptionsfilm
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Image fusion integrates essential information from multiple images into a single composite, enhancing structures, textures, and refining imperfections. Existing methods predominantly focus on pixel-level and semantic visual features for recognition, but often overlook the deeper text-level semantic information beyond vision. Therefore, we introduce a novel fusion paradigm named image Fusion via vIsion-Language Model (FILM), for the first time, utilizing explicit textual information from source images to guide the fusion process. Specifically, FILM generates semantic prompts from images and inputs them into ChatGPT for comprehensive textual descriptions. These descriptions are fused within the textual domain and guide the visual information fusion, enhancing feature extraction and contextual understanding, directed by textual semantic information via cross-attention. FILM has shown promising results in four image fusion tasks: infrared-visible, medical, multi-exposure, and multi-focus image fusion. We also propose a vision-language dataset containing ChatGPT-generated paragraph descriptions for the eight image fusion datasets across four fusion tasks, facilitating future research in vision-language model-based image fusion. Code and dataset are available at https://github.com/Zhaozixiang1228/IF-FILM.

Discussion (0). Sign in to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. MPerS: Dynamic MLLM MixExperts Perception-Guided Remote Sensing Scene Segmentation

    cs.CV 2026-05 unverdicted novelty 6.0 of 10

    MPerS dynamically mixes semantic guidance from MLLM-generated RS captions with DINOv3 features via MixExperts and Linguistic Query Guided Attention to achieve superior semantic segmentation on three public remote sens...

  2. Coarse-to-fine Framework for Generative MEF via Implicit Neural Representation

    cs.CV 2026-07 conditional novelty 5.0 of 10

    A coarse-to-fine multi-exposure fusion method that fuses low-res diffusion output with implicit-neural high-res detail reconstruction, achieving ~3.5x speedup over a diffusion-only baseline.

  3. Adding Thermal Awareness to Visual Systems in Real-Time via Distilled Diffusion Models

    cs.CV 2026-05 unverdicted novelty 5.0 of 10

    FusionProxy is a distilled diffusion-based fusion module that adds thermal awareness to RGB vision systems in real time as an independent plug-and-play component.

  4. CNN-ViT Fusion with Adaptive Attention Gate for Brain Tumor MRI Classification: A Hybrid Deep Learning Model

    cs.CV 2026-04 unverdicted novelty 5.0 of 10

    Hybrid CNN-ViT with adaptive attention gate achieves 97.6% accuracy on brain tumor MRI classification, outperforming baselines.

  5. FMRFusion: Frequency-Aware Multi-View Representation Learning for Heterogeneous Image Fusion

    cs.CV 2026-06 unverdicted novelty 3.0 of 10

    The paper introduces FMRFusion, a frequency-aware multi-view representation learning network for infrared-visible image fusion that claims superior performance on benchmarks especially at night.

Pith tools