The reviewed record of science sign in
Pith

arxiv: 2607.06552 · v1 · pith:ZRBXEEZS · submitted 2026-07-07 · cs.CV

MonoIR-RS: Infrared Remote Sensing Vision-Language Learning with CLIP and VLM Adaptation

Jiaju Han , Ma Yaqi , Yahui Chai , Xuemeng Sun , Xin Li , Qike Zhang , Yingying Zhao , Xiang Chen
show 3 more authors
Luwei Yang Chengyin Hu Jiahuan Long
This is my paper

Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel 2026-07-08 01:51 UTCglm-5.2pith:ZRBXEEZSrecord.jsonopen to challenge →

classification cs.CV
keywords infraredimagerymonoir-rsremote-sensingvision-languageadaptationclipir-aware
0
0 comments X

The pith

Rewriting captions for infrared evidence lifts CLIP retrieval by up to 12.8 points

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper argues that the core bottleneck in infrared remote-sensing vision-language learning is not just a visual domain shift but a supervision shift: captions written for visible-band images describe color and surface appearance that infrared imagery cannot support. The authors construct MonoIR-RS, a dataset of 600,000 synthetic infrared images paired with 59,032 IR-aware captions that rewrite supervision around grayscale structure, intensity contrast, and object-background separation rather than RGB appearance. They then fine-tune five CLIP backbones and six VLM backbones under a train-only protocol, keeping the infrared image as the sole model-facing modality. The central mechanism is the IR-aware caption: by replacing color-centric language with descriptions of what infrared evidence actually shows, the text side of the image-text alignment problem becomes consistent with the visual input. The authors show that this rewrite is load-bearing — in a development ablation, switching from original captions to IR-aware captions raises CLIP retrieval mean recall from 30.2% to 43.0% averaged across five backbones. On the formal filtered test split, IR-aware adaptation yields +3.5 to +12.8 mean-recall gains over zero-shot baselines, with the best model reaching 19.2%. For VLMs, fine-tuning drives the rate of infrared-vocabulary usage in captioning to 100% across all six backbones while reducing visible-color word leakage to zero. The paper also validates that the synthetic infrared images produced by DiffV2IR are closer to real thermal imagery than simple grayscale conversion under FID and histogram-distance metrics on the AVIID benchmark, and reports a small auxiliary transfer check showing the fine-tuned CLIP improves paired retrieval with real infrared queries.

Core claim

The paper's central finding is that infrared vision-language alignment fails when captions describe RGB evidence the model cannot see, and succeeds when captions are rewritten to match the infrared modality's actual visual evidence. This supervision shift — not just the pixel-level domain shift — is what the authors identify as the decisive factor. The ablation showing IR-aware captions nearly halve the gap to a joint RGB-IR diagnostic while outperforming original captions by 12.8 mean-recall points on development retrieval is the cleanest evidence for this claim. The paper also establishes that synthetic infrared imagery from a diffusion model carries enough modality-specific structure to支持

What carries the argument

The IR-aware caption — a text description rewritten from visible-centric source captions to describe grayscale intensity patterns, thermal contrast, object-background separation, and scene layout instead of color — is the central object carrying the argument. The DiffV2IR diffusion model generates the synthetic infrared images, and the Qwen2.5-VL-72B model performs the caption rewriting. The evaluation protocol separates CLIP retrieval (bidirectional image-text matching on a filtered 9,720-image test split) from VLM lexical diagnostics (keyword-match rates for infrared terms, color terms, overclaim terms, class tokens, and response length).

If this is right

  • If supervision-text rewriting is the decisive lever, then any modality with a systematic mismatch between visual evidence and caption vocabulary — synthetic aperture radar, hyperspectral, depth maps — could benefit from the same IR-aware-caption approach, not just infrared.
  • The finding that RGB-pretrained remote-sensing models (RemoteCLIP) transfer poorly to infrared even after fine-tuning suggests that domain-specific RGB pretraining can encode color shortcuts that actively interfere with non-visible-band adaptation.
  • The modest absolute retrieval scores (best 19.2% mean recall) indicate that current CLIP-scale vision-language models still lack the visual capacity to discriminate many remote-sensing scenes from infrared evidence alone, leaving substantial headroom for architectures designed around intensity-structure rather than color.
  • The partially circular VLM evaluation — training on captions written to contain infrared vocabulary, then measuring whether outputs contain that vocabulary — means the 100% IR-cue rate may overstate genuine infrared grounding, and future work will need semantic correctness benchmarks to separate vocabulary adoption from visual evidence understanding.

Load-bearing premise

The VLM evaluation assumes that lexical keyword matching — counting whether model outputs contain infrared vocabulary terms like 'grayscale,' 'intensity,' 'contrast,' 'thermal' — is a meaningful proxy for infrared grounding quality. Because the training captions are specifically written to contain these same vocabulary terms, the 100% IR-cue rate may largely reflect that models learned to reproduce the vocabulary style of their training data rather than correctly grounding红外

What would settle it

If IR-aware captions produced no retrieval gain over original captions in the ablation, or if the lexical diagnostics showed no difference between fine-tuned and zero-shot VLMs, the central claim that supervision rewriting is the decisive lever would be unsupported.

Figures

Figures reproduced from arXiv: 2607.06552 by Chengyin Hu, Jiahuan Long, Jiaju Han, Luwei Yang, Ma Yaqi, Qike Zhang, Xiang Chen, Xin Li, Xuemeng Sun, Yahui Chai, Yingying Zhao.

Figure 1
Figure 1. Figure 1: Overview of the MonoIR-RS construction workflow. The shared FusionRS source pool is converted into an infrared image-text corpus, IR-aware text, and evalu￾ation tasks covering retrieval, VLM understanding, and dataset-quality checks. visible-band imagery, where discriminative evidence includes object-background separation and grayscale response rather than color. This makes infrared imagery valuable for ni… view at source ↗
Figure 2
Figure 2. Figure 2: Representative MonoIR-RS task formats derived from infrared remote-sensing supervision, including VLM understanding, CLIP retrieval, and visual-evidence QA with paired prompts and target responses. 3.3 Split Hygiene and Quality Control We constrain all fine-tuning data to the train split; validation and test images are excluded before CLIP or VLM training, and empty or non-train entries are treated as inva… view at source ↗
Figure 3
Figure 3. Figure 3: Two-stage model adaptation used for MonoIR-RS. The left branch fine-tunes CLIP-style encoders with infrared image-text contrastive alignment, while the right branch adapts VLM backbones with infrared visual tokens, a projector, and LoRA￾based instruction tuning. 4.2 Training Data Interface The same infrared source boundary is exposed to the two model families through different interfaces. For CLIP, each re… view at source ↗
Figure 4
Figure 4. Figure 4: Prompt card for IR-aware caption generation. C.3 CLIP Fine-Tuning Hyperparameters All IR-CLIP backbones are fine-tuned with a frozen text encoder, AdamW, weight decay 0.1, gradient clipping 1.0, fp16 mixed precision, and a linear￾warmup (20 steps) plus cosine-decay schedule. The input resolution follows each backbone’s native CLIP processor (2242 ). Per-backbone settings are listed in [PITH_FULL_IMAGE:fig… view at source ↗
read the original abstract

Infrared remote-sensing imagery captures intensity structure, object-background contrast, and illumination-invariant cues often invisible in RGB imagery. Yet, most remote-sensing vision-language resources and models focus on visible-band semantics, leaving infrared vision-language understanding underexplored. We introduce MonoIR-RS, a large-scale infrared remote-sensing vision-language dataset and benchmark that couples IR-aware data construction with CLIP-style contrastive adaptation and VLM instruction tuning. Built from the same source pool and split as FusionRS, MonoIR-RS retains the infrared image as the model-facing modality, yielding 600,000 synthesized infrared images and 59,032 retained IR-aware caption records. The model experiments use this retained language-supervision subset, whose captions rewrite supervision around grayscale structure and infrared-style contrast instead of RGB appearance. We show that the synthesized infrared imagery is markedly closer to real thermal imagery than a grayscale conversion on the AVIID benchmark. We fine-tune five CLIP backbones and six VLM backbones, and calibrate them against zero-shot behavior: IR-aware adaptation lifts CLIP mean recall by up to 12.8 points and drives VLM captioning IR-cue coverage to 100% while reducing residual RGB-color leakage to near zero. By isolating the infrared modality from RGB-IR dual-modal learning, MonoIR-RS offers a controlled, reproducible testbed for aligning infrared remote-sensing evidence with language.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 9 minor

Summary. The paper introduces MonoIR-RS, a large-scale infrared remote-sensing vision-language dataset and benchmark. The dataset comprises 600,000 synthetic infrared images generated from visible-band remote-sensing sources via DiffV2IR, paired with 59,032 IR-aware caption records rewritten by Qwen2.5-VL-72B to emphasize grayscale structure and thermal contrast rather than RGB appearance. The authors fine-tune five CLIP backbones for contrastive retrieval and six VLM backbones for instruction-following, evaluating against zero-shot baselines. The CLIP retrieval results show consistent gains (+3.5 to +12.8 mR) across all backbones, with a seed-stability check on one backbone. The VLM results are evaluated via lexical proxy diagnostics (IR-cue rate, RGB-color rate, overclaim rate, class-hit rate). The paper also validates synthetic IR realism against the AVIID benchmark (FID 85.2 vs 126.3 for grayscale baseline) and includes an auxiliary paired-retrieval transfer check on AVIID.

Significance. The paper addresses a genuine gap: infrared remote-sensing vision-language resources are scarce, and existing RGB-centric captions are inappropriate for the infrared modality. The CLIP retrieval evaluation is the strongest contribution: it uses a genuine retrieval task with zero-shot baselines providing a non-circular comparison point, consistent gains across five heterogeneous backbones, and a seed-stability check (11.88±0.02 on ViT-B/32). The synthetic-IR realism check against AVIID (Table 2) is reasonable and the auxiliary transfer sanity check (Table 14, +6.5 mR on real-IR queries) provides supporting evidence. The dataset construction pipeline—synthetic IR generation, IR-aware caption rewriting, split hygiene, and RGB-leakage auditing—is reproducible and documented with algorithmic pseudocode and hyperparameter tables. The release package including images, captions, split files, and evaluation scripts is a positive signal for reproducibility.

major comments (3)
  1. §5.3, Eqs. (33)–35, Fig. 4, Table 9: The VLM captioning evaluation is substantially circular. The IR-aware caption generation prompt (Fig. 4) explicitly instructs the model to use terms like 'grayscale intensity,' 'bright/dark intensity responses,' 'low-texture regions,' 'structural outlines,' and 'contrast.' The evaluation keyword set K_IR (Appendix A.7) checks for {infrared, thermal, grayscale, intensity, contrast, texture}. These vocabulary lists overlap almost entirely. The 100% IR-cue rate for captioning (Table 9) is thus nearly guaranteed by construction: the model is trained to produce specific terms and then evaluated on whether it produces those same terms. The paper acknowledges this ('a high IR-cue rate indicates infrared-style phrasing rather than verified grounding,' §5.3), but the abstract and conclusion still prominently feature the 100% IR-cue claim as a central result. I
  2. §5.3, Table 9, Table 16: The only metric that could indicate genuine semantic understanding—class-hit rate—is near-zero across all tasks and backbones (0.4–1.2% for captioning, 0.2–0.6% for scene questions, 0.4–1.0% for object presence, Table 16). The paper dismisses this as 'conservative by design' (§5.3), but the consequence is that the VLM results provide no positive evidence of correct scene or object recognition in IR images. The VLM claims thus rest entirely on lexical style adoption, not infrared grounding. The paper should either (a) add a semantic evaluation with human-verified labels on a subset, or (b) substantially reframe the VLM contribution as a style-transfer diagnostic rather than an infrared understanding result. As it stands, the VLM half of the paper's central claim is not independently supported.
  3. §4.2, Table 5: The CLIP fine-tuning uses different batch sizes across backbones (768 for OpenAI B/32, 128 for OpenAI L/14, 1024 for OpenCLIP/RemoteCLIP/GeoRSCLIP) and OpenAI ViT-L/14 receives an additional stage-2 resume (6 epochs, lr 5e-7). This makes cross-backbone comparisons in Table 6 not fully controlled: the strongest model (ViT-L/14, 19.2% mR) also has the most favorable training schedule. The paper should acknowledge this confound when interpreting the relative ranking, or ideally re-run with matched effective batch sizes.
minor comments (9)
  1. Abstract: 'drives VLM captioning IR-cue coverage to 100%' is technically accurate but misleading given the circularity discussed above. Consider qualifying this claim in the abstract.
  2. Table 1: The 'Scale' column mixes units (15K pairs, 1.2K images, 5M pairs). Standardizing to a consistent format would improve readability.
  3. §3.1: The claim that DiffV2IR 'attains strong fidelity (low FID/PSNR/SSIM against real thermal benchmarks)' cites [25] but the paper itself only reports FID and histogram metrics (Table 2). PSNR/SSIM values are not reported in this manuscript.
  4. Table 8: The supervision ablation uses a 'smaller development retrieval split' that is 'not directly comparable to the formal 9,720-sample test split.' The absolute numbers (35.1, 30.2, 43.0) are therefore not interpretable alongside Table 6. Consider adding the development-split zero-shot baseline for context.
  5. §5.2: 'This gap shows that the reported scores reflect genuine infrared adaptation rather than residual visible-domain priors' is slightly overstated. The zero-shot baseline uses original pretrained weights on synthetic IR images; the gain could also reflect adaptation to the synthetic IR domain rather than to infrared evidence per se. The AVIID transfer check partially addresses this but is a small-scale paired retrieval.
  6. Table 7 caption: 'Small pools (RSICD, RSITMD) inflate recall and are not directly comparable across sources' — this is a good caveat but could be stated more prominently, perhaps in the main text rather than only the caption.
  7. Appendix C.5: The DiffV2IR generation uses seed 1234 for all images. It would be useful to note whether any qualitative or quantitative sensitivity to seed was checked.
  8. Fig. 1: The workflow diagram is dense and some text is difficult to read. Consider simplifying or enlarging.
  9. References: Several 2025–2026 references are cited; ensure these are final published versions rather than arXiv preprints where published versions exist.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for a careful and constructive report. The referee identifies three substantive issues: (1) circularity in the VLM captioning evaluation due to vocabulary overlap between the generation prompt and the IR-cue keyword set, (2) near-zero class-hit rates undermining the VLM semantic-understanding claim, and (3) uncontrolled batch sizes and an extra training stage for ViT-L/14 confounding cross-backbone CLIP comparisons. We agree with all three points in substance. For (1), we will reframe the 100% IR-cue claim as a style-adoption result rather than evidence of infrared grounding, and adjust the abstract and conclusion accordingly. For (2), we will add a human-verified semantic evaluation on a subset and reframe the VLM contribution as a style-transfer diagnostic with a limited semantic check. For (3), we will add an explicit confound acknowledgment and, if compute permits, re-run with matched effective batch sizes. No standing objections remain.

read point-by-point responses

  1. Referee: §5.3, Eqs. (33)–35, Fig. 4, Table 9: The VLM captioning evaluation is substantially circular. The IR-aware caption generation prompt (Fig. 4) explicitly instructs the model to use terms like 'grayscale intensity,' 'bright/dark intensity responses,' 'low-texture regions,' 'structural outlines,' and 'contrast.' The evaluation keyword set K_IR (Appendix A.7) checks for {infrared, thermal, grayscale, intensity, contrast, texture}. These vocabulary lists overlap almost entirely. The 100% IR-cue rate for captioning (Table 9) is thus nearly guaranteed by construction: the model is trained to produce specific terms and then evaluated on whether it produces those same terms. The paper acknowledges this ('a high IR-cue rate indicates infrared-style phrasing rather than verified grounding,' §5.3), but the abstract and conclusion still prominently feature the 100% IR-cue claim as a central result.

    Authors: The referee is correct that the vocabulary overlap between the caption generation prompt and the K_IR keyword set makes the 100% IR-cue rate largely guaranteed by construction for the captioning task. We acknowledge that the abstract and conclusion currently present this number as a central result in a way that overstates its evidential value. We will revise the manuscript as follows: (1) The abstract will reframe the 100% IR-cue claim as evidence of style adoption—i.e., that IR-aware fine-tuning reliably shifts VLM output language from RGB-centric to infrared-style phrasing—rather than as evidence of infrared understanding. (2) The conclusion will be revised to match. (3) Section 5.3 will add an explicit statement that the captioning IR-cue rate is near-tautological given the prompt–keyword overlap, and that the meaningful diagnostic signal lies in the zero-shot comparison (Table 11), where IR-cue rates rise from as low as 4.0% to 100% and RGB-color leakage drops to zero across all six backbones. The zero-shot comparison demonstrates that the fine-tuning, not the base model, is responsible for the style shift, which is a non-trivial behavioral result even if it does not establish semantic grounding. We agree this should not be presented as a central understanding result. revision: yes

  2. Referee: §5.3, Table 9, Table 16: The only metric that could indicate genuine semantic understanding—class-hit rate—is near-zero across all tasks and backbones (0.4–1.2% for captioning, 0.2–0.6% for scene questions, 0.4–1.0% for object presence, Table 16). The paper dismisses this as 'conservative by design' (§5.3), but the consequence is that the VLM results provide no positive evidence of correct scene or object recognition in IR images. The VLM claims thus rest entirely on lexical style adoption, not infrared grounding. The paper should either (a) add a semantic evaluation with human-verified labels on a subset, or (b) substantially reframe the VLM contribution as a style-transfer diagnostic rather than an infrared understanding result. As it stands, the VLM half of the paper's central claim is not independently supported.

    Authors: We agree that the near-zero class-hit rates mean the VLM results currently provide no positive evidence of semantic scene or object recognition in IR images. The paper's existing framing of class-hit as 'conservative by design' is accurate but insufficient—it does not address the referee's core point that the VLM contribution lacks an independent semantic evaluation. We will take both actions the referee suggests: (a) We will add a human-verified semantic evaluation on a subset of 200–300 held-out infrared images, where annotators judge whether VLM-generated captions and scene classifications are semantically correct (not just lexically IR-styled). This will be reported as a new table in the revised manuscript. (b) Pending the outcome of that evaluation, we will substantially reframe the VLM contribution as a style-transfer diagnostic with a limited semantic check, rather than claiming infrared understanding. The abstract and conclusion will be revised to match: the VLM results will be described as demonstrating that IR-aware fine-tuning shifts model language toward infrared-style evidence and eliminates RGB-color leakage, with semantic grounding assessed only on the annotated subset. We agree that, as currently written, the VLM half of the paper's central claim is not independently supported by the lexical diagnostics alone. revision: partial

  3. Referee: §4.2, Table 5: The CLIP fine-tuning uses different batch sizes across backbones (768 for OpenAI B/32, 128 for OpenAI L/14, 1024 for OpenCLIP/RemoteCLIP/GeoRSCLIP) and OpenAI ViT-L/14 receives an additional stage-2 resume (6 epochs, lr 5e-7). This makes cross-backbone comparisons in Table 6 not fully controlled: the strongest model (ViT-L/14, 19.2% mR) also has the most favorable training schedule. The paper should acknowledge this confound when interpreting the relative ranking, or ideally re-run with matched effective batch sizes.

    Authors: The referee is correct that the differing batch sizes and the additional stage-2 resume for ViT-L/14 introduce a confound in the cross-backbone comparison in Table 6. The strongest model (ViT-L/14, 19.2% mR) does have the most favorable training schedule, and the current manuscript does not acknowledge this confound when interpreting the relative ranking. We will add an explicit acknowledgment in Section 5.2 that the ViT-L/14 result benefits from both a larger model capacity and an extended training schedule, and that the cross-backbone ranking should therefore not be interpreted as a controlled comparison of backbone quality under matched budgets. Regarding the suggestion to re-run with matched effective batch sizes: we will attempt this for at least the two strongest backbones (ViT-L/14 and GeoRSCLIP ViT-B/32) under a common batch size, and report the results as a supplementary table. However, we note that memory constraints on a single GPU (as specified in our setup) may limit the feasible batch size for ViT-L/14, so a fully matched comparison may require gradient accumulation tuning that we cannot guarantee will be ready for the next revision. At minimum, the confound will be explicitly stated in the main text. revision: partial

Circularity Check

1 steps flagged

VLM captioning IR-cue rate is partially circular: training captions are generated to contain infrared vocabulary, then evaluation checks for the same vocabulary. CLIP retrieval results are independently grounded.

specific steps
  1. fitted input called prediction [Sec. 5.3, Eqs. 33-35; Fig. 4 (IR-aware caption generation prompt); Table 9]
    "IR-cue rate is the fraction of answers with an infrared term (e.g. infrared, thermal, grayscale, intensity, contrast, texture)... Explicitly describe infrared-style visual cues, such as grayscale intensity, bright/dark intensity responses, low-texture regions, structural outlines, and contrast."

    The IR-aware caption generation prompt (Fig. 4) instructs the LLM to 'explicitly describe infrared-style visual cues, such as grayscale intensity, bright/dark intensity responses, low-texture regions, structural outlines, and contrast.' The evaluation metric IR-cue rate (Eq. 33) checks whether generated answers contain keywords from K_IR = {infrared, thermal, grayscale, intensity, contrast, texture}. These are essentially the same vocabulary sets. VLMs are trained on captions written to contain these terms, then evaluated on whether their outputs contain these terms. The 100% IR-cue rate for captioning (Table 9, Table 11) is thus largely guaranteed by construction: the model learned to reproduce the vocabulary style of its training data. The paper acknowledges this: 'a high IR-cue rate is,

full rationale

The paper has two main evaluation tracks. The CLIP retrieval track (Table 6) is independently grounded: the model must match images to text among 9,720 candidates, zero-shot baselines provide non-circular comparison points, and the +12.8 mean-recall gain reflects genuine adaptation. The VLM captioning track has a partially circular component: IR-aware captions are generated to contain infrared vocabulary (grayscale, intensity, contrast, thermal), and then VLM outputs are scored on whether they contain that same vocabulary (Eqs. 33-35). The 100% IR-cue rate for captioning is thus largely forced by construction. However, the paper is notably transparent about this limitation, explicitly stating 'a high IR-cue rate indicates infrared-style phrasing rather than verified grounding' and that diagnostics are 'lexical proxies rather than human-verified correctness.' The only semantic metric (class-hit rate) is near-zero across all tasks (0.4-1.2%), which the paper acknowledges as 'conservative by design.' The circularity is real but partial and self-disclosed, not hidden. The central CLIP retrieval claim retains independent content. Score 4 reflects: one metric partially circular by construction, but the paper's main contributions (dataset construction, CLIP adaptation gains, zero-shot calibration) have independent grounding.

Axiom & Free-Parameter Ledger

4 free parameters · 3 axioms · 1 invented entities

The paper introduces no new physical entities or forces. The main 'invented entity' is the dataset itself, which is a constructed artifact rather than a postulated phenomenon. The key assumptions are about the fidelity of synthetic IR and the validity of lexical proxies.

free parameters (4)
  • DiffV2IR generation settings = text scale 7.5, image/seg scales 1.5, 20 steps, Euler-ancestral, seed 1234
    Fixed generation hyperparameters for synthetic IR; not fitted to downstream task performance but chosen from DiffV2IR defaults.
  • CLIP learning rates = 2e-6 (B/32 family), 1e-6 (L/14), 5e-7 (stage-2 resume)
    Per-backbone learning rates chosen for training stability; standard fine-tuning values.
  • VLM LoRA parameters = r=32, alpha=64, dropout=0.05, lr=2e-4
    Standard LoRA configuration applied uniformly across all six VLM backbones.
  • API-v3 multitask data size = 32,000 conversations (30K text QA + 2K vision-audited)
    Selected from development ablation (Table 10) as the best balance of IR-cue coverage and task diversity.
axioms (3)
  • domain assumption DiffV2IR synthetic infrared captures meaningful infrared-style visual structure beyond simple desaturation
    Sec. 3.1: The paper treats DiffV2IR outputs as a 'controlled, reproducible infrared testbed' rather than radiometrically calibrated measurements. This is partially validated against AVIID (Table 2) but remains an assumption for the broader benchmark.
  • ad hoc to paper Lexical keyword matching is a meaningful proxy for infrared grounding quality in VLM outputs
    Sec. 5.3 and Appendix A.7: VLM evaluation uses keyword sets K_IR, K_RGB, K_over to measure behavior. The paper acknowledges this is not semantic correctness but uses it as the primary VLM evidence.
  • domain assumption Qwen2.5-VL-72B produces reliable IR-aware captions that preserve scene semantics
    Sec. 3.2: IR-aware captions are generated by Qwen2.5-VL-72B-Instruct from source text and infrared evidence. No human verification of caption quality is performed; filtering is automatic.
invented entities (1)
  • MonoIR-RS dataset independent evidence
    purpose: Large-scale synthetic infrared remote-sensing vision-language dataset and benchmark
    The dataset is validated against the external AVIID benchmark (Table 2) and includes split-integrity checks. However, the infrared modality is synthetic, not sensor-captured.

pith-pipeline@v1.1.0-glm · 28897 in / 4164 out tokens · 325991 ms · 2026-07-08T01:51:42.973010+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

36 extracted references · 36 canonical work pages · 1 internal anchor

  1. [1]

    In: Proceedings of the Com- puter Vision and Pattern Recognition Conference

    Astruc, G., Gonthier, N., Mallet, C., Landrieu, L.: Anysat: One earth observation model for many resolutions, scales, and modalities. In: Proceedings of the Com- puter Vision and Pattern Recognition Conference. pp. 19530–19540 (2025)

    work page 2025

  2. [2]

    Bai, S., Chen, K., Liu, X., Wang, J., Ge, W., Song, S., Dang, K., Wang, P., Wang, S., Tang, J., Zhong, H., Zhu, Y., Yang, M., Wan, J., Ding, W., Fu, Z., Xu, Y., Ye, J., Zhang, X., Xie, T., Cheng, Z., Zhang, H., Yang, Z., Xu, H., Lin, J.: Qwen2.5-vl technical report (2025)

    work page 2025

  3. [3]

    In: Proceedings of the 63rd Annual Meeting of the Asso- ciation for Computational Linguistics (Volume 1: Long Papers)

    Bi, J., Wang, Y., Chen, H., Xiao, X., Hecker, A., Tresp, V., Ma, Y.: Llava steer- ing: Visual instruction tuning with 500x fewer parameters through modality linear representation-steering. In: Proceedings of the 63rd Annual Meeting of the Asso- ciation for Computational Linguistics (Volume 1: Long Papers). pp. 15230–15250 (2025)

    work page 2025

  4. [4]

    Sensors26(2), 390 (2026)

    Cao, J., Liu, X., Xue, R.: Firemm-ir: An infrared-enhanced multi-modal large lan- guage model for comprehensive scene understanding in remote sensing forest fire monitoring. Sensors26(2), 390 (2026)

    work page 2026

  5. [5]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Cao, Q., Xu, Z., Chen, Y., Ma, C., Yang, X.: Domain prompt learning with quater- nion networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 26637–26646 (2024)

    work page 2024

  6. [6]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision

    Cao, Z., Zhang, J., Zhang, R.: Irgpt: Understanding real-world infrared image with bi-cross-modal curriculum on large-scale benchmark. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 166–176 (2025)

    work page 2025

  7. [7]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2025)

    Chen, W., Chen, J., Deng, Y., Chen, J., Feng, Y., Xi, Z., Liu, D., Li, K., Meng, Y.: Lrsclip: A vision-language foundation model for aligning remote sensing image with longer text. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2025)

    work page 2025

  8. [8]

    Proceedings of the IEEE105(10), 1865–1883 (2017)

    Cheng, G., Han, J., Lu, X.: Remote sensing image scene classification: Benchmark and state of the art. Proceedings of the IEEE105(10), 1865–1883 (2017)

    work page 2017

  9. [9]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision

    Danish, M., Munir, M.A., Shah, S.R.A., Kuckreja, K., Khan, F.S., Fraccaro, P., Lacoste, A., Khan, S.: Geobench-vlm: Benchmarking vision-language models for geospatial tasks. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 7132–7142 (2025)

    work page 2025

  10. [10]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Guo, X., Lao, J., Dang, B., Zhang, Y., Yu, L., Ru, L., Zhong, L., Huang, Z., Wu, K., Hu, D., et al.: Skysense: A multi-modal remote sensing foundation model towards universal interpretation for earth observation imagery. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 27672–27683 (2024)

    work page 2024

  11. [11]

    In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems

    Ha, Q., Watanabe, K., Karasawa, T., Ushiku, Y., Harada, T.: Mfnet: Towards real- time semantic segmentation for autonomous vehicles with multi-spectral scenes. In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems. pp. 5108–5115. IEEE (2017)

    work page 2017

  12. [12]

    arXiv preprint arXiv:2606.17020 (2026)

    Han, J., Zhang, B., Sun, X., Zhang, Q., Dong, Y., Hu, C., Zhang, F., Wei, Y., Guo, J.: Fusionrs: A large-scale rgb-infrared remote sensing dataset for dual-modal vision-language foundation models. arXiv preprint arXiv:2606.17020 (2026)

    work page arXiv 2026

  13. [13]

    Journal of Remote Sensing3, 0096 (2023)

    Han, Z., Zhang, Z., Zhang, S., Zhang, G., Mei, S.: Aerial visible-to-infrared image translation: Dataset, evaluation, and baseline. Journal of Remote Sensing3, 0096 (2023)

    work page 2023

  14. [14]

    In: Proceedings of the IEEE/CVF International Con- ference on Computer Vision Workshops (ICCVW)

    Jia, X., Zhu, C., Li, M., Tang, W., Zhou, W.: Llvip: A visible-infrared paired dataset for low-light vision. In: Proceedings of the IEEE/CVF International Con- ference on Computer Vision Workshops (ICCVW). pp. 3496–3504 (2021) MonoIR-RS 17

    work page 2021

  15. [15]

    In: Find- ingsoftheAssociationforComputationalLinguistics:EMNLP2024.pp.8573–8591 (2024)

    Jiang, S., Chen, Z., Liang, J., Zhao, Y., Liu, M., Qin, B.: Infrared-llava: Enhancing understanding of infrared images in multi-modal large language models. In: Find- ingsoftheAssociationforComputationalLinguistics:EMNLP2024.pp.8573–8591 (2024)

    work page 2024

  16. [16]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Kuckreja, K., Danish, M.S., Naseer, M., Das, A., Khan, S., Khan, F.S.: Geochat: Grounded large vision-language model for remote sensing. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 27831– 27840 (2024)

    work page 2024

  17. [17]

    In: Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition

    Li, X., Hong, D., Chanussot, J.: S2mae: A spatial-spectral pretraining foundation model for spectral remote sensing data. In: Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition. pp. 24088–24097 (2024)

    work page 2024

  18. [18]

    IEEE Transactions on Geoscience and Remote Sensing62, 1–16 (2024)

    Liu, F., Chen, D., Guan, Z., Zhou, X., Zhu, J., Ye, Q., Fu, L., Zhou, J.: Remoteclip: A vision language foundation model for remote sensing. IEEE Transactions on Geoscience and Remote Sensing62, 1–16 (2024)

    work page 2024

  19. [19]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Liu, H., Li, C., Li, Y., Lee, Y.J.: Improved baselines with visual instruction tun- ing. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 26296–26306 (2024)

    work page 2024

  20. [20]

    In: Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition

    Liu, J., Fan, X., Huang, Z., Wu, G., Liu, R., Zhong, W., Luo, Z.: Target-aware dual adversarial learning and a multi-scenario multi-modality benchmark to fuse infrared and visible for object detection. In: Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition. pp. 5802–5811 (2022)

    work page 2022

  21. [21]

    IEEE Transactions on Geoscience and Remote Sensing 56(4), 2183–2195 (2017)

    Lu, X., Wang, B., Zheng, X., Li, X.: Exploring models and data for remote sensing image caption generation. IEEE Transactions on Geoscience and Remote Sensing 56(4), 2183–2195 (2017)

    work page 2017

  22. [22]

    In: Proceedings of the IEEE/CVF International Con- ference on Computer Vision

    Luo, J., Zhang, Y., Yang, X., Wu, K., Zhu, Q., Liang, L., Chen, J., Li, Y.: When large vision-language model meets large remote sensing imagery: Coarse-to-fine text-guided token pruning. In: Proceedings of the IEEE/CVF International Con- ference on Computer Vision. pp. 9206–9217 (2025)

    work page 2025

  23. [23]

    In: International Conference on Learning Representations

    Mall, U.K., Phoo, C.P., Liu, M., Vondrick, C., Hariharan, B., Bala, K.: Remote sensing vision-language foundation models without annotations via ground remote alignment. In: International Conference on Learning Representations. vol. 2024, pp. 49294–49314 (2024)

    work page 2024

  24. [24]

    In: Proceedings of the AAAI Conference on Artificial Intelligence

    Pang, C., Weng, X., Wu, J., Li, J., Liu, Y., Sun, J., Li, W., Wang, S., Feng, L., Xia, G.S., He, C.: Vhm: Versatile and honest vision language model for remote sensing image analysis. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 39, pp. 6381–6388 (2025)

    work page 2025

  25. [25]

    DiffV2IR: Visible-to-Infrared Diffusion Model via Vision-Language Understanding

    Ran, L., Wang, L., Wang, G., Wang, P., Zhang, Y.: Diffv2ir: Visible-to-infrared dif- fusion model via vision-language understanding. arXiv preprint arXiv:2503.19012 (2025)

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  26. [26]

    Journal of Visual Communication and Image Representation 34, 187–203 (2016)

    Razakarivony, S., Jurie, F.: Vehicle detection in aerial imagery: A small target detection benchmark. Journal of Visual Communication and Image Representation 34, 187–203 (2016)

    work page 2016

  27. [27]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Safaei, B., Siddiqui, F., Xu, J., Patel, V.M., Lo, S.Y.: Filter images first, gener- ate instructions later: Pre-instruction data selection for visual instruction tuning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 14247–14256 (2025)

    work page 2025

  28. [28]

    In: Proceedings of the Computer Vision and Pattern Recognition Conference

    Soni, S., Dudhane, A., Debary, H., Fiaz, M., Munir, M.A., Danish, M.S., Fraccaro, P., Watson, C.D., Klein, L.J., Khan, F.S., et al.: Earthdial: Turning multi-sensory earth observations to interactive dialogues. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 14303–14313 (2025) 18 J. Han et al

    work page 2025

  29. [29]

    IEEE Transactions on Circuits and Systems for Video Technology32(10), 6700–6713 (2022)

    Sun, Y., Cao, B., Zhu, P., Hu, Q.: Drone-based rgb-infrared cross-modality vehi- cle detection via uncertainty-aware learning. IEEE Transactions on Circuits and Systems for Video Technology32(10), 6700–6713 (2022)

    work page 2022

  30. [30]

    In: Proceedings of the AAAI Conference on Artificial Intelligence

    Wang, Z., Prabha, R., Huang, T., Wu, J., Rajagopal, R.: Skyscript: A large and semantically diverse vision-language dataset for remote sensing. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 38, pp. 5805–5813 (2024)

    work page 2024

  31. [31]

    Advances in Neural Information Processing Systems37, 36567–36601 (2024)

    Wen, X., Zhao, B., Chen, Y., Pang, J., Qi, X.: What makes clip more robust to long-tailed pre-training data? a controlled study for transferable insights. Advances in Neural Information Processing Systems37, 36567–36601 (2024)

    work page 2024

  32. [32]

    IEEE Transactions on Geoscience and Remote Sensing60, 1–19 (2021)

    Yuan, Z., Zhang, W., Fu, K., Li, X., Deng, C., Wang, H., Sun, X.: Exploring a fine- grained multiscale method for cross-modal remote sensing image retrieval. IEEE Transactions on Geoscience and Remote Sensing60, 1–19 (2021)

    work page 2021

  33. [33]

    Advances in Neural Information Processing Systems37, 62223–62256 (2024)

    Zanella, M., Gérin, B., Ayed, I.B.: Boosting vision-language models with transduc- tion. Advances in Neural Information Processing Systems37, 62223–62256 (2024)

    work page 2024

  34. [34]

    In: International Conference on Learning Representations

    Zhang, S., Fang, Q., Yang, Y., Feng, Y.: Llava-mini: Efficient image and video large multimodal models with one vision token. In: International Conference on Learning Representations. vol. 2025, pp. 53285–53310 (2025)

    work page 2025

  35. [35]

    IEEE Transactions on Geoscience and Remote Sensing62, 1–23 (2024)

    Zhang, Z., Zhao, T., Guo, Y., Yin, J.: Rs5m and georsclip: A large-scale vision- language dataset and a large vision-language model for remote sensing. IEEE Transactions on Geoscience and Remote Sensing62, 1–23 (2024)

    work page 2024

  36. [36]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Zhu, Q., Lao, J., Ji, D., Luo, J., Wu, K., Zhang, Y., Ru, L., Wang, J., Chen, J., Yang, M., et al.: Skysense-o: Towards open-world remote sensing interpretation with vision-centric visual-language modeling. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 14733–14744 (2025) MonoIR-RS 19 A Mathematical Formulation ...

    work page 2025