Pith. sign in

REVIEW 5 major objections 9 minor 77 references

Remote Sensing Large Vision-Language Model: Semantic-augmented Multi-level Alignment and Semantic-aware Expert Modeling

T0 review · 5 major / 9 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that remote-sensing vision-language models improve consistently across scene classification, visual question answering, visual grounding, and image captioning when scene descriptions are retrieved from a text database and…

desk verdict Plausible new architecture for RS LVLMs, but the evaluation leaks instruction-tuning data into the test sets, so the headline claim is unverifiable from the reported numbers. read the letter →

arxiv 2506.21863 v1 pith:Q3CVGRLK submitted 2025-06-27 cs.CV

Sungjune Park , Yeongyun Kim , Se Yeon Kim , Yong Man Ro This is my paper
classification cs.CV
keywords remotesensinglargevision-languagemodelsemanticretrievalhierarchicalsemanticsexpertssceneclassificationvisualquestionansweringgrounding
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Remote-sensing images carry meaning at several scales at once: a scene can be an airport and also a collection of planes, roads, and hangars. The paper argues that general vision-language models miss this hierarchy, and that a remote-sensing LVLM should explicitly handle coarse scene-level context and fine object-level detail together. It proposes two linked components: a semantic augmentation module that retrieves relevant descriptions from a remote-sensing caption database and fuses them with multi-level visual features, and semantic-aware expert modeling that routes each semantic level through its own low-rank expert inside the language model. The authors report consistent gains over prior remote-sensing LVLMs on scene classification, visual question answering, visual grounding, and image captioning benchmarks. The claim is that explicit coarse-to-fine semantic modeling, rather than single-scale alignment, is what lets a general LVLM adapt to the remote-sensing domain.

What carries the argument

The load-bearing mechanism is the pairing of the Semantic Augmentation Module with Semantic-aware Expert Modeling. The augmentation module is a retrieval pipeline: a database of text descriptions, a fine-tuned contrastive retriever that scores descriptions against the input image, and a multi-level visual prompter in which 144 aggregation tokens first attend to the user query, then to the retrieved text, then separately to visual features from the 1/3, 2/3, and final layers of the vision encoder, producing level-specific semantic-augmented tokens. The expert module inserts low-rank experts into the language model's feed-forward path and uses a router that masks the semantic-augmented tokens so that each of the three experts sees only its assigned semantic level together with image and query tokens; the gated expert outputs are added to the standard feed-forward output. Each expert is a down-projection followed by an up-projection (3584 to 512 to 3584 dimensions), which the paper reports costs about 1.8% of the parameters of a conventional mixture-of-experts expert.

What would settle it

Hold out a set of test images and run the pipeline twice: once with the top-5 retrieved descriptions and once with randomly sampled descriptions from the same database. If the random-description version performs as well as the retrieved version on RSVQA-HR and DIOR-RSVG, then semantic relevance of the retrieved text is not what produces the reported gains.

Watch

Extended reading notes

Core claim

The central claim is that explicitly organizing remote-sensing semantics from coarse to fine improves LVLMs. The framework retrieves the top-5 text descriptions most relevant to an input image from a database built from LHRS-Align-Recap captions, then uses learnable aggregation tokens to attend to the user query, the retrieved text, and visual features from three layers of the vision encoder. The resulting semantic-augmented tokens are routed by a mask to three low-rank experts, each handling one semantic level while still seeing the image and query tokens. On the reported benchmarks, this design yields an average accuracy of 87.4% on the four scene-classification datasets shared with the strongest prior baselines, 90.3% on RSVQA-LR and 93.1% on RSVQA-HR, and 71.1% precision at IoU 0.5 on DIOR-RSVG visual grounding. The authors interpret these results as evidence that hierarchical semantic modeling is the key to adapting general LVLMs to remote sensing.

Load-bearing premise

The framework assumes that the text descriptions it retrieves from the database are actually semantically relevant to the image being looked at, even though the authors could not obtain the images paired with most of the database captions.

Editorial extensions

If this is right

  • On coarse-grained scene classification, the framework reports 87.4% average accuracy on the four datasets shared with the strongest prior baselines, compared with 80.3% and 82.8% for those baselines.
  • On fine-grained visual grounding, it reports 71.1% precision at IoU 0.5 on DIOR-RSVG, against 56.2% for the best prior result listed.
  • On VQA, it reports 90.3% accuracy on RSVQA-LR and 93.1% on RSVQA-HR, leading in both low- and high-resolution settings.
  • On image captioning, the framework obtains the best or second-best score in six of the eight metric comparisons across NWPU-Captions and UCM-Captions.
  • The low-rank expert design adds roughly 1.8% of the parameters of a standard MoE expert, suggesting that hierarchical semantic routing need not be parameter-heavy.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The retrieval step is deliberately built on captions whose paired images are unavailable, so the same augmentation recipe could transfer to other visual domains that have abundant text or metadata but few aligned image-text pairs.
  • The paper ties each semantic level to a fixed layer of the vision encoder; testing other backbones and layer selections would clarify how general the coarse-to-fine assignment is.
  • A head-to-head comparison of matched retrieved captions versus randomly sampled captions from the same database would isolate how much of the gain comes from semantic relevance rather than from simply injecting extra text.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 9 minor

Summary. The paper proposes a remote sensing large vision-language model (LVLM) framework with two core components: Semantic-augmented Multi-level Alignment, which retrieves textual descriptions from a database and aggregates them with user query and multi-level visual features via a multi-level visual prompter, and Semantic-aware Expert Modeling, which routes different semantic levels of tokens to separate low-rank experts inside the LLM. The model is trained in two stages (image-text alignment and instruction tuning) and evaluated on scene classification, VQA, visual grounding, and captioning benchmarks. The central claim is that the framework achieves consistent improvements across multiple semantic levels and tasks.

Significance. If the reported results are reproducible and the comparisons are fair, the framework would offer a practical recipe for injecting hierarchical semantic cues into remote sensing LVLMs, with an interesting low-parameter expert design (about 1.8% of standard MoE parameters per expert, per Section V). The paper also transparently discusses the data limitation that motivated retrieval-based augmentation. However, the empirical validation currently has multiple load-bearing gaps: evaluation benchmarks overlap with instruction-tuning data, hyperparameters are selected on test benchmarks, no ablations isolate the two core components, and there is a direct contradiction about whether paired images were available for retriever fine-tuning. These gaps prevent verification of the headline claim.

major comments (5)
  1. [Section IV-A1 and Sections IV-B to IV-E] The instruction-tuning data listed in Table II includes UCM, NWPU, METER-ML, fMoW, NWPU-Captions, RSVQA-LR, RSVQA-HR, RSVG, and DIOR-RSVG. The evaluation sections report results on exactly these datasets: NWPU, METER-ML, and fMoW (Table III), RSVQA-LR/HR (Table IV), DIOR-RSVG (Table V), and NWPU-Captions and UCM (Table VI). No train/test split or exclusion is stated anywhere in the paper. As a result, the headline claim of 'consistent improvements' is unverifiable because the reported gains could reflect memorization of the instruction-tuning data. Please state the splits used for each benchmark, or run the evaluation on held-out subsets and report those numbers.
  2. [Section IV-F, Table VII] The default number of aggregation tokens (144) is selected using NWPU, RSVQA-HR, and UCM-Captions, which are the same benchmarks used for the final evaluation in Tables III, IV, and VI. This constitutes tuning a hyperparameter on the test sets, which can inflate the reported performance. Please select hyperparameters on validation splits disjoint from all reported test benchmarks, or report results for all tested aggregation-token settings on held-out data.
  3. [Sections III-A and III-B] The paper proposes two core components, Semantic-augmented Multi-level Alignment and Semantic-aware Expert Modeling, and lists them as the main contributions. No ablation study isolates the contribution of either component. To support the claims in the contributions list, please provide experiments with (i) the full model, (ii) the model without semantic augmentation, (iii) the model without expert modeling, (iv) the model without both, and ideally a control with randomly retrieved descriptions.
  4. [Section III-A1 vs Section V] Section III-A1 states that the CLIP-based retriever is 'fine-tune[d] ... with paired RS image and text data (i.e., a subset of LHRS-Align-Recap)', while Section V states that the authors 'were unable to acquire most of the corresponding RS images from Google Earth API' and therefore devised a retrieval-based approach that works 'even in the absence of paired RS images'. These statements are contradictory. Please clarify how the retriever was trained: if fine-tuning was done on a subset with images, specify its size and source; if not, state that the retriever is used off-the-shelf or zero-shot. This is load-bearing for the semantic augmentation module.
  5. [Section IV-B, Table III] On the scene-classification datasets that are not in the instruction-tuning list (AID, WHU-RS19, SIRI-WHU), the proposed method is second-best on AID (90.8 vs 91.3 for LHRS-Bot) and below LHRS-Bot on fMoW (54.6 vs 56.6). The abstract and conclusion claim 'consistent improvements' across tasks. This pattern is not consistent on held-out data. Please qualify the claim or provide an analysis of why the method underperforms on these datasets.
minor comments (9)
  1. [Abstract] Typo: 'hider' should be 'hinder' in 'These discrepancies hider the effective understanding'.
  2. [Section III-A1] Grammar: 'these descriptions includes' should be 'these descriptions include'.
  3. [Section IV-A2] Typo: 'is is adopted' should be 'is adopted' in the visual grounding metric description.
  4. [Section IV-E and Table VI] The metric is misspelled as 'ROGUE-1' in the text and table header; it should be 'ROUGE-1'.
  5. [Section IV-E] Grammar: 'the proposed framework consistently outperforms other model' should be 'other models'.
  6. [Fig. 1 caption] The caption says 'enabling the LLM to encoder and reason'; 'encoder' should be 'encode'.
  7. [Section III-B, Eq. (11)] The notation 'FFN(Xl)' uses an undefined 'Xl'; it should likely be 'FFN(\tilde{X}^l)' or 'FFN(X)'. Please clarify which input is meant.
  8. [Section III-B, Eq. (10) and Section IV-A3] The gating weight gl is applied to hl in Eq. (10), but the implementation details say gl is only applied to the image and user query tokens. This is inconsistent and should be reconciled.
  9. [Table III] Several entries are missing (marked '-') for GeoChat and RSUniVLM, which makes the comparison incomplete; please provide the missing numbers if available.

Circularity Check

2 steps flagged · score 6.0 of 10

Reported gains on NWPU, METER-ML, fMoW, UCM, RSVQA, and DIOR-RSVG are not out-of-sample: those datasets are also the instruction-tuning data, and the aggregation-token count is tuned on them.

  1. fitted input called prediction [Section IV-A1 (instruction-tuning dataset list) vs. Section IV-A1 evaluation list and Tables III-VI]
    "Then we use 395.4K data for the instruction-tuning step with following datasets: UCM [58], NWPU [59], [60], RSITMD [61], METER-ML [62], fMoW [63], ... NWPU-Captions [60], ... RSVQA-LR [65], RSVQA-HR [65], ... DIOR-RSVG [67] ... ... the proposed framework is evaluated on six scene classification datasets; AID [70], WHU-RS19 [71], NWPU [59], SIRI-WHU [72], METER-ML [62], and fMoW [63], two image captioning datasets; NWPU-Captions [60] and UCM [58], two VQA datasets; RSVQA-LR and -HR [65], and a visual grounding dataset; DIOR-RSVG [68]."

    The same benchmarks used for instruction tuning are later reported as evaluation results: NWPU, METER-ML, fMoW, NWPU-Captions, UCM, RSVQA-LR/HR, and DIOR-RSVG all appear in both the training list and the evaluation list. No train/test split or exclusion is stated. The abstract's claim of 'consistent improvements' on these datasets is therefore not an out-of-sample prediction; as reported, it is a measure of fit to the same data. Only AID, WHU-RS19, and SIRI-WHU are outside the listed tuning data, and on those the pattern is mixed (e.g., AID 90.8 vs. LHRS-Bot's 91.3), so the central claim is not independently evidenced.

  2. fitted input called prediction [Section IV-F, Table VII (aggregation-token exploration)]
    "In this section, we explore the impact of the number of aggregation tokens, in order to determine the default number of the tokens. ... The performances are evaluated on three benchmarks (i.e., NWPU [59], RSVQA-HR [65], and UCM-Captions [58]) with 72, 144, and 288 aggregation tokens. ... Based on this observation, 144 aggregation tokens are adopted as our default configuration."

    The aggregation-token count is a hyperparameter selected by maximizing performance on NWPU, RSVQA-HR, and UCM-Captions. These are the same datasets on which the final 'Ours' numbers are reported (NWPU 90.4 in Table III, RSVQA-HR 93.1 in Table IV, UCM-Captions BERT 90.2 in Table VI). Thus the reported improvements on those benchmarks are partly constructed from the evaluation data: the configuration was chosen because it scored well on those exact metrics, making the subsequent 'evaluation' on them a fitted result rather than an independent prediction.

full rationale

This is an empirical systems paper, not a derivation-based paper: the retrieval module, prompt aggregation, and semantic-expert routing are defined architecturally and trained, and none of the equations (Eqs. 1-11) reduce to a fitted quantity by construction. There is no self-citation chain or imported uniqueness theorem; the authors do not rely on their own prior work to justify the central premise. The main circularity is in the evaluation logic. The instruction-tuning set (Table II) contains nearly all of the benchmarks later reported as evaluation results (NWPU, METER-ML, fMoW, NWPU-Captions, UCM, RSVQA-LR/HR, DIOR-RSVG), and no train/test split or exclusion is stated, so those reported numbers cannot be verified as out-of-sample. In addition, the aggregation-token count is tuned directly on NWPU, RSVQA-HR, and UCM-Captions, and the same datasets are then used to report final performance. These are not cases of the model's mathematical derivation being equivalent to its inputs; rather, the empirical evidence for the headline 'consistent improvements' is partially fitted to the evaluation data. The method still has independent content and is not forced by definition or self-citation, so a score of 6 is appropriate rather than 8 or 10.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The framework relies on several hand-chosen hyperparameters (aggregation tokens, top-k, expert count, rank) and domain assumptions about semantic relevance and layer granularity, none of which are systematically ablated.

free parameters (5)
  • Number of aggregation tokens (Na) = 144
    Tuned on NWPU, RSVQA-HR, and UCM-Captions in Table VII; the same datasets appear in final evaluation tables.
  • Top-k retrieved descriptions (k) = 5
    Chosen without ablation; affects the semantic tokens injected into the prompter.
  • Number of semantic experts (L) = 3
    Set to match three visual feature levels from layers at 1/3, 2/3, and last of SigLIP2; not ablated.
  • Rank of semantic experts (dr) = 512
    Chosen for efficiency; not ablated.
  • LoRA rank and scaling factor = 128 and 256
    Standard choices for LLM fine-tuning; not evaluated.
assumptions (4)
  • domain assumption The three visual encoder layers (1/3, 2/3, last) correspond to coarse, intermediate, and fine semantic levels.
    Section IV-A.3 states features are extracted from these layers; no analysis supports the semantic-level interpretation.
  • domain assumption Retrieved descriptions from LHRS-Align-Recap are semantically relevant to evaluation images from other datasets via CLIP cosine similarity.
    Section III-A1; no retrieval quality evaluation is provided.
  • domain assumption Instruction tuning on the listed datasets generalizes to the same benchmarks under standard splits.
    Section IV-A.1 lists training and evaluation datasets; splits are not specified.
  • ad hoc to paper Masking tokens in Eq. (7-8) and merging expert outputs in Eq. (10-11) preserves the LLM's ability to reason over the full context.
    The design is introduced for this paper; no theoretical or empirical validation of the masking and routing behavior.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Remote Sensing Large Vision-Language Model: Semantic-augmented Multi-level Alignment and Semantic-aware Expert Modeling." pith.science (2026). https://pith.science/paper/Q3CVGRLK

@misc{pith2026250621863,
  author       = {Pith},
  title        = {Pith review of: Remote Sensing Large Vision-Language Model: Semantic-augmented Multi-level Alignment and Semantic-aware Expert Modeling},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Q3CVGRLK}},
  note         = {Machine review of arXiv:2506.21863}
}
read the original abstract

Large Vision and Language Models (LVLMs) have shown strong performance across various vision-language tasks in natural image domains. However, their application to remote sensing (RS) remains underexplored due to significant domain differences in visual appearances, object scales, and semantics. These discrepancies hider the effective understanding of RS scenes, which contain rich, multi-level semantic information spanning from coarse-to-fine levels. Hence, it limits the direct adaptation of existing LVLMs to RS imagery. To address this gap, we propose a novel LVLM framework tailored for RS understanding, incorporating two core components: Semantic-augmented Multi-level Alignment and Semantic-aware Expert Modeling. First, to align multi-level visual features, we introduce the retrieval-based Semantic Augmentation Module which enriches the visual features with relevant semantics across fine-to-coarse levels (e.g., object- and scene-level information). It is designed to retrieve relevant semantic cues from a RS semantic knowledge database, followed by aggregation of semantic cues with user query and multi-level visual features, resulting in semantically enriched representation across multiple levels. Second, for Semantic-aware Expert Modeling, we design semantic experts, where each expert is responsible for processing semantic representation at different levels separately. This enables hierarchical semantic understanding from coarse to fine levels. Evaluations across multiple RS tasks-including scene classification and VQA, etc.-demonstrate that the proposed framework achieves consistent improvements across multiple semantic levels. This highlights its capability and effectiveness in bridging the gap between general LVLMs and unique demands of RS-specific vision-language understanding.

Figures

Figures reproduced from arXiv: 2506.21863 by the authors.

Figure 1
Figure 1. Overview of the proposed framework. The Semantic Augmentation Module retrieves the descriptions, most relevant to the given RS image using a [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The upper figure shows the overall process in the semantic retriever. By [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. The details of Semantic-aware Expert Modeling. The input tokens [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: The qualitative visualization results of remote sensing scene classification obtained from the proposed framework. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: The qualitative visualization results of remote sensing visual question answering (VQA) obtained from the proposed framework. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: The qualitative visualization results of remote sensing visual grounding obtained from the proposed framework. The red and green boxes denote the [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

77 extracted references · 30 canonical work pages

  1. [1]

    Llama: Open and efficient foundation language models,

    H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozi `ere, N. Goyal, E. Hambro, F. Azhar et al. , “Llama: Open and efficient foundation language models,” arXiv preprint arXiv:2302.13971, 2023

    arXiv 2023

  2. [2]

    Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality,

    W.-L. Chiang, Z. Li, Z. Lin, Y . Sheng, Z. Wu, H. Zhang, L. Zheng, S. Zhuang, Y . Zhuang, J. E. Gonzalez et al., “Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality,” See https://vicuna. lmsys. org (accessed 14 April 2023) , vol. 2, no. 3, p. 6, 2023

    2023

  3. [3]

    Phi-3 technical report: A highly capable language model locally on your phone,

    M. Abdin, J. Aneja, H. Awadalla, A. Awadallah, A. A. Awan, N. Bach, A. Bahree, A. Bakhtiari, J. Bao, H. Behl et al., “Phi-3 technical report: A highly capable language model locally on your phone,” arXiv preprint arXiv:2404.14219, 2024

    arXiv 2024

  4. [4]

    Stanford alpaca: An instruction-following llama model,

    R. Taori, I. Gulrajani, T. Zhang, Y . Dubois, X. Li, C. Guestrin, P. Liang, and T. B. Hashimoto, “Stanford alpaca: An instruction-following llama model,” 2023

    2023

  5. [5]

    Gpt-4 technical report,

    J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat et al. , “Gpt-4 technical report,” arXiv preprint arXiv:2303.08774 , 2023

    arXiv 2023

  6. [6]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark et al., “Learning transferable visual models from natural language supervision,” in International Conference on Machine Learning . PMLR, 2021, pp. 8748–8763

    2021

  7. [7]

    Blip-2: Bootstrapping language- image pre-training with frozen image encoders and large language models,

    J. Li, D. Li, S. Savarese, and S. Hoi, “Blip-2: Bootstrapping language- image pre-training with frozen image encoders and large language models,” in International Conference on Machine Learning . PMLR, 2023, pp. 19 730–19 742

    work page 2023

  8. [8]

    Instructblip: towards general-purpose vision-language mod- els with instruction tuning,

    W. Dai, J. Li, D. Li, A. M. H. Tiong, J. Zhao, W. Wang, B. Li, P. Fung, and S. Hoi, “Instructblip: towards general-purpose vision-language mod- els with instruction tuning,” Advances in Neural Information Processing Systems, pp. 49 250–49 267, 2023

    work page 2023

Show all 77 references
  1. [9]

    Visual instruction tuning,

    H. Liu, C. Li, Q. Wu, and Y . J. Lee, “Visual instruction tuning,” Ad- vances in Neural Information Processing Systems , vol. 36, pp. 34 892– 34 916, 2023

    2023

  2. [10]

    Cogvlm: Visual expert for pretrained language models,

    W. Wang, Q. Lv, W. Yu, W. Hong, J. Qi, Y . Wang, J. Ji, Z. Yang, L. Zhao, S. XiXuan et al. , “Cogvlm: Visual expert for pretrained language models,” Advances in Neural Information Processing Systems , vol. 37, pp. 121 475–121 499, 2024

    2024

  3. [11]

    mplug-owl2: Revolutionizing multi-modal large language model with modality collaboration,

    Q. Ye, H. Xu, J. Ye, M. Yan, A. Hu, H. Liu, Q. Qian, J. Zhang, and F. Huang, “mplug-owl2: Revolutionizing multi-modal large language model with modality collaboration,” in Proceedings of the ieee/cvf conference on computer vision and pattern recognition , 2024, pp. 13 040–13 051

    2024

  4. [12]

    Flamingo: a visual language model for few-shot learning,

    J.-B. Alayrac, J. Donahue, P. Luc, A. Miech, I. Barr, Y . Hasson, K. Lenc, A. Mensch, K. Millican, M. Reynolds et al. , “Flamingo: a visual language model for few-shot learning,” Advances in Neural Information Processing Systems, vol. 35, pp. 23 716–23 736, 2022

    2022

  5. [13]

    Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks,

    Z. Chen, J. Wu, W. Wang, W. Su, G. Chen, S. Xing, M. Zhong, Q. Zhang, X. Zhu, L. Lu et al. , “Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition ,...

    2024

  6. [14]

    How far are we to gpt-4v? closing the gap to commercial multimodal models with open-source suites,

    Z. Chen, W. Wang, H. Tian, S. Ye, Z. Gao, E. Cui, W. Tong, K. Hu, J. Luo, Z. Ma et al. , “How far are we to gpt-4v? closing the gap to commercial multimodal models with open-source suites,” Science China Information Sciences, vol. 67, no. 12, p. 220101, 2024

    2024

  7. [15]

    Qwen2. 5-vl technical report,

    S. Bai, K. Chen, X. Liu, J. Wang, W. Ge, S. Song, K. Dang, P. Wang, S. Wang, J. Tang et al., “Qwen2. 5-vl technical report,” arXiv preprint arXiv:2502.13923, 2025

    2025 arXiv

  8. [16]

    Multimodal chain-of-thought reasoning in language models,

    Z. Zhang, A. Zhang, M. Li, H. Zhao, G. Karypis, and A. Smola, “Multimodal chain-of-thought reasoning in language models,” arXiv preprint arXiv:2302.00923, 2023

    2023 arXiv

  9. [17]

    Deepseek-vl2: Mixture-of-experts vision-language models for advanced multimodal understanding,

    Z. Wu, X. Chen, Z. Pan, X. Liu, W. Liu, D. Dai, H. Gao, Y . Ma, C. Wu, B. Wang et al. , “Deepseek-vl2: Mixture-of-experts vision-language models for advanced multimodal understanding,” arXiv preprint arXiv:2412.10302, 2024

    2024 arXiv

  10. [18]

    Visionllm: Large language model is also an open-ended decoder for vision-centric tasks,

    W. Wang, Z. Chen, X. Chen, J. Wu, X. Zhu, G. Zeng, P. Luo, T. Lu, J. Zhou, Y . Qiao et al. , “Visionllm: Large language model is also an open-ended decoder for vision-centric tasks,” Advances in Neural Information Processing Systems , vol. 36, pp. 61 501–61 513, 2023

    2023

  11. [19]

    Vision-r1: Incentivizing reasoning capability in multimodal large language models,

    W. Huang, B. Jia, Z. Zhai, S. Cao, Z. Ye, F. Zhao, Z. Xu, Y . Hu, and S. Lin, “Vision-r1: Incentivizing reasoning capability in multimodal large language models,” arXiv preprint arXiv:2503.06749 , 2025

    2025 arXiv

  12. [20]

    Learning source-invariant deep hashing convolutional neural networks for cross-source remote sensing image retrieval,

    Y . Li, Y . Zhang, X. Huang, and J. Ma, “Learning source-invariant deep hashing convolutional neural networks for cross-source remote sensing image retrieval,” IEEE Transactions on Geoscience and Remote Sensing, vol. 56, no. 11, pp. 6521–6536, 2018

    2018

  13. [21]

    Automatic radiometric normalization for multitemporal remote sensing imagery with iterative slow feature anal- 12 ysis,

    L. Zhang, C. Wu, and B. Du, “Automatic radiometric normalization for multitemporal remote sensing imagery with iterative slow feature anal- 12 ysis,” IEEE Transactions on Geoscience and Remote Sensing , vol. 52, no. 10, pp. 6141–6155, 2014

    2014

  14. [22]

    A supervised progressive growing generative adversarial network for remote sensing image scene classification,

    A. Ma, N. Yu, Z. Zheng, Y . Zhong, and L. Zhang, “A supervised progressive growing generative adversarial network for remote sensing image scene classification,” IEEE Transactions on Geoscience and Remote Sensing, vol. 60, pp. 1–18, 2022

    2022

  15. [23]

    Multimodal remote sensing image matching combining learning features and delaunay triangula- tion,

    Y . Zhang, Y . Liu, H. Zhang, and G. Ma, “Multimodal remote sensing image matching combining learning features and delaunay triangula- tion,” IEEE Transactions on Geoscience and Remote Sensing , vol. 60, pp. 1–17, 2022

    2022

  16. [24]

    Urban flood-related remote sensing: research trends, gaps and opportunities,

    W. Zhu, Z. Cao, P. Luo, Z. Tang, Y . Zhang, M. Hu, and B. He, “Urban flood-related remote sensing: research trends, gaps and opportunities,” Remote Sensing, vol. 14, no. 21, p. 5505, 2022

    2022

  17. [25]

    Remote sensing-based proxies for urban disaster risk management and resilience: A review,

    S. Ghaffarian, N. Kerle, and T. Filatova, “Remote sensing-based proxies for urban disaster risk management and resilience: A review,” Remote Sensing, vol. 10, no. 11, p. 1760, 2018

    2018

  18. [26]

    Remote sensing in multirisk assess- ment: Improving disaster preparedness,

    M. M. Harb and F. Dell’Acqua, “Remote sensing in multirisk assess- ment: Improving disaster preparedness,” IEEE Geoscience and Remote Sensing Magazine, vol. 5, no. 1, pp. 53–65, 2017

    2017

  19. [27]

    Geochat: Grounded large vision-language model for remote sensing,

    K. Kuckreja, M. S. Danish, M. Naseer, A. Das, S. Khan, and F. S. Khan, “Geochat: Grounded large vision-language model for remote sensing,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 27 831–27 840

    2024

  20. [28]

    Lhrs-bot: Empowering remote sensing with vgi-enhanced large multimodal language model,

    D. Muhtar, Z. Li, F. Gu, X. Zhang, and P. Xiao, “Lhrs-bot: Empowering remote sensing with vgi-enhanced large multimodal language model,” in European Conference on Computer Vision. Springer, 2024, pp. 440– 457

    2024

  21. [29]

    Skyeyegpt: Unifying remote sensing vision-language tasks via instruction tuning with large language model,

    Y . Zhan, Z. Xiong, and Y . Yuan, “Skyeyegpt: Unifying remote sensing vision-language tasks via instruction tuning with large language model,” ISPRS Journal of Photogrammetry and Remote Sensing , vol. 221, pp. 64–77, 2025

    2025

  22. [30]

    H2rsvlm: Towards helpful and honest remote sensing large vision language model,

    C. Pang, J. Wu, J. Li, Y . Liu, J. Sun, W. Li, X. Weng, S. Wang, L. Feng, G.-S. Xia et al., “H2rsvlm: Towards helpful and honest remote sensing large vision language model,” arXiv e-prints, pp. arXiv–2403, 2024

    2024

  23. [31]

    Rs- llava: A large vision-language model for joint captioning and question answering in remote sensing imagery,

    Y . Bazi, L. Bashmal, M. M. Al Rahhal, R. Ricci, and F. Melgani, “Rs- llava: A large vision-language model for joint captioning and question answering in remote sensing imagery,” Remote Sensing, vol. 16, no. 9, p. 1477, 2024

    2024

  24. [32]

    Skysensegpt: A fine-grained instruction tuning dataset and model for remote sensing vision-language understanding,

    J. Luo, Z. Pang, Y . Zhang, T. Wang, L. Wang, B. Dang, J. Lao, J. Wang, J. Chen, Y . Tan et al. , “Skysensegpt: A fine-grained instruction tuning dataset and model for remote sensing vision-language understanding,” arXiv preprint arXiv:2406.10100 , 2024

    2024 arXiv

  25. [33]

    Rsgpt: A remote sensing vision language model and benchmark,

    Y . Hu, J. Yuan, C. Wen, X. Lu, Y . Liu, and X. Li, “Rsgpt: A remote sensing vision language model and benchmark,” ISPRS Journal of Photogrammetry and Remote Sensing , vol. 224, pp. 272–286, 2025

    2025

  26. [34]

    Rsunivlm: A unified vision language model for remote sensing via granularity-oriented mixture of experts,

    X. Liu and Z. Lian, “Rsunivlm: A unified vision language model for remote sensing via granularity-oriented mixture of experts,” arXiv preprint arXiv:2412.05679, 2024

    2024 arXiv

  27. [35]

    Llama 2: Open foundation and fine-tuned chat models,

    H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y . Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale et al. , “Llama 2: Open foundation and fine-tuned chat models,” arXiv preprint arXiv:2307.09288, 2023

    2023 arXiv

  28. [36]

    The llama 3 herd of models,

    A. Grattafiori, A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, A. Mathur, A. Schelten, A. Vaughan et al. , “The llama 3 herd of models,” arXiv preprint arXiv:2407.21783 , 2024

    2024 arXiv

  29. [37]

    Qwen technical report,

    J. Bai, S. Bai, Y . Chu, Z. Cui, K. Dang, X. Deng, Y . Fan, W. Ge, Y . Han, F. Huang et al. , “Qwen technical report,” arXiv preprint arXiv:2309.16609, 2023

    2023 arXiv

  30. [38]

    Internlm: A multilingual language model with progressively enhanced capabilities,

    I. Team, “Internlm: A multilingual language model with progressively enhanced capabilities,” 2023

    2023

  31. [39]

    Llava-next: Improved reasoning, ocr, and world knowledge,

    H. Liu, C. Li, Y . Li, B. Li, Y . Zhang, S. Shen, and Y . J. Lee, “Llava-next: Improved reasoning, ocr, and world knowledge,” January 2024. [Online]. Available: https://llava-vl.github.io/blog/2024-01-30-llava-next/

    2024

  32. [40]

    Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models,

    J. Zhu, W. Wang, Z. Chen, Z. Liu, S. Ye, L. Gu, Y . Duan, H. Tian, W. Su, J. Shao et al. , “Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models,” arXiv preprint arXiv:2504.10479, 2025

    2025 arXiv

  33. [41]

    Minigpt-4: Enhancing vision-language understanding with advanced large language models,

    D. Zhu, J. Chen, X. Shen, X. Li, and M. Elhoseiny, “Minigpt-4: Enhancing vision-language understanding with advanced large language models,” arXiv preprint arXiv:2304.10592 , 2023

    2023 arXiv

  34. [42]

    Mini-gemini: Mining the potential of multi-modality vision language models,

    Y . Li, Y . Zhang, C. Wang, Z. Zhong, Y . Chen, R. Chu, S. Liu, and J. Jia, “Mini-gemini: Mining the potential of multi-modality vision language models,” arXiv preprint arXiv:2403.18814 , 2024

    2024 arXiv

  35. [43]

    Gemini: a family of highly capable multimodal models,

    G. Team, R. Anil, S. Borgeaud, J.-B. Alayrac, J. Yu, R. Soricut, J. Schalkwyk, A. M. Dai, A. Hauth, K. Millican et al., “Gemini: a family of highly capable multimodal models,”arXiv preprint arXiv:2312.11805, 2023

    2023 arXiv

  36. [44]

    Kosmos-2.5: A multimodal literate model,

    T. Lv, Y . Huang, J. Chen, Y . Zhao, Y . Jia, L. Cui, S. Ma, Y . Chang, S. Huang, W. Wang et al., “Kosmos-2.5: A multimodal literate model,” arXiv preprint arXiv:2309.11419 , 2023

    2023 arXiv

  37. [45]

    Scaling up visual and vision-language representation learning with noisy text supervision,

    C. Jia, Y . Yang, Y . Xia, Y .-T. Chen, Z. Parekh, H. Pham, Q. Le, Y .-H. Sung, Z. Li, and T. Duerig, “Scaling up visual and vision-language representation learning with noisy text supervision,” in International conference on machine learning . PMLR, 2021, pp. 4904–4916

    2021

  38. [46]

    Sharegpt4v: Improving large multi-modal models with better captions,

    L. Chen, J. Li, X. Dong, P. Zhang, C. He, J. Wang, F. Zhao, and D. Lin, “Sharegpt4v: Improving large multi-modal models with better captions,” in European Conference on Computer Vision. Springer, 2024, pp. 370– 387

    2024

  39. [47]

    Minigpt-v2: large language model as a unified interface for vision-language multi-task learning,

    J. Chen, D. Zhu, X. Shen, X. Li, Z. Liu, P. Zhang, R. Krishnamoorthi, V . Chandra, Y . Xiong, and M. Elhoseiny, “Minigpt-v2: large language model as a unified interface for vision-language multi-task learning,” arXiv preprint arXiv:2310.09478 , 2023

    arXiv

  40. [48]

    Internlm-xcomposer: A vision- language large model for advanced text-image comprehension and composition,

    P. Zhang, X. Dong, B. Wang, Y . Cao, C. Xu, L. Ouyang, Z. Zhao, H. Duan, S. Zhang, S. Ding et al. , “Internlm-xcomposer: A vision- language large model for advanced text-image comprehension and composition,” arXiv preprint arXiv:2309.15112 , 2023

    2023 arXiv

  41. [49]

    Gpt4roi: Instruction tuning large language model on regionof-interest, 2024,

    S. Zhang, P. Sun, S. Chen, M. Xiao, W. Shao, W. Zhang, Y . Liu, K. Chen, and P. Luo, “Gpt4roi: Instruction tuning large language model on regionof-interest, 2024,” in URL https://openreview. net/forum

    2024

  42. [50]

    Glamm: Pixel grounding large multimodal model,

    H. Rasheed, M. Maaz, S. Shaji, A. Shaker, S. Khan, H. Cholakkal, R. M. Anwer, E. Xing, M.-H. Yang, and F. S. Khan, “Glamm: Pixel grounding large multimodal model,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 13 009–13 018

    2024

  43. [51]

    Lhrs- bot-nova: Improved multimodal large language model for remote sensing vision-language interpretation,

    Z. Li, D. Muhtar, F. Gu, X. Zhang, P. Xiao, G. He, and X. Zhu, “Lhrs- bot-nova: Improved multimodal large language model for remote sensing vision-language interpretation,” arXiv preprint arXiv:2411.09301, 2024

    2024 arXiv

  44. [52]

    Earth- marker: Visual prompt learning for region-level and point-level remote sensing imagery comprehension,

    W. Zhang, M. Cai, T. Zhang, J. Li, Y . Zhuang, and X. Mao, “Earth- marker: Visual prompt learning for region-level and point-level remote sensing imagery comprehension,” arXiv preprint arXiv:2407.13596 , 2024

    2024 arXiv

  45. [53]

    Geollava: Efficient fine-tuned vision-language models for temporal change detection in remote sensing,

    H. Elgendy, A. Sharshar, A. Aboeitta, Y . Ashraf, and M. Guizani, “Geollava: Efficient fine-tuned vision-language models for temporal change detection in remote sensing,” arXiv preprint arXiv:2410.19552 , 2024

    2024 arXiv

  46. [54]

    Teochat: A large vision-language assistant for temporal earth observation data,

    J. A. Irvin, E. R. Liu, J. C. Chen, I. Dormoy, J. Kim, S. Khanna, Z. Zheng, and S. Ermon, “Teochat: A large vision-language assistant for temporal earth observation data,” arXiv preprint arXiv:2410.06234 , 2024

    2024 arXiv

  47. [55]

    Earthgpt: A universal multi-modal large language model for multi-sensor image comprehension in remote sensing domain,

    W. Zhang, M. Cai, T. Zhang, Y . Zhuang, and X. Mao, “Earthgpt: A universal multi-modal large language model for multi-sensor image comprehension in remote sensing domain,” IEEE Transactions on Geo- science and Remote Sensing , 2024

    2024

  48. [56]

    Ringmogpt: A unified remote sensing foundation model for vision, language, and grounded tasks,

    P. Wang, H. Hu, B. Tong, Z. Zhang, F. Yao, Y . Feng, Z. Zhu, H. Chang, W. Diao, Q. Ye et al., “Ringmogpt: A unified remote sensing foundation model for vision, language, and grounded tasks,” IEEE Transactions on Geoscience and Remote Sensing , 2024

    2024

  49. [57]

    Skyscript: A large and semantically diverse vision-language dataset for remote sens- ing,

    Z. Wang, R. Prabha, T. Huang, J. Wu, and R. Rajagopal, “Skyscript: A large and semantically diverse vision-language dataset for remote sens- ing,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 38, no. 6, 2024, pp. 5805–5813

    2024

  50. [58]

    Deep semantic understanding of high resolution remote sensing image,

    B. Qu, X. Li, D. Tao, and X. Lu, “Deep semantic understanding of high resolution remote sensing image,” in 2016 International conference on computer, information and telecommunication systems (Cits) . IEEE, 2016, pp. 1–5

    2016

  51. [59]

    Remote sensing image scene classifi- cation: Benchmark and state of the art,

    G. Cheng, J. Han, and X. Lu, “Remote sensing image scene classifi- cation: Benchmark and state of the art,” Proceedings of the IEEE , vol. 105, no. 10, pp. 1865–1883, 2017

    2017

  52. [60]

    Nwpu- captions dataset and mlca-net for remote sensing image captioning,

    Q. Cheng, H. Huang, Y . Xu, Y . Zhou, H. Li, and Z. Wang, “Nwpu- captions dataset and mlca-net for remote sensing image captioning,” IEEE Transactions on Geoscience and Remote Sensing , vol. 60, pp. 1– 19, 2022

    2022

  53. [61]

    Exploring a fine-grained multiscale method for cross-modal remote sensing image retrieval,

    Z. Yuan, W. Zhang, K. Fu, X. Li, C. Deng, H. Wang, and X. Sun, “Exploring a fine-grained multiscale method for cross-modal remote sensing image retrieval,” arXiv preprint arXiv:2204.09868 , 2022

    2022 arXiv

  54. [62]

    Meter-ml: a multi-sensor earth observation benchmark for automated methane source mapping,

    B. Zhu, N. Lui, J. Irvin, J. Le, S. Tadwalkar, C. Wang, Z. Ouyang, F. Y . Liu, A. Y . Ng, and R. B. Jackson, “Meter-ml: a multi-sensor earth observation benchmark for automated methane source mapping,” arXiv preprint arXiv:2207.11166, 2022

    2022 arXiv

  55. [63]

    Functional map of the world,

    G. Christie, N. Fendley, J. Wilson, and R. Mukherjee, “Functional map of the world,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2018, pp. 6172–6180

    2018

  56. [64]

    Exploring models and data for remote sensing image caption generation,

    X. Lu, B. Wang, X. Zheng, and X. Li, “Exploring models and data for remote sensing image caption generation,” IEEE Transactions on Geoscience and Remote Sensing , vol. 56, no. 4, pp. 2183–2195, 2017. 13

    2017

  57. [65]

    Rsvqa: Visual question answering for remote sensing data,

    S. Lobry, D. Marcos, J. Murray, and D. Tuia, “Rsvqa: Visual question answering for remote sensing data,” IEEE Transactions on Geoscience and Remote Sensing , vol. 58, no. 12, pp. 8555–8566, 2020

    2020

  58. [66]

    Visual grounding in remote sensing images,

    Y . Sun, S. Feng, X. Li, Y . Ye, J. Kang, and X. Huang, “Visual grounding in remote sensing images,” inProceedings of the 30th ACM International conference on Multimedia , 2022, pp. 404–412

    2022

  59. [67]

    Rsvg: Exploring data and models for visual grounding on remote sensing data,

    Y . Zhan, Z. Xiong, and Y . Yuan, “Rsvg: Exploring data and models for visual grounding on remote sensing data,” IEEE Transactions on Geoscience and Remote Sensing , vol. 61, pp. 1–13, 2023

    2023

  60. [68]

    Object detection in optical remote sensing images: A survey and a new benchmark,

    K. Li, G. Wan, G. Cheng, L. Meng, and J. Han, “Object detection in optical remote sensing images: A survey and a new benchmark,” ISPRS journal of photogrammetry and remote sensing , vol. 159, pp. 296–307, 2020

    2020

  61. [69]

    Dota: A large-scale dataset for object detection in aerial images,

    G.-S. Xia, X. Bai, J. Ding, Z. Zhu, S. Belongie, J. Luo, M. Datcu, M. Pelillo, and L. Zhang, “Dota: A large-scale dataset for object detection in aerial images,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2018, pp. 3974–3983

    2018

  62. [70]

    Aid: A benchmark data set for performance evaluation of aerial scene classification,

    G.-S. Xia, J. Hu, F. Hu, B. Shi, X. Bai, Y . Zhong, L. Zhang, and X. Lu, “Aid: A benchmark data set for performance evaluation of aerial scene classification,” IEEE Transactions on Geoscience and Remote Sensing , vol. 55, no. 7, pp. 3965–3981, 2017

    2017

  63. [71]

    Satellite image classification via two-layer sparse coding with biased image representation,

    D. Dai and W. Yang, “Satellite image classification via two-layer sparse coding with biased image representation,” IEEE Geoscience and remote sensing letters, vol. 8, no. 1, pp. 173–176, 2010

    2010

  64. [72]

    Bag-of-visual- words scene classifier with local and global features for high spatial resolution remote sensing imagery,

    Q. Zhu, Y . Zhong, B. Zhao, G.-S. Xia, and L. Zhang, “Bag-of-visual- words scene classifier with local and global features for high spatial resolution remote sensing imagery,” IEEE Geoscience and Remote Sensing Letters, vol. 13, no. 6, pp. 747–751, 2016

    2016

  65. [73]

    Improved baselines with visual instruction tuning,

    H. Liu, C. Li, Y . Li, and Y . J. Lee, “Improved baselines with visual instruction tuning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 26 296–26 306

    2024

  66. [74]

    Qwen-vl: A frontier large vision-language model with versatile abilities,

    J. Bai, S. Bai, S. Yang, S. Wang, S. Tan, P. Wang, J. Lin, C. Zhou, and J. Zhou, “Qwen-vl: A frontier large vision-language model with versatile abilities,” arXiv preprint arXiv:2308.12966, vol. 1, no. 2, p. 3, 2023

    2023 arXiv

  67. [75]

    Siglip 2: Multilingual vision-language encoders with improved semantic understanding, localization, and dense features,

    M. Tschannen, A. Gritsenko, X. Wang, M. F. Naeem, I. Alabdul- mohsin, N. Parthasarathy, T. Evans, L. Beyer, Y . Xia, B. Mustafa et al. , “Siglip 2: Multilingual vision-language encoders with improved semantic understanding, localization, and dense features,” arXiv preprint arX...

    2025 arXiv

  68. [76]

    Qwen2.5: A party of foundation models,

    Q. Team, “Qwen2.5: A party of foundation models,” September 2024. [Online]. Available: https://qwenlm.github.io/blog/qwen2.5/

    2024

  69. [77]

    Vhm: Versatile and honest vision language model for remote sensing image analysis,

    C. Pang, X. Weng, J. Wu, J. Li, Y . Liu, J. Sun, W. Li, S. Wang, L. Feng, G.-S. Xia et al., “Vhm: Versatile and honest vision language model for remote sensing image analysis,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 39, no. 6, 2025, pp. 6381–6388

    2025

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.