REVIEW 6 major objections 5 minor 28 references
Large Vision-Language Models for Remote Sensing Visual Question Answering
T0 review · 6 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read A generative LVLM with domain-adaptive pretraining and prompt-based finetuning outperforms prior RSVQA models on every question type in the RSVQAxBEN benchmark.
desk verdict A standard LVLM-for-RSVQA recipe whose only empirical claim is unverifiable because the baselines, backbone, and training details are all unspecified. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is a two-step training recipe applied to a pretrained LVLM composed of a visual encoder $f_{\text{enc}}$ and a language decoder $g_{\text{dec}}$. First, domain-adaptive pretraining minimizes the negative log-likelihood of answer tokens on a curated remote sensing dataset $\mathcal{D}$; second, prompt-based finetuning refines the model on prompts of the form "Given the image features V, answer the question: Q". Inference uses beam search over the token-level conditional probabilities, and the total objective is $\lambda_1 \mathcal{L}_{\text{pretrain}} + \lambda_2 \mathcal{L}_{\text{finetune}}$. The role of this machinery is to inject remote sensing-specific visual knowledge into a general-purpose LVLM and then steer it toward concise answer generation, which is what the paper argues produces the reported gains over prompt-only and discriminative baselines.
What would settle it
Re-run the exact RSVQAxBEN split with publicly available, properly tuned baselines (for example the released checkpoints of GeoChat or SkyEyeGPT) over multiple seeds and report confidence intervals; if any baseline matches or beats 90.2% yes/no, 84.9% multiple-choice, and 75.4% open-ended F1, the paper's superiority claim falls. A second check is to audit the pretraining dataset $\mathcal{D}$ for overlap with the test split; any leakage would invalidate the numbers.
Extended reading notes
Core claim
The central claim is that a generative LVLM trained with domain-adaptive pretraining followed by prompt-based finetuning achieves state-of-the-art performance on RSVQAxBEN across yes/no, multiple-choice, and open-ended questions. The paper frames the task as maximizing the conditional probability $p(A|I,Q;\theta)$ of a natural language answer $A$ given satellite image $I$ and question $Q$, generated token by token by a language decoder conditioned on visual features from an encoder. The implementation reported in Table 1 reaches 90.2% yes/no accuracy, 84.9% multiple-choice accuracy, and 75.4% open-ended F1, and the ablation in Table 2 shows that removing either training step lowers all three numbers. The authors conclude that both domain adaptation and prompt engineering contribute, and that the generative formulation handles open-ended questions more flexibly than discriminative models with rigid answer classes.
Load-bearing premise
The comparison's validity rests on the unstated assumption that the three baselines in Section 4.1 are legitimate, well-tuned state-of-the-art systems, since none of them is defined, cited, or given error bars; the paper also assumes that the curated pretraining dataset $\mathcal{D}$ exists, is large enough to drive adaptation, and is disjoint from the test split.
Editorial extensions
If this is right
- Open-ended RSVQA no longer needs a fixed answer vocabulary; any question whose answer can be expressed in language is in principle addressable by the same generative model.
- The two-step recipe should transfer to other remote sensing vision-language tasks such as captioning, land-use classification by description, and region-grounded dialogue, since it only changes the prompt and finetuning data.
- Domain-adaptive pretraining plus prompt finetuning may reduce the amount of labeled VQA data needed for a new geographic area or sensor type, because the pretraining step carries the domain knowledge.
- Benchmark comparisons in RSVQA should include generative LVLM baselines alongside discriminative classifiers, since the best reported results now come from the generative family.
Reading between the lines
- If the reported gaps replicate, the main source of gain is probably the domain-adaptive pretraining step rather than the generative architecture itself; a direct test would be to freeze the base LVLM and vary only the pretraining corpus.
- The paper leaves the pretraining dataset $\mathcal{D}$ unspecified; a natural extension is to measure how performance scales with the size and diversity of that corpus, and whether mixing in non-remote-sensing data helps or hurts.
- Because the baselines are not defined or cited, an independent replication should first pin down what "state-of-the-art" means on RSVQAxBEN; until then, the practical implication is that generative LVLMs are competitive, not that this specific configuration is uniquely optimal.
- The same two-step prompting recipe could be tested on temporal or multispectral inputs, where the visual encoder would need extension; the paper's formulation already allows $C$ spectral bands in $I\in\mathbb{R}^{H\times W\times C}$.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a generative Large Vision-Language Model (LVLM) approach for Remote Sensing Visual Question Answering (RSVQA). The method consists of two training stages: domain-adaptive pretraining on remote sensing data and prompt-based finetuning, followed by beam-search decoding. The authors evaluate on RSVQAxBEN and report performance across yes/no, multiple-choice, and open-ended question types (Table 1), an ablation study of the two training components (Table 2), and a human evaluation of output quality (Table 3). The central claim, stated in Section 4.4, is that the proposed method outperforms three baselines on all question types and achieves the highest human ratings.
Significance. If the empirical results were verifiable, the paper would provide a useful data point on adapting generative LVLMs to the remote sensing domain, particularly the combination of domain-adaptive pretraining and prompt-based finetuning. The manuscript is clearly structured and evaluates on a public benchmark, and it reports both automatic metrics and human judgments. However, the central contribution cannot be assessed as written: the three baselines are not defined, the base LVLM is never identified, the training objective is internally inconsistent, and no experimental details (splits, hyperparameters, variance, statistical tests) are provided. These omissions are load-bearing, not cosmetic, because the paper's only substantive claim is empirical superiority. The significance of the work therefore remains unestablished.
major comments (6)
- [§4.1, Table 1] The comparative claim is unverifiable because the three baselines (Baseline-1, Prompt-RSVQA, Discriminative-RSVQA) are introduced by name only, with no model identity, architecture, training procedure, citations, or hyperparameters. The manuscript's own related-work section (§2.2) reviews GeoChat [21], SkyEyeGPT [22], H2RSVLM [26], RSGPT [28], and Ground Remote Alignment [27], yet none of these appears in Table 1. Since the paper's central claim is 'superior performance compared to state-of-the-art baselines,' the comparison must be against identifiable, properly configured systems; as written, it is not.
- [§3.2, Eqs. (2)–(3)] The base LVLM is never identified: Section 3.2 only states that the model uses 'a pre-trained LVLM' with a visual encoder and a language decoder. No model name, parameter scale, architecture details, or pretraining corpus are given. Without this information, the reported numbers in Tables 1–3 cannot be reproduced, and the contribution of the proposed domain-adaptive pretraining and prompt-based finetuning cannot be isolated from the choice of the base model.
- [§3.3, §3.5, Eq. (8)] There is an internal inconsistency between the described training procedure and the stated objective. Section 3.3 describes a two-step sequential strategy (domain-adaptive pretraining followed by prompt-based finetuning), with separate losses in Eqs. (4) and (6). Section 3.5 then defines a joint objective L_total = λ1·L_pretrain + λ2·L_finetune. If the stages are sequential, no joint loss is ever optimized; if Eq. (8) is actually optimized, then the two-step description is inaccurate. The values of λ1, λ2, and beam size k are also never specified, so the method is underdetermined.
- [Tables 1–3] All reported results are point estimates with no error bars, confidence intervals, statistical significance tests, number of runs, random seeds, or dataset split details. The performance gaps in Table 1 are modest (e.g., 90.2 vs. 88.3 yes/no accuracy, 75.4 vs. 72.6 open-ended F1), so without variance information the claimed superiority may not be significant. Similarly, the ablation differences in Table 2 (e.g., 90.2 vs. 88.6) could fall within run-to-run noise. The manuscript does not report how the RSVQAxBEN data were split into training and test sets.
- [§3.3, dataset D] The domain-adaptive pretraining dataset D = {(I_i, Q_i, A_i)} is described only as 'curated,' with no source, size, composition, or explicit statement that it is disjoint from the RSVQAxBEN test split. Because the pretraining data include question-answer pairs, the possibility of overlap with the evaluation data is not addressed. The reported contribution of domain-adaptive pretraining (Table 2) is therefore uninterpretable without clarity on this dataset.
- [§4.3, Table 3] The human evaluation is insufficiently described: the number of annotators, their expertise, the annotation instructions, the sampling procedure for the 100 questions, and inter-annotator agreement are all omitted. All scores are reported as averages with no variance or statistical tests. Given the small score differences (e.g., 4.4 vs. 4.1 for correctness), the claim that the method produces 'more accurate, relevant, and fluent' answers is not supported as reported.
minor comments (5)
- [§2.2] The text attributes a dataset named 'DDFA V' to reference [23], but reference [23] is the FAIR1M benchmark paper; the dataset name and citation do not match.
- [References] References [6] and [7] are identical (both are Wornow et al., 'The shaky foundations of clinical foundation models'), so the related-work citations need to be corrected and de-duplicated.
- [References] Reference [16] has a garbled title, with multiple unrelated phrases concatenated; it should be replaced with the actual title of the cited work.
- [Eq. (5)] The prompt template 'Given the image features V, answer the question: Q' uses the formal feature-vector symbol V rather than a natural-language description; the authors should clarify whether V is replaced by a caption, a feature summary, or something else when the prompt is presented to the model.
- [Introduction and Abstract] The abstract and introduction emphasize that the generative approach avoids 'predefined answer categories,' yet the evaluation includes multiple-choice accuracy, which by definition uses predefined options; this point should be clarified to avoid appearing contradictory.
Circularity Check
No significant circularity: the paper's claims are empirical benchmark results, not derivations from fitted inputs or self-citations.
full rationale
The paper makes no attempt to derive an empirical quantity from fitted constants or to rename a known result as a prediction. The proposed method is a standard two-stage training pipeline (domain-adaptive pretraining and prompt-based finetuning) with losses in Eqs. (4), (6), and (8) that are conventional maximum-likelihood objectives; nothing in these equations defines the test metric in terms of a fitted parameter. Table 1 and Table 3 report direct evaluations on the RSVQAxBEN benchmark and a human study, and the ablation in Table 2 removes components from the same pipeline, which is a standard ablation rather than a circular prediction. There are no self-citations at all, and no uniqueness theorem or prior work is invoked to forbid alternatives. The main weakness identified by the reader—that the three baselines are unnamed, uncited, and not described—is a verifiability and correctness concern, not a circularity concern, because the reported numbers are claimed empirical measurements rather than consequences of a self-referential definition. The paper is self-contained as an empirical study, so the circularity score is 0.
Assumptions & free parameters
free parameters (3)
- lambda_1
- lambda_2
- beam_size_k
assumptions (4)
- standard math Conditional probability factorization of answer tokens (Eq. 3).
- domain assumption A pretrained LVLM can be adapted to remote sensing via the two-step objective in Eqs. 4-6.
- domain assumption A curated remote sensing dataset D of image-question-answer triples exists for domain-adaptive pretraining.
- ad hoc to paper The prompt template in Eq. 5 ('Given the image features V, answer the question: Q') is effective.
Cite this review
Pith. "Pith review of Large Vision-Language Models for Remote Sensing Visual Question Answering." pith.science (2026). https://pith.science/paper/DTYLCYUP
@misc{pith2026241110857,
author = {Pith},
title = {Pith review of: Large Vision-Language Models for Remote Sensing Visual Question Answering},
year = {2026},
howpublished = {\url{https://pith.science/paper/DTYLCYUP}},
note = {Machine review of arXiv:2411.10857}
}
read the original abstract
Remote Sensing Visual Question Answering (RSVQA) is a challenging task that involves interpreting complex satellite imagery to answer natural language questions. Traditional approaches often rely on separate visual feature extractors and language processing models, which can be computationally intensive and limited in their ability to handle open-ended questions. In this paper, we propose a novel method that leverages a generative Large Vision-Language Model (LVLM) to streamline the RSVQA process. Our approach consists of a two-step training strategy: domain-adaptive pretraining and prompt-based finetuning. This method enables the LVLM to generate natural language answers by conditioning on both visual and textual inputs, without the need for predefined answer categories. We evaluate our model on the RSVQAxBEN dataset, demonstrating superior performance compared to state-of-the-art baselines. Additionally, a human evaluation study shows that our method produces answers that are more accurate, relevant, and fluent. The results highlight the potential of generative LVLMs in advancing the field of remote sensing analysis.
Reference graph
Works this paper leans on
-
[21]
, Khan, F.S.: Geochat: Grounded large vision-language model for re mote sens- ing
Kuckreja, K., Danish, M.S., Naseer, M., Das, A., Khan, S. , Khan, F.S.: Geochat: Grounded large vision-language model for re mote sens- ing. In: IEEE/CVF Conference on Computer Vision and Pattern Recog- nition, CVPR 2024, Seattle, W A, USA, June 16-22, 2024. pp. 27 831– 27840. IEEE (2024). https://doi.org/10.1109/CVPR52733.2024.02629, https://doi.org/10.1...
arXiv 2024
-
[22]
L VLMs for Remote Sensing VQA 13 CoRR abs/2401.09712 (2024)
Zhan, Y., Xiong, Z., Yuan, Y.: Skyeyegpt: Unifying remot e sensing vision-language tasks via instruction tuning with large la nguage model. L VLMs for Remote Sensing VQA 13 CoRR abs/2401.09712 (2024). https://doi.org/10.48550/ARXIV.2401.09712, https://doi.org/10.48550/arXiv.2401.09712
-
[26]
Pang, C., Wu, J., Li, J., Liu, Y., Sun, J., Li, W., Weng, X., Wang, S., Feng, L., Xia, G., He, C.: H2RSVLM: towards help- ful and honest remote sensing large vision language model. C oRR abs/2403.20213 (2024). https://doi.org/10.48550/ARXIV.2403.20213, https://doi.org/10.48550/arXiv.2403.20213
-
[28]
Hu, Y., Yuan, J., Wen, C., Lu, X., Li, X.: RSGPT: A re- mote sensing vision language model and benchmark. CoRR abs/2307.15266 (2023). https://doi.org/10.48550/ARXIV.2307.15266, https://doi.org/10.48550/arXiv.2307.15266
-
[27]
Mall, U., Phoo, C.P., Liu, M.K., Vondrick, C., Hariharan , B., Bala, K.: Remote sensing vision-language foundation models without annota tions via ground re- mote alignment. In: The Twelfth International Conference o n Learning Repre- sentations, ICLR 2024, Vienna, Austria, May 7-11, 2024. Ope nReview.net (2024), https://openreview.net/forum?id=w9tc699w3Z
work page 2024
-
[1]
Narayanan, A., Rao, A., Prasad, A., Subramanyam, N.: VQA a s a factoid question answering problem: A novel approach for kn owledge- aware and explainable visual question answering. Image Vis . Comput. 116, 104328 (2021). https://doi.org/10.1016/J.IMAVIS.2021.104328, https://doi.org/10.1016/j.imavis.2021.104328
arXiv 2021
-
[2]
Lobry, S., Marcos, D., Murray, J., Tuia, D.: RSVQA: visual ques- tion answering for remote sensing data. IEEE Trans. Geosci. Remote. Sens. 58(12), 8555–8566 (2020). https://doi.org/10.1109/TGRS.2020.2988782, https://doi.org/10.1109/TGRS.2020.2988782
arXiv 2020
-
[3]
Lobry, S., Demir, B., Tuia, D.: RSVQA meets bigearthnet: A new, large-scale, visual question answering dataset for remote sensing. In: IEEE International Geoscience and Remote Sensing Sympo sium, IGARSS 2021, Brussels, Belgium, July 11-16, 2021. pp. 1218– 1221. IEEE (2021). https://doi.org/10.1109/IGARSS47720.2021.9553307, https://doi.org/10.1109/IGARSS47...
arXiv 2021
Show all 28 references
-
[4]
In: Proceedings of the 60th Annual Meeting o f the Association for Computational Linguistics (Volume 1: Long Papers)
Zhou, Y., Shen, T., Geng, X., Long, G., Jiang, D.: Claret: P re-training a correlation-aware context-to-event transformer for even t-centric generation and classification. In: Proceedings of the 60th Annual Meeting o f the Association for Computational Linguistics (Volume 1: Lo...
2022
-
[5]
In: Proceedings of the ACM Web Conference 2022
Zhou, Y., Geng, X., Shen, T., Long, G., Jiang, D.: Eventber t: A pre-trained model for event correlation reasoning. In: Proceedings of the ACM Web Conference 2022. pp. 850–859 (2022)
2022
-
[7]
CoRR abs/2303.12961 (2023)
Wornow, M., Xu, Y., Thapa, R., Patel, B.S., Steinberg, E., Fleming, S.L., Pf- effer, M.A., Fries, J.A., Shah, N.H.: The shaky foundations o f clinical founda- tion models: A survey of large language models and foundatio n models for emrs. CoRR abs/2303.12961 (2023). https://doi...
-
[8]
CoRR abs/2404.11973 (2024)
Moradi, M., Yan, K., Colwell, D., Samwald, M., Asgari, R.: Exploring the landscape of large language models: Foundations, techniqu es, and challenges. CoRR abs/2404.11973 (2024). https://doi.org/10.48550/ARXIV.2404.11973, https://doi.org/10.48550/arXiv.2404.11973
2024 doi
-
[9]
CoRR abs/2401.14656 (2024)
Zhang, Q., Ding, K., Lyv, T., Wang, X., Yin, Q., Zhang, Y., Y u, J., Wang, Y., Li, X., Xiang, Z., Zhuang, X., Wang, Z., Qin, M., Zha ng, M., Zhang, J., Cui, J., Xu, R., Chen, H., Fan, X., Xing, H., Chen, H .: Sci- entific large language models: A survey on biological & chemi ca...
-
[10]
In : Proceedings of the AAAI Conference on Artificial Intelligence
Zhou, Y., Shen, T., Geng, X., Tao, C., Shen, J., Long, G., X u, C., Jiang, D.: Fine-grained distillation for long document retrieval. In : Proceedings of the AAAI Conference on Artificial Intelligence. vol. 38, pp. 19732–1 9740 (2024)
2024
-
[11]
In: Findings of the Associ ation for Computational Linguistics: ACL 2023
Zhou, Y., Shen, T., Geng, X., Tao, C., Xu, C., Long, G., Jia o, B., Jiang, D.: Towards robust ranker for text retrieval. In: Findings of the Associ ation for Computational Linguistics: ACL 2023. pp. 5387–5401 (2023) 12 S. Siripong et al
2023
-
[12]
In: Pro- ceedings of the 2021 Conference of the North American Chapte r of the Associa- tion for Computational Linguistics: Human Language Techno logies
Zhou, Y., Geng, X., Shen, T., Zhang, W., Jiang, D.: Improv ing zero-shot cross- lingual transfer for multilingual question answering over knowledge graph. In: Pro- ceedings of the 2021 Conference of the North American Chapte r of the Associa- tion for Computational Linguistics...
2021
-
[13]
arXiv preprint arXiv :2311.08734 (2023)
Zhou, Y., Geng, X., Shen, T., Tao, C., Long, G., Lou, J.G., Shen, J.: Thread of thought unraveling chaotic contexts. arXiv preprint arXiv :2311.08734 (2023)
2023 arXiv
-
[14]
: Modeling event- pair relations in external knowledge graphs for script reas oning
Zhou, Y., Geng, X., Shen, T., Pei, J., Zhang, W., Jiang, D. : Modeling event- pair relations in external knowledge graphs for script reas oning. Findings of the Association for Computational Linguistics: ACL-IJCNLP 20 21 (2021)
2021
-
[15]
In: Ku , L., Mar- tins, A., Srikumar, V
Zhang, D., Yu, Y., Dong, J., Li, C., Su, D., Chu, C., Yu, D.: Mm-llms: Recent advances in multimodal large language models. In: Ku , L., Mar- tins, A., Srikumar, V. (eds.) Findings of the Association fo r Computa- tional Linguistics, ACL 2024, Bangkok, Thailand and virtua l me...
2024 doi
-
[16]
In: Proceedings of the 2023 4th Internationa l Symposium on Artifi- cial Intelligence for Medicine Science, ISAIMS 2023, Cheng du, China, October 20- 22, 2023
Zhou, C., Gong, Q., Zhu, J., Luan, H.: Research and applic ation of large language models in healthcarecurrent development of large language models in the health- care fielda framework for applying large language models and the opportunities and challenges of large language mod...
2023
-
[17]
, Liu, J., Xu, Y., Xu, J., Nie, J., Liu, Y.: A survey on large lan- guage models with multilingualism: Recent advances and new frontiers
Huang, K., Mo, F., Li, H., Li, Y., Zhang, Y., Yi, W., Mao, Y. , Liu, J., Xu, Y., Xu, J., Nie, J., Liu, Y.: A survey on large lan- guage models with multilingualism: Recent advances and new frontiers. CoRR abs/2405.10936 (2024). https://doi.org/10.48550/ARXIV.2405.10936, https:...
-
[18]
arXiv preprin t arXiv:2410.19732 (2024)
Zhou, Y., Rao, Z., Wan, J., Shen, J.: Rethinking visual de pendency in long-context reasoning for large vision-language models. arXiv preprin t arXiv:2410.19732 (2024)
2024 arXiv
-
[19]
In: Findings of the Association for Comput ational Linguistics, ACL 2024, Bangkok, Thailand and virtual meeting, August 11- 16, 2024
Zhou, Y., Li, X., Wang, Q., Shen, J.: Visual in-context le arning for large vision- language models. In: Findings of the Association for Comput ational Linguistics, ACL 2024, Bangkok, Thailand and virtual meeting, August 11- 16, 2024. pp. 15890– 15902. Association for Computat...
2024
-
[20]
In: Companion of the 2024 International Conference on Management of Data
Chen, D., Huang, Y., Ma, Z., Chen, H., Pan, X., Ge, C., Gao, D., Xie, Y., Liu, Z., Gao, J., et al.: Data-juicer: A one-stop data processing sys tem for large language models. In: Companion of the 2024 International Conference on Management of Data. pp. 120–134 (2024)
2024
-
[23]
ISPRS Journal o f Photogrammetry and Remote Sensing 184, 116–130 (2022)
Sun, X., Wang, P., Yan, Z., Xu, F., Wang, R., Diao, W., Chen , J., Li, J., Feng, Y., Xu, T., et al.: Fair1m: A benchmark dataset for fine-grain ed object recognition in high-resolution remote sensing imagery. ISPRS Journal o f Photogrammetry and Remote Sensing 184, 116–130 (2022)
2022
-
[24]
IEEE Trans
Zhang, Z., Zhao, T., Guo, Y., Yin, J.: RS5M and georsclip: A large-scale vision- language dataset and a large vision-la nguage model for remote sensing. IEEE Trans. Geosci. Remote. Sens. 62, 1–23 (2024). https://doi.org/10.1109/TGRS.2024.3449154, https://doi.org/10.1109/TGRS.2...
2024
-
[25]
In: Wooldridge, M.J., Dy, J.G., Natarajan, S
Wang, Z., Prabha, R., Huang, T., Wu, J., Rajagopal, R.: Sk yscript: A large and semantically diverse vision-language dataset for remote s ensing. In: Wooldridge, M.J., Dy, J.G., Natarajan, S. (eds.) Thirty-Eighth AAAI Con ference on Artificial Intelligence, AAAI 2024, Thirty-Si...
2024
- [29]
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.