REVIEW 4 major objections 5 minor 1 cited by
A remote-sensing vision-language model can train itself by reading OpenStreetMap tiles, matching or beating costlier teacher-based pipelines.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
A remote-sensing VLM can be adapted by having it read rendered OpenStreetMap maps paired with satellite images, then fine-tuning it on satellite images alone.
T0 review reviewed 2026-08-04 challenge →
load-bearing objection Novel map-rendering approach with real generalization gains, but the 'self-contained' story doesn't survive the Qwen2.5-72B labeling step. the 4 major comments →
OSMDA: OpenStreetMap-based Domain Adaptation for Remote Sensing VLMs
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
Core claim
On the paper's own terms, the central discovery is that aligning a strong foundation VLM with crowd-sourced geographic data is a practical, scalable path to remote-sensing domain adaptation. The model reads semantic labels, road networks, and land-use polygons directly from co-registered OSM-carto map tiles, converts them into detailed captions, and then learns to produce the same geographic descriptions from RGB images alone. Jointly training on these self-generated captions mixed with real benchmark data yields a model that generalizes better than models trained on data distilled from much larger teachers.
What carries the argument
The central mechanism is the map-augmented self-labeling loop: satellite images are paired with rendered OpenStreetMap tiles whose text labels have been replaced by short semantic descriptors generated from OSM tags; the base VLM is prompted to read the map and write a dense, declarative caption; the map is then discarded and the model is fine-tuned on the resulting caption corpus. The key carrier is the model's own OCR and chart-comprehension capability, which turns a freely available geographic database into structured visual supervision.
Load-bearing premise
The pipeline assumes the base VLM can read the rendered map's semantic labels accurately enough to produce captions that are correct training targets; if the model misreads or hallucinates under the prompt's confident, declarative tone, those errors are baked into the training corpus and reinforced by fine-tuning.
What would settle it
Take a sample of satellite images where OSM annotations are known to be wrong or sparse, run the OSMDA captioning step, and score the captions against human ground truth. If the self-generated captions systematically contain hallucinated or map-driven errors, or if fine-tuning on the OSMDA corpus degrades performance relative to fine-tuning on human captions of the same images, the central claim fails.
If this is right
- Teacher-dependent pseudo-labeling becomes unnecessary: any strong base VLM can bootstrap its own remote-sensing supervision from OpenStreetMap, removing API costs and the teacher's performance ceiling.
- Fine-tuning on OSMDA-Captions acts as an effective intermediate training stage, so subsequent fine-tuning on downstream benchmarks starts from a better initialization.
- The resulting model is robust to instruction paraphrases, whereas several evaluated baselines degrade or refuse to answer under unified prompts.
- The method improves zero-shot generalization on classification benchmarks such as Million-AID and XLRS-Bench, not just in-distribution performance.
- Because the pipeline is self-contained and cheap, it scales to larger corpora or updated base models without re-querying expensive proprietary teachers.
Where Pith is reading between the lines
- If the base model's OCR is the bottleneck, the method's gains should grow as base VLMs improve; upgrading the backbone requires no changes to the data pipeline.
- The map-induced biases the paper concedes (degradation in sparse mixed-use areas, vocabulary bias) suggest that blending OSMDA-Captions with other noisy, freely available signals—for example land-cover products or temporal imagery—could spread coverage beyond well-annotated areas.
- A direct test: compare OSMDA-VLM to a model fine-tuned on the same captions after human correction. If performance does not change, the self-generated supervision is the effective carrier; if it changes a lot, the gains come from the OSM signal rather than from self-labeling per se.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes OSMDA, a method for remote-sensing domain adaptation. A base VLM (InternVL3.5-8B) is prompted with co-registered satellite imagery and rendered OpenStreetMap (OSM) tiles to generate a 200k-image caption corpus (OSMDA-Captions). The same VLM is then fine-tuned on a mixture of this corpus and human-annotated training splits from several downstream benchmarks, producing OSMDA-VLM. The paper claims the pipeline is self-contained, requires no manual labeling and no stronger external VLM teacher, and reports state-of-the-art results on six of ten evaluated remote-sensing benchmarks under a unified evaluation protocol. Extensive ablations compare OSMDA to fine-tuning on benchmark data alone and to a gemma-3-27b teacher-distillation baseline.
Significance. The empirical study is potentially valuable. The idea of rendering OSM cartography, exploiting a VLM's OCR/chart capabilities to read semantic labels from map tiles, and using the resulting self-generated captions as training signal is creative and much cheaper than frontier-teacher distillation. The unified evaluation of ten models across ten benchmarks is a useful contribution, and the planned release of data and weights would enable reproducibility. The paper's own ablations, particularly the comparison against a gemma-3 teacher, help ground the claims. However, the central narrative contains factual overstatements that are load-bearing: the pipeline is not self-contained because Qwen2.5-72B-Instruct supplies the semantic labels that make the maps legible, and OSMDA-VLM is not trained without manual labels because it uses human-annotated benchmark training splits. The fine-tuning-split results also do not uniformly support the claim that OSMDA improves in-distribution performance. These issues need to be resolved before the paper can be accepted.
major comments (4)
- [Abstract, §1, §3.1, Table 1] The central claim that OSMDA is self-contained and requires no external model is contradicted by the semantic-labeling stage in §3.1. Qwen2.5-72B-Instruct is used to convert raw OSM tag sets (e.g., amenity=fuel; canopy=yes) into the 2–3 word labels that are then substituted into the rendered map tiles in §3.2. The base VLM's OCR pathway reads these Qwen-generated labels; without them the map would show raw, non-natural tags or no text. The abstract and §1 state that the pipeline requires 'no external model in the loop' and the title 'eliminates this dependency,' but Qwen is a strong external 72B language model. Table 1 lists only InternVL3.5-8B as the teacher and omits Qwen. This is not cosmetic: the claimed novelty of self-containment is precisely what is violated. Please either (a) add an ablation replacing Qwen with rule-based label construction or with the base model itself, or (b) r
- [Abstract, §3.4, §1] The claim 'requires no manual labeling' is not supported by the method as described. §3.4 states that OSMDA-VLM is trained on 'a joint mixture of OSMDA-Captions and real labelled data from the training splits of the downstream benchmarks.' Those benchmark training splits are human-annotated. The abstract also says the model is 'fine-tuned on the resulting corpus with satellite imagery alone,' which is inconsistent with the joint-training protocol used for OSMDA-VLM. The 'ours' model that uses only OSMDA-Captions is much weaker (see Table 3). The paper should clearly separate the two variants: OSMDA-Captions-only fine-tuning requires no new manual labels, but OSMDA-VLM does use existing human labels. The abstract and contribution list need to be rephrased accordingly.
- [Table 3, §4.2, Figure 4] The fine-tuning-split results do not support the statement that the OSMDA method 'improves fine-tuned performance on downstream tasks.' In Table 3, comparing 'ours-joint-training (OSMDA-VLM)' with 'base-fine-tuned' on all six fine-tuning-split metrics (NWPU-Captions, UCM-Captions, RSVQA-LR, RSVQA-HR, VRSBench caption, VRSBench vqa), OSMDA-VLM is worse in every case, e.g., RSVQA-LR agg 0.806 vs 0.823, VRS caption 0.429 vs 0.434. Moreover, training on OSMDA-Captions alone ('ours') degrades the base model on several benchmarks, e.g., RSVQA-LR agg 0.664 vs 0.796 and VRS caption 0.190 vs 0.243. The genuine gains from OSMDA appear in the generalization-split, where OSMDA-VLM improves over base-ft (EuroSAT, SkyScript, Million-AID, XLRS-Bench). The paper should state this precisely and avoid claiming that OSMDA improves both fine-tuned and zero-shot performance; the claim should be that it mainl
- [§3.3, §5] The self-training loop deserves more discussion. The caption corpus is generated by the same model that is later fine-tuned on it, with a prompt that explicitly forbids hedging words ('possibly,' 'likely') and requires a 'confident, declarative tone.' This can amplify hallucination. The paper's own §5 'Map-Induced Biases' concedes degradation in mixed-use areas and bias toward OSM label vocabulary, but no quantitative analysis of caption hallucination rate is provided. Since the G-Eval judge is also a Qwen LLM (§4.1), there is no independent human verification that the OSMDA-Captions are accurate. At minimum, add a human-annotated sample or a targeted analysis of caption correctness, especially for categories where map labels are sparse.
minor comments (5)
- [§3.2 / Supplementary] The 'magical numbers' t1, t2, t3 and the K-means parameter K are free hyperparameters; they are described in the supplementary but no sensitivity analysis is provided. Please rename them to 'hyperparameters' and, if possible, report a small sensitivity study.
- [Figure 4] The barplot legend refers to 'average rank' but neither the figure nor the text defines how ties are handled or how the rank is aggregated across benchmarks. Add a precise definition in the caption or in §4.2.
- [§3.3] The caption-generation model is called a 'teacher' even though it is the same model later fine-tuned. Consider using 'annotator' or 'base VLM' to avoid confusion with the teacher-distillation baselines.
- [Table 1] The cost estimate for OSMDA-VLM lists only the 200k-caption generation cost. If the Qwen semantic-labeling step incurs API or compute cost, include it in the cost estimate so that Figure 1(a) is complete and reproducible.
- [Abstract / §1] The abstract says 'no stronger external model,' while §1 says 'no external model in the loop.' These phrasings are inconsistent. Please align them with the actual pipeline after addressing the Qwen dependency.
Circularity Check
No significant circularity: the self-training loop is a genuine bootstrap with external map structure and independent benchmarks; the Qwen labeling step is an inconsistency in the 'self-contained' claim, not a circular reduction.
full rationale
The claimed derivation chain is: (1) curate OSM objects; (2) render them as map tiles; (3) prompt the base VLM with satellite+map to produce OSMDA-Captions; (4) fine-tune the same VLM on those captions with maps removed; (5) evaluate on ten external benchmarks. Step (4) is a self-training/bootstrap loop, but it is not circular in the sense used here: the captions are not a restatement of the fine-tuning input. The generator sees a strictly richer input (satellite image + co-registered rendered map with Qwen-generated semantic labels) than the student sees at fine-tuning (satellite image alone), and the map carries external geographic structure from OpenStreetMap that is independent of the model. No fitted parameter is later renamed as a prediction; no benchmark label is used to construct the captions (the joint mixture uses official training splits, which is standard supervised adaptation); and no uniqueness theorem or ansatz is imported from the authors' prior work. The sole self-citation [33] (FireScope) appears in a contextual list of reasoning/application works and is not load-bearing. The paper's own §5 'Map-Induced Biases' concedes that OSM-derived supervision biases the model toward OSM vocabulary and degrades in sparse mixed-use areas; that is a stated limitation, not evidence of circularity. The most serious weakness is factual rather than circular: §3.1 states 'We process each unique set of object tags with Qwen2.5-72B-Instruct' to produce the 48k semantic labels, and Table 1's teacher column omits Qwen. This contradicts the abstract's 'no stronger external VLM teacher'/'no external model' claim, but it does not make any benchmark result equivalent to its input by construction. Therefore the circularity score is low; the self-containment claim should be scrutinized as a correctness/support problem, not as a circular derivation.
Axiom & Free-Parameter Ledger
free parameters (5)
- Meta-CLIP balancing targets t1, t2, t3 =
700, 4000, 15
- Number of DINOv3 clusters K =
25000
- Visibility-filter pixel thresholds =
polygon area < 1 pixel; linestring length < 1 pixel
- Caption generation temperature =
1.0
- Fine-tuning hyperparameters =
LoRA rank 16, dropout 0.05, lr 1e-4, batch 32, 1 epoch, bf16, 16 H200 GPUs
axioms (5)
- domain assumption SkyScript georeferenced images can be accurately co-registered with OSM objects via bounding-box spatial queries.
- domain assumption The OSM-carto style rendered tile is visually legible to InternVL3.5-8B's OCR and chart-comprehension capabilities, so the model extracts correct semantic labels from the map.
- domain assumption Base-model captions conditioned on the map are accurate enough to serve as ground truth, and the banned hedging language will not induce systematic hallucination.
- domain assumption G-Eval using Qwen2.5-32B-Instruct without chain-of-thought yields scores that agree with human judgment on these caption/VQA benchmarks.
- domain assumption After fine-tuning on satellite images alone, visual features are sufficient to recover OSM-derived semantic labels.
Cite this review
Pith. "Pith review of OSMDA: OpenStreetMap-based Domain Adaptation for Remote Sensing VLMs." pith.science (2026). https://pith.science/paper/MME76GH7
@misc{pith2026260311804,
author = {Pith},
title = {Pith review of: OSMDA: OpenStreetMap-based Domain Adaptation for Remote Sensing VLMs},
year = {2026},
howpublished = {\url{https://pith.science/paper/MME76GH7}},
note = {Machine review of arXiv:2603.11804}
}
read the original abstract
Vision-Language Models (VLMs) adapted to remote sensing rely heavily on domain-specific image-text supervision, yet high-quality annotations for satellite and aerial imagery remain scarce and expensive to produce. Prevailing pseudo-labeling pipelines address this gap by distilling knowledge from large frontier models, but this dependence on large teachers is costly, limits scalability, and caps achievable performance at the ceiling of the teacher. We propose OSMDA: a self-contained domain adaptation framework that eliminates this dependency. Our key insight is that a capable base VLM can serve as its own annotation engine: by pairing aerial images with rendered OpenStreetMap (OSM) tiles, we leverage optical character recognition and chart comprehension capabilities of the model to generate captions enriched by OSM's vast auxiliary metadata. The model is then fine-tuned on the resulting corpus with satellite imagery alone, yielding OSMDA-VLM, a domain-adapted VLM that requires no manual labeling and no stronger external VLM teacher. We conduct exhaustive evaluations spanning six zero-shot and five in-distribution benchmarks across vision-language tasks, where OSMDA leads to substantial improvement. We further compare against nine competitive baselines, demonstrating that our method achieves superior overall performance, while being substantially cheaper to train than teacher-dependent alternatives. These results suggest that, given a strong foundation model, alignment with crowd-sourced geographic data is a practical and scalable path towards remote sensing domain adaptation. Dataset and model weights will be made publicly available upon acceptance.
Figures
Forward citations
Cited by 1 Pith paper
-
SearchEyes: Towards Frontier Multimodal Deep Search Intelligence via Search World Simulation
SearchEyes unifies multimodal search-agent training via Perception-Knowledge Chains on Wikidata5M and Hop-Anchored Policy Optimization, claiming a 6.2-point average gain over the strongest open-source baseline on six ...
Reference graph
Works this paper leans on
-
[1]
AI9Stars: Xlrs-bench: Benchmarking multimodal llms in ultra-high-resolution re- mote sensing.https://github.com/AI9Stars/XLRS-Bench(2025), gitHub reposi- tory 33
2025
-
[2]
Andy Allan and contributors: OpenStreetMap Carto: A global map style for Open- StreetMap maps.https://github.com/openstreetmap-carto/openstreetmap- carto,https://github.com/openstreetmap-carto/openstreetmap-carto3, 8, 25
-
[3]
Bai, L., Cai, Z., Cao, M., Cao, W., Chen, C., et al.: Intern-s1: A scientific multi- modal foundation model (2025),https://arxiv.org/abs/2508.157632, 10
arXiv 2025
-
[4]
Chen, L., Li, J., Dong, X., Zhang, P., He, C., Wang, J., Zhao, F., Lin, D.: Sharegpt4v: Improving large multi-modal models with better captions. In: Com- puter Vision – ECCV 2024: 18th European Conference, Milan, Italy, Septem- ber 29–October 4, 2024, Proceedings, Part XVII. p. 370–387. Springer-Verlag, Berlin, Heidelberg (2024).https://doi.org/10.1007/97...
-
[5]
IEEE Transactions on Geo- science and Remote Sensing60, 1–19 (2022).https://doi.org/10.1109/TGRS
Cheng, Q., Huang, H., Xu, Y., Zhou, Y., Li, H., Wang, Z.: Nwpu-captions dataset and mlca-net for remote sensing image captioning. IEEE Transactions on Geo- science and Remote Sensing60, 1–19 (2022).https://doi.org/10.1109/TGRS. 2022.320147410, 28
arXiv 2022
-
[6]
In: Proceedings of the 37th International Conference on Neural Information Processing Systems
Dai, W., Li, J., Li, D., Tiong, A.M.H., Zhao, J., Wang, W., Li, B., Fung, P., Hoi, S.: Instructblip: towards general-purpose vision-language models with instruction tuning. In: Proceedings of the 37th International Conference on Neural Information Processing Systems. NIPS ’23, Curran Associates Inc., Red Hook, NY, USA (2023) 4
2023
-
[7]
In: Globerson, A., Mackey, L., Belgrave, D., Fan, A., Pa- quet, U., Tomczak, J., Zhang, C
Dong, X., Zhang, P., Zang, Y., Cao, Y., Wang, B., Ouyang, L., Zhang, S., Duan, H., Zhang, W., Li, Y., Yan, H., Gao, Y., Chen, Z., Zhang, X., Li, W., Li, J., Wang, W., Chen, K., He, C., Zhang, X., Dai, J., Qiao, Y., Lin, D., Wang, J.: Internlm- xcomposer2-4khd: A pioneering large vision-language model handling resolutions from 336 pixels to 4k hd. In: Glob...
2024
-
[8]
In: In- ternational Conference on Learning Representations (2021),https://openreview
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., Houlsby, N.: An image is worth 16x16 words: Transformers for image recognition at scale. In: In- ternational Conference on Learning Representations (2021),https://openreview. net/forum?id=YicbFdNTTy4 16...
2021
-
[9]
Remote Sensing of Environment120, 25–36 (2012)
Drusch, M., Del Bello, U., Carlier, S., Colin, O., Fernandez, V., Gascon, F., Ho- ersch, B., Isola, C., Laberinti, P., Martimort, P., Meygret, A., Spoto, F., Sy, O., Marchese, F., Bargellini, P.: Sentinel-2: Esa’s optical high-resolution mission for gmes operational services. Remote Sensing of Environment120, 25–36 (2012). https://doi.org/10.1016/j.rse.20...
-
[10]
Earth Resources Observation and Science (EROS) Center: National agriculture imagery program (naip) (2017).https://doi.org/10.5066/F7QN651G,https: //doi.org/10.5066/F7QN651G6
doi:10.5066/f7qn651g 2017
-
[11]
Feng, J., Luo, H., Gu, Z.: Improving semi-supervised remote sensing scene clas- sification via multilevel feature fusion and pseudo-labeling. International Journal of Applied Earth Observation and Geoinformation136, 104335 (2025).https: //doi.org/https://doi.org/10.1016/j.jag.2024.104335,https://www. sciencedirect.com/science/article/pii/S15698432240069395
arXiv 2025
-
[12]
ISPRS Journal of Photogrammetry and Remote Sensing226, 146–163 (2025).https://doi.org/https://doi.org/10
Ge, J., Zhang, X., Zheng, Y., Guo, K., Liang, J.: Rsteller: Scaling up visual lan- guage modeling in remote sensing with rich linguistic semantics from openly avail- able data and large language models. ISPRS Journal of Photogrammetry and Remote Sensing226, 146–163 (2025).https://doi.org/https://doi.org/10. 1016/j.isprsjprs.2025.05.002,https://www.science...
2025
-
[13]
Helber, P., Bischke, B., Dengel, A., Borth, D.: Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing12(7), 2217– 2226 (2019).https://doi.org/10.1109/JSTARS.2019.291824210, 28
arXiv 2019
-
[14]
In: International Con- ference on Learning Representations (2022),https://openreview.net/forum?id= nZeVKeeFYf911
Hu, E.J., yelong shen, Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W.: LoRA: Low-rank adaptation of large language models. In: International Con- ference on Learning Representations (2022),https://openreview.net/forum?id= nZeVKeeFYf911
2022
-
[16]
arXiv preprint arXiv:2401.04088 (2024) 6
Jiang, A.Q., Sablayrolles, A., Roux, A., Mensch, A., Savary, B., Bamford, C., Chaplot, D.S., Casas, D.d.l., Hanna, E.B., Bressand, F., Lengyel, G., et al.: Mixtral of experts. arXiv preprint arXiv:2401.04088 (2024) 6
Pith/arXiv arXiv 2024
-
[17]
Kage, P., Rothenberger, J.C., Andreadis, P., Diochnos, D.I.: A review of pseudo- labeling for computer vision (2025),https://arxiv.org/abs/2408.072215
arXiv 2025
-
[18]
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Kuckreja, K., Danish, M.S., Naseer, M., Das, A., Khan, S., Khan, F.S.: Geochat: Grounded large vision-language model for remote sensing. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 27831–27840 (June 2024) 2, 4, 5
2024
-
[19]
In: Thirty-seventh Conference on Neural Information Pro- cessing Systems Datasets and Benchmarks Track (2023),https://openreview
Li, C., Wong, C., Zhang, S., Usuyama, N., Liu, H., Yang, J., Naumann, T., Poon, H., Gao, J.: LLaVA-med: Training a large language-and-vision assistant for biomedicine in one day. In: Thirty-seventh Conference on Neural Information Pro- cessing Systems Datasets and Benchmarks Track (2023),https://openreview. net/forum?id=GSuP99u2kR4
2023
-
[20]
In: Krause, A., Brunskill, E., Cho, K., Engelhardt, B., Sabato, S., Scarlett, J
Li, J., Li, D., Savarese, S., Hoi, S.: BLIP-2: Bootstrapping language-image pre- training with frozen image encoders and large language models. In: Krause, A., Brunskill, E., Cho, K., Engelhardt, B., Sabato, S., Scarlett, J. (eds.) Proceedings of the 40th International Conference on Machine Learning. Proceedings of Machine OSM-based Domain Adaptation for ...
2023
-
[21]
Li, K., Xin, Z., Pang, L., Pang, C., Deng, Y., Yao, J., Xia, G., Meng, D., Wang, Z., Cao, X.: Segearth-r1: Geospatial pixel reasoning via large language model (2025), https://arxiv.org/abs/2504.096445
Pith/arXiv arXiv 2025
-
[22]
Li, W., Xiang, X., Wen, Z., Zhou, G., Niu, B., Wang, F., Huang, L., Wang, Q., Hu, Y.: Georeason: Aligning thinking and answering in remote sensing vision-language models via logical consistency reinforcement learning (2026),https://arxiv.org/ abs/2601.041185
arXiv 2026
-
[23]
In: Globerson, A., Mackey, L., Belgrave, D., Fan, A., Paquet, U., Tomczak, J., Zhang, C
Li, X., Ding, J., Elhoseiny, M.: Vrsbench: A versatile vision-language bench- mark dataset for remote sensing image understanding. In: Globerson, A., Mackey, L., Belgrave, D., Fan, A., Paquet, U., Tomczak, J., Zhang, C. (eds.) Advances in Neural Information Processing Systems. vol. 37, pp. 3229–
-
[24]
Li, Z., Muhtar, D., Gu, F., He, Y., Zhang, X., Xiao, P., He, G., Zhu, X.: Lhrs-bot-nova: Improved multimodal large language model for remote sensing vision-language interpretation. ISPRS Journal of Photogrammetry and Remote Sensing227, 539–550 (2025).https://doi.org/https://doi.org/10.1016/j. isprsjprs.2025.06.003,https://www.sciencedirect.com/science/art...
doi:10.1016/j 2025
-
[25]
In: Advances in Neural Information Processing Systems (NeurIPS) (2023) 4
Liu, H., Li, C., Wu, Q., Lee, Y.J.: Visual instruction tuning. In: Advances in Neural Information Processing Systems (NeurIPS) (2023) 4
2023
-
[26]
Liu, J., Sun, L., Fu, R., Yang, B.: Towards faithful reasoning in remote sensing: A perceptually-grounded geospatial chain-of-thought for vision-language models (2026),https://arxiv.org/abs/2509.222215
arXiv 2026
-
[27]
In: Bouamor, H., Pino, J., Bali, K
Liu, Y., Iter, D., Xu, Y., Wang, S., Xu, R., Zhu, C.: G-Eval: NLG evaluation using GPT-4 with better human alignment. In: Bouamor, H., Pino, J., Bali, K. (eds.) Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. pp. 2511–2522. Association for Computational Linguistics, Singapore (Dec 2023).https://doi.org/10.18653/v1/2...
-
[28]
Lobry, S., Marcos, D., Murray, J., Tuia, D.: Rsvqa: Visual question answering for remote sensing data. IEEE Transactions on Geoscience and Remote Sensing 58(12), 8555–8566 (2020).https://doi.org/10.1109/TGRS.2020.298878210, 28
arXiv 2020
-
[29]
Long, Y., Xia, G.S., Li, S., Yang, W., Yang, M.Y., Zhu, X.X., Zhang, L., Li, D.: Oncreatingbenchmarkdatasetforaerialimageinterpretation:Reviews,guidances, and million-aid. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing14, 4205–4230 (2021).https://doi.org/10.1109/JSTARS. 2021.307036810, 28
arXiv 2021
-
[30]
preprint arXiv:1711.05101 (2017) 12
Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. preprint arXiv:1711.05101 (2017) 12
Pith/arXiv arXiv 2017
-
[31]
Luo, J., Pang, Z., Zhang, Y., Wang, T., Wang, L., Dang, B., Lao, J., Wang, J., Chen, J., Tan, Y., Li, Y.: Skysensegpt: A fine-grained instruction tuning dataset and model for remote sensing vision-language understanding (2024),https:// arxiv.org/abs/2406.101004, 5
Pith/arXiv arXiv 2024
-
[32]
Ailuro et al
Luo, J., Zhang, Y., Yang, X., Wu, K., Zhu, Q., Liang, L., Chen, J., Li, Y.: When large vision-language model meets large remote sensing imagery: Coarse-to-fine 18 S.M. Ailuro et al. text-guided token pruning. In: Proceedings of the IEEE/CVF International Con- ference on Computer Vision (ICCV). pp. 9206–9217 (October 2025) 4
2025
-
[33]
Markov, M., Ailuro, S.M., Gool, L.V., Schindler, K., Paudel, D.P.: Firescope: Wild- fireriskpredictionwithachain-of-thoughtoracle(2025),https://arxiv.org/abs/ 2511.171715
Pith/arXiv arXiv 2025
-
[34]
Machine Learning with Applications18, 100605 (2024)
Moslemi, A., Briskina, A., Dang, Z., Li, J.: A survey on knowledge distillation: Recent advancements. Machine Learning with Applications18, 100605 (2024). https://doi.org/https://doi.org/10.1016/j.mlwa.2024.100605,https:// www.sciencedirect.com/science/article/pii/S26668270240008115
arXiv 2024
-
[35]
In: Leonardis, A., Ricci, E., Roth, S., Russakovsky, O., Sattler, T., Varol, G
Muhtar, D., Li, Z., Gu, F., Zhang, X., Xiao, P.: Lhrs-bot: Empowering remote sensing with vgi-enhanced large multimodal language model. In: Leonardis, A., Ricci, E., Roth, S., Russakovsky, O., Sattler, T., Varol, G. (eds.) Computer Vision – ECCV 2024. pp. 440–457. Springer Nature Switzerland, Cham (2025) 5
2024
-
[36]
OpenAI: Gpt-4 technical report (2023),https://arxiv.org/abs/2303.087742, 6
Pith/arXiv arXiv 2023
-
[37]
OpenAI: Gpt-4o system card. Tech. rep., OpenAI (2024),https://openai.com/ index/gpt-4o-system-card/, accessed: 2025-03-05 5
2024
-
[38]
OpenStreetMap contributors: Planet dump retrieved from https://planet.osm.org .https://www.openstreetmap.org(2025) 3, 6, 21
2025
-
[39]
Ou, R., Hu, Y., Zhang, F., Chen, J., Liu, Y.: Geopix: A multimodal large language model for pixel-level image understanding in remote sensing. IEEE Geoscience and Remote Sensing Magazine13(3), 324–337 (2025).https://doi.org/10.1109/ MGRS.2025.35602932, 4, 5
arXiv 2025
-
[40]
Pang, C., Weng, X., Wu, J., Li, J., Liu, Y., Sun, J., Li, W., Wang, S., Feng, L., Xia, G.S., He, C.: Vhm: Versatile and honest vision language model for remote sens- ing image analysis. Proceedings of the AAAI Conference on Artificial Intelligence 39(6), 6381–6388 (Apr 2025).https://doi.org/10.1609/aaai.v39i6.32683, https://ojs.aaai.org/index.php/AAAI/art...
-
[41]
Mapnik Project (11 2025),https://mapnik.org/, version 4.1.4 3, 8
Pavlenko, A., The Mapnik Contributors: Mapnik: Open Source Toolkit for Devel- oping Mapping Applications. Mapnik Project (11 2025),https://mapnik.org/, version 4.1.4 3, 8
2025
-
[42]
In: 2016 International Conference on Computer, Information and Telecommunication Systems (CITS)
Qu, B., Li, X., Tao, D., Lu, X.: Deep semantic understanding of high resolution remote sensing image. In: 2016 International Conference on Computer, Information and Telecommunication Systems (CITS). pp. 1–5 (2016).https://doi.org/10. 1109/CITS.2016.754639710, 28
arXiv 2016
-
[43]
In: Meila, M., Zhang, T
Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., Sutskever, I.: Learning transferable visual models from natural language supervision. In: Meila, M., Zhang, T. (eds.) Proceedings of the 38th International Conference on Machine Learning. Proceed- ings of Machine Learning Res...
2021
-
[44]
Siméoni, O., Vo, H.V., Seitzer, M., Baldassarre, F., Oquab, M., Jose, C., Khalidov, V., Szafraniec, M., Yi, S., Ramamonjisoa, M., Massa, F., Haziza, D., Wehrstedt, L., Wang, J., Darcet, T., Moutakanni, T., Sentana, L., Roberts, C., Vedaldi, A., Tolan, J., Brandt, J., Couprie, C., Mairal, J., Jégou, H., Labatut, P., Bojanowski, P.: DINOv3 (2025),https://ar...
Pith/arXiv arXiv 2025
-
[45]
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Soni, S., Dudhane, A., Debary, H., Fiaz, M., Munir, M.A., Danish, M.S., Frac- caro, P., Watson, C.D., Klein, L.J., Khan, F.S., Khan, S.: Earthdial: Turning multi-sensory earth observations to interactive dialogues. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 14303–14313 (June 2025) 4, 5, 6 OSM-based Do...
2025
-
[46]
Journal of Machine Learning Research15, 1929–1958 (2014) 11
Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research15, 1929–1958 (2014) 11
1929
-
[47]
arXiv preprint arXiv:2312.11805 (2023),https://arxiv.org/abs/2312.118052, 6
Team, G., et al.: Gemini: A family of highly capable multimodal models. arXiv preprint arXiv:2312.11805 (2023),https://arxiv.org/abs/2312.118052, 6
Pith/arXiv arXiv 2023
-
[48]
arXiv preprint arXiv:2503.19786 (2025),https://arxiv.org/abs/2503
Team, G., Kamath, A., Ferret, J., Pathak, S., Vieillard, N., Merhej, R., Perrin, S., Matejovicova, T., Ramé, A., Rivière, M., Rouillard, L., Mesnard, T., Cideron, G., Grill, J.B., Ramos, S., Yvinec, E., Casbon, M., Pot, E., Penchev, I., Liu, G., Visin, F., Kenealy, K., Beyer, L., Zhai, X., Tsitsulin, A., Busa-Fekete, R., Feng, A., Sachdeva, N., Coleman, B...
Pith/arXiv arXiv 2025
-
[49]
Butterworths, London (1979),https: //www.dcs.gla.ac.uk/Keith/Preface.html10
Van Rijsbergen, C.J.: Information Retrieval. Butterworths, London (1979),https: //www.dcs.gla.ac.uk/Keith/Preface.html10
1979
-
[50]
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Wada, Y., Kaneda, K., Saito, D., Sugiura, K.: Polos: Multimodal metric learning from human feedback for image captioning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 13559– 13568 (June 2024) 28
2024
-
[51]
Wang, D., Liu, S., Jiang, W., Wang, F., Liu, Y., Qin, X., Luo, Z., Zhou, C., Guo, H., Zhang, J., Du, B., Tao, D., Zhang, L.: Geozero: Incentivizing reasoning from scratch on geospatial scenes (2026),https://arxiv.org/abs/2511.226455
arXiv 2026
-
[52]
Wang, F., Wang, H., Guo, Z., Wang, D., Wang, Y., Chen, M., Ma, Q., Lan, L., Yang, W., Zhang, J., Liu, Z., Sun, M.: Xlrs-bench: Could your multimodal llms understand extremely large ultra-high-resolution remote sensing imagery? In: Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion (CVPR). pp. 14325–14336 (June 2025) 2, 10, 28
2025
-
[53]
Wang, L., Yoon, K.J.: Knowledge Distillation and Student-Teacher Learning for Visual Intelligence: A Review and New Outlooks . IEEE Transactions on Pattern Analysis & Machine Intelligence44(06), 3048–3068 (Jun 2022).https: //doi.org/10.1109/TPAMI.2021.3055564,https://doi.ieeecomputersociety. org/10.1109/TPAMI.2021.30555645
arXiv 2022
-
[54]
arXiv preprint arXiv:2508.18265 (2025).https://doi
Wang, W., Gao, Z., Gu, L., Pu, H., Cui, L., Wei, X., Liu, Z., Jing, L., Ye, S., Shao, J., et al.: Internvl3.5: Advancing open-source multimodal models in versatility, reasoning, and efficiency. arXiv preprint arXiv:2508.18265 (2025).https://doi. org/10.48550/arXiv.2508.18265,https://arxiv.org/abs/2508.182654, 11
-
[55]
Wang, Z., Prabha, R., Huang, T., Wu, J., Rajagopal, R.: Skyscript: a large and semantically diverse vision-language dataset for remote sensing. In: Pro- ceedings of the Thirty-Eighth AAAI Conference on Artificial Intelligence and Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence and Fourteenth Symposium on Educational Advances ...
doi:10.1609/aaai.v 2024
-
[56]
IEEE Transactions on Geoscience and Remote Sensing55(7), 3965–3981 (2017)
Xia, G.S., Hu, J., Hu, F., Shi, B., Bai, X., Zhong, Y., Zhang, L., Lu, X.: Aid: A benchmark data set for performance evaluation of aerial scene classification. IEEE Transactions on Geoscience and Remote Sensing55(7), 3965–3981 (2017). https://doi.org/10.1109/TGRS.2017.268594510, 28 20 S.M. Ailuro et al
arXiv 2017
-
[57]
In: The Twelfth In- ternational Conference on Learning Representations (2024),https://openreview
Xu, H., Xie, S., Tan, X., Huang, P.Y., Howes, R., Sharma, V., Li, S.W., Ghosh, G., Zettlemoyer, L., Feichtenhofer, C.: Demystifying CLIP data. In: The Twelfth In- ternational Conference on Learning Representations (2024),https://openreview. net/forum?id=5BCFlnfE1g8, 25
2024
-
[58]
Xue, X., Wei, G., Chen, H., Zhang, H., Lin, F., Shen, C., Zhu, X.X.: Reo-vlm: Transformingvlmtomeetregressionchallengesinearthobservation(2024),https: //arxiv.org/abs/2412.165835
Pith/arXiv arXiv 2024
-
[59]
arXiv preprint arXiv:2412.15115 (2024) 8, 10, 24, 28
Yang, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Li, C., Liu, D., Huang, F., Wei, H., Lin, H., Yang, J., Tu, J., Zhang, J., Yang, J., Yang, J., Zhou, J., Lin, J., Dang, K., Lu, K., Bao, K., Yang, K., Yu, L., Li, M., Xue, M., Zhang, P., Zhu, Q., Men, R., Lin, R., Li, T., Xia, T., Ren, X., Ren, X., Fan, Y., Su, Y., Zhang, Y., Wan, Y., Liu, Y., Cui...
Pith/arXiv arXiv 2024
-
[60]
Yuan, Z., Xiong, Z., Mou, L., Zhu, X.X.: Chatearthnet: a global-scale image–text dataset empowering vision–language geo-foundation models. Earth System Science Data17(3), 1245–1263 (2025).https://doi.org/10.5194/essd-17-1245-2025, https://essd.copernicus.org/articles/17/1245/2025/6
-
[61]
Zanaga, D., Van De Kerchove, R., Daems, D., De Keersmaecker, W., Brockmann, C., Kirches, G., Wevers, J., Cartus, O., Santoro, M., Fritz, S., Lesiv, M., Herold, M., Tsendbazar, N., Xu, P., Ramoino, F., Arino, O.: ESA WorldCover 10 m 2021 v200 (2022).https://doi.org/10.5281/zenodo.7254221,https://doi.org/10. 5281/zenodo.72542216
-
[62]
Zhan, Y., Xiong, Z., Yuan, Y.: Skyeyegpt: Unifying remote sensing vision-language tasks via instruction tuning with large language model. ISPRS Journal of Pho- togrammetry and Remote Sensing221, 64–77 (2025).https://doi.org/https: //doi.org/10.1016/j.isprsjprs.2025.01.020,https://www.sciencedirect. com/science/article/pii/S09242716250002062, 4, 5
-
[63]
IEEE Transactions on Geoscience and Remote Sensing62, 1–20 (2024)
Zhang, W., Cai, M., Zhang, T., Zhuang, Y., Mao, X.: Earthgpt: A universal multi- modallargelanguagemodelformultisensorimagecomprehensioninremotesensing domain. IEEE Transactions on Geoscience and Remote Sensing62, 1–20 (2024). https://doi.org/10.1109/TGRS.2024.34096242
arXiv 2024
-
[64]
arXiv preprint arXiv:2402.14289 (2024),https://arxiv.org/abs/2402.142895
Zhou, B., Hu, Y., Weng, X., Jia, J., Luo, J., Liu, X., Wu, J., Huang, L.: Tinyllava: A framework of small-scale large multimodal models. arXiv preprint arXiv:2402.14289 (2024),https://arxiv.org/abs/2402.142895
Pith/arXiv arXiv 2024
-
[65]
Zhu, D., Chen, J., Shen, X., Li, X., Elhoseiny, M.: Minigpt-4: Enhancing vision- language understanding with advanced large language models. arXiv preprint arXiv:2304.10592 (2023) 4 OSM-based Domain Adaptation for Remote Sensing VLMs 21 Supplementary Material 7 Detailed Benchmarking Results DetailedablationresultsarereportedinTable3.DetailedperformanceonR...
Pith/arXiv arXiv 2023
-
[67]
1" to "5
The same image with a map overlay where objects are outlined and textually identified for reference only. Mandatory Rules (Follow Exactly) - Describe only what is visible from a top-down, aerial perspective, given the <res> m resolution. - Use only visual evidence from the RGB image and the provided map. - Do NOT infer, speculate, or guess. Avoid all unce...
-
[68]
Analyze the Ground Truth for core objects and counts
-
[69]
Imaginary Objects
Check the Prediction for “Imaginary Objects” (Hallucinations) not present in the GT
-
[70]
Verify if object counts and spatial relationships match the GT
-
[71]
Scoring Rubric: 1 (Critical Failure): Major hallucination (imaginary objects) or completely wrong scene classification
Assign a strict score from 1-5 using the rubric below. Scoring Rubric: 1 (Critical Failure): Major hallucination (imaginary objects) or completely wrong scene classification. 2 (Poor): Correct scene type, but severe errors in object counting or wrong object attributes. 3 (Fair): Captures the main gist, but has minor hallucinations or noticeable counting e...
-
[72]
Analyze the Question to understand what information is required
-
[73]
Examine the Ground Truth Answer for key facts, values, and constraints
-
[74]
Check the Predicted Answer for hallucinations (information not supported by the GT)
-
[75]
Verify correctness, precision, and completeness of the Predicted Answer
-
[76]
Scoring Rubric: 1 (Critical Failure): Incorrect answer or major hallucination; does not address the question
Assign a strict score from 1-5 using the rubric below. Scoring Rubric: 1 (Critical Failure): Incorrect answer or major hallucination; does not address the question. 2 (Poor): Partially related but mostly incorrect; major factual errors or missing key elements. 3 (Fair): Captures the general idea but contains minor errors, ambiguity, or incomplete details....
-
[77]
Output MUST be a single sentence under 15 words
-
[78]
The image shows
Do NOT use phrases like "The image shows" or "In this picture"
-
[79]
Do NOT describe surroundings, colors, or lighting
-
[80]
There is a piece of terrain
Select ONE of the following sentence patterns based on the image content: Pattern A (For Land/Terrain): "There is a piece of terrain." Pattern B (For Vehicles/Objects): "Quantity objects are stopped/parked arrangement at the location." (Note: Use "dispersedly" or "neatly" for arrangement). Pattern C (For Facilities/Structures): "It is a facility compose o...
-
[3242]
Curran Associates, Inc. (2024).https://doi.org/10.52202/079017- 0106,https://proceedings.neurips.cc/paper_files/paper/2024/file/ 05b7f821234f66b78f99e7803fffa78a-Paper-Datasets_and_Benchmarks_Track. pdf10, 28
doi:10.52202/079017- 2024
This paper was first reviewed by deepseek-v4-flash on August 4, 2026.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.