Pith. sign in

REVIEW 25 cited by

SkySenseGPT: A Fine-Grained Instruction Tuning Dataset and Model for Remote Sensing Vision-Language Understanding

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2406.10100 v2 pith:6H2UN2LY submitted 2024-06-14 cs.CV cs.AI

Junwei Luo , Zhen Pang , Yongjun Zhang , Tingzhu Wang , Linlin Wang , Bo Dang , Jiangwei Lao , Jian Wang
show 3 more authors
Jingdong Chen Yihua Tan Yansheng Li
This is my paper · ORCID
classification cs.CVcs.AI
keywords comprehensionremoterslmmssensingdatasetfit-rsinstructioncomplex
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Remote Sensing Large Multi-Modal Models (RSLMMs) are developing rapidly and showcase significant capabilities in remote sensing imagery (RSI) comprehension. However, due to the limitations of existing datasets, RSLMMs have shortcomings in understanding the rich semantic relations among objects in complex remote sensing scenes. To unlock RSLMMs' complex comprehension ability, we propose a large-scale instruction tuning dataset FIT-RS, containing 1,800,851 instruction samples. FIT-RS covers common interpretation tasks and innovatively introduces several complex comprehension tasks of escalating difficulty, ranging from relation reasoning to image-level scene graph generation. Based on FIT-RS, we build the FIT-RSFG benchmark. Furthermore, we establish a new benchmark to evaluate the fine-grained relation comprehension capabilities of LMMs, named FIT-RSRC. Based on combined instruction data, we propose SkySenseGPT, which achieves outstanding performance on both public datasets and FIT-RSFG, surpassing existing RSLMMs. We hope the FIT-RS dataset can enhance the relation comprehension capability of RSLMMs and provide a large-scale fine-grained data source for the remote sensing community. The dataset will be available at https://github.com/Luo-Z13/SkySenseGPT

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 25 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. TESSERA v2: Scaling Pixel-wise Earth Foundation Models

    cs.CV 2026-07 conditional novelty 7.0 of 10

    Downstream-driven scaling of pixel-wise Barlow Twins EO models favors large encoders and matched data over projectors, and distillation yields compact Matryoshka students that lead multi-task embedding benchmarks.

  2. NeSy-Route: A Neuro-Symbolic Benchmark for Constrained Route Planning in Remote Sensing

    cs.AI 2026-03 reject novelty 6.5 of 10

    NeSy-Route supplies 10,821 optimally labeled remote-sensing route-planning tasks plus a three-level neuro-symbolic protocol that reveals major perception and planning deficits in current MLLMs.

  3. Beyond Zooming: Learning Multi-Tool Visual Reasoning for Ultra-High-Resolution Remote Sensing

    cs.CV 2026-07 conditional novelty 6.0 of 10

    Training on multi-tool visual reasoning trajectories (zoom, grounding, lines) with an attention-focused RL objective improves UHR remote-sensing VQA accuracy over single-tool zoom-in and larger base models.

  4. Filling Before Advancing: Capability-Gap-Driven Post-Training for Scenario-Specialized Remote Sensing MLLMs

    cs.CV 2026-07 conditional novelty 6.0 of 10

    An ordered three-stage post-training route (FBA) improves harbor-scenario performance of RS-MLLMs over direct and collapsed fine-tuning on the authors' HarborEval benchmark.

  5. Multimodal Large Language Models for Remote Sensing Image Understanding: Domain-Specific or General-Purpose?

    cs.CV 2026-07 conditional novelty 6.0 of 10

    Under one zero-shot protocol, general-purpose MLLMs match or outperform remote-sensing-specific MLLMs on several RS benchmarks, while RS-MLLMs keep advantages in visual grounding, RS-VQA, and ultra-high-resolution und...

  6. Memory-Augmented Multimodal Large Language Models for Small Object Understanding in Streaming Aerial Videos

    cs.CV 2026-07 conditional novelty 6.0 of 10

    A new pixel-level aerial streaming referring-segmentation dataset (DroneEyes) and an MLLM (SkyAnchor) with learned token routing and hierarchical memory report SOTA on DroneEyes and large zero-shot gains on SkyFind.

  7. SkyVLaM: Multimodal Large Language Model for UAV Video Understanding in Remote Sensing

    cs.CV 2026-07 conditional novelty 6.0 of 10

    SkyVLaM introduces a temporal basis perceiver and adaptive dense selection to improve language-conditioned video segmentation in UAV scenes, and contributes the SkyVid dataset.

  8. GeoChrono: Benchmarking and Rethinking Long-Term Temporal Understanding in Remote Sensing

    cs.CV 2026-07 conditional novelty 6.0 of 10

    ChronoBench decomposes long-term remote sensing understanding into four cognitive levels, and the GeoChrono model, using per-location temporal trajectories, achieves 78.34% accuracy—over 20 points above prior MLLMs—bu...

  9. OSMDA: OpenStreetMap-based Domain Adaptation for Remote Sensing VLMs

    cs.CV 2026-03 conditional novelty 6.0 of 10

    A remote-sensing VLM can be adapted by having it read rendered OpenStreetMap maps paired with satellite images, then fine-tuning it on satellite images alone.

  10. OpenEarthAgent: A Unified Framework for Tool-Augmented Geospatial Agents

    cs.CV 2026-02 conditional novelty 6.0 of 10

    A 4B model fine-tuned on tool-augmented geospatial reasoning traces outperforms larger general-purpose models on executable GIS/spectral tool-use benchmarks and matches frontier models on trajectory fidelity.

  11. VectorLLM: Human-like Extraction of Structured Building Contours vis Multimodal LLMs

    cs.CV 2025-07 conditional novelty 6.0 of 10

    VectorLLM, a multimodal LLM that regresses building contour vertices token by token, reports gains of 5.6 to 13.6 AP over prior polygon extraction methods on WHU, WHU-Mix, and CrowdAI.

  12. GeoProg3D: Compositional Visual Reasoning for City-Scale 3D Language Fields

    cs.CV 2025-06 conditional novelty 6.0 of 10

    GeoProg3D combines a georeferenced hierarchical 3D language field, geographic vision APIs, and LLM-generated programs to answer natural-language queries about city-scale 3D scenes, and includes a new 952-query benchma...

  13. UrbanLLaVA: A Multi-modal Large Language Model for Urban Intelligence with Spatial Reasoning and Understanding

    cs.CV 2025-06 conditional novelty 6.0 of 10

    A fine-tuned small multimodal LLM outperforms much larger general models on urban tasks in a new benchmark, with caveats about benchmark overlap with training data.

  14. GeoPix: Multi-Modal Large Language Model for Pixel-level Image Understanding in Remote Sensing

    cs.CV 2025-01 conditional novelty 6.0 of 10

    GeoPix extends multimodal language models for remote sensing to pixel-level referring segmentation through a mask predictor, a class-wise learnable memory, and a new 65,463-image instruction dataset.

  15. EarthDial: Turning Multi-sensory Earth Observations to Interactive Dialogues

    cs.CV 2024-12 conditional novelty 6.0 of 10

    EarthDial is a 4B-parameter remote sensing chatbot trained on 11.11M instruction pairs to handle multi-resolution, multi-spectral, and multi-temporal satellite imagery, and it reports gains over prior VLMs on dozens o...

  16. RSUniVLM: A Unified Vision Language Model for Remote Sensing via Granularity-oriented Mixture of Experts

    cs.CV 2024-12 conditional novelty 6.0 of 10

    RSUniVLM is a 1-billion-parameter remote sensing vision-language model that unifies image-, region-, and pixel-level tasks plus multi-image change analysis, achieving state-of-the-art visual grounding on VRSBench and ...

  17. GEOBench-VLM: Benchmarking Vision-Language Models for Geospatial Tasks

    cs.CV 2024-11 conditional novelty 6.0 of 10

    State-of-the-art vision-language models answer only 41.7% of GEOBench-VLM multiple-choice questions correctly, roughly double random chance.

  18. More with Less: a Large Scale Remote Sensing VLM with a Simple Recipe

    cs.CV 2026-07 conditional novelty 5.0 of 10

    An unmodified general-purpose VLM trained with multi-task RL and a SAM3 tool reaches top results on most remote sensing zero-shot benchmarks, with gains the paper attributes to training-data diversity rather than arch...

  19. Remote Sensing Large Vision-Language Model: Semantic-augmented Multi-level Alignment and Semantic-aware Expert Modeling

    cs.CV 2025-06 conditional novelty 5.0 of 10

    A remote sensing LVLM that augments visual features with retrieved captions and routes them through level-specific experts improves performance on several RS vision-language benchmarks.

  20. A Vision-Language Framework for Multispectral Scene Representation Using Language-Grounded Features

    cs.CV 2025-01 conditional novelty 5.0 of 10

    A lightweight projection layer trained to align frozen SpectralGPT multispectral features with LLaMA-3 text embeddings markedly improves EuroSAT classification and enables multispectral scene description.

  21. A Simple Aerial Detection Baseline of Multimodal Language Models

    cs.CV 2025-01 conditional novelty 5.0 of 10

    Fine-tuned Florence-2 detects rotated aerial objects in text form and matches conventional detectors under the paper's confidence-free mAPnc and mF1 metrics.

  22. UniRS: Unifying Multi-temporal Remote Sensing Tasks through Vision Language Models

    cs.CV 2024-12 conditional novelty 5.0 of 10

    A single vision-language model is fine-tuned to handle three remote sensing input types and reports state-of-the-art results on VQA, change captioning, and video classification benchmarks.

  23. LHRS-Bot-Nova: Improved Multimodal Large Language Model for Remote Sensing Vision-Language Interpretation

    cs.CV 2024-11 conditional novelty 5.0 of 10

    A remote sensing chatbot with recaptioned image-text data and a mixture-of-experts visual bridge reports gains over prior general and remote sensing models, but no code or data are released yet.

  24. REO-VLM: Transforming VLM to Meet Regression Challenges in Earth Observation

    cs.CV 2024-12 reject novelty 4.0 of 10

    A vision-language model for Earth observation that adds numeric biomass regression to generative question answering, with R^2 up to 0.36 and patch counting still failing.

  25. Vision-Language Modeling Meets Remote Sensing: Models, Datasets and Perspectives

    cs.CV 2025-05 conditional novelty 3.0 of 10

    A structured review of remote sensing vision-language models, organizing contrastive, instruction-tuned, and generative approaches alongside their datasets and benchmarks.

Pith tools