Pith. sign in

REVIEW 3 major objections 5 minor 51 references

The paper claims that the key to language-conditioned UAV video segmentation is how sparse and dense visual tokens are allocated, and that its residual-guided split beats uniform sampling.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 18:06 UTC pith:SEDGDZQI

load-bearing objection Credible incremental method for UAV video grounding, but the headline comparison to prior work is under-specified because baseline fine-tuning status is never stated. the 3 major comments →

arxiv 2607.17386 v1 pith:SEDGDZQI submitted 2026-07-19 cs.CV

SkyVLaM: Multimodal Large Language Model for UAV Video Understanding in Remote Sensing

Kaiwen Jing , Ruixu Jia , Bingyao Li , Ruizhe Ou , Ming Wu , Chuang Zhang This is my paper
classification cs.CV
keywords multimodal large language modelsUAV video understandingreferring video object segmentationvisual token allocationtemporal basis perceiverdense selectionremote sensingvideo grounded conversation generation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that the main obstacle to language-driven segmentation of objects in drone videos is not model capacity but how the limited visual token budget is split between a global temporal summary and high-resolution frame details. It proposes SkyVLaM, which compresses the whole video into a small set of sparse basis tokens, then uses the mismatch between each frame and those basis tokens to pick one contiguous segment for dense inspection. On a new dataset built from an existing UAV benchmark, this allocation beats uniform sampling and previously published video segmentation MLLMs across grounding and captioning metrics, supporting the claim that token allocation is the central lever. The work also contributes the SkyVid dataset of pixel-level masks and grounded text annotations for 101 UAV videos.

Core claim

The central discovery claim is that for UAV language-conditioned video segmentation, the visual token budget should be spent on a sparse temporal abstraction plus a carefully chosen dense segment, rather than uniform sparse-dense sampling. SkyVLaM constructs sparse tokens from patch-level video representations through a temporal basis perceiver—one global token plus window-level basis tokens—with a diversity regularization that forces the basis to carry complementary temporal cues. An adaptive dense selector then scores each frame by the squared residual between its summary and the projection onto the sparse basis, and retains the contiguous segment with the highest accumulated score for ful

What carries the argument

The temporal basis perceiver and the adaptive dense selector form the engine. The perceiver turns full-video patch tokens into one global token plus R learnable query tokens per temporal window, regularized to be mutually decorrelated via an off-diagonal Gram-matrix penalty. The selector computes each frame's residual to the spanned basis space and picks the highest-scoring contiguous K-frame segment; those full-resolution patch tokens form the dense tokens. Together they implement a learned, query-free allocation of the token budget between temporal coverage and spatial fidelity.

Load-bearing premise

The load-bearing assumption is that the compared baselines were evaluated under the same training conditions (same fine-tuning, same token budget) as SkyVLaM; if they were run zero-shot, the reported performance gaps reflect a training-data advantage, not architectural superiority.

What would settle it

Fine-tune each compared baseline on the same SkyVid-RVOS training split, give each the same total visual token budget and the same segmentation mask decoder, and rerun the comparison. If SkyVLaM no longer leads on gIoU, the claim that learned sparse-dense allocation is superior to uniform sampling is falsified.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Replacing uniform sparse-dense sampling with learned temporal-basis sparsification plus residual-guided dense selection raises referring video object segmentation gIoU from 20.68 to 23.52 on the new benchmark.
  • The basis diversity regularization and adaptive dense selection improve different aspects: the regularizer helps average mask quality (mIoU), while adaptive selection helps global grounding (gIoU).
  • The learned basis compresses at a rate set by window stride and basis capacity, not video length, so longer videos do not inflate the sparse token count.
  • SkyVid provides 1.53M pixel-level instances and 916 QA pairs for UAV video grounding, enabling training of this class of model.
  • The same sparse-dense representation extends to grounded conversation generation, improving caption quality metrics (CIDEr 82.6) alongside mask quality.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the residual scores could serve as a cheap automatic 'interestingness' signal for UAV footage—moments the sparse summary fails to explain are likely where new objects appear or targets maneuver.
  • Editorial inference: the same sparse-basis-plus-residual-selection recipe is transferable to other long-video MLLMs and to egocentric or surveillance video, not just aerial views.
  • Editorial inference: the evaluation is confined to one benchmark built from one source; whether the 2-3 point gIoU gain persists on other drone or scene datasets is untested.
  • Editorial inference: if baselines were not fine-tuned on the same training split, the reported margins may overstate the architecture's advantage; the strongest test is an equal-budget fine-tuned comparison.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes SkyVLaM, a multimodal large language model for UAV video understanding that combines a temporal basis perceiver (which summarizes patch-level video tokens into a compact sparse representation with diversity regularization) with an adaptive dense selector (which chooses a temporally coherent, high-information segment for dense inspection). The model is trained on a new SkyVid dataset built from VisDrone, comprising 101 videos, 33.6K frames, and 1.53M pixel-level object instances, with two annotation strategies (HPA-STE, BVAF). The paper reports superior results on video grounded conversation generation and referring video object segmentation compared with VideoGLaMM, VideoLISA, VISA, and VRS-HQ, and presents ablations showing each proposed component contributes to the final performance.

Significance. If the results are taken at face value, the sparse-dense token allocation formulation and the new UAV video grounding benchmark are useful contributions. The internal ablations (Tables 3–7) are controlled and demonstrate monotonic improvements as the proposed components are added, which supports internal coherence. The dataset construction addresses a real gap: existing UAV benchmarks lack language-conditioned pixel-level video annotations. However, the headline claim of outperforming prior methods rests on an evaluation setup whose fairness is not established, and the lack of error bars makes even the internal ordering partly fragile. The paper does not provide code or release plans, which limits immediate reproducibility, though the method description is reasonably detailed.

major comments (3)
  1. [§5.2–5.3, Tables 1–2] The paper never states whether the compared baselines (VideoGLaMM, VideoLISA, VISA, VRS-HQ) were fine-tuned on the SkyVid training split. Notably, the first row of Table 3 (no proposed components) is numerically identical to VideoLISA+SAM2 in Table 2 (gIoU 20.68, cIoU 37.31, mIoU 22.91), suggesting that baseline was trained with the same protocol, but the paper does not say so explicitly. If VISA, VideoGLaMM, and VRS-HQ were evaluated zero-shot or under their original training, the large gaps (e.g., SkyVLaM 23.52 vs VideoGLaMM 19.60 gIoU in Table 2) may reflect training-data exposure rather than architectural superiority. This is load-bearing for the central claim of 'best overall performance' and must be addressed by reporting the fine-tuning protocol per baseline and, where applicable, adding fine-tuned baselines.
  2. [§5.4, Tables 3–7] No error bars, variance, or significance tests are reported. The test set is only 20 videos / 714 QA pairs, and several adjacent results are close (e.g., Table 3: 22.83 vs 23.52 gIoU; Table 7: 23.52 vs 23.47 gIoU for 1+4N vs 1+8N). Without multiple seeds or a significance measure, the claimed ordering of ablations and the conclusion that the default budget is the 'best trade-off' are not statistically supported. Report mean ± std over at least three seeds, or another justified protocol.
  3. [§5.3, Table 2] The comparison to prior RVOS methods is additionally confounded by the absence of any description of the baselines' input sampling or output mask resolution. For example, VRS-HQ was designed for high-quality temporal reasoning segmentation and VideoGLaMM for visual grounding in longer videos; the paper does not state how these were adapted to the SkyVid frame length and token budget. If the baselines were run with default settings that are not matched to the UAV scenario, the reported gaps again may not reflect a fair comparison. Please specify the adaptation details and the inference-time settings for each baseline.
minor comments (5)
  1. [§3.2, Eq. (2)] The notation bZ(n) denotes a row-normalized matrix, but the normalization dimension and the norm used are not defined. Please clarify whether rows are L2-normalized over the feature dimension.
  2. [§5.1] The initialization is described as 'LLaVA [19, 26]' but [26] is LLaVA++. Specify the exact checkpoint and the LoRA rank/alpha, since these affect token efficiency claims.
  3. [Tables 1–2] The metrics gIoU, cIoU, and mIoU are never defined. mIoU is standard, but gIoU and cIoU need explicit definitions (e.g., global IoU and cumulative IoU) and the evaluation code or metric description should be provided.
  4. [Figure 2] The figure has duplicated panel labels (a) and (b) in different places and the text contains placeholder symbols (??, ?). This makes the architecture description unnecessarily hard to read. Please redraw and fix the labels.
  5. [§4, Dataset] The paper states annotations were generated by GPT-4o and 'manually reviewed', but no inter-annotator agreement or post-refinement quality measures are reported. A small annotation-quality study (e.g., agreement on mask correction, caption grounding consistency) would strengthen confidence in the benchmark.

Circularity Check

0 steps flagged

No significant circularity: the model is evaluated on held-out SkyVid test data and the central comparisons are controlled ablations, not fits renamed as predictions.

full rationale

SkyVLaM is an empirical architecture paper; its claims are supported by training on the SkyVid training split and evaluating on the retained VisDrone test split, not by a derivation that reduces to its inputs. The temporal basis perceiver, diversity regularization, and adaptive dense selector are defined by explicit equations (Eqs. 1-4) and trained with language, segmentation, and diversity losses (Eq. 7); performance gains in Tables 3-7 are measured against ablations that hold the training protocol fixed. The first row of Table 3 (20.68/37.31/22.91) equals the VideoLISA+SAM2 row in Table 2 because the paper explicitly defines the fully ablated model as the VideoLISA-style sparse-dense baseline; this is internal consistency, not circularity. The only author-overlapping citation, GeoPix [24], appears as ordinary related-work context in the Introduction and does not carry a load-bearing assumption for SkyVLaM's design or results. Concerns that baseline fine-tuning status is not reported would affect comparative validity, but they do not demonstrate that any prediction is equivalent to a fitted input or that any claim is forced by self-citation. No circular step can be exhibited.

Axiom & Free-Parameter Ledger

5 free parameters · 3 axioms · 0 invented entities

The method has several hand-set hyperparameters. The key domain assumptions are about mask and caption quality; the comparison assumption is the main load-bearing premise for the empirical claims.

free parameters (5)
  • R (basis tokens per temporal window) = 4
    Default sparse token count 1+4N implies R=4 per window; chosen by experiments in Table 7.
  • N (number of temporal windows) = 8
    Table 7 caption states N=8 for the default configuration.
  • K (dense frames selected) = 6
    Default dense frames = 6 from Table 7; Table 6 ablates this choice.
  • λ_div (diversity loss weight) = 0.05
    Best variant in Table 5 uses window-wise decorrelation with λ_div=0.05.
  • λ_lm, λ_bce, λ_dice weights = not stated
    Loss weights in Eq. (6)-(7) are not given numerical values in the paper.
axioms (3)
  • domain assumption SAM2-generated masks corrected by annotators are a valid ground truth for referring segmentation.
    The dataset and evaluation metrics rely on the quality of these masks (Section 4.1). No quality statistics for mask accuracy are reported.
  • domain assumption GPT-4o captions with manual refinement are reliable textual annotations.
    The VGCG task depends on these captions; the paper does not provide inter-annotator agreement.
  • ad hoc to paper The compared baselines are evaluated under conditions making performance comparable to SkyVLaM.
    The paper never states whether baselines were fine-tuned on SkyVid; without this, the comparison is not interpretable.

pith-pipeline@v1.3.0-alltime-deepseek · 12965 in / 10290 out tokens · 101412 ms · 2026-08-01T18:06:44.950581+00:00 · methodology

0 comments
read the original abstract

Recent advances in Multimodal Large Language Models (MLLMs) have significantly improved remote sensing (RS) multimodal understanding. Language-conditioned segmentation is crucial for fine-grained target understanding in Unmanned Aerial Vehicle (UAV) videos. However, this task remains challenging due to the prevalence of small, visually ambiguous targets and dynamic aerial perspectives. In this paper, we propose SkyVLaM, a multimodal large language model for UAV video understanding. SkyVLaM constructs sparse tokens directly from patch-level video representations through a temporal basis perceiver, regularizes the sparse basis to encourage complementary temporal cues, and adaptively selects a temporally coherent dense segment for high-resolution inspection. The resulting sparse and dense tokens are jointly processed by a large language model for query-conditioned segmentation. We further build SkyVid, consisting of SkyVid-VGCG and SkyVid-RVOS for video grounded conversation generation and referring video object segmentation, respectively. SkyVid contains 101 videos, 33.6K frames, and 1.53M pixel-level object instances. Experiments show that SkyVLaM provides a more effective allocation of the visual token budget and improves language-conditioned video segmentation in UAV scenarios.

Figures

Figures reproduced from arXiv: 2607.17386 by Bingyao Li, Chuang Zhang, Kaiwen Jing, Ming Wu, Ruixu Jia, Ruizhe Ou.

Figure 1
Figure 1. Figure 1: Qualitative example of SkyVLaM on UAV video pixel grounding. Given a UAV video and a user instruction, SkyVLaM generates a grounded response while associating referred categories with pixel-level masks. The example shows that the model can identify the primary visual subject in the video, describe surrounding contextual objects, and ground multiple categories such as cars, pedestrians, vans, and motors acr… view at source ↗
Figure 2
Figure 2. Figure 2: Overview of SkyVLaM. Given a UAV video and a text prompt, SkyVLaM constructs sparse [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Examples of SkyVid-VGCG. The left panel displays representative frames sampled from a UAV video sequence. The upper and lower parts of the center panel present textual descriptions generated under the Hierarchical Pyramid & Spatio-temporal Evolution (HPA-STE) and Biomimetic Visual Attention Flow (BVAF) strategies, respectively. The right panel visualizes representative object masks, with blue, red, and gre… view at source ↗
Figure 4
Figure 4. Figure 4: Video objects statistics and word cloud. (a) Objects count distribution across all categories. (b) Cumulative distribution function of instance counts. (c) Word cloud of textual annotations [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Qualitative results of SkyVLaM on SkyVid-VGCG. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

51 extracted references · 9 linked inside Pith

  1. [1]

    One token to seg them all: Language instructed reasoning seg- mentation in videos.Advances in Neural Information Processing Systems, 37:6833–6859, 2024

    Zechen Bai, Tong He, Haiyang Mei, Pichao Wang, Ziteng Gao, Joya Chen, Lei Liu, Zheng Zhang, and Mike Z Shou. One token to seg them all: Language instructed reasoning seg- mentation in videos.Advances in Neural Information Processing Systems, 37:6833–6859, 2024

    2024

  2. [2]

    Meteor: An automatic metric for mt evaluation with improved correlation with human judgments

    Satanjeev Banerjee and Alon Lavie. Meteor: An automatic metric for mt evaluation with improved correlation with human judgments. InProceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization, pages 65–72, 2005

    2005

  3. [3]

    End-to-end referring video object segmentation with multimodal transformers

    Adam Botach, Evgenii Zheltonozhskii, and Chaim Baskin. End-to-end referring video object segmentation with multimodal transformers. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4985–4995, 2022

    2022

  4. [4]

    Streamingtom: Streaming token compres- sion for efficient video understanding.arXiv preprint arXiv:2510.18269, 2025

    Xueyi Chen, Keda Tao, Kele Shao, and Huan Wang. Streamingtom: Streaming token compres- sion for efficient video understanding.arXiv preprint arXiv:2510.18269, 2025

    arXiv 2025

  5. [5]

    Mevis: A large-scale benchmark for video segmentation with motion expressions

    Henghui Ding, Chang Liu, Shuting He, Xudong Jiang, and Chen Change Loy. Mevis: A large-scale benchmark for video segmentation with motion expressions. InProceedings of the IEEE/CVF international conference on computer vision, pages 2694–2703, 2023

    2023

  6. [6]

    The unmanned aerial vehicle benchmark: Object detection and tracking

    Dawei Du, Yuankai Qi, Hongyang Yu, Yifan Yang, Kaiwen Duan, Guorong Li, Weigang Zhang, Qingming Huang, and Qi Tian. The unmanned aerial vehicle benchmark: Object detection and tracking. InProceedings of the European conference on computer vision (ECCV), pages 370–386, 2018

    2018

  7. [7]

    Framefusion: Combining similarity and importance for video token reduction on large vision language models

    Tianyu Fu, Tengxuan Liu, Qinghao Han, Guohao Dai, Shengen Yan, Huazhong Yang, Xuefei Ning, and Yu Wang. Framefusion: Combining similarity and importance for video token reduction on large vision language models. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 22654–22663, 2025

    2025

  8. [8]

    Chain-of-frames: Advancing video understanding in multimodal llms via frame-aware reasoning.arXiv preprint arXiv:2506.00318, 2025

    Sara Ghazanfari, Francesco Croce, Nicolas Flammarion, Prashanth Krishnamurthy, Farshad Khorrami, and Siddharth Garg. Chain-of-frames: Advancing video understanding in multimodal llms via frame-aware reasoning.arXiv preprint arXiv:2506.00318, 2025

    Pith/arXiv arXiv 2025

  9. [9]

    The devil is in temporal token: High quality video reasoning segmentation

    Sitong Gong, Yunzhi Zhuge, Lu Zhang, Zongxin Yang, Pingping Zhang, and Huchuan Lu. The devil is in temporal token: High quality video reasoning segmentation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 29183–29192, 2025

    2025

  10. [10]

    Rsgpt: A remote sensing vision language model and benchmark.ISPRS Journal of Photogrammetry and Remote Sensing, 224:272–286, 2025

    Yuan Hu, Jianlong Yuan, Congcong Wen, Xiaonan Lu, Yu Liu, and Xiang Li. Rsgpt: A remote sensing vision language model and benchmark.ISPRS Journal of Photogrammetry and Remote Sensing, 224:272–286, 2025

    2025

  11. [11]

    Prunevid: Visual token pruning for efficient video large language models

    Xiaohu Huang, Hao Zhou, and Kai Han. Prunevid: Visual token pruning for efficient video large language models. InFindings of the Association for Computational Linguistics: ACL 2025, pages 19959–19973, 2025

    2025

  12. [12]

    Multi-granular spatio-temporal token merging for training-free acceleration of video llms

    Jeongseok Hyun, Sukjun Hwang, Su Ho Han, Taeoh Kim, Inwoong Lee, Dongyoon Wee, Joon- Young Lee, Seon Joo Kim, and Minho Shim. Multi-granular spatio-temporal token merging for training-free acceleration of video llms. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 23990–24000, 2025

    2025

  13. [13]

    Teochat: A large vision-language assistant for temporal earth observation data.arXiv preprint arXiv:2410.06234, 2024

    Jeremy Andrew Irvin, Emily Ruoyu Liu, Joyce Chuyi Chen, Ines Dormoy, Jinyoung Kim, Samar Khanna, Zhuo Zheng, and Stefano Ermon. Teochat: A large vision-language assistant for temporal earth observation data.arXiv preprint arXiv:2410.06234, 2024

    Pith/arXiv arXiv 2024

  14. [14]

    Lisa: Reasoning segmentation via large language model

    Xin Lai, Zhuotao Tian, Yukang Chen, Yanwei Li, Yuhui Yuan, Shu Liu, and Jiaya Jia. Lisa: Reasoning segmentation via large language model. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9579–9589, 2024. 12

    2024

  15. [15]

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. InInternational conference on machine learning, pages 19730–19742. PMLR, 2023

    2023

  16. [16]

    Referdino: Referring video object segmentation with visual grounding foundations

    Tianming Liang, Kun-Yu Lin, Chaolei Tan, Jianguo Zhang, Wei-Shi Zheng, and Jian-Fang Hu. Referdino: Referring video object segmentation with visual grounding foundations. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 20009– 20019, 2025

    2025

  17. [17]

    Video-llava: Learning united visual representation by alignment before projection

    Bin Lin, Yang Ye, Bin Zhu, Jiaxi Cui, Munan Ning, Peng Jin, and Li Yuan. Video-llava: Learning united visual representation by alignment before projection. InProceedings of the 2024 conference on empirical methods in natural language processing, pages 5971–5984, 2024

    2024

  18. [18]

    Glus: Global-local reasoning unified into a single large language model for video segmentation

    Lang Lin, Xueyang Yu, Ziqi Pang, and Yu-Xiong Wang. Glus: Global-local reasoning unified into a single large language model for video segmentation. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 8658–8667, 2025

    2025

  19. [19]

    Visual instruction tuning.Advances in neural information processing systems, 36:34892–34916, 2023

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning.Advances in neural information processing systems, 36:34892–34916, 2023

    2023

  20. [20]

    Rsunivlm: A unified vision language model for remote sensing via granularity-oriented mixture of experts.arXiv preprint arXiv:2412.05679, 2024

    Xu Liu and Zhouhui Lian. Rsunivlm: A unified vision language model for remote sensing via granularity-oriented mixture of experts.arXiv preprint arXiv:2412.05679, 2024

    Pith/arXiv arXiv 2024

  21. [21]

    Skysensegpt: A fine-grained instruction tuning dataset and model for remote sensing vision-language understanding.arXiv preprint arXiv:2406.10100, 2024

    Junwei Luo, Zhen Pang, Yongjun Zhang, Tingzhu Wang, Linlin Wang, Bo Dang, Jiangwei Lao, Jian Wang, Jingdong Chen, Yihua Tan, et al. Skysensegpt: A fine-grained instruction tuning dataset and model for remote sensing vision-language understanding.arXiv preprint arXiv:2406.10100, 2024

    Pith/arXiv arXiv 2024

  22. [22]

    Video-chatgpt: Towards detailed video understanding via large vision and language models

    Muhammad Maaz, Hanoona Rasheed, Salman Khan, and Fahad Khan. Video-chatgpt: Towards detailed video understanding via large vision and language models. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12585–12602, 2024

    2024

  23. [23]

    Videoglamm: A large multimodal model for pixel-level visual grounding in videos

    Shehan Munasinghe, Hanan Gani, Wenqi Zhu, Jiale Cao, Eric Xing, Fahad Shahbaz Khan, and Salman Khan. Videoglamm: A large multimodal model for pixel-level visual grounding in videos. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 19036–19046, 2025

    2025

  24. [24]

    Geopix: A multimodal large language model for pixel-level image understanding in remote sensing.IEEE Geoscience and Remote Sensing Magazine, 2025

    Ruizhe Ou, Yuan Hu, Fan Zhang, Jiaxin Chen, and Yu Liu. Geopix: A multimodal large language model for pixel-level image understanding in remote sensing.IEEE Geoscience and Remote Sensing Magazine, 2025

    2025

  25. [25]

    Vhm: Versatile and honest vision language model for remote sensing image analysis

    Chao Pang, Xingxing Weng, Jiang Wu, Jiayu Li, Yi Liu, Jiaxing Sun, Weijia Li, Shuai Wang, Litong Feng, Gui-Song Xia, et al. Vhm: Versatile and honest vision language model for remote sensing image analysis. InProceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 6381–6388, 2025

    2025

  26. [26]

    Llava++: extending visual capabilities with llama-3 and phi-3 (2024).URL https://github.com/mbzuai-oryx/LLaVA- pp, 2024

    Hanoona Rasheed, Muhammad Maaz, Salman Khan, and Fahad S Khan. Llava++: extending visual capabilities with llama-3 and phi-3 (2024).URL https://github.com/mbzuai-oryx/LLaVA- pp, 2024

    2024

  27. [27]

    Glamm: Pixel grounding large multimodal model

    Hanoona Rasheed, Muhammad Maaz, Sahal Shaji, Abdelrahman Shaker, Salman Khan, Hisham Cholakkal, Rao M Anwer, Eric Xing, Ming-Hsuan Yang, and Fahad S Khan. Glamm: Pixel grounding large multimodal model. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13009–13018, 2024

    2024

  28. [28]

    Sam 2: Segment anything in images and videos.arXiv preprint arXiv:2408.00714, 2024

    Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, et al. Sam 2: Segment anything in images and videos.arXiv preprint arXiv:2408.00714, 2024

    Pith/arXiv arXiv 2024

  29. [29]

    Pixellm: Pixel reasoning with large multimodal model

    Zhongwei Ren, Zhicheng Huang, Yunchao Wei, Yao Zhao, Dongmei Fu, Jiashi Feng, and Xiaojie Jin. Pixellm: Pixel reasoning with large multimodal model. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26374–26383, 2024. 13

    2024

  30. [30]

    Moviechat: From dense token to sparse memory for long video understanding

    Enxin Song, Wenhao Chai, Guanhong Wang, Yucheng Zhang, Haoyang Zhou, Feiyang Wu, Haozhe Chi, Xun Guo, Tian Ye, Yanting Zhang, et al. Moviechat: From dense token to sparse memory for long video understanding. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18221–18232, 2024

    2024

  31. [31]

    Earthdial: Turning multi-sensory earth observations to interactive dialogues

    Sagar Soni, Akshay Dudhane, Hiyam Debary, Mustansar Fiaz, Muhammad Akhtar Munir, Muhammad Sohail Danish, Paolo Fraccaro, Campbell D Watson, Levente J Klein, Fahad Shah- baz Khan, et al. Earthdial: Turning multi-sensory earth observations to interactive dialogues. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 14303–14313, 2025

    2025

  32. [32]

    Drone-based rgb-infrared cross-modality vehicle detection via uncertainty-aware learning.IEEE Transactions on Circuits and Systems for Video Technology, 32(10):6700–6713, 2022

    Yiming Sun, Bing Cao, Pengfei Zhu, and Qinghua Hu. Drone-based rgb-infrared cross-modality vehicle detection via uncertainty-aware learning.IEEE Transactions on Circuits and Systems for Video Technology, 32(10):6700–6713, 2022

    2022

  33. [33]

    Adaptive keyframe sampling for long video understanding

    Xi Tang, Jihao Qiu, Lingxi Xie, Yunjie Tian, Jianbin Jiao, and Qixiang Ye. Adaptive keyframe sampling for long video understanding. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 29118–29128, 2025

    2025

  34. [34]

    Cider: Consensus-based image description evaluation

    Ramakrishna Vedantam, C Lawrence Zitnick, and Devi Parikh. Cider: Consensus-based image description evaluation. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 4566–4575, 2015

    2015

  35. [35]

    Geollava-8k: scaling remote-sensing multimodal large language models to 8k resolution.arXiv preprint arXiv:2505.21375, 2025

    Fengxiang Wang, Mingshuo Chen, Yueying Li, Di Wang, Haotian Wang, Zonghao Guo, Zefan Wang, Boqi Shan, Long Lan, Yulin Wang, et al. Geollava-8k: scaling remote-sensing multimodal large language models to 8k resolution.arXiv preprint arXiv:2505.21375, 2025

    arXiv 2025

  36. [36]

    Instructseg: Unifying instructed visual segmentation with multi-modal large language models

    Cong Wei, Yujie Zhong, Haoxian Tan, Yingsen Zeng, Yong Liu, Hongfa Wang, and Yujiu Yang. Instructseg: Unifying instructed visual segmentation with multi-modal large language models. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 20193–20203, 2025

    2025

  37. [37]

    Longvlm: Efficient long video understanding via large language models

    Yuetian Weng, Mingfei Han, Haoyu He, Xiaojun Chang, and Bohan Zhuang. Longvlm: Efficient long video understanding via large language models. InEuropean Conference on Computer Vision, pages 453–470. Springer, 2024

    2024

  38. [38]

    Language as queries for referring video object segmentation

    Jiannan Wu, Yi Jiang, Peize Sun, Zehuan Yuan, and Ping Luo. Language as queries for referring video object segmentation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4974–4984, 2022

    2022

  39. [39]

    Slowfast-llava: A strong training-free baseline for video large language models.arXiv preprint arXiv:2407.15841, 2024

    Mingze Xu, Mingfei Gao, Zhe Gan, Hong-You Chen, Zhengfeng Lai, Haiming Gang, Kai Kang, and Afshin Dehghan. Slowfast-llava: A strong training-free baseline for video large language models.arXiv preprint arXiv:2407.15841, 2024

    Pith/arXiv arXiv 2024

  40. [40]

    Visa: Reasoning video object segmentation via large language models

    Cilin Yan, Haochen Wang, Shilin Yan, Xiaolong Jiang, Yao Hu, Guoliang Kang, Weidi Xie, and Efstratios Gavves. Visa: Reasoning video object segmentation via large language models. In European Conference on Computer Vision, pages 98–115. Springer, 2024

    2024

  41. [41]

    Referred by multi-modality: A unified temporal transformer for video object segmentation

    Shilin Yan, Renrui Zhang, Ziyu Guo, Wenchao Chen, Wei Zhang, Hongyang Li, Yu Qiao, Hao Dong, Zhongjiang He, and Peng Gao. Referred by multi-modality: A unified temporal transformer for video object segmentation. InProceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 6449–6457, 2024

    2024

  42. [42]

    Self-chained image-language model for video localization and question answering.Advances in Neural Information Processing Systems, 36:76749–76771, 2023

    Shoubin Yu, Jaemin Cho, Prateek Yadav, and Mohit Bansal. Self-chained image-language model for video localization and question answering.Advances in Neural Information Processing Systems, 36:76749–76771, 2023

    2023

  43. [43]

    Frame-voyager: Learning to query frames for video large language models.arXiv preprint arXiv:2410.03226, 2024

    Sicheng Yu, Chengkai Jin, Huanyu Wang, Zhenghao Chen, Sheng Jin, Zhongrong Zuo, Xiaolei Xu, Zhenbang Sun, Bingni Zhang, Jiawei Wu, et al. Frame-voyager: Learning to query frames for video large language models.arXiv preprint arXiv:2410.03226, 2024

    Pith/arXiv arXiv 2024

  44. [44]

    Sa2va: Marrying sam2 with llava for dense grounded understanding of images and videos.arXiv preprint arXiv:2501.04001, 2025

    Haobo Yuan, Xiangtai Li, Tao Zhang, Yueyi Sun, Zilong Huang, Shilin Xu, Shunping Ji, Yunhai Tong, Lu Qi, Jiashi Feng, et al. Sa2va: Marrying sam2 with llava for dense grounded understanding of images and videos.arXiv preprint arXiv:2501.04001, 2025. 14

    Pith/arXiv arXiv 2025

  45. [45]

    Skyeyegpt: Unifying remote sensing vision- language tasks via instruction tuning with large language model.ISPRS Journal of Photogram- metry and Remote Sensing, 221:64–77, 2025

    Yang Zhan, Zhitong Xiong, and Yuan Yuan. Skyeyegpt: Unifying remote sensing vision- language tasks via instruction tuning with large language model.ISPRS Journal of Photogram- metry and Remote Sensing, 221:64–77, 2025

    2025

  46. [46]

    Video-llama: An instruction-tuned audio-visual language model for video understanding

    Hang Zhang, Xin Li, and Lidong Bing. Video-llama: An instruction-tuned audio-visual language model for video understanding. InProceedings of the 2023 conference on empirical methods in natural language processing: system demonstrations, pages 543–553, 2023

    2023

  47. [47]

    Georsmllm: A multimodal large language model for vision-language tasks in geoscience and remote sensing.arXiv preprint arXiv:2503.12490, 2025

    Zilun Zhang, Haozhan Shen, Tiancheng Zhao, Bin Chen, Zian Guan, Yuhao Wang, Xu Jia, Yux- iang Cai, Yongheng Shang, and Jianwei Yin. Georsmllm: A multimodal large language model for vision-language tasks in geoscience and remote sensing.arXiv preprint arXiv:2503.12490, 2025

    Pith/arXiv arXiv 2025

  48. [48]

    Tifre: Text-guided video frame reduction for efficient video multi-modal large language models.arXiv preprint arXiv:2602.08861, 2026

    Xiangtian Zheng, Zishuo Wang, and Yuxin Peng. Tifre: Text-guided video frame reduction for efficient video multi-modal large language models.arXiv preprint arXiv:2602.08861, 2026

    arXiv 2026

  49. [49]

    Reason: Reinforced causal search with information bottleneck for video understanding

    Yuan Zhou, Litao Hua, Shilong Jin, Wentao Huang, and Haoran Duan. Reason: Reinforced causal search with information bottleneck for video understanding. InProceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 13791–13799, 2026

    2026

  50. [50]

    Detection and tracking meet drones challenge.IEEE transactions on pattern analysis and machine intelligence, 44(11):7380–7399, 2021

    Pengfei Zhu, Longyin Wen, Dawei Du, Xiao Bian, Heng Fan, Qinghua Hu, and Haibin Ling. Detection and tracking meet drones challenge.IEEE transactions on pattern analysis and machine intelligence, 44(11):7380–7399, 2021

    2021

  51. [51]

    Focus: Efficient keyframe selection for long video understanding.arXiv preprint arXiv:2510.27280, 2025

    Zirui Zhu, Hailun Xu, Yang Luo, Yong Liu, Kanchan Sarkar, Zhenheng Yang, and Yang You. Focus: Efficient keyframe selection for long video understanding.arXiv preprint arXiv:2510.27280, 2025. 15

    arXiv 2025