Pith. sign in

REVIEW 3 major objections 4 minor 51 references

This paper claims that current vision-language models fail at continuous remote-sensing video understanding — 29.0% average accuracy versus 69.7% on natural videos — and that an evidence-focused two-stage RL framework recovers up to 9.01 po

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 16:10 UTC pith:S4YLADI3

load-bearing objection A genuinely useful benchmark and a serious paper, but the headline numbers rest on unverified gold labels and same-distribution tuning; the dataset release and a human baseline should decide it. the 3 major comments →

arxiv 2608.02039 v1 pith:S4YLADI3 submitted 2026-08-03 cs.CV

RSVideo: Are Your Vision-Language Models Ready for Remote Sensing Videos?

Hongjie Zhou , Shiqin Wang , Haoyang Chen , Haonan Guo , Di Wang , Juhua Liu , Fu Lin , Yong Luo This is my paper
classification cs.CV
keywords remote sensing videovideo question answeringvision-language modelsspatiotemporal evidencereinforcement learningbenchmarksmall-target perceptionUAV and satellite imagery
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper's core claim is that current vision-language models are not ready for remote-sensing video: across 26 backbones they average 29.0% accuracy on the new RSVideo-Bench but 69.7% on natural-video benchmarks under the same five-choice protocol — a 40.7-point gap the authors trace to small targets, short-lived state changes, and scene-constrained spatial relations that models systematically overlook. To make the problem trainable, they build RSVideo-10K, a 10,773-instance dataset (17.02 hours, UAV and satellite footage) in which 4,638 instances carry explicit temporal and spatial evidence annotations. They then propose RSVideo, a two-stage evidence-focused framework — supervised fine-tuning on the evidence-answer format, followed by GRPO reinforcement learning with a composite reward for answer correctness, evidence alignment, and background compression — and report consistent gains across all tested backbones, up to 9.01 points absolute, with transfer gains on four external video benchmarks. Why it matters: if right, the paper shows the bottleneck in overhead-video understanding is not scale but attention to sparse evidence, and that a publicly specified training recipe can close a meaningful slice of the gap.

Core claim

The paper's central claim: continuous remote-sensing video understanding is limited not by model scale or context length but by recovering sparse, weak spatiotemporal evidence — tiny targets, short-lived state changes, scene-constrained spatial relations. On new benchmark RSVideo-Bench, open-source models average 29.0% vs 69.7% on natural-video Video-MME, a 40.7-point gap, traced to four findings: sparse evidence is easily missed; scene context is necessary but redundant; frame-level recognition is insufficient for spatiotemporal reasoning; correct answers do not guarantee grounded decisions. The proposed two-stage framework (evidence-aware SFT plus GRPO RL with rewards for answer correctnes

What carries the argument

The load-bearing mechanism is the evidence-aware reward operating under a fixed visual-token budget. Each video token is scored by fusing global saliency (visual self-attention), question relevance (cross-modal similarity), temporal change (local frame-to-frame inconsistency), and a time–region cell prior; the top-K tokens are retained and the remainder are compressed through question-conditioned cross-attention into M background slots. The reward R = λ_ans R_ans + g_ans(λ_st R_st + λ_bg R_bg − λ_cost C_cost) couples answer correctness to overlap between the model's emitted evidence tag and the human-annotated temporal windows and spatial cells (T*, G*), while the background term rewards rou

Load-bearing premise

Everything hinges on the human-annotated gold answers and evidence labels being correct and uniquely determined by the released video frames — an assumption the paper supports with a three-expert adjudication pipeline but not with agreement statistics, and one that cannot yet be checked because the dataset is not released for audit (Appendix G.6).

What would settle it

Independently re-annotate a random sample of the 2,731 RSVideo-Bench items and measure expert agreement on gold answers and evidence windows; if agreement is low, or if text-only and single-frame inputs already reproduce most of the 29.0% baseline (the paper's own audit shows 23.49% and 33.17%), the 40.7-point gap would be a benchmark artifact rather than a model deficiency — and the RL gains should be re-tested with the evidence-alignment reward removed.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Existing off-the-shelf VLMs are not deployable for remote-sensing video analysis as-is; the 29.0% baseline quantifies the shortfall, and the paper's finding that capacity scaling from 1B to 241B barely moves the needle suggests the bottleneck is evidence selection rather than parameter count.
  • Evidence-annotated instruction data plus evidence-aligned reinforcement learning is a repeatable recipe: it beats outcome-only GRPO, T-GRPO, and GSPO on every one of the 26 backbones tested, so the gap-closing mechanism is the evidence reward, not the RL optimizer alone.
  • The policy transfers beyond the training distribution: RSVideo training improves average accuracy on MVBench (+0.66), Video-MME (+0.37), UrbanVideo-Bench (+1.52), and SIS-Bench (+1.64) with no external benchmark data used in training.
  • Answer accuracy and evidence grounding are separable and jointly optimizable: the full reward lifts accuracy to 40.63% while raising temporal-hit and region-hit grounding diagnostics to 57.4% and 54.8%, so model decisions become more auditable, not just more correct.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the paper's own validity audit (Appendix E.2) shows a random single frame reaches 33.17% versus 36.59% for the full video, and text-only input scores 23.49% against 20% chance — suggesting part of the headline 40.7-point gap reflects failed perception of small or confusable targets and reliance on static scene priors rather than temporal reasoning alone; a capability-resolved
  • Editorial inference: the evidence-tag format and the T-Hit/R-Hit metrics are portable diagnostics — any video QA system could be audited for whether its correct answers are grounded in the right frames and regions, independent of the proposed RL training — so the benchmark doubles as a grounding test for future models.
  • Editorial inference: because all footage comes from eight public UAV/satellite sources, the strongest stress test is coverage generalization — evaluating the trained policy on unseen platforms, geographic regions, or event types (e.g., maritime or wildlife footage) would reveal whether evidence-focused selection is a general mechanism or a prior fitted to those eight sources.
  • Editorial inference: the dataset's deliberate 'insufficient evidence' answer option — correct only when no released frame resolves the question — makes RSVideo-Bench a calibration test for overconfidence; a model that never selects that option is overclaiming, and the paper does not report this distribution, which could be a useful deployment summary metric.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces RSVideo-10K, a remote-sensing video QA dataset of 10,773 five-choice instances (8,042 train/validation, 2,731 locked test) built from eight public UAV/satellite video sources, with a taxonomy of two capability dimensions, seven groups, and 17 tasks. It reports that existing VLMs average 29.0% on RSVideo-Bench versus 69.7% on Video-MME, a 40.7-point gap, and proposes RSVideo, a two-stage training method combining evidence-aware SFT with GRPO-based reinforcement learning that uses spatiotemporal evidence rewards to focus on question-relevant tokens. The method is reported to improve all 26 evaluated backbones, with up to 9.01% absolute gain and a best accuracy of 40.63% on Qwen3.6-27B. The paper also includes transfer experiments to MVBench, Video-MME, UrbanVideo-Bench, and SIS-Bench, plus a validity audit using text-only, random-frame, shuffled-frame, and full-video inputs.

Significance. If the dataset and annotations are valid, the benchmark fills a genuine gap: existing remote-sensing benchmarks are mostly image-based or multi-temporal, while continuous UAV/satellite video QA is underrepresented. The paper is unusually careful about evaluation protocol: Appendix C fixes decoding, sampling, parsing, and option ordering, and Appendix E.2 reports an input-degradation audit showing that temporal order matters. The training method is evaluated across 26 backbones, an unusually broad sweep, and the transfer table provides a check against out-of-distribution overfitting. These are real strengths. However, both headline claims—the 40.7-point natural-to-remote-sensing gap and the causal role of evidence-aligned RL—rest on the correctness and uniqueness of human gold labels and evidence annotations. The manuscript reports no human baseline, no inter-annotator agreement, and the dataset is not yet released, so the benchmark's validity cannot currently be audited. The proposed reward formulation also contains an ambiguity in the background-compression term. The central ideas are plausible and the paper is publishable in principle, but the evidence needed to support the load-

major comments (3)
  1. [Appendix A.1, B.3, G.6; Table 10] The benchmark's validity rests on the assertion in Appendix A.1 that clips are retained only when 'the correct answer can be established from the released visual input.' The manuscript describes a three-expert pipeline in Appendix B.3 but reports no inter-annotator agreement statistics, no human accuracy on the 2,731 RSVideo-Bench items, and no released sample that an external reviewer can audit. Appendix G.6 states the dataset 'is not yet publicly distributed.' Without a human baseline, the headline 29.0% versus 69.7% gap cannot be interpreted: if human experts score near 90% the gap reflects model deficiency, but if they score near chance the benchmark is not measuring what the paper claims. The validity audit in Table 10 shows that models use video content and temporal order, but it does not establish that the gold answers are uniquely determinable from the video. This is load-bearing
  2. [§4.2, Eq. (7)] Equation (7) defines R_bg = |I_rem ∩ I*_bg|/|I*_bg| + (|I_rem|/L) M. The text states that 'the second favors summarizing more residual tokens with fewer background slots,' but the formula contains M in the numerator, so it rewards a larger number of background slots M, not fewer. As written, the reward is inconsistent with the stated objective of sparse background compression. Also, the first term is recall-only over non-target tokens, and the overall reward in Eq. (8) only gates evidence rewards on correct answers, so the mechanism by which the method learns to discard background for incorrect trajectories is unclear. The authors should correct the formula or the description and provide a derivation of what the second term is intended to measure.
  3. [§4.2, Eqs. (6)–(8); Table 3; Table 4] The evidence-alignment reward R_st in Eq. (6) is computed against T* and G* derived from RSVideo-Instruct, and the headline results are measured on RSVideo-Bench, which shares the same eight sources, taxonomy, and annotation protocol as the training set. The reward is also gated by g_ans = I[A=A*] I[E is valid], so trajectories with wrong answers receive no evidence-alignment signal. Consequently, the 1.95-point improvement attributed to R_st in Table 3 may reflect learning the RSVideo-specific tag format and answer distribution rather than a general spatiotemporal evidence-grounding capability. The transfer results in Table 4 show small positive gains on external benchmarks, but those benchmarks do not require evidence tags, so they do not test evidence grounding. To support the claim that RSVideo improves evidence-focused reasoning, the authors should evaluate evidence-alignment metric
minor comments (4)
  1. [§3.2] Typo: 'datset' should be 'dataset'.
  2. [Appendix F, Leaf 15 and Leaf 19] Minor typos: 'from start to end??' has a double question mark, and 'bule' should be 'blue'.
  3. [Appendix G.6] The datasheet says the dataset is not yet distributed, while the abstract and Section 1 state codes will be available. Please state explicitly in the main text what will be released at review time (e.g., a sample, evaluation scripts, prediction files) and what will be released at publication.
  4. [Table 10] The validity audit shows that 'Random single frame' outperforms 'Full video' on SP, TP, and CER. The text acknowledges this, but a brief explanation in the main text would help readers interpret the non-monotonicity and avoid the impression that the full-video advantage is inconsistent.

Circularity Check

1 steps flagged

Partial metric-level circularity in TH/RH grounding diagnostics; central accuracy and transfer claims remain independent.

specific steps
  1. fitted input called prediction [Eq. (6) and Appendix C.3; Table 3]
    "The first key reward aligns the predicted evidence with annotated temporal positions and spatial cells. Let bT(E) and bG(E) be the temporal indices and spatial cells parsed from E, and let T∗ and G∗ denote the corresponding annotations. We compute RT = | bT(E)∩ T∗|/|T∗|, RG = | bG(E)∩ G∗|/|G∗| ... Temporal Hit (T-Hit) is the fraction of items for which T̂ overlaps the annotated key frame or temporal window. Region Hit (R-Hit) is the fraction for which R̂ overlaps the annotated target region."

    The RL reward Rst (Eq. 6) is literally a coverage/overlap measure between the model's predicted evidence tag and the human-annotated T*, G*. The paper's reported grounding diagnostics T-Hit and R-Hit are the same overlap measure against the same annotation fields. Therefore, Table 3's finding that adding Rst increases TH/RH by 4.2/4.5 points is not an independent confirmation of 'evidence grounding': the model was trained to maximize exactly the quantity used as the diagnostic. This is a metric-level circularity. It does not, however, collapse the paper's headline accuracy claims, because accuracy on the held-out RSVideo-Bench and the external transfer results (MVBench, Video-MME, UrbanVideo-Bench, SIS-Bench) are not by construction tied to the evidence-overlap reward.

full rationale

The central claims of the paper are empirical rather than derivational: (1) current VLMs score substantially lower on RSVideo-Bench than on Video-MME, and (2) RSVideo training improves accuracy on the held-out RSVideo-Bench and transfers to external benchmarks. Neither claim reduces to the paper's inputs by construction. The benchmark gap is a measurement, and the accuracy gains are obtained on a locked test set with additional external-benchmark transfer evaluation (Table 4), which the paper explicitly motivates in Section 5.4 by noting that in-domain results alone would not establish generalization. The only circularity I can exhibit with the paper's own equations is narrower: the evidence-grounding rewards (Eq. 6-7) and the T-Hit/R-Hit diagnostics (Appendix C.3) are defined as the same overlap with the same annotations, so improvements in those diagnostics partly reflect fitting to the annotation scheme rather than an independent test of grounding. This is a secondary, metric-level issue and does not undermine the independent content of the accuracy and transfer results. No load-bearing self-citation or imported uniqueness theorem is present; references to the authors' prior benchmarks are contextual, not argumentative.

Axiom & Free-Parameter Ledger

6 free parameters · 5 axioms · 1 invented entities

The paper imports standard RL machinery (GRPO, KL-penalized policy optimization, LoRA) and standard vision encoders; these are listed as standard_math. Its domain premises are that remote-sensing videos are characterized by small targets, repetitive backgrounds, and scene-constrained relations, and that the eight public sources plus a three-expert annotation pipeline produce uniquely answerable questions. The latter is the load-bearing premise and is currently unauditable. The paper's own invented construct is the coarse temporal/spatial evidence-tag format, over which the rewards are defined; it has no falsifiable handle outside this benchmark. The free-parameter list reflects the validation sweeps that set eta, rho, reward weights, temperature, group size, and KL; the fusion weights alpha_sal/alpha_rel/alpha_chg/alpha_cell appear in the headline equations but their values are not reported in the released text.

free parameters (6)
  • Evidence score fusion weights alpha_sal, alpha_rel, alpha_chg (Eq. 2) = not reported in text
    Hand-chosen or learned weights balancing saliency, question-relevance, and temporal-change signals; values are not given in the main text or Appendix D.1, so the score driving token selection is not fully specified.
  • Region prior weight alpha_cell (Eq. 3) = not reported
    Weight on the coarse time-region cell prior that rescues weak single-token evidence; unspecified in the provided text.
  • Token pooling ratio eta = 0.10
    Selected on RSVideo-Instruct validation by sweeping {0.05, 0.10, 0.20}; accuracy, TH, and RH all peak at 0.10 (Appendix E.3), so the headline number depends on this choice.
  • Visual token budget rho = 0.40
    Selected on validation by sweeping {0.30, 0.40, 0.50}; 0.40 wins (Appendix E.4, Table 12).
  • Reward weights lambda_ans/lambda_st/lambda_bg/lambda_cost = 0.40/0.40/0.15/0.05
    Selected on validation from a weight grid (Appendix E.8, Table 18); the reported 40.63% is the best cell of this sweep.
  • GRPO group size, sampling temperature, KL coefficient = G=4, temp 0.7, KL 0.03
    All chosen by separate validation sweeps (Appendix E.7, Tables 15-17); each changes accuracy by 0.5-1.3 points, so the reported configuration is the best cell of a tuning procedure.
axioms (5)
  • standard math GRPO clipped policy objective with KL penalty to the SFT policy (Appendix D.5, Eq. 18) is a valid optimizer for the evidence-aware return.
    Taken from Shao et al. (DeepSeekMath), cited as [27]; the paper contributes the return definition, not the optimization algorithm.
  • domain assumption Remote-sensing videos are dominated by small targets, large repetitive backgrounds, subtle short-lived state changes, and scene-constrained spatial relations (Sections 2.2, 4.1).
    This premise motivates both the benchmark design (which tasks to include) and the method (why score fusion and background compression are needed). If real RS video understanding is not dominated by these factors, the benchmark under-samples the problem.
  • domain assumption Every retained clip's gold answer and evidence window are uniquely determinable from the released frames (Appendix A.1: clips retained 'only when the correct answer can be established from the released visual input').
    Load-bearing for both the 40.7% gap claim and the reward in Eq. 6-7; supported only by an unreleased three-expert adjudication description with no inter-annotator agreement statistics.
  • domain assumption The eight public sources (AU-AIR, DTB70, ERA, OOTB, Okutama, SatSOT, Stanford Drone, VISO) provide a representative sample of remote-sensing video understanding, and clip-level split isolation prevents leakage (Appendix A.1, A.3).
    Benchmark conclusions generalize only as far as these sources do; the datasheet (G.2.3) acknowledges the set is 'a curated sample'.
  • ad hoc to paper Question-relevant evidence can be faithfully represented by coarse temporal indices and spatial cells of a fixed grid, 'Evidence: T08-R05/R06' (Section 4.2; Eq. 12).
    The RL rewards (Eq. 6-7) and the T-Hit/R-Hit metrics are defined on this quantization; the method's success is measured in terms of its own representational convention.
invented entities (1)
  • Spatiotemporal evidence tag format (T/G index pairs, e.g., T08-R05/R06) no independent evidence
    purpose: New output convention forcing the model to state where and when its answer is grounded, and the target of the Rst reward in Eq. 6.
    T-Hit and R-Hit are measured only on the paper's own benchmark, so the convention has no falsifiable handle outside this dataset. This is an annotation convention, not a physical entity, and the paper introduces no new particles, forces, or mediators.

pith-pipeline@v1.3.0-daily-deepseek · 41370 in / 22083 out tokens · 192636 ms · 2026-08-04T16:10:59.994400+00:00 · methodology

0 comments
read the original abstract

Remote-sensing videos enable real-time observation of changes in target attributes, short-term activities, and scene evolution. They record motion, actions, interactions, and scene changes that cannot be captured by isolated images. Existing models primarily target single images or discrete temporal observations spanning a long time range. However, a unified evaluation setting for assessing vision-language models on continuous remote-sensing video understanding remains lacking. We introduce RSVideo-10K, a remote-sensing video dataset comprising 10,773 instances, 1.47 million frames, and 17.02 hours of footage, containing both unmanned aerial vehicles and satellite platforms. Its fixed evaluation benchmark, RSVideo-Bench, contains 2,731 test instances and evaluates two complementary aspects of remote-sensing video understanding: L1 Perception and L2 Reasoning, spanning seven capability groups and 17 tasks. Evaluations show that current vision-language models still struggle to recover small local evidence, track short-lived states, and use scene-constrained spatial relations. Based on this analysis, we further propose RSVideo, a reinforcement learning framework for small-target spatiotemporal focusing that selects question-relevant regions across frames and suppresses redundant background tokens. RSVideo achieves a maximum absolute improvement of 9.01% with InternVL3.5-14B and attains the highest accuracy of 40.63% with Qwen3.6-27B across 26 open-source vision-language backbones.Codes will be available at https://github.com/HongjieZhou0329/RSVideo.

Figures

Figures reproduced from arXiv: 2608.02039 by Di Wang, Fu Lin, Haonan Guo, Haoyang Chen, Hongjie Zhou, Juhua Liu, Shiqin Wang, Yong Luo.

Figure 1
Figure 1. Figure 1: Accuracy of representative vision-language models on [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of RSVideo-10K. The dataset contains 10,773 five-choice QA instances grounded in 4,629 audited evidence clips [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Performance of representative vision-language models [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Overview of RSVideo. The visual encoder and text encoder first generate visual tokens and question tokens. RSVideo combines [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Qualitative comparison of question-conditioned visual [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Complete RSVideo data-construction pipeline. Public videos first pass source registration, decoding, quality screening, and [PITH_FULL_IMAGE:figures/full_fig_p015_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Qualitative example for Leaf 01, scene type. [PITH_FULL_IMAGE:figures/full_fig_p026_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Qualitative example for Leaf 02, viewpoint type. [PITH_FULL_IMAGE:figures/full_fig_p027_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Qualitative example for Leaf 03, spatial relation between static land objects. [PITH_FULL_IMAGE:figures/full_fig_p028_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Qualitative example for Leaf 04, target identification. [PITH_FULL_IMAGE:figures/full_fig_p029_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Qualitative example for Leaf 05, target counting. [PITH_FULL_IMAGE:figures/full_fig_p030_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Qualitative example for Leaf 06, reference-image target localization. [PITH_FULL_IMAGE:figures/full_fig_p031_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Qualitative example for Leaf 07, action recognition. [PITH_FULL_IMAGE:figures/full_fig_p032_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Qualitative example for Leaf 08, action transition. [PITH_FULL_IMAGE:figures/full_fig_p033_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: Qualitative example for Leaf 09, action counting. [PITH_FULL_IMAGE:figures/full_fig_p034_15.png] view at source ↗
Figure 16
Figure 16. Figure 16: Qualitative example for Leaf 10, environmental constraint reasoning. [PITH_FULL_IMAGE:figures/full_fig_p035_16.png] view at source ↗
Figure 17
Figure 17. Figure 17: Qualitative example for Leaf 11, target state inference. [PITH_FULL_IMAGE:figures/full_fig_p036_17.png] view at source ↗
Figure 18
Figure 18. Figure 18: Qualitative example for Leaf 12, main visible activity recognition. [PITH_FULL_IMAGE:figures/full_fig_p037_18.png] view at source ↗
Figure 19
Figure 19. Figure 19: Qualitative example for Leaf 13, disaster evidence monitoring. [PITH_FULL_IMAGE:figures/full_fig_p038_19.png] view at source ↗
Figure 20
Figure 20. Figure 20: Qualitative example for Leaf 14, abnormal behavior. [PITH_FULL_IMAGE:figures/full_fig_p039_20.png] view at source ↗
Figure 21
Figure 21. Figure 21: Qualitative example for Leaf 15, long-term trajectory summary for small moving targets. [PITH_FULL_IMAGE:figures/full_fig_p040_21.png] view at source ↗
Figure 22
Figure 22. Figure 22: Qualitative example for Leaf 16, displacement and speed estimation. [PITH_FULL_IMAGE:figures/full_fig_p041_22.png] view at source ↗
Figure 23
Figure 23. Figure 23: Qualitative example for Leaf 17, temporal ordering. [PITH_FULL_IMAGE:figures/full_fig_p042_23.png] view at source ↗
Figure 24
Figure 24. Figure 24: Qualitative example for Leaf 18, action-duration comparison. [PITH_FULL_IMAGE:figures/full_fig_p043_24.png] view at source ↗
Figure 25
Figure 25. Figure 25: Qualitative example for Leaf 19, concurrent visibility interval for multiple targets. [PITH_FULL_IMAGE:figures/full_fig_p044_25.png] view at source ↗
Figure 26
Figure 26. Figure 26: Qualitative example for Leaf 20, single-event spatiotemporal localization. [PITH_FULL_IMAGE:figures/full_fig_p045_26.png] view at source ↗
Figure 27
Figure 27. Figure 27: Qualitative example for Leaf 21, regional trajectory co-occurrence. [PITH_FULL_IMAGE:figures/full_fig_p046_27.png] view at source ↗
Figure 28
Figure 28. Figure 28: Qualitative example for Leaf 22, relation-constrained candidate localization. [PITH_FULL_IMAGE:figures/full_fig_p047_28.png] view at source ↗
Figure 29
Figure 29. Figure 29: Qualitative example for Leaf 23, reasoned counting of small moving targets. [PITH_FULL_IMAGE:figures/full_fig_p048_29.png] view at source ↗
Figure 30
Figure 30. Figure 30: Qualitative example for Leaf 24, cross-window constraint satisfiability. [PITH_FULL_IMAGE:figures/full_fig_p049_30.png] view at source ↗
Figure 31
Figure 31. Figure 31: Qualitative example for Leaf 25, target re-identification after occlusion. [PITH_FULL_IMAGE:figures/full_fig_p050_31.png] view at source ↗
Figure 32
Figure 32. Figure 32: Qualitative example for Leaf 26, minimal constraint relaxation. [PITH_FULL_IMAGE:figures/full_fig_p051_32.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

51 extracted references · 16 linked inside Pith

  1. [1]

    Temporal chain of thought: Long- video understanding by thinking in frames

    Anurag Arnab, Ahmet Iscen, Mathilde Caron, Alireza Fathi, and Cordelia Schmid. Temporal chain of thought: Long- video understanding by thinking in frames. InAdvances in Neural Information Processing Systems, 2025. 3

    2025

  2. [2]

    Qwen3-VL technical report.arXiv preprint arXiv:2511.21631, 2025

    Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, et al. Qwen3-VL technical report.arXiv preprint arXiv:2511.21631, 2025. 18

    Pith/arXiv arXiv 2025

  3. [3]

    Qwen2.5-VL technical report.arXiv preprint arXiv:2502.13923, 2025

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2.5-VL technical report.arXiv preprint arXiv:2502.13923, 2025. 1, 18

    Pith/arXiv arXiv 2025

  4. [4]

    Okutama-action: An aerial view video dataset for concurrent human action detection

    Mohammadamin Barekatain, Miquel Mart ´ı, Hsueh-Fu Shih, Samuel Murray, Kotaro Nakayama, Yutaka Matsuo, and Hel- mut Prendinger. Okutama-action: An aerial view video dataset for concurrent human action detection. InProceed- ings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pages 28–35, 2017. 1, 2, 13

    2017

  5. [5]

    Al Rahhal, Mo- hammad Zuair, and Farid Melgani

    Laila Bashmal, Yakoub Bazi, Mohamad M. Al Rahhal, Mo- hammad Zuair, and Farid Melgani. CapERA: Captioning events in aerial videos.Remote Sensing, 15(8):2139, 2023

    2023

  6. [6]

    AU-AIR: A multi- modal unmanned aerial vehicle dataset for low altitude traf- fic surveillance

    Ilker Bozcan and Erdal Kayacan. AU-AIR: A multi- modal unmanned aerial vehicle dataset for low altitude traf- fic surveillance. InProceedings of the IEEE International Conference on Robotics and Automation, pages 8504–8510,

  7. [7]

    Satel- lite video single object tracking: A systematic review and an oriented object tracking benchmark.ISPRS Journal of Pho- togrammetry and Remote Sensing, 210:212–240, 2024

    Yuzeng Chen, Yuqi Tang, Yi Xiao, Qiangqiang Yuan, Yuwei Zhang, Fengqing Liu, Jiang He, and Liangpei Zhang. Satel- lite video single object tracking: A systematic review and an oriented object tracking benchmark.ISPRS Journal of Pho- togrammetry and Remote Sensing, 210:212–240, 2024. 3, 13

    2024

  8. [8]

    Video-R1: Rein- forcing video reasoning in MLLMs.arXiv preprint arXiv:2503.21776, 2025

    Kaituo Feng, Kaixiong Gong, Bohao Li, Zonghao Guo, Yibing Wang, Tianshuo Peng, Junfei Wu, Xiaoying Zhang, Benyou Wang, and Xiangyu Yue. Video-R1: Rein- forcing video reasoning in MLLMs.arXiv preprint arXiv:2503.21776, 2025. 7

    Pith/arXiv arXiv 2025

  9. [9]

    Video- MME: The first-ever comprehensive evaluation benchmark of multi-modal LLMs in video analysis

    Chaoyou Fu, Yuhan Dai, Yongdong Luo, et al. Video- MME: The first-ever comprehensive evaluation benchmark of multi-modal LLMs in video analysis. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 24108–24118, 2025. 1, 2, 3, 8

    2025

  10. [10]

    VideoTIR: Ac- curate understanding for long videos with efficient tool- integrated reasoning.arXiv preprint arXiv:2603.25021,

    Zhe Gao, Shiyu Shen, Taifeng Chai, et al. VideoTIR: Ac- curate understanding for long videos with efficient tool- integrated reasoning.arXiv preprint arXiv:2603.25021,

    Pith/arXiv arXiv

  11. [11]

    TEOChat: A large vision-language as- sistant for temporal earth observation data

    Jeremy Andrew Irvin, Emily Ruoyu Liu, Joyce Chuyi Chen, Ines Dormoy, Jinyoung Kim, Samar Khanna, Zhuo Zheng, and Stefano Ermon. TEOChat: A large vision-language as- sistant for temporal earth observation data. InInternational Conference on Learning Representations, 2025. 3

    2025

  12. [12]

    GeoChat: Grounded large vision-language model for remote sensing

    Kartik Kuckreja, Muhammad Sohail Danish, Muzammal Naseer, Abhijit Das, Salman Khan, and Fahad Shahbaz Khan. GeoChat: Grounded large vision-language model for remote sensing. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 27831– 27840, 2024. 1, 2

    2024

  13. [13]

    LLaV A-OneVision: Easy visual task transfer.arXiv preprint arXiv:2408.03326, 2024

    Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Zi- wei Liu, and Chunyuan Li. LLaV A-OneVision: Easy visual task transfer.arXiv preprint arXiv:2408.03326, 2024. 1, 18

    Pith/arXiv arXiv 2024

  14. [14]

    MVBench: A com- prehensive multi-modal video understanding benchmark

    Kunchang Li, Yali Wang, Yinan He, et al. MVBench: A com- prehensive multi-modal video understanding benchmark. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, 2024. 2, 3, 8

    2024

  15. [15]

    Visual object tracking for un- manned aerial vehicles: A benchmark and new motion mod- els

    Siyi Li and Dit-Yan Yeung. Visual object tracking for un- manned aerial vehicles: A benchmark and new motion mod- els. InProceedings of the AAAI Conference on Artificial In- telligence, 2017. 13 9

    2017

  16. [16]

    Vrsbench: A versatile vision-language benchmark dataset for remote sens- ing image understanding

    Xiang Li, Jian Ding, and Mohamed Elhoseiny. Vrsbench: A versatile vision-language benchmark dataset for remote sens- ing image understanding. InAdvances in Neural Information Processing Systems, 2024. 1, 2, 3

    2024

  17. [17]

    UniRS: Unifying multi- temporal remote sensing tasks through vision language mod- els.arXiv preprint arXiv:2412.20742, 2024

    Yujie Li, Wenjia Xu, Guangzuo Li, Zijian Yu, Zhiwei Wei, Jiuniu Wang, and Mugen Peng. UniRS: Unifying multi- temporal remote sensing tasks through vision language mod- els.arXiv preprint arXiv:2412.20742, 2024. 1

    Pith/arXiv arXiv 2024

  18. [18]

    TempCom- pass: Do video LLMs really understand videos? InFindings of the Association for Computational Linguistics: ACL 2024, pages 8731–8772, Bangkok, Thailand, 2024

    Yuanxin Liu, Shicheng Li, Yi Liu, Yuxiang Wang, Shuhuai Ren, Lei Li, Sishuo Chen, Xu Sun, and Lu Hou. TempCom- pass: Do video LLMs really understand videos? InFindings of the Association for Computational Linguistics: ACL 2024, pages 8731–8772, Bangkok, Thailand, 2024. Association for Computational Linguistics. 3

    2024

  19. [19]

    RSVQA: Visual question answering for remote sensing data

    Sylvain Lobry, Diego Marcos, Jesse Murray, and Devis Tuia. RSVQA: Visual question answering for remote sensing data. IEEE Transactions on Geoscience and Remote Sensing, 58 (12):8555–8566, 2020. 1, 2, 3

    2020

  20. [20]

    VLRS-Bench: A vision-language reasoning benchmark for remote sensing.arXiv preprint arXiv:2602.07045, 2026

    Zhiming Luo, Di Wang, Haonan Guo, Jing Zhang, and Bo Du. VLRS-Bench: A vision-language reasoning benchmark for remote sensing.arXiv preprint arXiv:2602.07045, 2026. 3

    Pith/arXiv arXiv 2026

  21. [21]

    Ma, Willis Guo, Aditya Agrawal, Ankit Gupta, Paul Pu Liang, Ruslan Salakhutdinov, and Louis-Philippe Morency

    Martin Q. Ma, Willis Guo, Aditya Agrawal, Ankit Gupta, Paul Pu Liang, Ruslan Salakhutdinov, and Louis-Philippe Morency. Video active perception: Effective inference-time long-form video understanding with vision-language mod- els.arXiv preprint arXiv:2605.01662, 2026. 3

    Pith/arXiv arXiv 2026

  22. [22]

    EgoSchema: A diagnostic benchmark for very long- form video language understanding

    Karttikeya Mangalam, Raiymbek Akshulakov, and Jitendra Malik. EgoSchema: A diagnostic benchmark for very long- form video language understanding. InAdvances in Neural Information Processing Systems, pages 46212–46244, 2023. 2

    2023

  23. [23]

    ERA: A data set and deep learning benchmark for event recognition in aerial videos.IEEE Geoscience and Remote Sensing Magazine, 8(4):125–133, 2020

    Lichao Mou, Yuansheng Hua, Pu Jin, and Xiao Xiang Zhu. ERA: A data set and deep learning benchmark for event recognition in aerial videos.IEEE Geoscience and Remote Sensing Magazine, 8(4):125–133, 2020. 1, 2, 13

    2020

  24. [24]

    UHR-Micro: Diagnosing and mit- igating the resolution illusion in earth observation VLMs

    Shuo Ni, Tong Wang, Jing Zhang, He Chen, Haonan Guo, Ning Zhang, and Bo Du. UHR-Micro: Diagnosing and mit- igating the resolution illusion in earth observation VLMs. arXiv preprint arXiv:2605.12237, 2026. 1, 2

    Pith/arXiv arXiv 2026

  25. [25]

    Qwen3.6-27B: Flagship-level coding in a 27B dense model

    Qwen Team. Qwen3.6-27B: Flagship-level coding in a 27B dense model. https://qwen.ai/blog?id=qwen3.6-27b, 2026. Accessed July 2026. 18

    2026

  26. [26]

    Learning social etiquette: Human tra- jectory understanding in crowded scenes

    Alexandre Robicquet, Amir Sadeghian, Alexandre Alahi, and Silvio Savarese. Learning social etiquette: Human tra- jectory understanding in crowded scenes. InProceedings of the European Conference on Computer Vision, pages 549– 565, 2016. 1, 13

    2016

  27. [27]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y . K. Li, Y . Wu, and Daya Guo. DeepSeekMath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024. 7

    Pith/arXiv arXiv 2024

  28. [28]

    UA VReason: A unified, large-scale benchmark for multimodal aerial scene reasoning and generation.arXiv preprint arXiv:2604.05377, 2026

    Jintao Sun, Hu Zhang, Donglin Di, Gangyi Ding, and Zhe- dong Zheng. UA VReason: A unified, large-scale benchmark for multimodal aerial scene reasoning and generation.arXiv preprint arXiv:2604.05377, 2026. 2, 3

    Pith/arXiv arXiv 2026

  29. [29]

    RefDrone: A challenging benchmark for referring ex- pression comprehension in drone scenes.arXiv preprint arXiv:2502.00392, 2025

    Zhaorui Sun, Yuhang Liu, Haolin Zhu, Yuxuan Gu, Yim- ing Zou, Zhen Liu, Gui-Song Xia, Bo Du, and Yongchao Xu. RefDrone: A challenging benchmark for referring ex- pression comprehension in drone scenes.arXiv preprint arXiv:2502.00392, 2025. 2

    arXiv 2025

  30. [30]

    DynamicEarthNet: Daily multi-spectral satellite dataset for semantic change segmentation

    Aysim Toker, Lukas Kondmann, Mark Weber, Marvin Eisen- berger, Andr´es Camero, Jingliang Hu, Ariadna Pregel Hoder- lein, C ¸ a˘glar S ¸enaras, Timothy Davis, Daniel Cremers, Gio- vanni Marchisio, Xiao Xiang Zhu, and Laura Leal-Taix ´e. DynamicEarthNet: Daily multi-spectral satellite dataset for semantic change segmentation. InProceedings of the IEEE/CVF C...

    2022

  31. [31]

    Advancing plain vision transformer toward remote sensing foundation model.IEEE Transactions on Geoscience and Remote Sensing, 61:1–15,

    Di Wang, Qiming Zhang, Yufei Xu, Jing Zhang, Bo Du, Dacheng Tao, and Liangpei Zhang. Advancing plain vision transformer toward remote sensing foundation model.IEEE Transactions on Geoscience and Remote Sensing, 61:1–15,

  32. [32]

    Fengxiang Wang, Hongzhen Wang, Mingshuo Chen, Di Wang, Yulin Wang, Zonghao Guo, Qiang Ma, Long Lan, Wenjing Yang, Jing Zhang, Zhiyuan Liu, and Maosong Sun. XLRS-Bench: Could your multimodal LLMs understand ex- tremely large ultra-high-resolution remote sensing imagery? InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025. 2, 3

    2025

  33. [33]

    Earthvqa: Towards queryable earth via re- lational reasoning-based remote sensing visual question an- swering

    Junjue Wang, Zhuo Zheng, Zihang Chen, Ailong Ma, and Yanfei Zhong. Earthvqa: Towards queryable earth via re- lational reasoning-based remote sensing visual question an- swering. InProceedings of the AAAI Conference on Artificial Intelligence, pages 5481–5489, 2024. 1, 2, 3

    2024

  34. [34]

    InternVL3.5: Advancing open-source multimodal models in versatility, reasoning, and efficiency

    Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingguang Wei, Zhaoyang Liu, Linglin Jing, Shenglong Ye, Jie Shao, et al. InternVL3.5: Advancing open-source multimodal models in versatility, reasoning, and efficiency. arXiv preprint arXiv:2508.18265, 2025. 18

    Pith/arXiv arXiv 2025

  35. [35]

    Skyscript: A large and semantically diverse vision-language dataset for remote sensing

    Zhecheng Wang, Rajanie Prabha, Tianyuan Huang, Jiajun Wu, and Ram Rajagopal. Skyscript: A large and semantically diverse vision-language dataset for remote sensing. InPro- ceedings of the AAAI Conference on Artificial Intelligence, pages 5805–5813, 2024. 1, 2

    2024

  36. [36]

    Videotree: Adaptive tree-based video representation for LLM reasoning on long videos

    Ziyang Wang, Shoubin Yu, Elias Stengel-Eskin, Jaehong Yoon, Feng Cheng, Gedas Bertasius, and Mohit Bansal. Videotree: Adaptive tree-based video representation for LLM reasoning on long videos. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025. 3

    2025

  37. [37]

    Dy- namicVL: Benchmarking multimodal large language models for dynamic city understanding

    Weihao Xuan, Junjue Wang, Heli Qi, Zihang Chen, Zhuo Zheng, Yanfei Zhong, Junshi Xia, and Naoto Yokoya. Dy- namicVL: Benchmarking multimodal large language models for dynamic city understanding. InAdvances in Neural In- formation Processing Systems, 2025. 1, 2, 3

    2025

  38. [38]

    Gupta, Rilyn Han, Li Fei-Fei, and Saining Xie

    Jihan Yang, Shusheng Yang, Anjali W. Gupta, Rilyn Han, Li Fei-Fei, and Saining Xie. Thinking in space: How mul- timodal large language models see, remember, and recall spaces. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10632–10643, 2025. 3 10

    2025

  39. [39]

    VCA: Video curious agent for long video un- derstanding

    Zeyuan Yang, Delin Chen, Xueyang Yu, Maohao Shen, and Chuang Gan. VCA: Video curious agent for long video un- derstanding. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 20168–20179, 2025. 3

    2025

  40. [40]

    Detecting and tracking small and dense moving objects in satellite videos: A benchmark.IEEE Transactions on Geoscience and Re- mote Sensing, 60:1–18, 2022

    Qian Yin, Qingyong Hu, Hao Liu, Feng Zhang, Yingqian Wang, Zaiping Lin, Wei An, and Yulan Guo. Detecting and tracking small and dense moving objects in satellite videos: A benchmark.IEEE Transactions on Geoscience and Re- mote Sensing, 60:1–18, 2022. 3, 13

    2022

  41. [41]

    VideoARM: Agentic reasoning over hierarchical memory for long-form video understanding

    Yufei Yin, Qianke Meng, Minghao Chen, Jiajun Ding, Zhen- wei Shao, and Zhou Yu. VideoARM: Agentic reasoning over hierarchical memory for long-form video understanding. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, 2026. 3

    2026

  42. [42]

    MiniCPM-V 4.5: Cooking efficient MLLMs via architecture, data, and training recipe.arXiv preprint arXiv:2509.18154, 2025

    Tianyu Yu, Zefan Wang, Chongyi Wang, Fuwei Huang, Zhi- hui He, Tianchi Cai, Weize Chen, Yuxiang Huang, Bokai Xu, Ning Ding, et al. MiniCPM-V 4.5: Cooking efficient MLLMs via architecture, data, and training recipe.arXiv preprint arXiv:2509.18154, 2025. 18

    Pith/arXiv arXiv 2025

  43. [43]

    FUSU: A multi-temporal-source land use change segmentation dataset for fine-grained urban semantic understanding

    Shuai Yuan, Guancong Lin, Lixian Zhang, Runmin Dong, Jinxiao Zhang, Shuang Chen, Juepeng Zheng, Jie Wang, and Haohuan Fu. FUSU: A multi-temporal-source land use change segmentation dataset for fine-grained urban semantic understanding. InAdvances in Neural Information Process- ing Systems, 2024. 1, 2

    2024

  44. [44]

    Where does it exist from the low-altitude: Spatial aerial video grounding

    Yang Zhan and Yuan Yuan. Where does it exist from the low-altitude: Spatial aerial video grounding. InAdvances in Neural Information Processing Systems, 2025. 2

    2025

  45. [45]

    UA VBench and UA VIT- 1M: Benchmarking and enhancing MLLMs for low-altitude UA V vision-language understanding.arXiv preprint arXiv:2603.14336, 2026

    Yang Zhan and Yuan Yuan. UA VBench and UA VIT- 1M: Benchmarking and enhancing MLLMs for low-altitude UA V vision-language understanding.arXiv preprint arXiv:2603.14336, 2026. 2, 3

    arXiv 2026

  46. [46]

    VideoLLaMA 3: Frontier mul- timodal foundation models for image and video understand- ing.arXiv preprint arXiv:2501.13106, 2025

    Boqiang Zhang, Kehan Li, Zesen Cheng, Zhiqiang Hu, Yuqian Yuan, Guanzheng Chen, Sicong Leng, Yuming Jiang, Hang Zhang, Xin Li, Peng Jin, Wenqi Zhang, Fan Wang, Li- dong Bing, and Deli Zhao. VideoLLaMA 3: Frontier mul- timodal foundation models for image and video understand- ing.arXiv preprint arXiv:2501.13106, 2025. 1, 18

    Pith/arXiv arXiv 2025

  47. [47]

    UrbanVideo-Bench: Benchmark- ing vision-language models on embodied intelligence with video data in urban spaces

    Baining Zhao, Jianjie Fang, Zichao Dai, Ziyou Wang, Jirong Zha, Weichen Zhang, Chen Gao, Yue Wang, Jinqiang Cui, Xinlei Chen, and Yong Li. UrbanVideo-Bench: Benchmark- ing vision-language models on embodied intelligence with video data in urban spaces. InProceedings of the 63rd An- nual Meeting of the Association for Computational Linguis- tics (Volume 1:...

    2025

  48. [48]

    SatSOT: A benchmark dataset for satellite video single object tracking.IEEE Trans- actions on Geoscience and Remote Sensing, 60:1–11, 2022

    Manqi Zhao, Shengyang Li, Shiyu Xuan, Longxuan Kou, Shuai Gong, and Zhuang Zhou. SatSOT: A benchmark dataset for satellite video single object tracking.IEEE Trans- actions on Geoscience and Remote Sensing, 60:1–11, 2022. 1, 3, 13

    2022

  49. [49]

    Group sequence policy optimization.arXiv preprint arXiv:2507.18071, 2025

    Chujie Zheng, Shixuan Liu, Mingze Li, Xiong-Hui Chen, Bowen Yu, Chang Gao, Kai Dang, Yuqiong Liu, Rui Men, An Yang, Jingren Zhou, and Junyang Lin. Group sequence policy optimization.arXiv preprint arXiv:2507.18071, 2025. 7

    Pith/arXiv arXiv 2025

  50. [50]

    A.I.R.: Enabling adaptive, itera- tive, and reasoning-based frame selection for video question answering

    Yuanhao Zou, Shengji Jin, Andong Deng, Youpeng Zhao, Jun Wang, and Chen Chen. A.I.R.: Enabling adaptive, itera- tive, and reasoning-based frame selection for video question answering. InInternational Conference on Learning Repre- sentations, 2026. 3

    2026

  51. [51]

    first appearing near the top side of the lake

    Zhishan Zou, Guoyan Sun, Zhiwei Wei, Jiancheng Pan, Yujie Li, Mugen Peng, and Wenjia Xu. Self in space: Benchmark- ing self-awareness and spatial cognition in UA V embodied intelligence.arXiv preprint arXiv:2607.12477, 2026. 3, 8 11 Appendix Contents Appendix A. Dataset Construction Details. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ...

    Pith/arXiv arXiv 2026