Pith. sign in

REVIEW 3 major objections 4 minor 10 cited by

OpenEarthAgent shows that a 4-billion-parameter agent, trained on 14,538 executable geospatial reasoning traces, matches or beats much larger models on structured tool use in remote sensing.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 22:07 UTC pith:MGEUJBZL

load-bearing objection OpenEarthAgent ships a valuable dataset and trained agent, but the headline SOTA claims rest on a self-built benchmark with an LLM judge—worth refereeing, with the caveat that the authors need to rule out train/test leakage. the 3 major comments →

arxiv 2602.17665 v4 pith:MGEUJBZL submitted 2026-02-19 cs.CV

OpenEarthAgent: A Unified Framework for Tool-Augmented Geospatial Agents

Akashah Shabbir , Muhammad Umer Sheikh , Muhammad Akhtar Munir , Hiyam Debary , Mustansar Fiaz , Muhammad Zaigham Zaheer , Paolo Fraccaro , Fahad Shahbaz Khan
show 3 more authors
Muhammad Haris Khan Xiao Xiang Zhu Salman Khan
This is my paper
classification cs.CV
keywords geospatial agentsremote sensingtool-augmented reasoningsatellite imageryGIS operationsspectral indicessupervised fine-tuningbenchmark
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to establish that a small, openly available language model can become a strong geospatial reasoning agent through supervised fine-tuning on validated tool-use trajectories. It introduces OpenEarthAgent, a framework pairing a unified registry of geospatial tools (GIS, spectral-index, and image-perception operations) with a corpus of 14,538 training and 1,169 evaluation instances spanning over 100,000 reasoning steps. The central claim is that trajectory-based policy learning with deterministic replay validation yields near-frontier tool-selection accuracy (99.5%), argument correctness (62.1%), and competitive final-answer accuracy (45.3%) despite the model's small size. A sympathetic reader would care because it suggests structured reasoning in specialized domains can be trained into compact models rather than requiring ever-larger API models.

Core claim

The discovery is that a 4-billion-parameter agent, fine-tuned on LLM-generated reasoning traces that interleave natural-language thoughts with executable tool calls, can outperform much larger general-purpose models on geospatial tool use. In step-by-step and end-to-end tests, it leads in instance accuracy (99.51), tool selection (97.18), argument-name accuracy (96.08), argument-value accuracy (62.10), and trajectory-order fidelity, while its final-answer accuracy (45.26) stays competitive with frontier models. The framework couples a unified tool registry—where every perception, GIS, spectral, and raster operation shares one callable schema—with supervised fine-tuning on replay-validated tr

What carries the argument

The central mechanism is the unified tool registry: a standardized callable schema M=(xin, yout, ψ) that wraps heterogeneous geospatial operations (bounding-box detection, POI layer construction, NDVI/NBR index computation, distance measurement, GeoTIFF rendering) under one JSON contract, with a central orchestrator that executes tool calls, validates arguments, and caches intermediate outputs for deterministic replay. On top of this, trajectory-based policy learning fits the agent by maximum likelihood over verified action sequences, masking tool observations from the loss so the model learns to produce executable, spatially consistent chains rather than free-form answers.

Load-bearing premise

The load-bearing premise is that the LLM-generated reasoning traces and test answers are correct and faithful ground truth; if they encode wrong GIS logic or mis-measured results, the trained agent and the benchmark's scores inherit those errors.

What would settle it

Take a random sample of training traces and independently recompute their claimed measurements (distances, areas, burn percentages) with standard GIS software; if a meaningful fraction disagree with the ground-truth values, the training signal and benchmark accuracy are both unreliable. Alternatively, run a substring or near-duplicate check between training and test queries; high overlap would explain the agent's test-set edge.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • A compact 4B agent can reach or exceed frontier-model performance on geospatial tool selection and argument correctness, implying that domain-specific agentic skill can be compressed into small models.
  • The deterministic replay validation makes every reasoning step in the corpus executable and spatially checkable, so the same pipeline could audit or certify agent outputs.
  • New tools can be added to the registry without retraining the backbone, so the framework can grow to new sensors or analyses.
  • The corpus itself becomes a benchmark for structured EO reasoning, letting future work compare trajectory fidelity and argument validity rather than only final answers.
  • Training on a mix of short and long trajectories improves logical consistency and tool-order fidelity more than either alone, according to the paper's ablations.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the trace corpus is representative, this points to a general recipe: build a small, domain-specific tool-using agent by generating validated traces with a frontier model and distilling them via SFT; the same recipe may transfer to other tool-heavy domains such as medical imaging or document analysis.
  • Because the answer-accuracy judge is itself a weaker language model, an independent human audit of a random test subset would clarify whether the 45% answer accuracy reflects genuine geospatial competence or judge leniency.
  • The tool registry is interface-standardized but the underlying functions are deterministic; perturbing tool outputs during evaluation would reveal whether the agent can adapt to noisy or adversarial feedback, a test the paper does not run.
  • No train/test deduplication analysis is reported, so some evaluation gains could come from query overlap; checking this would separate memorization from generalization.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces OpenEarthAgent, a unified framework for tool-augmented geospatial reasoning, together with a large corpus of 14,538 training and 1,169 evaluation instances containing over 107K reasoning steps. The framework standardizes perceptual, GIS, spectral, and georeferenced raster operations under a common tool registry and trains a Qwen3-4B backbone by supervised fine-tuning on LLM-generated reasoning traces validated by deterministic replay. The authors report that OpenEarthAgent-4B achieves state-of-the-art step-level tool metrics (Inst. 99.51, Tool. 97.18, ArgN. 96.08, ArgV. 62.10) on their benchmark, substantially improves end-to-end answer accuracy over the Qwen3 baseline, and transfers to the Earth-Agent benchmark (Table A6).

Significance. If the benchmark is accepted as a valid measure of geospatial tool-use reasoning, the contribution is significant: it provides a publicly available, large-scale corpus, a unified and extensible tool interface, deterministic replay validation, and evidence that a 4B model can learn structured multi-step geospatial workflows from trajectory supervision. The paper's strengths include deterministic replay validation of tool calls, ablations on data composition, tool schema, tool order, and high-level planning, and a cross-benchmark evaluation on Earth-Agent. These design choices are a useful step beyond static remote-sensing VLMs.

major comments (3)
  1. [Sec. 4.1 and Sec. S8] The final answer accuracy metric, which underlies the headline end-to-end gains (Table 4, Ans. 45.26), is produced by an LLM judge (gpt-4o-mini) using a 'standardized elaboration evaluation prompt.' Sec. S8 shows that an open-source judge (Qwen3-30B) assigns systematically higher absolute scores, though rankings are preserved. No human-validated calibration of this judge on the OpenEarthAgent test set is reported. I request a human-annotated subset (e.g., 100–200 instances) with inter-annotator agreement and judge–human correlation, plus an error analysis of judge failures. Without this, the end-to-end accuracy numbers are not verified against objective ground truth.
  2. [Sec. 3.1 and Fig. 2] The same automated pipeline (question synthesizer and conversational 1-shot generator) is used to create both the 14,538 training and the 1,169 test instances. The paper reports no analysis of overlap or similarity between the two splits — e.g., near-duplicate queries, identical regions/POIs, or repeated tool sequences. Since the model is fine-tuned on this distribution, the large improvements in Tables 3–4 could partially reflect train/test distributional overlap rather than learned geospatial reasoning. Please add a deduplication and similarity analysis (query text, image source, trace-level) and, ideally, evaluate on a held-out set built from unseen regions/templates. The cross-benchmark result (Table A6) is reassuring but uses reformatted tools and curated exemplars, so it does not eliminate this concern.
  3. [Sec. 3.1] The reasoning traces that serve as both training supervision and evaluation references are generated by an LLM module and validated mainly by deterministic replay and manual inspection of test samples only. No human audit of training traces or of the correctness of final reference answers is reported. If the generator encodes wrong GIS logic, hallucinated measurements, or biased question distributions, both the training signal and the benchmark are systematically corrupted. Please report a human audit on a random sample of training and test traces, including inter-annotator agreement, and provide examples of corrected or discarded traces. This would establish that the benchmark measures geospatial correctness rather than imitation of the generator's trace style.
minor comments (4)
  1. [Sec. 4.2] The phrase 'state-of-the-art Inst./Tool./ArgN./and ArgV.' should be qualified as 'state-of-the-art among evaluated models on the self-built OpenEarthAgent benchmark.' Also, Summarization Accuracy (Summ.) in Table 3 is not improved over frontier models (83.64 vs. o4-mini's 89.48), so the 'narrows the gap' claim should be scoped to the tool-level metrics.
  2. [Sec. S3, Table A7] Table A7 reports near-saturated F1 scores (e.g., GIS 99.53) that differ sharply from the main benchmark results in Table 4. Please clarify in the caption or text that these numbers are on the internal 8% validation split used for model selection, not on the 1,169-instance benchmark test set.
  3. [Sec. 3.1] The manual inspection of test samples is mentioned but not quantified. State how many test samples were inspected, by how many annotators, and what specific criteria were applied for 'realistic geospatial measurements.'
  4. [Throughout] Minor typographical and formatting issues: 'T able 1' (page 5), 'OpenEar thAgent' (page 2), inconsistent capitalization of 'Answer Acc.' vs. 'Answer Accuracy,' and the figure caption references in Fig. A1 are not fully self-contained.

Circularity Check

0 steps flagged

No significant circularity: performance claims are empirical, and independent cross-benchmark evidence exists.

full rationale

OpenEarthAgent is an empirical systems paper: it constructs a dataset, fine-tunes a model, and measures performance. There is no derivation chain in which a predicted quantity is definitionally equal to a fitted input. The training and evaluation sets are separate instances generated by the same LLM-based pipeline, which raises a legitimate distributional-validity concern about benchmark independence, but that is not a circularity of the kind required here (e.g., Eq. X = Eq. Y by construction, or a fitted parameter renamed as a prediction). The paper also evaluates on the external Earth-Agent benchmark, where OpenEarthAgent outperforms open baselines and approaches GPT-4o, providing independent grounding. The only self-citation with overlapping authors is ThinkGeo [29], used to motivate the research gap rather than as load-bearing evidence for the main claims. Under the stated standard, no circular step can be exhibited, so the appropriate finding is no significant circularity with score 0.

Axiom & Free-Parameter Ledger

0 free parameters · 4 axioms · 0 invented entities

The central claim rests on the correctness of the LLM-generated training/test traces, the validity of the self-built benchmark, the reliability of the LLM judge, and the determinism of external geospatial services. No free parameters are fitted in a scientific derivation; the threshold of seven tool-calling turns is only an ablation split.

axioms (4)
  • domain assumption The LLM-synthesized reasoning traces are correct after automated replay validation.
    Sec. 3.1 states traces are generated by an LLM question-synthesizer and validated by executing tool calls; no human audit of training traces is described.
  • domain assumption The 1,169-item test set is a valid, contamination-free benchmark for the claims.
    Sec. 3.1 says test samples pass manual inspection, but no deduplication/overlap analysis between training and test sets is reported; both splits come from the same pipeline.
  • domain assumption LLM-judge scores (gpt-4o-mini; Qwen3-30B check) are a faithful measure of final-answer correctness.
    S8/Fig. A6: cross-judge comparison preserves ranking but Qwen3-30B is systematically more lenient; no human-validated calibration of the judge is provided.
  • domain assumption External geospatial services (OpenStreetMap, Google Earth Engine) and the tool execution environment behave deterministically across replay and evaluation.
    Sec. 3.2 relies on replay validation and caching for deterministic execution; live external services are outside the authors' control.

pith-pipeline@v1.3.0-alltime-deepseek · 23952 in / 9943 out tokens · 99433 ms · 2026-08-02T22:07:03.736481+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of OpenEarthAgent: A Unified Framework for Tool-Augmented Geospatial Agents." pith.science (2026). https://pith.science/paper/MGEUJBZL

@misc{pith2026260217665,
  author       = {Pith},
  title        = {Pith review of: OpenEarthAgent: A Unified Framework for Tool-Augmented Geospatial Agents},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MGEUJBZL}},
  note         = {Machine review of arXiv:2602.17665}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Recent progress in multimodal reasoning has enabled agents that interpret imagery, connect it with language, and execute structured analytical tasks. Extending these capabilities to remote sensing remains challenging, as models must reason over spatial scale, geographic structures, and multispectral indices while maintaining coherent multi-step logic. To address this gap, we introduce \textit{OpenEarthAgent}, a unified framework for tool-augmented geospatial reasoning trained on satellite imagery, natural-language queries, and structured reasoning traces. Beyond serving as a benchmark, OpenEarthAgent establishes a cohesive agentic architecture built around a unified executable tool registry and trajectory-based policy learning. The framework standardizes heterogeneous visual, spectral, GIS, and georeferenced raster operations under a consistent callable schema, enabling modular orchestration and deterministic execution. Training is performed via supervised fine-tuning on structured reasoning trajectories with deterministic replay validation to ensure executability and spatial correctness. The accompanying corpus comprises 14,538 training and 1,169 evaluation instances with over 107K reasoning steps, spanning urban, environmental, disaster, and infrastructure domains and incorporating GIS operations alongside index analyses such as NDVI, NBR, and NDBI. Grounded in explicit reasoning traces, the learned agent demonstrates structured reasoning, stable spatial understanding, and interpretable tool-driven behaviour across diverse EO scenarios. We report consistent improvements over a strong baseline and competitive performance against recent open and closed-source models. Our code, data and trained models are publicly available: https://github.com/mbzuai-oryx/OpenEarthAgent

Figures

Figures reproduced from arXiv: 2602.17665 by Akashah Shabbir, Fahad Shahbaz Khan, Hiyam Debary, Muhammad Akhtar Munir, Muhammad Haris Khan, Muhammad Umer Sheikh, Muhammad Zaigham Zaheer, Mustansar Fiaz, Paolo Fraccaro, Salman Khan, Xiao Xiang Zhu.

Figure 1
Figure 1. Figure 1: Comparison of OpenEarthAgent-4B and Qwen3-4B on a complex GIS reasoning task. OpenEarthAgent correctly sequences tool calls with proper dependencies and feedback, while Qwen3 fails due to misordered tool usage and inconsistent reasoning. Abstract. Recent progress in multimodal reasoning has enabled agents that interpret imagery, connect it with language, and execute struc￾tured analytical tasks. Extending … view at source ↗
Figure 2
Figure 2. Figure 2: Unified data-curation pipeline for the OpenEarthAgent dataset. RGB/SAR imagery, GIS spatial layers, and index-driven datasets are processed through dedi￾cated filtering (annotation checks, POI constraints, and spectral-change signals such as NDVI/NBR/NDBI). The resulting samples are merged into unified JSON records with imagery metadata and geopackage information. A question synthesizer and con￾versational… view at source ↗
Figure 3
Figure 3. Figure 3: Representative query-reasoning trajectories from dataset, showing interleaving of natural-language thoughts, tool executions, and observations across tasks such as object localization, direction estimation, spectral mapping, change detection, and GIS￾based analysis. Examples are simplified excerpts of longer multi-step reasoning traces. (a) Data distribution across categories. 1,169 benchmarking samples 14… view at source ↗
Figure 4
Figure 4. Figure 4: Dataset overview: category distribution (left) and corpus statistics (right). statistics), object-level reasoning (counting, comparison, orientation, proximity), spectral index computation (NDVI, NBR, NDBI), and temporal change detec￾tion. Reasoning depth and tool sequence length vary across samples, but each trajectory provides a complete chain of intermediate steps, enabling transparent multi-step geospa… view at source ↗
Figure 5
Figure 5. Figure 5: Overview of the proposed OpenEarthAgent framework. The figure depicts tool deployment, where user queries over RGB, SAR, paired change-detection (CD), or GIS/indexed imagery are processed by the geospatial reasoning engine and tool or￾chestrator, which invoke appropriate tools and integrate feedback. 3.2 Methodology Overview: We present OpenEarthAgent, a unified agentic learning framework for multimodal re… view at source ↗
Figure 6
Figure 6. Figure 6: Tool-call performance: Open￾source models show lower success rates, while GPT family and OpenEarthAgent achieve substantially higher success rates [PITH_FULL_IMAGE:figures/full_fig_p014_6.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 10 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. TerraBench: Can Agents Reason Over Heterogeneous Earth-System Data?

    cs.AI 2026-06 unverdicted novelty 7.0

    TerraBench provides 403 executable tasks across three tracks to evaluate agents on unified Earth-system reasoning with tool-use and tolerance-aware metrics.

  2. TerraBench: Can Agents Reason Over Heterogeneous Earth-System Data?

    cs.AI 2026-06 unverdicted novelty 7.0

    TerraBench is a new benchmark with 403 tasks across Earth-science domains that evaluates LLM agents on coordinating heterogeneous data using executable ReAct-style workflows and process-level metrics.

  3. Can LLM Agents Respond to Disasters? Benchmarking Heterogeneous Geospatial Reasoning in Emergency Operations

    cs.AI 2026-05 unverdicted novelty 7.0

    DORA is the first end-to-end agentic benchmark for LLM-based disaster response, covering perception, spatial analysis, evacuation planning, temporal reasoning, and report generation over heterogeneous geospatial data,...

  4. RemoteAgent: Bridging Vague Human Intents and Earth Observation with RL-based Agentic MLLMs

    cs.CV 2026-04 unverdicted novelty 7.0

    RemoteAgent uses RL fine-tuning on VagueEO to align MLLMs for vague EO intent recognition, handling simple tasks internally and routing dense predictions to tools via Model Context Protocol.

  5. Agentic AI for Remote Sensing: Technical Challenges and Research Directions

    cs.CV 2026-04 unverdicted novelty 6.0

    Agentic AI faces structural challenges in remote sensing due to geospatial data properties and workflow constraints, requiring EO-native agents built around structured state, tool-aware reasoning, and validity-aware e...

  6. GeoDisaster: Benchmarking Orchestrated Agents for Operational Disaster Geo-Intelligence

    cs.CV 2026-06 unverdicted novelty 5.0

    GeoDisaster provides a new benchmark for operational disaster geo-intelligence and proposes an RCEA-trained multi-agent framework with 18 geospatial tools that improves tool use and decision consistency over existing RS-VLMs.

  7. Agentic AI for Remote Sensing: Technical Challenges and Research Directions

    cs.CV 2026-04 unverdicted novelty 5.0

    Agentic AI for remote sensing requires new designs centered on structured geospatial state, tool-aware reasoning, verifier-guided execution, and physical validity rather than generic extensions.

  8. Agentic AI for Remote Sensing: Technical Challenges and Research Directions

    cs.CV 2026-04 unverdicted novelty 4.0

    Position paper identifies structural challenges in applying generic agentic AI to Earth Observation and outlines design principles for EO-native agents focused on geospatial state and validity.

  9. Earth Science Foundation Models: From Perception to Reasoning and Discovery

    astro-ph.IM 2026-05 unverdicted novelty 3.0

    The paper delivers a unified review and roadmap of Earth science foundation models, structured by capability depth from perception to agentic reasoning and by application breadth across atmosphere, hydrosphere, lithos...

  10. Earth Science Foundation Models: From Perception to Reasoning and Discovery

    astro-ph.IM 2026-05 unverdicted novelty 2.0

    A review of Earth science foundation models covering capability evolution from perception to discovery, applications across atmosphere/hydrosphere/lithosphere/biosphere/anthroposphere/cryosphere, over 200 datasets, an...

Reference graph

Works this paper leans on

51 extracted references · 15 linked inside Pith · cited by 6 Pith papers

  1. [1]

    In: Proceedings of the Com- puter Vision and Pattern Recognition Conference

    Astruc, G., Gonthier, N., Mallet, C., Landrieu, L.: Anysat: One earth observation model for many resolutions, scales, and modalities. In: Proceedings of the Com- puter Vision and Pattern Recognition Conference. pp. 19530–19540 (2025)

    2025

  2. [2]

    Cai, Z., Cao, M., Chen, H., Chen, K., Chen, K., Chen, X., Chen, X., Chen, Z., Chen, Z., Chu, P., Dong, X., Duan, H., Fan, Q., Fei, Z., Gao, Y., Ge, J., Gu, C., Gu, Y., Gui, T., Guo, A., Guo, Q., He, C., Hu, Y., Huang, T., Jiang, T., Jiao, P., Jin, Z., Lei, Z., Li, J., Li, J., Li, L., Li, S., Li, W., Li, Y., Liu, H., Liu, J., Hong, J., Liu, K., Liu, K., Li...

    2024

  3. [3]

    In: Proceedings of the IEEE/CVF international conference on computer vision

    Caron, M., Touvron, H., Misra, I., Jégou, H., Mairal, J., Bojanowski, P., Joulin, A.: Emerging properties in self-supervised vision transformers. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 9650–9660 (2021)

    2021

  4. [4]

    Advances in Neural Information Processing Systems35, 197–211 (2022)

    Cong, Y., Khanna, S., Meng, C., Liu, P., Rozi, E., He, Y., Burke, M., Lobell, D., Ermon, S.: Satmae: Pre-training transformers for temporal and multi-spectral satellite imagery. Advances in Neural Information Processing Systems35, 197–211 (2022)

    2022

  5. [5]

    Advances in neural information processing systems36, 49250–49267 (2023) 16

    Dai, W., Li, J., Li, D., Tiong, A., Zhao, J., Wang, W., Li, B., Fung, P.N., Hoi, S.: Instructblip: Towards general-purpose vision-language models with instruction tuning. Advances in neural information processing systems36, 49250–49267 (2023) 16

    2023

  6. [6]

    arXiv preprint arXiv:2506.06281 (2025)

    Danish, M.S., Munir, M.A., Shah, S.R.A., Khan, M.H., Anwer, R.M., Laaksonen, J., Khan, F.S., Khan, S.: Terrafm: A scalable foundation model for unified multi- sensor earth observation. arXiv preprint arXiv:2506.06281 (2025)

    Pith/arXiv arXiv 2025

  7. [7]

    arXiv preprint arXiv:2509.23141 (2025)

    Feng, P., Lv, Z., Ye, J., Wang, X., Huo, X., Yu, J., Xu, W., Zhang, W., Bai, L., He, C., et al.: Earth-agent: Unlocking the full landscape of earth observation with agents. arXiv preprint arXiv:2509.23141 (2025)

    arXiv 2025

  8. [8]

    Advances in Neural Information Processing Systems36, 5506–5538 (2023)

    Fuller, A., Millard, K., Green, J.: Croma: Remote sensing representations with contrastive radar-optical masked autoencoders. Advances in Neural Information Processing Systems36, 5506–5538 (2023)

    2023

  9. [9]

    Google Earth Engine: Google earth engine.https://earthengine.google.com/ (2025)

    2025

  10. [10]

    Nature645(8081), 633–638 (2025)

    Guo, D., Yang, D., Zhang, H., Song, J., Wang, P., Zhu, Q., Xu, R., Zhang, R., Ma, S., Bi, X., et al.: Deepseek-r1 incentivizes reasoning in llms through reinforcement learning. Nature645(8081), 633–638 (2025)

    2025

  11. [11]

    In: IGARSS 2024-2024 IEEE International Geoscience and Remote Sensing Symposium

    Guo, H., Su, X., Wu, C., Du, B., Zhang, L., Li, D.: Remote sensing chatgpt: Solving remote sensing tasks with chatgpt and visual models. In: IGARSS 2024-2024 IEEE International Geoscience and Remote Sensing Symposium. pp. 11474–11478. IEEE (2024)

    2024

  12. [12]

    arXiv preprint arXiv:1911.09296 (2019)

    Gupta, R., Hosfelt, R., Sajeev, S., Patel, N., Goodman, B., Doshi, J., Heim, E., Choset, H., Gaston, M.: xbd: A dataset for assessing building damage from satellite imagery. arXiv preprint arXiv:1911.09296 (2019)

    Pith/arXiv arXiv 1911

  13. [13]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    He, K., Chen, X., Xie, S., Li, Y., Dollár, P., Girshick, R.: Masked autoencoders are scalable vision learners. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 16000–16009 (2022)

    2022

  14. [14]

    arXiv preprint arXiv:2505.20289 (2025)

    Huang, Z., Ji, Y., Rajan, A.S., Cai, Z., Xiao, W., Wang, H., Hu, J., Lee, Y.J.: Vi- sualtoolagent (vista): A reinforcement learning framework for visual tool selection. arXiv preprint arXiv:2505.20289 (2025)

    Pith/arXiv arXiv 2025

  15. [15]

    arXiv preprint arXiv:2310.18660 (2023)

    Jakubik, J., Roy, S., Phillips, C., Fraccaro, P., Godwin, D., Zadrozny, B., Szwar- cman, D., Gomes, C., Nyirjesy, G., Edwards, B., et al.: Foundation models for generalist geospatial artificial intelligence. arXiv preprint arXiv:2310.18660 (2023)

    Pith/arXiv arXiv 2023

  16. [16]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Kuckreja, K., Danish, M.S., Naseer, M., Das, A., Khan, S., Khan, F.S.: Geochat: Grounded large vision-language model for remote sensing. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 27831– 27840 (2024)

    2024

  17. [17]

    arXiv preprint arXiv:2408.03326 (2024)

    Li, B., Zhang, Y., Guo, D., Zhang, R., Li, F., Zhang, H., Zhang, K., Zhang, P., Li, Y., Liu, Z., et al.: Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.03326 (2024)

    Pith/arXiv arXiv 2024

  18. [18]

    In: International conference on machine learning

    Li, J., Li, D., Savarese, S., Hoi, S.: Blip-2: Bootstrapping language-image pre- training with frozen image encoders and large language models. In: International conference on machine learning. pp. 19730–19742. PMLR (2023)

    2023

  19. [19]

    ISPRS journal of photogrammetry and remote sensing159, 296–307 (2020)

    Li, K., Wan, G., Cheng, G., Meng, L., Han, J.: Object detection in optical remote sensing images: A survey and a new benchmark. ISPRS journal of photogrammetry and remote sensing159, 296–307 (2020)

    2020

  20. [20]

    Remote Sensing 15(11), 2825 (2023)

    Lin, X., Zhang, B., Wu, F., Wang, C., Yang, Y., Chen, H.: Sived: A sar image dataset for vehicle detection based on rotatable bounding box. Remote Sensing 15(11), 2825 (2023)

    2023

  21. [21]

    IEEE Transactions on Geoscience and Remote Sensing62, 1–16 (2024) 17

    Liu, F., Chen, D., Guan, Z., Zhou, X., Zhu, J., Ye, Q., Fu, L., Zhou, J.: Remoteclip: A vision language foundation model for remote sensing. IEEE Transactions on Geoscience and Remote Sensing62, 1–16 (2024) 17

    2024

  22. [22]

    arXiv preprint arXiv:2502.11271 (2025)

    Lu, P., Chen, B., Liu, S., Thapa, R., Boen, J., Zou, J.: Octotools: An agentic frame- work with extensible tools for complex reasoning. arXiv preprint arXiv:2502.11271 (2025)

    Pith/arXiv arXiv 2025

  23. [23]

    arXiv preprint arXiv:2406.10100 (2024)

    Luo, J., Pang, Z., Zhang, Y., Wang, T., Wang, L., Dang, B., Lao, J., Wang, J., Chen, J., Tan, Y., et al.: Skysensegpt: A fine-grained instruction tuning dataset and model for remote sensing vision-language understanding. arXiv preprint arXiv:2406.10100 (2024)

    Pith/arXiv arXiv 2024

  24. [24]

    In: Proceed- ings of the IEEE/CVF International Conference on Computer Vision

    Manas, O., Lacoste, A., Giró-i Nieto, X., Vazquez, D., Rodriguez, P.: Seasonal con- trast: Unsupervised pre-training from uncurated remote sensing data. In: Proceed- ings of the IEEE/CVF International Conference on Computer Vision. pp. 9414– 9423 (2021)

    2021

  25. [25]

    OpenAI: Introducing gpt.https://openai.com(2025)

    2025

  26. [26]

    OpenStreetMap contributors: Openstreetmap: Collaborative project to create a free editable map of the world.https://www.openstreetmap.org(2025)

    2025

  27. [27]

    arXiv preprint arXiv:2306.14824 (2023)

    Peng, Z., Wang, W., Dong, L., Hao, Y., Huang, S., Ma, S., Wei, F.: Kosmos- 2: Grounding multimodal large language models to the world. arXiv preprint arXiv:2306.14824 (2023)

    Pith/arXiv arXiv 2023

  28. [28]

    IEEE Access9, 89644–89654 (2021)

    Rahnemoonfar, M., Chowdhury, T., Sarkar, A., Varshney, D., Yari, M., Murphy, R.R.: Floodnet: A high resolution aerial imagery dataset for post flood scene un- derstanding. IEEE Access9, 89644–89654 (2021)

    2021

  29. [29]

    arXiv preprint arXiv:2505.23752 (2025)

    Shabbir, A., Munir, M.A., Dudhane, A., Sheikh, M.U., Khan, M.H., Fraccaro, P., Moreno, J.B., Khan, F.S., Khan, S.: Thinkgeo: Evaluating tool-augmented agents for remote sensing tasks. arXiv preprint arXiv:2505.23752 (2025)

    arXiv 2025

  30. [30]

    arXiv preprint arXiv:2504.07615 (2025)

    Shen, H., Liu, P., Li, J., Fang, C., Ma, Y., Liao, J., Shen, Q., Zhang, Z., Zhao, K., Zhang, Q., et al.: Vlm-r1: A stable and generalizable r1-style large vision-language model. arXiv preprint arXiv:2504.07615 (2025)

    Pith/arXiv arXiv 2025

  31. [31]

    In: Proceedings of the Computer Vision and Pattern Recognition Conference

    Soni, S., Dudhane, A., Debary, H., Fiaz, M., Munir, M.A., Danish, M.S., Fraccaro, P., Watson, C.D., Klein, L.J., Khan, F.S., et al.: Earthdial: Turning multi-sensory earth observations to interactive dialogues. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 14303–14313 (2025)

    2025

  32. [32]

    In: Pro- ceedings of the ACM SIGCAS/SIGCHI Conference on Computing and Sustainable Societies

    Stamoulis, D., Marculescu, D.: Geo-olm: Enabling sustainable earth observation studies with cost-efficient open language models & state-driven workflows. In: Pro- ceedings of the ACM SIGCAS/SIGCHI Conference on Computing and Sustainable Societies. pp. 608–619 (2025)

    2025

  33. [33]

    arXiv preprint arXiv:2505.08617 (2025)

    Su, Z., Li, L., Song, M., Hao, Y., Yang, Z., Zhang, J., Chen, G., Gu, J., Li, J., Qu, X., et al.: Openthinkimg: Learning to think with images via visual tool reinforce- ment learning. arXiv preprint arXiv:2505.08617 (2025)

    Pith/arXiv arXiv 2025

  34. [34]

    Nature Communications 14(1), 1444 (2023)

    Sun, X., Yin, D., Qin, F., Yu, H., Lu, W., Yao, F., He, Q., Huang, X., Yan, Z., Wang, P., et al.: Revealing influencing factors on global waste distribution via deep- learning based dumpsite detection from satellite imagery. Nature Communications 14(1), 1444 (2023)

    2023

  35. [35]

    arXiv preprint arXiv:2412.02732 (2024)

    Szwarcman, D., Roy, S., Fraccaro, P., Gíslason, Þ.E., Blumenstiel, B., Ghosal, R., de Oliveira, P.H., Almeida, J.L.d.S., Sedona, R., Kang, Y., et al.: Prithvi-eo-2.0: A versatile multi-temporal foundation model for earth observation applications. arXiv preprint arXiv:2412.02732 (2024)

    arXiv 2024

  36. [36]

    arXiv e-prints pp

    Tseng, G., Fuller, A., Reil, M., Herzog, H., Beukema, P., Bastani, F., Green, J.R., Shelhamer, E., Kerner, H., Rolnick, D.: Galileo: Learning global and local features in pretrained remote sensing models. arXiv e-prints pp. arXiv–2502 (2025)

    2025

  37. [37]

    In: Proceedings of the Computer Vision and Pattern Recognition Conference

    Waldmann, L., Shah, A., Wang, Y., Lehmann, N., Stewart, A., Xiong, Z., Zhu, X.X., Bauer, S., Chuang, J.: Panopticon: Advancing any-sensor foundation mod- 18 els for earth observation. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 2204–2214 (2025)

    2025

  38. [38]

    IEEE Geoscience and Remote Sensing Letters16(2), 310–314 (2018)

    Wang, C., Bai, X., Wang, S., Zhou, J., Ren, P.: Multiscale visual attention networks for object detection in vhr remote sensing images. IEEE Geoscience and Remote Sensing Letters16(2), 310–314 (2018)

    2018

  39. [39]

    arXiv preprint arXiv:2305.16291 (2023)

    Wang, G., Xie, Y., Jiang, Y., Mandlekar, A., Xiao, C., Zhu, Y., Fan, L., Anand- kumar, A.: Voyager: An open-ended embodied agent with large language models. arXiv preprint arXiv:2305.16291 (2023)

    Pith/arXiv arXiv 2023

  40. [40]

    arXiv preprint arXiv:2503.11849 (2025)

    Wang, Y., Xiong, Z., Liu, C., Stewart, A.J., Dujardin, T., Bountos, N.I., Zavras, A., Gerken, F., Papoutsis, I., Leal-Taixé, L., et al.: Towards a unified copernicus foundation model for earth vision. arXiv preprint arXiv:2503.11849 (2025)

    Pith/arXiv arXiv 2025

  41. [41]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops

    Waqas Zamir, S., Arora, A., Gupta, A., Khan, S., Sun, G., Shahbaz Khan, F., Zhu, F., Shao, L., Xia, G.S., Bai, X.: isaid: A large-scale dataset for instance segmentation in aerial images. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops. pp. 28–37 (2019)

    2019

  42. [42]

    arXiv preprint arXiv:2505.16421 (2025)

    Wei, Z., Yao, W., Liu, Y., Zhang, W., Lu, Q., Qiu, L., Yu, C., Xu, P., Zhang, C., Yin, B., et al.: Webagent-r1: Training web agents via end-to-end multi-turn reinforcement learning. arXiv preprint arXiv:2505.16421 (2025)

    arXiv 2025

  43. [43]

    In: Proceedings of the IEEE conference on computer vision and pattern recognition

    Xia, G.S., Bai, X., Ding, J., Zhu, Z., Belongie, S., Luo, J., Datcu, M., Pelillo, M., Zhang, L.: Dota: A large-scale dataset for object detection in aerial images. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 3974–3983 (2018)

    2018

  44. [44]

    IEEE Transactions on Geoscience and Remote Sensing55(7), 3965–3981 (2017)

    Xia, G.S., Hu, J., Hu, F., Shi, B., Bai, X., Zhong, Y., Zhang, L., Lu, X.: Aid: A benchmark data set for performance evaluation of aerial scene classification. IEEE Transactions on Geoscience and Remote Sensing55(7), 3965–3981 (2017)

    2017

  45. [45]

    arXiv preprint arXiv:2406.07089 (2024)

    Xu, W., Yu, Z., Mu, B., Wei, Z., Zhang, Y., Li, G., Peng, M.: Rs-agent: automating remote sensing tasks through intelligent agent. arXiv preprint arXiv:2406.07089 (2024)

    Pith/arXiv arXiv 2024

  46. [46]

    arXiv preprint arXiv:2505.09388 (2025)

    Yang, A., Li, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Gao, C., Huang, C., Lv, C., et al.: Qwen3 technical report. arXiv preprint arXiv:2505.09388 (2025)

    Pith/arXiv arXiv 2025

  47. [47]

    Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K.R., Cao, Y.: React: Synergizingreasoningandactinginlanguagemodels.In:Theeleventhinternational conference on learning representations (2022)

    2022

  48. [48]

    IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing15, 3365–3375 (2022)

    Zhang, P., Xu, H., Tian, T., Gao, P., Li, L., Zhao, T., Zhang, N., Tian, J.: Se- fepnet: Scale expansion and feature enhancement pyramid network for sar aircraft detection with small sample dataset. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing15, 3365–3375 (2022)

    2022

  49. [49]

    Remote Sensing13(18), 3690 (2021)

    Zhang, T., Zhang, X., Li, J., Xu, X., Wang, B., Zhan, X., Xu, Y., Ke, X., Zeng, T., Su, H., et al.: Sar ship detection dataset (ssdd): Official release and comprehensive data analysis. Remote Sensing13(18), 3690 (2021)

    2021

  50. [50]

    thinking with images

    Zheng, Z., Yang, M., Hong, J., Zhao, C., Xu, G., Yang, L., Shen, C., Yu, X.: Deepeyes: Incentivizing" thinking with images" via reinforcement learning. arXiv preprint arXiv:2505.14362 (2025)

    Pith/arXiv arXiv 2025

  51. [51]

    thought-only

    Zhu, H., Chen, X., Dai, W., Fu, K., Ye, Q., Jiao, J.: Orientation robust object detection in aerial images using deep convolutional neural network. In: 2015 IEEE international conference on image processing (ICIP). pp. 3735–3739. IEEE (2015) 1 Supplementary OpenEarthAgent: A Unified Framework for Tool-Augmented Geospatial Agents Supplementary material inc...

    arXiv 2015