REVIEW 2 major objections 5 minor 2 cited by
VectorLLM: Human-like Extraction of Structured Building Contours vis Multimodal LLMs
T0 review · 2 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read The paper claims a multimodal LLM that outputs building contours corner-point by corner-point beats multi-stage segmentation-vectorization pipelines by 5.6-13.6 AP and zero-shots to unseen objects.
desk verdict A genuinely new MLLM-based approach to building contour extraction, but the headline AP gains are inflated by an unfair crop protocol and the zero-shot claim outruns the evidence. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is corner-by-corner autoregressive decoding into a coordinate vocabulary: the LLM emits one special token per x- or y-value, which forces the model to commit to a spatial position at each step and lets it use the already-emitted corners as context, exactly like a human tracing a polygon. Three training stages carry the argument: pretraining aligns vision features to the LLM with plain coordinate-string targets, supervised fine-tuning imposes a fixed orientation and starting point (top-left, clockwise) so outputs are regular and stable, and direct preference optimization (DPO) on automatically constructed chosen/rejected pairs cleans up repeated outputs, missing small lines, and roof-plane interference. An upstream FCOS detector with a Swin-L backbone crops each building instance, with boxes enlarged 1.3x, so the model sees one prominent building per image. The 128x128 input resolution and one-token-per-coordinate formulation keep the design deliberately simple.
What would settle it
Run P2PFormer, Line2Poly, and BuildMapper under the identical protocol—FCOS/Swin-L detections, 1.3x enlarged crops, single-building inputs—and compare AP on WHU, WHU-Mix, and CrowdAI; if the margins shrink to near zero, the central claim of LLM superiority for contour extraction does not hold. A complementary check would be to feed VectorLLM full scene images without any detector crops and measure the resulting drop, which would quantify how much of the system's capability depends on the external detector.
Extended reading notes
Core claim
VectorLLM recasts regular building contour extraction as an autoregressive language modeling task: the model predicts the next corner point's x- and y-coordinates, expressed as special tokens, conditioned on the image and the previously emitted corners. The architecture attaches a RADIO vision encoder to a Qwen3 LLM through an MLP projector, adds learnable position embeddings to the vision tokens, and trains with next-token prediction on coordinate strings, then on instruction-style data, then with direct preference optimization on chosen versus rejected contours. On the three benchmark datasets the model reports 79.4 AP, 68.4/56.4 AP (in-domain/out-of-domain), and 79.6 AP with detector-supplied boxes, and 90.2, 78.5/64.6, and 87.7 AP with oracle boxes, exceeding prior methods across the board. The authors argue this demonstrates that LLM-based topological reasoning is a qualitative leap for vector extraction, not an incremental gain.
Load-bearing premise
The headline gains assume the comparison protocol is fair: VectorLLM is evaluated on single-building crops produced by a separately trained detector, while the prior methods it beats are evaluated in their native end-to-end setting on full images, and the paper does not report those baselines under the same crop-and-recall protocol.
Editorial extensions
If this is right
- Building contour extraction collapses from three pipeline stages to one trained model: no pixel segmentation, no separate vectorization, and no polygon refinement step.
- A single co-trained model with one set of weights transfers across datasets with different annotation standards and image styles, as shown by the out-of-domain WHU-Mix test-2 score.
- Zero-shot contours for aircraft, water bodies, oil tanks, roads, and general objects imply the corner-point skill generalizes beyond buildings, opening a route to unified vector extraction for multiple geographic feature types.
- Scaling the LLM from 0.6B to 1.7B parameters improves AP on every dataset, so further scaling is expected to yield more gains.
- Preference optimization contributes a measurable ~1.8 AP gain and specifically fixes corner cases like duplicated nodes and missed small edges, so alignment techniques transfer to geometric outputs.
Reading between the lines
- The reported gains compare detector-cropped VectorLLM against prior methods evaluated end-to-end on full images; with the same crop protocol, the baselines might narrow the gap, so the cleanest attribution of the improvement to the LLM itself requires a matched evaluation.
- Because VectorLLM cannot yet find the buildings it delineates, the system's end-to-end accuracy is capped by the external detector; making the LLM itself localize instances would be the natural next step toward a true single-model pipeline.
- The fixed 128x128 input resolution is the likely cause of the reported failures on large and complex buildings; a multi-resolution or patch-based visual front end could raise the ceiling on fine detail.
- The zero-shot success on objects with very different geometry suggests the model may have learned a general straight-line-polygon prior during building training; testing on non-polygonal or curved boundaries, such as meandering rivers, would show how general that prior really is.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. VectorLLM introduces a multimodal large language model that extracts regular building contours by autoregressively predicting corner-point coordinates from single-building image crops. The architecture combines a RADIO vision encoder, learnable positional embeddings, an MLP projector, and a Qwen3-based language model, with coordinate-specific tokens added to the vocabulary. Training proceeds through pretraining, supervised fine-tuning, and direct preference optimization, using the WHU, WHU-Mix, CrowdAI, and COCO datasets. The paper reports state-of-the-art AP on the three building datasets and shows qualitative zero-shot contour extraction on aircraft, ships, water bodies, and other objects. The work also includes ablations of the positional embedding, vision encoder, training stages, model size, coordinate tokenization, and data scaling.
Significance. If the reported comparisons were controlled, this would be a significant result: a detector-plus-autoregressive-LLM system could replace multi-stage segmentation, vectorization, and refinement pipelines, and the oracle-box results (90.2 AP on WHU) suggest that the contour regression component itself is highly accurate. The ablations are systematic and the DPO preference-data construction is standard. The main weakness is that the headline SOTA margins compare VectorLLM under a detector-crop protocol to baselines evaluated end-to-end on full images, so the margins cannot yet be cleanly attributed to the LLM or the point-by-point task formulation. The zero-shot claim is also weakened by the inclusion of COCO in pretraining. A controlled comparison under the same protocol is needed before the central claim is established.
major comments (2)
- [§3.5, §4.1, Tables 1–3] The headline SOTA margins are not based on a controlled comparison. In §3.5 and §4.1, VectorLLM is tested on single-building crops produced by an external FCOS/Swin-L detector with 1.3× box enlargement, while prior methods such as P2PFormer, Line2Poly, and BuildMapper are evaluated end-to-end from full images. The paper does not report the detector's recall or run any baseline on the same detector-crop protocol, so the 5.6, 7.1, and 13.6 AP margins in Tables 1–3 may reflect the external detector's localization and instance separation rather than the LLM's contour quality. The oracle-bbox row in Table 1 (90.2 vs. 79.4 AP) further shows that bounding-box quality alone changes AP by about 11 points, so detector effects are material. The revision should report AP for P2PFormer, Line2Poly, and BuildMapper on the same FCOS crops, and should also report detector recall, before claiming that the LLM formulation is responsible for the gains.
- [§4.1, §4.4, Figure 5] The zero-shot claim is overstated because the model is pretrained on COCO. Section 4.1 states that VectorLLM is jointly trained on WHU, WHU-Mix, CrowdAI, and COCO training datasets, and Section 4.4 describes the results on aircraft, ships, water bodies, and oil tanks as objects that 'never appear in the training set.' However, COCO contains airplanes and ships, so those categories are not unseen if COCO instance masks are converted into contour sequences under the pretraining protocol described in §3.2. The paper should either exclude COCO categories from the zero-shot evaluation, report which COCO categories are actually used, or qualify the 'unseen' claim to categories absent from all training data.
minor comments (5)
- [Title/Abstract] The title contains a typo ('vis' should be 'via'), and the abstract's 'significantly outperformed' should be rephrased as 'outperforms' or 'outperformed' with a clear subject.
- [Table 7] The column headers for WHU, CrowdAI, and WHU-Mix do not indicate the metric; 'AP' should be included in each numeric column or specified in the caption.
- [Figures 1, 5, and 6] The captions state that the black bounding boxes are manually provided by users, but Section 4.1 describes a detector-based test protocol; the paper should clarify how the manual-box visualizations relate to the quantitative FCOS-based results and whether the qualitative figures use manual or detector boxes.
- [§3.4] The DPO data-generation rule 'IoU less than 0.8' should specify whether the IoU is computed between the predicted polygon and the ground-truth polygon, and in which coordinate space (image pixels or normalized coordinates).
- [§4.3, Table 6] The check-mark rows in the tuning-strategy ablation are not self-explanatory; a row label such as 'Pretraining only' would help readers interpret the effect of each training stage.
Circularity Check
No circularity found: the central results rest on public benchmarks and independent baselines, and the protocol concerns raised are fairness issues, not circular reductions.
full rationale
The paper's derivation chain is self-contained against external evidence. VectorLLM is trained on public building datasets (WHU, WHU-Mix, CrowdAI) plus COCO and evaluated with standard AP metrics against published baselines; the headline margins therefore do not reduce by construction to the model's own training labels. The DPO stage (§3.4) uses the SFT model's own low-IoU outputs as rejected answers and labels as chosen answers, but this is a standard preference-optimization construction and evaluation is performed on held-out test sets, so it is not a fitted input being renamed as a prediction. The paper's self-citations (BuildMapper, P2PFormer, Line2Poly) are prior published methods used as comparison baselines, not as load-bearing justifications for the present claims, and no uniqueness theorem or ansatz is imported from the authors' own prior work. The bounding-box cropping protocol (§3.5, §4.1) raises a legitimate comparison-fairness concern because prior methods are evaluated end-to-end on full images, and the COCO pretraining in §4.1 weakens the zero-shot claim for objects that appear in COCO; however, these are experimental-design limitations, not cases where an output is equivalent to an input by definition or where a derivation is forced by self-citation. No circular step can be exhibited from the paper's own equations or citations, so the appropriate finding is no significant circularity.
Assumptions & free parameters
free parameters (4)
- Input resolution 128x128 =
128x128 (hand-chosen)
- Test cropping enlargement factor =
1.3x
- DPO KL penalty beta =
0.5
- DPO IoU threshold =
0.8
assumptions (5)
- domain assumption Building contours can be represented as a single closed polygon without holes.
- domain assumption A single prominent building per crop is a sufficient unit for the task.
- domain assumption The LLM's pretrained reasoning is the cause of the performance gain.
- domain assumption Objects shown as zero-shot are absent from all training data.
- ad hoc to paper COCO instance masks can be converted into contour sequences and used as pretraining data without changing the building-focused objective.
Cite this review
Pith. "Pith review of VectorLLM: Human-like Extraction of Structured Building Contours vis Multimodal LLMs." pith.science (2026). https://pith.science/paper/M3MABDGB
@misc{pith2026250704664,
author = {Pith},
title = {Pith review of: VectorLLM: Human-like Extraction of Structured Building Contours vis Multimodal LLMs},
year = {2026},
howpublished = {\url{https://pith.science/paper/M3MABDGB}},
note = {Machine review of arXiv:2507.04664}
}
read the original abstract
Automatically extracting vectorized building contours from remote sensing imagery is crucial for urban planning, population estimation, and disaster assessment. Current state-of-the-art methods rely on complex multi-stage pipelines involving pixel segmentation, vectorization, and polygon refinement, which limits their scalability and real-world applicability. Inspired by the remarkable reasoning capabilities of Large Language Models (LLMs), we introduce VectorLLM, the first Multi-modal Large Language Model (MLLM) designed for regular building contour extraction from remote sensing images. Unlike existing approaches, VectorLLM performs corner-point by corner-point regression of building contours directly, mimicking human annotators' labeling process. Our architecture consists of a vision foundation backbone, an MLP connector, and an LLM, enhanced with learnable position embeddings to improve spatial understanding capability. Through comprehensive exploration of training strategies including pretraining, supervised fine-tuning, and preference optimization across WHU, WHU-Mix, and CrowdAI datasets, VectorLLM significantly outperformed the previous SOTA methods by 5.6 AP, 7.1 AP, 13.6 AP, respectively in the three datasets. Remarkably, VectorLLM exhibits strong zero-shot performance on unseen objects including aircraft, water bodies, and oil tanks, highlighting its potential for unified modeling of diverse remote sensing object contour extraction tasks. Overall, this work establishes a new paradigm for vector extraction in remote sensing, leveraging the topological reasoning capabilities of LLMs to achieve both high accuracy and exceptional generalization. All the codes and weights will be published for promoting community development.
Figures
Figures from the paper (5 more)
Forward citations
Cited by 2 Pith papers
-
Actor as Its Own Critic: Unifying Region Understanding and Localization via CycleGRPO
A single MLLM jointly improves region captioning and localization by rewarding captions that let it reconstruct the original mask, needing only region inputs.
-
HoliTracer: Holistic Vectorization of Geographic Objects from Large-Size Remote Sensing Imagery
HoliTracer is a framework that directly vectorizes geographic objects from very high resolution images over 10,000 pixels wide, using multi-scale attention and contour-sequence tracing.
Reference graph
Works this paper leans on
-
[1]
write newline
" write newline "" before.all 'output.state := FUNCTION list.label.output make.list.label ", " * write FUNCTION article output.bibitem list.label.output list.year output new.block title output new.block journal emphasize output format.volume.number output pages output new.block url output new.block note output fin.entry FUNCTION book output.bibitem list.l...
-
[2]
Efficient interactive annotation of segmentation datasets with polygon-rnn++
Acuna, D., Ling, H., Kar, A., Fidler, S., 2018. Efficient interactive annotation of segmentation datasets with polygon-rnn++. Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, 859--868
2018
-
[3]
Bai, J., Bai, S., Chu, Y., Cui, Z., Dang, K., Deng, X., Fan, Y., Ge, W., Han, Y., Huang, F. et al., 2023a. Qwen technical report. arXiv preprint arXiv:2309.16609
-
[4]
Qwen-vl: A frontier large vision-language model with versatile abilities
Bai, J., Bai, S., Yang, S., Wang, S., Tan, S., Wang, P., Lin, J., Zhou, C., Zhou, J., 2023b. Qwen-vl: A frontier large vision-language model with versatile abilities. arXiv preprint arXiv:2308.12966
-
[5]
Multi-task learning for segmentation of building footprints with deep neural networks
Bischke, B., Helber, P., Folz, J., Borth, D., Dengel, A., 2019. Multi-task learning for segmentation of building footprints with deep neural networks. 2019 IEEE International Conference on Image Processing (ICIP), IEEE, 1480--1484
2019
-
[6]
J., 2019
Bolya, D., Zhou, C., Xiao, F., Lee, Y. J., 2019. Yolact: Real-time instance segmentation. Proceedings of the IEEE/CVF International Conference on Computer Vision, 9157--9166
2019
-
[7]
Cai, Z., Cao, M., Chen, H., Chen, K., Chen, K., Chen, X., Chen, X., Chen, Z., Chen, Z., Chu, P. et al., 2024. Internlm2 technical report. arXiv preprint arXiv:2403.17297
arXiv 2024
-
[8]
Annotating object instances with a polygon-rnn
Castrejon, L., Kundu, K., Urtasun, R., Fidler, S., 2017. Annotating object instances with a polygon-rnn. Proceedings of the IEEE conference on computer vision and pattern recognition, 5230--5238
2017
Show all 106 references
-
[9]
ASF-Net: Adaptive Screening Feature Network for Building Footprint Extraction From Remote-Sensing Images
Chen, J., Jiang, Y., Luo, L., Gong, W., 2022. ASF-Net: Adaptive Screening Feature Network for Building Footprint Extraction From Remote-Sensing Images. IEEE Transactions on Geoscience and Remote Sensing , 60, 1--13
2022
-
[10]
RSPrompter: Learning to prompt for remote sensing instance segmentation based on visual foundation model
Chen, K., Liu, C., Chen, H., Zhang, H., Li, W., Zou, Z., Shi, Z., 2024a. RSPrompter: Learning to prompt for remote sensing instance segmentation based on visual foundation model. IEEE Transactions on Geoscience and Remote Sensing
-
[11]
L., Liu, X., 2020
Chen, Q., Wang, L., Waslander, S. L., Liu, X., 2020. An end-to-end shape modeling framework for vectorized building outline generation from aerial images. ISPRS Journal of Photogrammetry and Remote Sensing , 170, 114--126
2020
-
[12]
CGSANet: A Contour-Guided and Local Structure-Aware Encoder--Decoder Network for Accurate Building Extraction From Very High-Resolution Remote Sensing Imagery
Chen, S., Shi, W., Zhou, M., Zhang, M., Xuan, Z., 2021. CGSANet: A Contour-Guided and Local Structure-Aware Encoder--Decoder Network for Accurate Building Extraction From Very High-Resolution Remote Sensing Imagery. IEEE Journal of Selected Topics in Applied Earth Observations...
2021
-
[13]
et al., 2024b
Chen, Z., Wang, W., Cao, Y., Liu, Y., Gao, Z., Cui, E., Zhu, J., Ye, S., Tian, H., Liu, Z. et al., 2024b. Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling. arXiv preprint arXiv:2412.05271
-
[14]
et al., 2024c
Chen, Z., Wang, W., Tian, H., Ye, S., Gao, Z., Cui, E., Tong, W., Hu, K., Luo, J., Ma, Z. et al., 2024c. How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suites. arXiv preprint arXiv:2404.16821
-
[15]
et al., 2024d
Chen, Z., Wu, J., Wang, W., Su, W., Chen, G., Xing, S., Zhong, M., Zhang, Q., Zhu, X., Lu, L. et al., 2024d. Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Re...
-
[16]
G., Kirillov, A., Girdhar, R., 2022
Cheng, B., Misra, I., Schwing, A. G., Kirillov, A., Girdhar, R., 2022. Masked-attention mask transformer for universal image segmentation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 1290--1299
2022
-
[17]
E., Stoica, I., Xing, E
Chiang, W.-L., Li, Z., Lin, Z., Sheng, Y., Wu, Z., Zhang, H., Zheng, L., Zhuang, S., Zhuang, Y., Gonzalez, J. E., Stoica, I., Xing, E. P., 2023. Vicuna: An open-source chatbot impressing gpt-4 with 90\
2023
-
[18]
et al., 2020
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S. et al., 2020. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929
2020 arXiv
-
[19]
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., Houlsby, N., 2021. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. ICLR
2021
-
[20]
et al., 2024
Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang, A., Fan, A. et al., 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783
2024 arXiv
-
[21]
Geollava: Efficient fine-tuned vision-language models for temporal change detection in remote sensing
Elgendy, H., Sharshar, A., Aboeitta, A., Ashraf, Y., Guizani, M., 2024. Geollava: Efficient fine-tuned vision-language models for temporal change detection in remote sensing. arXiv preprint arXiv:2410.19552
2024 arXiv
-
[22]
Instances as queries
Fang, Y., Yang, S., Wang, X., Li, Y., Fang, C., Shan, Y., Feng, B., Liu, W., 2021. Instances as queries. Proceedings of the IEEE/CVF International Conference on Computer Vision, 6910--6919
2021
-
[23]
GPT-3: Its nature, scope, limits, and consequences
Floridi, L., Chiriatti, M., 2020. GPT-3: Its nature, scope, limits, and consequences. Minds and Machines , 30, 681--694
2020
-
[24]
Polygonal building extraction by frame field learning
Girard, N., Smirnov, D., Solomon, J., Tarabalka, Y., 2021. Polygonal building extraction by frame field learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 5891--5900
2021
-
[25]
Remote sensing chatgpt: Solving remote sensing tasks with chatgpt and visual models
Guo, H., Su, X., Wu, C., Du, B., Zhang, L., Li, D., 2024. Remote sensing chatgpt: Solving remote sensing tasks with chatgpt and visual models. arXiv preprint arXiv:2401.09083
2024 arXiv
-
[26]
HigherNet-DST: Higher resolution network with dynamic scale training for rooftop delineation
He, H., Ma, L., Li, J., 2024. HigherNet-DST: Higher resolution network with dynamic scale training for rooftop delineation. IEEE Transactions on Geoscience and Remote Sensing
2024
-
[27]
Mask r-cnn
He, K., Gkioxari, G., Doll \'a r, P., Girshick, R., 2017. Mask r-cnn. Proceedings of the IEEE international conference on computer vision, 2961--2969
2017
-
[28]
Rsgpt: A remote sensing vision language model and benchmark
Hu, Y., Yuan, J., Wen, C., Lu, X., Liu, Y., Li, X., 2025. Rsgpt: A remote sensing vision language model and benchmark. ISPRS Journal of Photogrammetry and Remote Sensing , 224, 272--286
2025
-
[29]
Sequentially delineation of rooftops with holes from VHR aerial images using a convolutional recurrent neural network
Huang, W., Liu, Z., Tang, H., Ge, J., 2021a. Sequentially delineation of rooftops with holes from VHR aerial images using a convolutional recurrent neural network. Remote Sensing , 13(21), 4271
-
[30]
OEC-RNN: Object-oriented delineation of rooftops with edges and corners using the recurrent neural network from the aerial images
Huang, W., Tang, H., Xu, P., 2021b. OEC-RNN: Object-oriented delineation of rooftops with edges and corners using the recurrent neural network from the aerial images. IEEE Transactions on Geoscience and Remote Sensing , 60, 1--12
-
[31]
A., Liu, E
Irvin, J. A., Liu, E. R., Chen, J. C., Dormoy, I., Kim, J., Khanna, S., Zheng, Z., Ermon, S., 2024. Teochat: A large vision-language assistant for temporal earth observation data. arXiv preprint arXiv:2410.06234
2024 arXiv
-
[32]
Fully convolutional networks for multisource building extraction from an open aerial and satellite imagery data set
Ji, S., Wei, S., Lu, M., 2018. Fully convolutional networks for multisource building extraction from an open aerial and satellite imagery data set. IEEE Transactions on geoscience and remote sensing , 57(1), 574--586
2018
-
[33]
A scale robust convolutional neural network for automatic building extraction from aerial and satellite imagery
Ji, S., Wei, S., Lu, M., 2019. A scale robust convolutional neural network for automatic building extraction from aerial and satellite imagery. International journal of remote sensing , 40(9), 3308--3322
2019
-
[34]
C., Lo, W.-Y
Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A. C., Lo, W.-Y. et al., 2023a. Segment anything. arXiv preprint arXiv:2304.02643
-
[35]
C., Lo, W.-Y
Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A. C., Lo, W.-Y. et al., 2023b. Segment anything. Proceedings of the IEEE/CVF International Conference on Computer Vision, 4015--4026
-
[36]
E., 2017
Krizhevsky, A., Sutskever, I., Hinton, G. E., 2017. ImageNet classification with deep convolutional neural networks. Communications of the ACM , 60(6), 84--90
2017
-
[37]
S., Naseer, M., Das, A., Khan, S., Khan, F
Kuckreja, K., Danish, M. S., Naseer, M., Das, A., Khan, S., Khan, F. S., 2024. Geochat: Grounded large vision-language model for remote sensing. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 27831--27840
2024
-
[38]
Lisa: Reasoning segmentation via large language model
Lai, X., Tian, Z., Chen, Y., Li, Y., Yuan, Y., Liu, S., Jia, J., 2024. Lisa: Reasoning segmentation via large language model. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 9579--9589
2024
-
[39]
VRSBench: A Versatile Vision-Language Benchmark Dataset for Remote Sensing Image Understanding
Li, X., Ding, J., Elhoseiny, M., 2024. VRSBench: A Versatile Vision-Language Benchmark Dataset for Remote Sensing Image Understanding. arXiv preprint arXiv:2406.12384
2024 arXiv
-
[40]
D., Lucchi, A., 2018
Li, Z., Wegner, J. D., Lucchi, A., 2018. Polymapper: Extracting city maps using polygons. arXiv preprint arXiv:1812.01497 , 2
2018 arXiv
-
[41]
D., Lucchi, A., 2019
Li, Z., Wegner, J. D., Lucchi, A., 2019. Topological map extraction from overhead images. Proceedings of the IEEE/CVF International Conference on Computer Vision, 1715--1724
2019
-
[42]
Polytransform: Deep polygon transformer for instance segmentation
Liang, J., Homayounfar, N., Ma, W.-C., Xiong, Y., Hu, R., Urtasun, R., 2020. Polytransform: Deep polygon transformer for instance segmentation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 9131--9140
2020
-
[43]
L., 2014
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Doll \'a r, P., Zitnick, C. L., 2014. Microsoft coco: Common objects in context. European conference on computer vision, Springer, 740--755
2014
-
[44]
Draw-and-understand: Leveraging visual prompts to enable mllms to comprehend what you want
Lin, W., Wei, X., An, R., Gao, P., Zou, B., Luo, Y., Huang, S., Zhang, S., Li, H., 2024. Draw-and-understand: Leveraging visual prompts to enable mllms to comprehend what you want
2024
-
[45]
Change-agent: Towards interactive comprehensive remote sensing change interpretation and analysis
Liu, C., Chen, K., Zhang, H., Qi, Z., Zou, Z., Shi, Z., 2024. Change-agent: Towards interactive comprehensive remote sensing change interpretation and analysis. IEEE Transactions on Geoscience and Remote Sensing
2024
-
[46]
J., 2023a
Liu, H., Li, C., Li, Y., Lee, Y. J., 2023a. Improved baselines with visual instruction tuning
-
[47]
J., 2023b
Liu, H., Li, C., Wu, Q., Lee, Y. J., 2023b. Visual instruction tuning
-
[48]
Path aggregation network for instance segmentation
Liu, S., Qi, L., Qin, H., Shi, J., Jia, J., 2018. Path aggregation network for instance segmentation. Proceedings of the IEEE conference on computer vision and pattern recognition, 8759--8768
2018
-
[49]
Swin transformer: Hierarchical vision transformer using shifted windows
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., Guo, B., 2021. Swin transformer: Hierarchical vision transformer using shifted windows. Proceedings of the IEEE/CVF international conference on computer vision, 10012--10022
2021
-
[50]
Building Outline Delineation From VHR Remote Sensing Images Using the Convolutional Recurrent Neural Network Embedded With Line Segment Information
Liu, Z., Tang, H., Huang, W., 2022. Building Outline Delineation From VHR Remote Sensing Images Using the Convolutional Recurrent Neural Network Embedded With Line Segment Information. IEEE Transactions on Geoscience and Remote Sensing , 60, 1--13
2022
-
[51]
et al., 2024a
Luo, J., Pang, Z., Zhang, Y., Wang, T., Wang, L., Dang, B., Lao, J., Wang, J., Chen, J., Tan, Y. et al., 2024a. Skysensegpt: A fine-grained instruction tuning dataset and model for remote sensing vision-language understanding. arXiv preprint arXiv:2406.10100
-
[52]
Cross-spatiotemporal land-cover classification from VHR remote sensing images with deep learning based domain adaptation
Luo, M., Ji, S., 2022. Cross-spatiotemporal land-cover classification from VHR remote sensing images with deep learning based domain adaptation. ISPRS Journal of Photogrammetry and Remote Sensing , 191, 105--128
2022
-
[53]
SAM-RSIS: Progressively adapting SAM with box prompting to remote sensing image instance segmentation
Luo, M., Zhang, T., Wei, S., Ji, S., 2024b. SAM-RSIS: Progressively adapting SAM with box prompting to remote sensing image instance segmentation. IEEE Transactions on Geoscience and Remote Sensing
-
[54]
P., 2019 (accessed November 10, 2019)
Mohanty, S. P., 2019 (accessed November 10, 2019). CrowdAI mapping challenge 2018 dataset. https://www.crowdai.org/challenges/mapping-challenge
2019
-
[55]
Lhrs-bot: Empowering remote sensing with vgi-enhanced large multimodal language model
Muhtar, D., Li, Z., Gu, F., Zhang, X., Xiao, P., 2024. Lhrs-bot: Empowering remote sensing with vgi-enhanced large multimodal language model. arXiv preprint arXiv:2402.02544
2024 arXiv
-
[56]
et al., 2023
Oquab, M., Darcet, T., Moutakanni, T., Vo, H., Szafraniec, M., Khalidov, V., Fernandez, P., Haziza, D., Massa, F., El-Nouby, A. et al., 2023. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193
2023 arXiv
-
[57]
et al., 2024
Pang, C., Wu, J., Li, J., Liu, Y., Sun, J., Li, W., Weng, X., Wang, S., Feng, L., Xia, G.-S. et al., 2024. H2RSVLM: Towards Helpful and Honest Remote Sensing Large Vision Language Model. arXiv preprint arXiv:2403.20213
2024 arXiv
-
[58]
Deep snake for real-time instance segmentation
Peng, S., Jiang, W., Pi, H., Li, X., Bao, H., Zhou, X., 2020. Deep snake for real-time instance segmentation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 8533--8542
2020
-
[59]
W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J. et al., 2021. Learning transferable visual models from natural language supervision. International conference on machine learning, PMLR, 8748--8763
2021
-
[60]
D., Ermon, S., Finn, C., 2024
Rafailov, R., Sharma, A., Mitchell, E., Manning, C. D., Ermon, S., Finn, C., 2024. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems , 36
2024
-
[61]
Am-radio: Agglomerative vision foundation model reduce all domains into one
Ranzinger, M., Heinrich, G., Kautz, J., Molchanov, P., 2024. Am-radio: Agglomerative vision foundation model reduce all domains into one. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 12490--12500
2024
-
[62]
M., Xing, E., Yang, M.-H., Khan, F
Rasheed, H., Maaz, M., Shaji, S., Shaker, A., Khan, S., Cholakkal, H., Anwer, R. M., Xing, E., Yang, M.-H., Khan, F. S., 2024. Glamm: Pixel grounding large multimodal model. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 13009--13018
2024
-
[63]
Pixellm: Pixel reasoning with large multimodal model
Ren, Z., Huang, Z., Wei, Y., Zhao, Y., Fu, D., Feng, J., Jin, X., 2024. Pixellm: Pixel reasoning with large multimodal model. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 26374--26383
2024
-
[64]
Geollm-engine: A realistic environment for building geospatial copilots
Singh, S., Fore, M., Stamoulis, D., 2024. Geollm-engine: A realistic environment for building geospatial copilots. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 585--594
2024
-
[65]
Internlm: A multilingual language model with progressively enhanced capabilities
Team, I., 2023. Internlm: A multilingual language model with progressively enhanced capabilities
2023
-
[66]
Fcos: Fully convolutional one-stage object detection
Tian, Z., Shen, C., Chen, H., He, T., 2019. Fcos: Fully convolutional one-stage object detection. Proceedings of the IEEE/CVF international conference on computer vision, 9627--9636
2019
-
[67]
et al., 2023
Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S. et al., 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288
2023 arXiv
-
[68]
From image transfer to object transfer: Cross-domain instance segmentation based on center point feature alignment
Wang, J., Ji, S., Zhang, T., 2023. From image transfer to object transfer: Cross-domain instance segmentation based on center point feature alignment. IEEE Transactions on Geoscience and Remote Sensing , 61, 1--11
2023
-
[69]
Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution
Wang, P., Bai, S., Tan, S., Wang, S., Fan, Z., Bai, J., Chen, K., Liu, X., Wang, J., Ge, W., Fan, Y., Dang, K., Du, M., Ren, X., Men, R., Liu, D., Zhou, C., Zhou, J., Lin, J., 2024a. Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution. arXiv p...
-
[70]
et al., 2024b
Wang, P., Hu, H., Tong, B., Zhang, Z., Yao, F., Feng, Y., Zhu, Z., Chang, H., Diao, W., Ye, Q. et al., 2024b. RingMoGPT: A Unified Remote Sensing Foundation Model for Vision, Language, and grounded tasks. IEEE Transactions on Geoscience and Remote Sensing
-
[71]
Solo: Segmenting objects by locations
Wang, X., Kong, T., Shen, C., Jiang, Y., Li, L., 2020. Solo: Segmenting objects by locations. European Conference on Computer Vision, Springer, 649--665
2020
-
[72]
Skyscript: A large and semantically diverse vision-language dataset for remote sensing
Wang, Z., Prabha, R., Huang, T., Wu, J., Rajagopal, R., 2024c. Skyscript: A large and semantically diverse vision-language dataset for remote sensing. Proceedings of the AAAI Conference on Artificial Intelligence, 38number 6, 5805--5813
-
[73]
Graph convolutional networks for the automated production of building vector maps from aerial images
Wei, S., Ji, S., 2021. Graph convolutional networks for the automated production of building vector maps from aerial images. IEEE Transactions on Geoscience and Remote Sensing , 60, 1--11
2021
-
[74]
Toward automatic building footprint delineation from aerial images using CNN and regularization
Wei, S., Ji, S., Lu, M., 2019. Toward automatic building footprint delineation from aerial images using CNN and regularization. IEEE Transactions on Geoscience and Remote Sensing , 58(3), 2178--2189
2019
-
[75]
A Concentric Loop Convolutional Neural Network for Manual Delineation-Level Building Boundary Segmentation From Remote-Sensing Images
Wei, S., Zhang, T., Ji, S., 2021. A Concentric Loop Convolutional Neural Network for Manual Delineation-Level Building Boundary Segmentation From Remote-Sensing Images. IEEE Transactions on Geoscience and Remote Sensing , 60, 1--11
2021
-
[76]
BuildMapper: A fully learnable framework for vectorized building contour extraction
Wei, S., Zhang, T., Ji, S., Luo, M., Gong, J., 2023. BuildMapper: A fully learnable framework for vectorized building contour extraction. ISPRS Journal of Photogrammetry and Remote Sensing , 197, 87--104
2023
-
[77]
From lines to Polygons: Polygonal building contour extraction from High-Resolution remote sensing imagery
Wei, S., Zhang, T., Yu, D., Ji, S., Zhang, Y., Gong, J., 2024. From lines to Polygons: Polygonal building contour extraction from High-Resolution remote sensing imagery. ISPRS Journal of Photogrammetry and Remote Sensing , 209, 213--232
2024
-
[78]
A., 2022
Wu, Y., Xu, L., Chen, Y., Wong, A., Clausi, D. A., 2022. TAL: Topography-Aware Multi-Resolution Fusion Learning for Enhanced Building Footprint Extraction. IEEE Geoscience and Remote Sensing Letters , 19, 1--5
2022
-
[79]
Vectorizing historical maps with topological consistency: A hybrid approach using transformers and contour-based instance segmentation
Xia, X., Zhang, T., Heitzler, M., Hurni, L., 2024a. Vectorizing historical maps with topological consistency: A hybrid approach using transformers and contour-based instance segmentation. International Journal of Applied Earth Observation and Geoinformation , 129, 103837
-
[80]
Video instance segmentation is all you need for linking geographic entities from historical maps
Xia, X., Zhang, T., Hurni, L., 2024b. Video instance segmentation is all you need for linking geographic entities from historical maps. IGARSS 2024-2024 IEEE International Geoscience and Remote Sensing Symposium, IEEE, 8491--8494
2024
-
[81]
Polarmask: Single shot instance segmentation with polar representation
Xie, E., Sun, P., Song, X., Wang, W., Liu, X., Liang, D., Shen, C., Luo, P., 2020. Polarmask: Single shot instance segmentation with polar representation. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 12193--12202
2020
-
[82]
HiSup: Accurate polygonal mapping of buildings in satellite imagery with hierarchical supervision
Xu, B., Xu, J., Xue, N., Xia, G.-S., 2023. HiSup: Accurate polygonal mapping of buildings in satellite imagery with hierarchical supervision. ISPRS Journal of Photogrammetry and Remote Sensing , 198, 284--296
2023
-
[83]
et al., 2025
Yang, A., Li, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Gao, C., Huang, C., Lv, C. et al., 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388
2025 arXiv
-
[84]
et al., 2024
Yang, A., Yang, B., Hui, B., Zheng, B., Yu, B., Zhou, C., Li, C., Li, C., Liu, D., Huang, F. et al., 2024. Qwen2 technical report. arXiv preprint arXiv:2407.10671
2024 arXiv
-
[85]
A review of recurrent neural networks: LSTM cells and network architectures
Yu, Y., Si, X., Hu, C., Zhang, J., 2019. A review of recurrent neural networks: LSTM cells and network architectures. Neural computation , 31(7), 1235--1270
2019
-
[86]
Sa2VA: Marrying SAM2 with LLaVA for Dense Grounded Understanding of Images and Videos
Yuan, H., Li, X., Zhang, T., Huang, Z., Xu, S., Ji, S., Tong, Y., Qi, L., Feng, J., Yang, M.-H., 2025. Sa2VA: Marrying SAM2 with LLaVA for Dense Grounded Understanding of Images and Videos. arXiv preprint arXiv:2501.04001
2025 arXiv
-
[87]
Learning building extraction in aerial scenes with convolutional networks
Yuan, J., 2017. Learning building extraction in aerial scenes with convolutional networks. IEEE transactions on pattern analysis and machine intelligence , 40(11), 2793--2798
2017
-
[88]
Osprey: Pixel understanding with visual instruction tuning
Yuan, Y., Li, W., Liu, J., Tang, D., Luo, X., Qin, C., Zhang, L., Zhu, J., 2024. Osprey: Pixel understanding with visual instruction tuning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 28202--28211
2024
-
[89]
Skyeyegpt: Unifying remote sensing vision-language tasks via instruction tuning with large language model
Zhan, Y., Xiong, Z., Yuan, Y., 2024. Skyeyegpt: Unifying remote sensing vision-language tasks via instruction tuning with large language model. arXiv preprint arXiv:2401.09712
2024 arXiv
-
[90]
HiT: Building Mapping with Hierarchical Transformers
Zhang, M., Liu, Q., Wang, Y., 2024a. HiT: Building Mapping with Hierarchical Transformers. IEEE Transactions on Geoscience and Remote Sensing
-
[91]
Gpt4roi: Instruction tuning large language model on region-of-interest
Zhang, S., Sun, P., Chen, S., Xiao, M., Shao, W., Zhang, W., Liu, Y., Chen, K., Luo, P., 2025a. Gpt4roi: Instruction tuning large language model on region-of-interest. European Conference on Computer Vision, Springer, 52--70
-
[92]
C., Yan, S., 2024b
Zhang, T., Li, X., Fei, H., Yuan, H., Wu, S., Ji, S., Loy, C. C., Yan, S., 2024b. Omg-llava: Bridging image-level, object-level, pixel-level reasoning and understanding. arXiv preprint arXiv:2406.19389
-
[93]
Pixel-sail: Single transformer for pixel-grounded understanding
Zhang, T., Li, X., Huang, Z., Li, Y., Lei, W., Deng, X., Chen, S., Ji, S., Feng, J., 2025b. Pixel-sail: Single transformer for pixel-grounded understanding. arXiv preprint arXiv:2504.10465
-
[94]
E2ec: An end-to-end contour-based method for high-quality high-speed instance segmentation
Zhang, T., Wei, S., Ji, S., 2022. E2ec: An end-to-end contour-based method for high-quality high-speed instance segmentation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 4443--4452
2022
-
[95]
P2PFormer: A Primitive-to-polygon Method for Regular Building Contour Extraction from Remote Sensing Images
Zhang, T., Wei, S., Zhou, Y., Luo, M., Yu, W., Ji, S., 2024c. P2PFormer: A Primitive-to-polygon Method for Regular Building Contour Extraction from Remote Sensing Images. IEEE Transactions on Geoscience and Remote Sensing
-
[96]
Popeye: A Unified Visual-Language Model for Multi-Source Ship Detection from Remote Sensing Imagery
Zhang, W., Cai, M., Zhang, T., Lei, G., Zhuang, Y., Mao, X., 2024d. Popeye: A Unified Visual-Language Model for Multi-Source Ship Detection from Remote Sensing Imagery. arXiv preprint arXiv:2403.03790
-
[97]
Earthgpt: A universal multi-modal large language model for multi-sensor image comprehension in remote sensing domain
Zhang, W., Cai, M., Zhang, T., Zhuang, Y., Mao, X., 2024e. Earthgpt: A universal multi-modal large language model for multi-sensor image comprehension in remote sensing domain. IEEE Transactions on Geoscience and Remote Sensing
-
[98]
RS5M and GeoRSCLIP: A large scale vision-language dataset and a large vision-language model for remote sensing
Zhang, Z., Zhao, T., Guo, Y., Yin, J., 2024f. RS5M and GeoRSCLIP: A large scale vision-language dataset and a large vision-language model for remote sensing. IEEE Transactions on Geoscience and Remote Sensing
-
[99]
Building extraction from satellite images using mask r-cnn with building boundary regularization
Zhao, K., Kang, J., Jung, J., Sohn, G., 2018. Building extraction from satellite images using mask r-cnn with building boundary regularization. Proceedings of the IEEE conference on computer vision and pattern recognition workshops, 247--251
2018
-
[100]
Building instance segmentation and boundary regularization from high-resolution remote sensing images
Zhao, W., Persello, C., Stein, A., 2020. Building instance segmentation and boundary regularization from high-resolution remote sensing images. IGARSS 2020-2020 IEEE International Geoscience and Remote Sensing Symposium, IEEE, 3916--3919
2020
-
[101]
Building outline delineation: From aerial images to polygons with an improved end-to-end learning framework
Zhao, W., Persello, C., Stein, A., 2021. Building outline delineation: From aerial images to polygons with an improved end-to-end learning framework. ISPRS journal of photogrammetry and remote sensing , 175, 119--131
2021
-
[102]
GeoGround: A Unified Large Vision-Language Model
Zhou, Y., Lan, M., Li, X., Ke, Y., Jiang, X., Feng, L., Zhang, W., 2024. GeoGround: A Unified Large Vision-Language Model. for Remote Sensing Visual Grounding. arXiv preprint arXiv:2411.11904
2024 arXiv
-
[103]
et al., 2025
Zhu, J., Wang, W., Chen, Z., Liu, Z., Ye, S., Gu, L., Duan, Y., Tian, H., Su, W., Shao, J. et al., 2025. InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models. arXiv preprint arXiv:2504.10479
2025 arXiv
-
[104]
MAP-Net: Multiple attending path neural network for building footprint extraction from remote sensed imagery
Zhu, Q., Liao, C., Hu, H., Mei, X., Li, H., 2020. MAP-Net: Multiple attending path neural network for building footprint extraction from remote sensed imagery. IEEE Transactions on Geoscience and Remote Sensing , 59(7), 6169--6181
2020
-
[105]
Polyworld: Polygonal building extraction with graph neural networks in satellite images
Zorzi, S., Bazrafkan, S., Habenschuss, S., Fraundorfer, F., 2022. Polyworld: Polygonal building extraction with graph neural networks in satellite images. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 1848--1857
2022
-
[106]
Machine-learned regularization and polygonization of building segmentation masks
Zorzi, S., Bittner, K., Fraundorfer, F., 2021. Machine-learned regularization and polygonization of building segmentation masks. 2020 25th International Conference on Pattern Recognition (ICPR), IEEE, 3098--3105
2021
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.