Pith. sign in

REVIEW 3 major objections 4 minor 26 references

Zero-Shot Tree Detection and Segmentation from Aerial Forest Imagery

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Pith's one-line read A pretrained SAM2 model, with no fine-tuning, can segment individual tree crowns from aerial imagery and can turn tree-detector boxes into masks.

desk verdict A plausible zero-shot SAM2 application to tree delineation with an honest but underspecified benchmark; the 'natural synergy' claim in the abstract is not backed by quantitative results. read the letter →

arxiv 2506.03114 v1 pith:KHP4XAV7 submitted 2025-06-03 cs.CV

classification cs.CV
keywords zero-shotsegmentationSAM2treecrowndelineationaerialforestimageryfoundationmodelsobjectdetectionpromptingremotesensingmonitoring
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to show that a pretrained image segmentation model, used without any tree-specific training, can delineate individual trees from aerial forest photos, and that feeding it bounding boxes from specialized tree detectors transfers those detections into segmentation masks. If true, tree-crown mapping could scale to new ecosystems without collecting labeled segmentation data for every site. The authors benchmark this against two specialized CNN-based detectors on three datasets and report that zero-shot SAM2 reaches recall comparable to the specialized models, with lower precision due to over-segmentation.

What carries the argument

The central object is SAM2, a transformer-based promptable image segmentation model, here used with the Hiera-L image encoder and its automatic mask generator. In zero-shot mode the generator uniformly samples single-point prompts on a grid and yields candidate masks, which are post-processed to remove disconnected regions; in transfer mode the model is prompted with bounding boxes produced by a specialized tree detector, so the detector supplies locations and SAM2 supplies crown shapes. Non-maximum suppression on predicted polygons is applied before evaluation.

What would settle it

On a held-out set of aerial images with manual crown annotations, measure how many SAM2 automatic masks agree with a labeled crown at an intersection-over-union threshold of 0.4; if most masks do not correspond to crowns, or if changing the prompt grid spacing drastically changes the tree count, the central claim is falsified.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that SAM2's automatic mask generator, which places a grid of single-point prompts and then removes disconnected regions, produces tree-crown masks in aerial RGB imagery without any fine-tuning, and that the same pretrained model can serve as a segmentation backend for specialized tree detectors by taking their predicted bounding boxes as prompts. The paper frames this as evidence that a generic foundation model can generalize to a domain it was not trained on and can combine naturally with in-domain detection models.

Load-bearing premise

The pipeline relies on the automatic mask generator's grid-sampled prompts producing masks that correspond to individual tree crowns rather than arbitrary image segments, with the paper checking this mainly visually and leaving the grid spacing and post-processing unspecified.

Editorial extensions

If this is right

  • Tree-crown delineation would no longer require a newly labeled segmentation dataset for each geographic region.
  • Any tree detector that outputs bounding boxes could be extended to output crown masks by routing its boxes through SAM2.
  • Zero-shot performance on an unseen site with oblique camera angles suggests the approach may tolerate varied aerial perspectives, though the evidence is qualitative.
  • The low precision scores indicate the raw zero-shot masks need filtering or non-maximum suppression before they can support tree counting.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An extension the authors leave implicit: if box-prompt transfer is reliable, the same recipe could equip any object detector with segmentation ability, not just tree detectors.
  • A testable consequence of their pipeline is that the number and quality of zero-shot tree masks should depend on the prompt grid spacing, which the paper does not report; varying that spacing would reveal how much of the result is tuning-dependent.
  • The qualitative results on oblique drone imagery point toward low-altitude surveys, but the paper has not quantitatively compared oblique and top-down settings.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper investigates whether a pretrained SAM2 model with a Hiera-L encoder, used without fine-tuning, can perform individual tree detection and segmentation from aerial RGB imagery. Two tasks are considered: standalone zero-shot segmentation using SAM2's automatic mask generator, and zero-shot transfer by prompting SAM2 with DeepForest bounding boxes. The authors compare against DeepForest and Detectree2 on the NEON TreeEvaluation and Detectree2 benchmark datasets, and show qualitative results on the Emerald Point dataset. Table 1 reports precision and recall at an IoU threshold of 0.4 for standalone SAM2, finding low precision but recall comparable to the specialized detectors on the Detectree2 dataset. The paper concludes that SAM2 shows strong generalization and can form a natural synergy with specialized tree detection models.

Significance. The contribution is an early empirical probe of whether a generic foundation model can substitute for domain-specific training in tree delineation. The paper's strengths are that it uses external public benchmarks, states the metric protocol for Table 1, applies NMS to all methods, and releases code. If the central synergy claim were quantitatively supported, this would be a useful result for the remote sensing community. However, the abstract's headline claim that SAM2 'can form a natural synergy' with specialized detectors is never measured, and the standalone zero-shot pipeline depends on unreported parameters that directly control the reported precision and recall. The contribution is therefore currently an interesting demonstration with unsupported central claims, rather than a complete empirical study.

major comments (3)
  1. [Section 3, Table 1 and Figures 1d, 3e] The abstract's central claim that SAM2 'can form a natural synergy with specialized methods trained on in-domain labeled data' is never quantitatively evaluated. The DeepForest+SAM2 transfer pathway is introduced in Section 2 as Task (2), but no precision, recall, or mask-quality metric is reported for it in Section 3 or Table 1; Figures 1d and 3e are qualitative only. Without a numeric comparison of DeepForest+SAM2 against SAM2 alone and against the specialized baselines, the synergy claim is unsupported. Please add a quantitative evaluation of the prompted transfer pathway, including the metric used and the number of images evaluated.
  2. [Section 2, 'Zero-shot Tree Segmentation'] The automatic mask generation pipeline is not reproducible as described. The paper states that SAM2's automatic mask generator 'uniformly samples single-point prompts in a grid structure' and that cropped sections receive 'standard post-processing to remove disconnected regions,' but it does not report the grid spacing, crop size and overlap, the threshold for discarding disconnected regions, or how these interact with the reported NMS IoU threshold of 0.05. These choices directly determine the number of candidate masks and therefore the precision and recall in Table 1. Without them, the low precision of SAM2 cannot be attributed to the model itself rather than to prompt density and post-processing settings. Please report these hyperparameters or provide code paths that fix them.
  3. [Section 3, 'Generalization to Emerald Point' and Figure 2] The claim that 'SAM2 significantly outperforms existing models' on the Emerald Point dataset is based solely on visual inspection, with no quantitative metric such as detection count, precision, recall, or IoU on that dataset. The authors acknowledge that 'more extensive evaluations are needed,' but this hedge does not support the strong wording in the Introduction and Abstract. Either provide quantitative results on Emerald Point or temper the claim to state that the improvement is a qualitative observation requiring further validation.
minor comments (4)
  1. [Title and Section 2 heading] The title and the Section 2 heading contain spacing errors: 'ZERO-SHOTTREEDETECTION ANDSEGMENTATION' and 'ZERO-SHOTPREDICTION ANDTRANSFER' should be separated by spaces.
  2. [Section 3, 'Baselines'] The word 'non-maxmimum' should be 'non-maximum,' and the acronym 'IOU' should be consistently written as 'IoU' throughout the text and Table 1.
  3. [Figure 1 caption] The caption begins 'SAM's prompt-based segmentation framework' while the paper refers to SAM2 throughout; please make the captions consistent with the model name.
  4. [Table 1] Table 1 reports point estimates without standard deviations or the number of images used; adding these would help assess the stability of the reported differences, especially given the modest margins in the Detectree2 recall column.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper is an external empirical benchmark with pretrained models, no fitted parameters, and no self-cited load-bearing claims.

full rationale

The paper's derivation chain is an empirical evaluation, not a mathematical or definitional derivation. SAM2, DeepForest, and Detectree2 are pretrained third-party models with public weights, and the authors state explicitly that 'the data we evaluate all methods on was not used during training.' No parameter in the paper is fitted to the evaluation data, and no result is defined in terms of the quantity it is claimed to predict. The main positive claim—that SAM2 shows 'impressive generalization' and can 'form a natural synergy with specialized methods'—is supported only by qualitative figures for the DeepForest→SAM2 transfer pathway, and the authors hedge this with 'While more extensive evaluations are needed.' That absence of quantitative evidence is a completeness or correctness concern, not circularity. The use of the authors' prior Emerald Point data (Young et al., 2022) is limited to qualitative generalization illustration and is not used to fit any parameter or to define the evaluation metric, so the self-citation is not load-bearing. The unspecified automatic-mask-generator grid spacing and post-processing details affect reproducibility but do not make the benchmark circular, because the compared methods are evaluated under the same externally defined protocol rather than under a criterion constructed from the paper's own outputs. No equation reduces to another equation, no fitted parameter is renamed as a prediction, and no uniqueness theorem is imported from the authors' prior work. The central claims therefore retain independent empirical content, and the correct circularity finding is a clean non-finding.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

No new entities are introduced. The paper relies on existing pretrained models and public datasets. The free parameters listed are evaluation-protocol choices rather than fitted model parameters, but they influence the reported scores.

free parameters (3)
  • SAM2 automatic mask generator prompt grid spacing and post-processing thresholds = not reported
    The zero-shot segmentation uses uniformly sampled grid points as prompts and 'standard post-processing' to remove disconnected regions; exact values are not given, so results depend on unspecified implementation choices.
  • Detection confidence threshold = 0.1
    All methods are evaluated with a minimum confidence of 0.1 for detections, which the authors note does not exactly match prior work; this choice affects reported precision and recall.
  • NMS IoU threshold for polygons = 0.05
    Non-maximum suppression is applied to predicted polygons with an IoU threshold of 0.05; the authors state this retains more detections compared to bounding-box NMS, affecting precision and recall.
assumptions (3)
  • domain assumption SAM2's pretrained representations transfer from natural images to overhead forest imagery.
    The entire zero-shot evaluation presupposes that a model trained on generic images produces sensible masks for aerial tree canopies; the paper only shows qualitative support.
  • domain assumption Manual bounding-box annotations in NEON and Detectree2 datasets are accurate ground truth for individual trees.
    Quantitative precision and recall use these annotations without auditing label quality; inconsistencies across sites would affect all methods.
  • domain assumption Tree crowns can be delineated from RGB aerial imagery at the resolutions tested.
    The task assumes visual separability of individual crowns; this is an ecological premise not established in the paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Zero-Shot Tree Detection and Segmentation from Aerial Forest Imagery." pith.science (2026). https://pith.science/paper/KHP4XAV7

@misc{pith2026250603114,
  author       = {Pith},
  title        = {Pith review of: Zero-Shot Tree Detection and Segmentation from Aerial Forest Imagery},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KHP4XAV7}},
  note         = {Machine review of arXiv:2506.03114}
}
read the original abstract

Large-scale delineation of individual trees from remote sensing imagery is crucial to the advancement of ecological research, particularly as climate change and other environmental factors rapidly transform forest landscapes across the world. Current RGB tree segmentation methods rely on training specialized machine learning models with labeled tree datasets. While these learning-based approaches can outperform manual data collection when accurate, the existing models still depend on training data that's hard to scale. In this paper, we investigate the efficacy of using a state-of-the-art image segmentation model, Segment Anything Model 2 (SAM2), in a zero-shot manner for individual tree detection and segmentation. We evaluate a pretrained SAM2 model on two tasks in this domain: (1) zero-shot segmentation and (2) zero-shot transfer by using predictions from an existing tree detection model as prompts. Our results suggest that SAM2 not only has impressive generalization capabilities, but also can form a natural synergy with specialized methods trained on in-domain labeled data. We find that applying large pretrained models to problems in remote sensing is a promising avenue for future progress. We make our code available at: https://github.com/open-forest-observatory/tree-detection-framework.

Figures

Figures reproduced from arXiv: 2506.03114 by the authors.

Figure 1
Figure 1. SAM’s prompt-based segmentation framework enables (c) [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Segmentation results using aerial images from the Emerald Point dataset. Each row in [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Tree crown detection results using RGB images from the NEON (Row 1) and Detectree2 [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

26 extracted references · 18 canonical work pages

  1. [1]

    Accurate delineation of individual tree crowns in tropical forests from aerial rgb imagery using mask r-cnn

    James GC Ball, Sebastian HM Hickman, Tobias D Jackson, Xian Jing Koay, James Hirst, William Jay, Matthew Archer, M \'e laine Aubry-Kientz, Gr \'e goire Vincent, and David A Coomes. Accurate delineation of individual tree crowns in tropical forests from aerial rgb imagery using mask r-cnn. Remote Sensing in Ecology and Conservation, 9 0 (5): 0 641--655, 2023

  2. [2]

    Depth any canopy: Leveraging depth foundation models for canopy height estimation

    Daniele Rege Cambrin, Isaac Corley, and Paolo Garza. Depth any canopy: Leveraging depth foundation models for canopy height estimation. arXiv preprint arXiv:2408.04523, 2024

  3. [3]

    Mechanisms of forest resilience

    Donald A Falk, Philip J van Mantgem, Jon E Keeley, Rachel M Gregg, Christopher H Guiterman, Alan J Tepley, Derek JN Young, and Laura A Marshall. Mechanisms of forest resilience. Forest Ecology and Management, 512: 0 120129, 2022

  4. [4]

    Remote sensing in forestry: current challenges, considerations and directions

    Fabian Ewald Fassnacht, Joanne C White, Michael A Wulder, and Erik N sset. Remote sensing in forestry: current challenges, considerations and directions. Forestry: An International Journal of Forest Research, 97 0 (1): 0 11--37, 2024

  5. [5]

    Emerging signals of declining forest resilience under climate change

    Giovanni Forzieri, Vasilis Dakos, Nate G McDowell, Alkama Ramdane, and Alessandro Cescatti. Emerging signals of declining forest resilience under climate change. Nature, 608 0 (7923): 0 534--539, 2022

  6. [6]

    Automated detection of conifer seedlings in drone imagery using convolutional neural networks

    Michael Fromm, Matthias Schubert, Guillermo Castilla, Julia Linke, and Greg McDermid. Automated detection of conifer seedlings in drone imagery using convolutional neural networks. Remote Sensing, 11 0 (21): 0 2585, 2019

  7. [7]

    Tree crown detection and delineation in a temperate deciduous forest from uav rgb imagery using deep learning approaches: Effects of spatial resolution and species characteristics

    Yi Gan, Quan Wang, and Atsuhiro Iio. Tree crown detection and delineation in a temperate deciduous forest from uav rgb imagery using deep learning approaches: Effects of spatial resolution and species characteristics. Remote Sensing, 15 0 (3): 0 778, 2023

  8. [8]

    No more training: Sam’s zero-shot transfer capabilities for cost-efficient medical image segmentation

    Juan D Guti \'e rrez, Roberto Rodriguez-Echeverria, Emilio Delgado, Miguel \'A ngel Suero Rodrigo, and Fernando S \'a nchez-Figueroa. No more training: Sam’s zero-shot transfer capabilities for cost-efficient medical image segmentation. IEEE Access, 2024

Show all 26 references
  1. [9]

    Mask r-cnn

    Kaiming He, Georgia Gkioxari, Piotr Doll \'a r, and Ross Girshick. Mask r-cnn. In Proceedings of the IEEE international conference on computer vision, pp.\ 2961--2969, 2017

  2. [10]

    Masked autoencoders are scalable vision learners

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Doll \'a r, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.\ 16000--16009, 2022

  3. [11]

    Machine learning based wildfire susceptibility mapping using remotely sensed fire data and gis: A case study of adana and mersin provinces, turkey

    Muzaffer Can Iban and Aliihsan Sekertekin. Machine learning based wildfire susceptibility mapping using remotely sensed fire data and gis: A case study of adana and mersin provinces, turkey. Ecological Informatics, 69: 0 101647, 2022

  4. [12]

    Segment anything

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al. Segment anything. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.\ 4015--4026, 2023

  5. [13]

    Applications in remote sensing to forest ecology and management

    Alex M Lechner, Giles M Foody, and Doreen S Boyd. Applications in remote sensing to forest ecology and management. One Earth, 2 0 (5): 0 405--412, 2020

  6. [14]

    Focal loss for dense object detection

    T Lin. Focal loss for dense object detection. arXiv preprint arXiv:1708.02002, 2017

  7. [15]

    Sam 2: Segment anything in images and videos

    Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman R \"a dle, Chloe Rolland, Laura Gustafson, et al. Sam 2: Segment anything in images and videos. arXiv preprint arXiv:2408.00714, 2024

  8. [16]

    Hiera: A hierarchical vision transformer without the bells-and-whistles

    Chaitanya Ryali, Yuan-Ting Hu, Daniel Bolya, Chen Wei, Haoqi Fan, Po-Yao Huang, Vaibhav Aggarwal, Arkabandhu Chowdhury, Omid Poursaeed, Judy Hoffman, et al. Hiera: A hierarchical vision transformer without the bells-and-whistles. In International Conference on Machine Learning...

  9. [17]

    Imputation of individual longleaf pine (pinus palustris mill.) tree attributes from field and lidar data

    Carlos A Silva, Andrew T Hudak, Lee A Vierling, E Louise Loudermilk, Joseph J O’Brien, J Kevin Hiers, Steve B Jack, Carlos Gonzalez-Benecke, Heezin Lee, Michael J Falkowski, et al. Imputation of individual longleaf pine (pinus palustris mill.) tree attributes from field and li...

  10. [18]

    Attention is all you need

    A Vaswani. Attention is all you need. Advances in Neural Information Processing Systems, 2017

  11. [19]

    Individual tree-crown detection in rgb imagery using semi-supervised deep learning neural networks

    Ben G Weinstein, Sergio Marconi, Stephanie Bohlman, Alina Zare, and Ethan White. Individual tree-crown detection in rgb imagery using semi-supervised deep learning neural networks. Remote Sensing, 11 0 (11): 0 1309, 2019

  12. [20]

    Deepforest: A python package for rgb deep learning tree crown delineation

    Ben G Weinstein, Sergio Marconi, M \'e laine Aubry-Kientz, Gregoire Vincent, Henry Senyondo, and Ethan P White. Deepforest: A python package for rgb deep learning tree crown delineation. Methods in Ecology and Evolution, 11 0 (12): 0 1743--1751, 2020

  13. [21]

    A remote sensing derived data set of 100 million individual tree crowns for the national ecological observatory network

    Ben G Weinstein, Sergio Marconi, Stephanie A Bohlman, Alina Zare, Aditya Singh, Sarah J Graves, and Ethan P White. A remote sensing derived data set of 100 million individual tree crowns for the national ecological observatory network. Elife, 10: 0 e62922, 2021

  14. [22]

    Optimizing aerial imagery collection and processing parameters for drone-based individual tree mapping in structurally complex conifer forests

    Derek JN Young, Michael J Koontz, and JonahMaria Weeks. Optimizing aerial imagery collection and processing parameters for drone-based individual tree mapping in structurally complex conifer forests. Methods in Ecology and Evolution, 13 0 (7): 0 1447--1463, 2022

  15. [23]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

  16. [24]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  17. [25]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  18. [26]

    c a1Y-X )8 KY5 ׷ z!q1J_˨ =r?x ύ_m= S7 |ʪV ר GYt ,mw1 : sH ;ksF#L-: Ƴ). <c)r<͈ i??B|Ϸ= ad 4B75 K=

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.