REVIEW 5 major objections 4 minor 24 references
SAMST: A Transformer framework based on SAM pseudo label filtering for remote sensing semi-supervised semantic segmentation
T0 review · 5 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read SAMST claims that a frozen SAM can be used to clean noisy pseudo-labels, lifting semi-supervised segmentation on Potsdam to 70.80 mIoU from a 67.00 baseline with only 1/32 of labels.
desk verdict Plausible SAM-based pseudo-label refinement recipe, but Table I is internally inconsistent and the evidence is under-specified. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the SAM-based Pseudo-label Refiner, a three-module pipeline: (1) the Threshold Filter Module zeroes out low-confidence predictions into an ignored label (255); (2) the Prompt Generation Module converts the filtered map into connected regions, computes maximum bounding rectangles expanded by Bn pixels, and samples positive/negative points with proximity and probability constraints to prompt SAM; (3) the Label Refinement Module takes SAM's masks, removes holes, and stitches them with the model's probability map using thresholds tc and to, updating or ignoring each pixel. This refiner is what converts SAM's class-agnostic, boundary-sensitive masks into class-aware pseudo-labels, and it is the component whose ablation would separate the paper's contribution from ordinary self-training.
What would settle it
Take the fully trained SAMST pipeline and run the refiner with SAM's masks replaced by the thresholded connected regions themselves (i.e., no SAM re-segmentation), keeping all thresholds identical. If the mIoU gain over the baseline nearly disappears, the gain is attributable to SAM's masks; if the gain persists, the filtering/stitching alone is responsible. Alternatively, measure the pixel accuracy of refined pseudo-labels against the ground truth on the held-out portion of unlabeled data: if the refinement does not raise accuracy relative to raw predictions, the reported gains cannot come from improved label quality.
Extended reading notes
Core claim
The paper's central claim is that the SAM-based Pseudo-label Refiner, composed of a Threshold Filter Module, a Prompt Generation Module, and a Label Refinement Module, turns noisy model predictions into substantially more accurate pseudo-labels, and that this refinement is the reason SAMST improves semi-supervised segmentation. The refiner first masks out pixels whose predicted class probability falls below class-specific thresholds, then finds connected regions for each surviving class, builds prompt boxes around them, places positive and negative points inside and outside the regions, and feeds these prompts to SAM to obtain object masks. The Label Refinement Module then assigns a pixel to a class only when the SAM mask's category agrees with the model's high-confidence prediction or when the competing class is very unlikely, otherwise marking the pixel as ignored. The paper reports that after one iteration this pipeline lifts mIoU from 67.00 to 70.80 on Potsdam at a 1/32 label ratio, surpassing LSST, ST++, and ClassHyPer.
Load-bearing premise
The method's gain rests on the assumption that SAM's boundary-preserving masks, when prompted by the noisy model's boxes and points and stitched back with the probability thresholds, improve pseudo-labels instead of injecting new errors.
Editorial extensions
If this is right
- SAM-based pseudo-label refinement yields a 5.67-point mIoU gain over the supervised baseline at a 1/32 label ratio on Potsdam.
- A single self-training iteration suffices to surpass the three compared semi-supervised methods (LSST, ST++, ClassHyPer) on the same setup.
- Per-class F1 rises for five of six classes, with the largest gain (17.68 points) on the clutter class; low vegetation decreases slightly.
- Because the refiner is prompt-based and class-agnostic, the same SAMST framework can be applied to other remote sensing datasets without retraining SAM, as long as the prompt and threshold hyperparameters are re-tuned.
Reading between the lines
- A controlled ablation that replaces SAM's masks with class-agnostic masks of identical coverage would reveal how much of the mIoU gain is due to SAM's boundary accuracy versus simple spatial smoothing; the paper does not report this ablation.
- The 255 ignore-label mechanism may act as a selective curriculum: by injecting a modest number of (or zero) pseudo-labels for difficult regions, the model avoids reinforcing its own errors. This could explain the large clutter-class gain and is testable by measuring training-set class distribution before and after refinement.
- The same refiner could be plugged into consistency-regularization methods, not just self-training, since it only touches pseudo-labels; testing SAMST's refiner inside an ST++-style pipeline would isolate whether the gains come from the refiner or the overall two-stage schedule.
- Dataset-specific thresholds (Bn, Pp, Pn, Tp, Pm, Tn, Nm, tc, to) suggest the method may transfer less cleanly to other resolutions or class taxonomies; a sensitivity analysis over these parameters would clarify how much tuning is needed.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes SAMST, a semi-supervised semantic segmentation method for remote sensing images that combines supervised self-training with a SAM-based pseudo-label refiner. The refiner consists of a Threshold Filter Module, a Prompt Generation Module that creates boxes and positive/negative points for SAM, and a Label Refinement Module that stitches SAM masks with model probabilities. A weighted composite loss balances labeled and pseudo-labeled training. The empirical claim is that after a single iteration on the Potsdam dataset with 1/32 labeled data, SAMST improves mIoU from 67.00 (baseline) to 70.80 and outperforms LSST, ST++, and ClassHyPer in mIoU and average F1.
Significance. If fully validated, the idea of using SAM to refine pseudo-labels in semi-supervised remote sensing segmentation would be a timely and useful contribution, and the proposed pipeline is clearly motivated. The method is not circular: pseudo-labels are generated on unlabeled data and the final evaluation is on a held-out test set. However, the current evidence is not sufficient to establish the contribution: the central table has a puzzling internal inconsistency, the refiner depends on several undisclosed hand-set thresholds, no ablations are provided, and comparison settings are not described. The paper is potentially interesting but requires substantial additional detail and experiments before its claims can be accepted.
major comments (5)
- [Table I, Section III.C] The reported baseline OA is 93.48, which is 9.65 to 12.91 points higher than the OA of every semi-supervised comparison method (LSST 80.57, ClassHyPer 83.62, ST++ 83.83), and SAMST's OA is 86.44, a 7.04-point drop from the baseline despite higher mIoU and higher per-class F1 for Building, Tree, and Surface. Since Building, Tree, and Surface dominate the Potsdam scene, this pattern is internally inconsistent under a matched evaluation protocol. The authors should explain the experimental setup that produces this pattern, report per-class recall and confusion matrices, and verify that all methods use the same test split, evaluation code, backbone, and training budget. Without this, the reported mIoU gain cannot be attributed to SAMST.
- [Section II.A] The core refiner depends on seven prompt-generation parameters (Bn, Pp, Pn, Tp, Pm, Tn, Nm) and two stitching thresholds (tc, to), but the paper only states that these are 'set through experimental outcomes' and gives neither their values nor the procedure used to choose them. The same omission applies to alpha in Eq. (1) and the class weights wl,c and wp,c in Eqs. (2)-(3). Without these details the method is not reproducible, and there is no way to assess whether the choices were made on a validation set or the test set. Please report the chosen values and the validation criterion used.
- [Section III.B] The text says that a checkpoint is saved every 1,000 iterations and that 'the best checkpoint' is used to generate pseudo-labels, but the selection criterion is not defined and no validation split is described. If the test set was used to select among the 80 saved checkpoints, the reported numbers are optimistically biased. Please specify the validation protocol and the metric used for checkpoint selection, and apply the same protocol to the baseline and comparison methods.
- [Section III.C and Table I] No ablation study is provided, so it is impossible to determine which component of the proposed framework (Threshold Filter Module, Prompt Generation Module, Label Refinement Module, or the composite loss) is responsible for the reported improvement. In addition, the paper claims an iterative framework but reports only a single full iteration. Please provide module-wise ablations and at least one multi-iteration experiment to support the central claim.
- [Section III.B] The implementation details are given only for SAMST; there is no description of how LSST, ST++, and ClassHyPer were configured for comparison. It is therefore unclear whether the comparison methods used the same backbone, the same labeled/unlabeled split, the same input resolution, the same optimizer settings, and the same training iterations. Without matched settings, the relative gains in Table I could be an artifact of weaker baselines. Please provide full training and evaluation details for all compared methods, and ideally repeat experiments over multiple seeds to report variance.
minor comments (4)
- [Fig. 2] The text in panel (d) labels the class as 'Imprevious surfaces'; this should be 'Impervious surfaces'.
- [Section III.A] The dataset split is ambiguous: 'we allocated 24 images for training and 14 for validation and testing' is followed by a statement that the testing set includes 2016 patches. Please clarify how the validation set and test set are separated, and whether the 2016 patches come from the 14 images or from a different partition.
- [Section II.A.3] The Label Refinement Module description says holes are removed from each mask to obtain mask m1, but the paper does not define what 'holes' means concretely or how the masks from all connected regions are concatenated. Please provide a precise algorithmic description or pseudo-code.
- [References] There is no dedicated related-work section; the introduction mentions consistency regularization and self-training but does not discuss the cited remote sensing semi-supervised methods in enough detail to position the novelty of SAMST. Consider adding a short related-work section.
Circularity Check
No significant circularity: SAMST's headline gains rest on held-out Potsdam test-set metrics, and no fitted parameter or refined pseudo-label is renamed as the predicted target.
full rationale
Walking the paper's claimed chain shows an empirical training/evaluation pipeline rather than a derivation that reduces to its inputs. An initial model is trained on 108 labeled Potsdam patches; pseudo-labels for the remaining 3348 unlabeled patches are generated and refined by the SAM-based Pseudo-label Refiner (Threshold Filter Module, Prompt Generation Module, Label Refinement Module); the model is retrained with the composite weighted loss; and performance is then measured on the separate test portion (2016 patches). Nothing in Equations (1)-(3) defines the reported mIoU, OA, or F1 in terms of the method's own pseudo-labels or thresholds, and the refined pseudo-labels are not the evaluation target. The threshold and prompt parameters are described as set through experimental outcomes and acknowledged as dataset-specific; this is ordinary hyperparameter selection, not a fitted input renamed as a prediction. The 'best checkpoint' choice and the internal inconsistency of Table I (baseline OA 93.48 versus all semi-supervised methods below 84) are experimental-matching and reporting-integrity concerns, not circularity, because they do not make SAMST's result true by construction. There is no load-bearing self-citation and no imported uniqueness theorem. The paper is self-contained against an external benchmark, so the honest finding is no significant circularity.
Assumptions & free parameters
free parameters (4)
- Threshold Filter per-class thresholds =
unspecified
- Prompt generation parameters (Bn, Pp, Pn, Tp, Pm, Tn, Nm) =
unspecified
- Label refinement thresholds (tc, to) =
unspecified
- Pseudo-loss weight alpha =
unspecified
assumptions (3)
- domain assumption SAM's zero-shot segmentation produces accurate object boundaries for remote sensing imagery when prompted with boxes and points.
- domain assumption Connected regions of high-confidence predictions correspond to semantically coherent objects suitable for prompting.
- domain assumption The threshold and prompt parameters are selected on a validation set and generalize to the test set.
Cite this review
Pith. "Pith review of SAMST: A Transformer framework based on SAM pseudo label filtering for remote sensing semi-supervised semantic segmentation." pith.science (2026). https://pith.science/paper/GETPIG2J
@misc{pith2026250711994,
author = {Pith},
title = {Pith review of: SAMST: A Transformer framework based on SAM pseudo label filtering for remote sensing semi-supervised semantic segmentation},
year = {2026},
howpublished = {\url{https://pith.science/paper/GETPIG2J}},
note = {Machine review of arXiv:2507.11994}
}
read the original abstract
Public remote sensing datasets often face limitations in universality due to resolution variability and inconsistent land cover category definitions. To harness the vast pool of unlabeled remote sensing data, we propose SAMST, a semi-supervised semantic segmentation method. SAMST leverages the strengths of the Segment Anything Model (SAM) in zero-shot generalization and boundary detection. SAMST iteratively refines pseudo-labels through two main components: supervised model self-training using both labeled and pseudo-labeled data, and a SAM-based Pseudo-label Refiner. The Pseudo-label Refiner comprises three modules: a Threshold Filter Module for preprocessing, a Prompt Generation Module for extracting connected regions and generating prompts for SAM, and a Label Refinement Module for final label stitching. By integrating the generalization power of large models with the training efficiency of small models, SAMST improves pseudo-label accuracy, thereby enhancing overall model performance. Experiments on the Potsdam dataset validate the effectiveness and feasibility of SAMST, demonstrating its potential to address the challenges posed by limited labeled data in remote sensing semantic segmentation.
Figures
Reference graph
Works this paper leans on
-
[19]
D. Yang, J. Ji, Y . Ma, T. Guo, H. Wang, X. Sun, and R. Ji, “Sam as the guide: mastering pseudo-label refinement in semi- supervised referring expression segmentation,” arXiv preprint arXiv:2406.01451, 2024
work page Pith review arXiv 2024
-
[1]
Weak-to-strong consistency learning for semisuper- vised image segmentation,
X. Lu, L. Jiao, L. Li, F. Liu, X. Liu, S. Yang, Z. Feng, and P. Chen, “Weak-to-strong consistency learning for semisuper- vised image segmentation,” IEEE Transactions on Geoscience and Remote Sensing , vol. 61, pp. 1–15, 2023
work page 2023
-
[2]
Y . He, J. Wang, C. Liao, B. Shan, and X. Zhou, “Classhyper: Classmix-based hybrid perturbations for deep semi-supervised semantic segmentation of remote sensing imagery,” Remote Sensing , vol. 14, no. 4, 2022. [Online]. Available: https://www.mdpi.com/2072-4292/14/4/879
work page 2022
-
[3]
Semisupervised semantic segmentation of remote sensing images with consistency self- training,
J. Li, B. Sun, S. Li, and X. Kang, “Semisupervised semantic segmentation of remote sensing images with consistency self- training,” IEEE Transactions on Geoscience and Remote Sens- ing, vol. 60, pp. 1–11, 2022
work page 2022
-
[4]
J. Kang, Z. Wang, R. Zhu, X. Sun, R. Fernandez-Beltran, and A. Plaza, “Picoco: Pixelwise contrast and consistency learning for semisupervised building footprint segmentation,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing , vol. 14, pp. 10 548–10 559, 2021
work page 2021
-
[5]
Virtual adversarial training: A regularization method for supervised and semi-supervised learning,
T. Miyato, S.-I. Maeda, M. Koyama, and S. Ishii, “Virtual adversarial training: A regularization method for supervised and semi-supervised learning,” IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 41, no. 8, pp. 1979– 1993, 2019
work page 1979
-
[6]
A. Tarvainen and H. Valpola, “Mean teachers are better role models: Weight-averaged consistency targets improve semi- supervised deep learning results,” Advances in neural informa- tion processing systems , vol. 30, 2017
work page 2017
-
[7]
Dynamic and adaptive self-training for semi-supervised remote sensing image semantic segmentation,
J. Jin, W. Lu, H. Yu, X. Rong, X. Sun, and Y . Wu, “Dynamic and adaptive self-training for semi-supervised remote sensing image semantic segmentation,” IEEE Transactions on Geoscience and Remote Sensing, vol. 62, pp. 1–14, 2024
work page 2024
Show all 24 references
-
[8]
Region-aware contrastive learning for semi-supervised semantic segmentation of remote sensing images,
Y . Luo, B. Sun, and S. Li, “Region-aware contrastive learning for semi-supervised semantic segmentation of remote sensing images,” in IGARSS 2024 - 2024 IEEE International Geoscience and Remote Sensing Symposium , 2024, pp. 8199–8203
2024
-
[9]
Semi- supervised remote sensing image semantic segmentation method based on deep learning,
L. Li, W. Zhang, X. Zhang, M. Emam, and W. Jing, “Semi- supervised remote sensing image semantic segmentation method based on deep learning,” Electronics, vol. 12, no. 2, p. 348, 2023
2023
-
[10]
Semi- supervised semantic segmentation of remote sensing images based on dual cross-entropy consistency,
M. Cui, K. Li, Y . Li, D. Kamuhanda, and C. J. Tessone, “Semi- supervised semantic segmentation of remote sensing images based on dual cross-entropy consistency,” Entropy, vol. 25, no. 4, p. 681, 2023
2023
-
[11]
St++: Make self- training work better for semi-supervised semantic segmenta- tion,
L. Yang, W. Zhuo, L. Qi, Y . Shi, and Y . Gao, “St++: Make self- training work better for semi-supervised semantic segmenta- tion,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2022, pp. 4268–4277
2022
-
[12]
Simple and efficient: A semisupervised learning framework for remote sensing image semantic segmentation,
X. Lu, L. Jiao, F. Liu, S. Yang, X. Liu, Z. Feng, L. Li, and P. Chen, “Simple and efficient: A semisupervised learning framework for remote sensing image semantic segmentation,” IEEE Transactions on Geoscience and Remote Sensing, vol. 60, pp. 1–16, 2022
2022
-
[13]
Semisupervised semantic segmentation of remote sensing images with consistency self- training,
J. Li, B. Sun, S. Li, and X. Kang, “Semisupervised semantic segmentation of remote sensing images with consistency self- training,” IEEE Transactions on Geoscience and Remote Sens- ing, vol. 60, pp. 1–11, 2021
2021
-
[14]
Semi-supervised hyperspectral image classification via spatial- regulated self-training,
Y . Wu, G. Mu, C. Qin, Q. Miao, W. Ma, and X. Zhang, “Semi-supervised hyperspectral image classification via spatial- regulated self-training,” Remote Sensing, vol. 12, no. 1, p. 159, 2020
2020
-
[15]
Semi-supervised semantic segmentation with cross-consistency training,
Y . Ouali, C. Hudelot, and M. Tami, “Semi-supervised semantic segmentation with cross-consistency training,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 12 674–12 684
2020
-
[16]
Semi-supervised semantic segmentation with high-and low-level consistency,
S. Mittal, M. Tatarchenko, and T. Brox, “Semi-supervised semantic segmentation with high-and low-level consistency,” IEEE transactions on pattern analysis and machine intelligence, vol. 43, no. 4, pp. 1369–1379, 2019
2019
-
[17]
Pseudo-label: The simple and efficient semi- supervised learning method for deep neural networks,
D.-H. Lee et al., “Pseudo-label: The simple and efficient semi- supervised learning method for deep neural networks,” in Work- shop on challenges in representation learning, ICML , vol. 3, no. 2. Atlanta, 2013, p. 896
2013
-
[18]
Segment anything,
A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y . Lo et al. , “Segment anything,” in Proceedings of the IEEE/CVF international conference on computer vision , 2023, pp. 4015– 4026
2023
-
[20]
The isprs benchmark on urban object classification and 3d building reconstruction,
F. Rottensteiner, G. Sohn, J. Jung, M. Gerke, C. Baillard, S. Benitez, and U. Breitkopf, “The isprs benchmark on urban object classification and 3d building reconstruction,” ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences; I-3 , vol. 1, no. ...
2012
-
[21]
What is a good evaluation measure for semantic segmentation?
G. Csurka, D. Larlus, F. Perronnin, and F. Meylan, “What is a good evaluation measure for semantic segmentation?.” in Bmvc, vol. 27, no. 2013. Bristol, 2013, pp. 10–5244
2013
-
[22]
Swin transformer: Hierarchical vision transformer using shifted windows,
Z. Liu, Y . Lin, Y . Cao, H. Hu, Y . Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: Hierarchical vision transformer using shifted windows,” in Proceedings of the IEEE/CVF in- ternational conference on computer vision , 2021, pp. 10 012– 10 022
2021
-
[23]
Unified perceptual parsing for scene understanding,
T. Xiao, Y . Liu, B. Zhou, Y . Jiang, and J. Sun, “Unified perceptual parsing for scene understanding,” in Proceedings of the European conference on computer vision (ECCV), 2018, pp. 418–434
2018
-
[24]
Context encoding for semantic segmentation,
H. Zhang, K. Dana, J. Shi, Z. Zhang, X. Wang, A. Tyagi, and A. Agrawal, “Context encoding for semantic segmentation,” in Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, 2018, pp. 7151–7160
2018
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.