MergeSAM: Unsupervised change detection of remote sensing images based on the Segment Anything Model

Chengxi Han; Lingzhi Lu; Meiqi Hu; Xiaoping Liu

REVIEW 4 major objections 5 minor 13 references

MergeSAM: Unsupervised change detection of remote sensing images based on the Segment Anything Model

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read MergeSAM claims that matching and splitting SAM's automatic masks across two dates yields label-free change detection that beats the SAM-based baseline AnyChange by about 7 F1 points on GZ_CD_data.

desk verdict Plausible SAM-based matching/splitting idea, but no ablation and a clear recall drop make the 7% F1 gain unattributable to the proposed strategies. read the letter →

arxiv 2507.22675 v2 pith:W5QQ7AQD submitted 2025-07-30 cs.CV

Meiqi Hu , Lingzhi Lu , Chengxi Han , Xiaoping Liu This is my paper

classification cs.CV

keywords changedetectionunsupervisedlearningSegmentAnythingModelMaskMatchingSplittingremotesensingbitemporalimageryhigh-resolutionsatelliteimages

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

MergeSAM aims to show that the Segment Anything Model, used without fine-tuning or labeled training data, can be turned into a strong unsupervised change detector for high-resolution remote sensing image pairs. The core move is to compare SAM's automatically generated object masks across the two dates: a MaskMatching step pairs masks of the same object by intersection-over-union, and a MaskSplitting step keeps the leftover intersecting pieces as candidate changed regions. The paper reports that on the GZ_CD_data high-resolution bitemporal dataset, MergeSAM raises the F1 score by about 7 percentage points over the SAM-based baseline AnyChange, mainly by improving precision while sacrificing relatively little recall. If this holds, label-free change detection can benefit directly from foundation-model segmentation, embedding the spatial structure of land cover into the detection process.

What carries the argument

The machinery is the pair of strategies MaskMatching and MaskSplitting operating on masks from the Segment Anything Model (SAM), a pretrained image segmentation model that produces object masks from a grid of points. MaskMatching aligns the same real-world object across dates by pairing masks with IoU at least 0.75, suppressing registration and segmentation-boundary noise. MaskSplitting removes those matched masks and treats the intersections of the leftover masks as candidate multitemporal regions, which is how splitting and merging—for instance a large barren land parcel partially turning into urban construction—enter the change map. These multitemporal masks are the analysis units; the average encoder feature inside each mask is compared with the corresponding feature in the other date using MSE, and Otsu thresholding converts the similarity scores into a final binary change map.

What would settle it

Run MergeSAM on GZ_CD_data after rotating or perturbing SAM's automatic point-prompt grid on one date; if the resulting change map shifts substantially, the detections depend on segmentation prompt geometry rather than ground change. A simpler check: apply the method to a pair of images of the same unchanged scene taken under different seasonal conditions—any nonzero change map concentrated along object boundaries would confirm that leftover mask intersections carry segmentation and radiometric noise into the change result.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that complex real-world changes—objects appearing, disappearing, splitting, merging, and partially transforming—can be captured by treating SAM's multitemporal masks as spatial analysis units. MaskMatching removes pairs of masks whose intersection-over-union exceeds 0.75, treating them as unchanged objects despite minor boundary or solar-angle differences. MaskSplitting then takes the remaining masks, finds their intersections, and forms a multitemporal mask set that represents changed parts; each mask's average feature embedding is compared across dates with mean squared error, and Otsu thresholding produces the final binary change map. This shifts change detection from pixel-level differencing to object-structure-level comparison, an assumption the paper validates on GZ_CD_data by reporting F1 of 31.65, 31.59, and 30.13 for ViT-B, ViT-L, and ViT-H backbones, outperforming CVA, CVA+SAM, and AnyChange on composite metrics while reducing false detections.

Load-bearing premise

The load-bearing assumption is that SAM's masks on the two dates are spatially consistent enough that an IoU of 0.75 reliably identifies the same object, and that the leftover intersecting mask pieces really are changed land cover rather than artifacts of seasonal, illumination, or boundary differences.

Editorial extensions

If this is right

With no labels and no fine-tuning, a SAM-based wrapper can beat the current SAM-based unsupervised baseline AnyChange by roughly 7 F1 points on GZ_CD_data.
Precision rises substantially while recall remains above 66, meaning the method trades some completeness for much cleaner change maps.
The F1 gain is stable across SAM backbones, with the best reported Kappa of 20.96 achieved with ViT-L.
Object-level rather than pixel-level comparison lets the method capture partial changes such as large parcels being split into changed and unchanged parts.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

We infer that the same MaskMatching/MaskSplitting recipe could be wrapped around any off-the-shelf segmentation model, making the paper's contribution a general postprocessing pattern rather than something tied to SAM's specific weights.
We infer that on datasets with stronger misregistration or smaller objects, the fixed IoU threshold of 0.75 and the Otsu thresholding would need per-scene tuning; the paper does not report such sensitivity.
A testable extension we see is filtering split masks by minimum area or adding a radiometric-normalization step before SAM, which should reduce boundary false positives and might raise the reported recall.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

What's actually new: the combination of IoU-based mask matching and intersection-based mask splitting as a wrapper on SAM's automatic masks for unsupervised change detection. That's a legitimate extension of AnyChange, not a new paradigm, but it's a sensible next step and the paper describes the pipeline clearly. It reports results across three SAM backbones and compares against CVA, CVA+SAM, and AnyChange, which is the right baseline set.

The soft spot is the one the stress-test flags, and it holds up on reading. MergeSAM changes several things at once relative to AnyChange: SAM proposal parameters, the matching step, the splitting step, and per-mask feature averaging with Otsu. There is no ablation that isolates MaskMatching or MaskSplitting, so the end-to-end F1 gain cannot be attributed to either strategy. Worse, Table 1 shows recall dropping sharply for MergeSAM (e.g., 79.98 to 66.18 for ViT-B), while the text claims 'less missed detections.' The F1 improvement is precision-driven, consistent with a more conservative detector, not with better recovery of split/merge changes. That internal contradiction matters.

Other issues are minor but real: one dataset, no error bars, no statistical test, no sensitivity analysis on the IoU threshold, and no code or data released. The MaskSplitting description is too vague to reproduce exactly. The supplied text also has garbled math notation in places. The absence of any co-registration or radiometric normalization step is a smaller concern given the dataset may already be aligned, but it should be stated.

The underlying idea is not bad. A control experiment replacing SAM masks with generic superpixels, or an ablation dropping MaskSplitting, would likely tell a clearer story. As it stands, the evidence does not support the causal claim.

Who is this for? Researchers working on foundation-model-based change detection, especially those looking for cheap post-hoc ways to use SAM. It is a workshop-level or short-letters contribution, not a definitive method. I would not cite it as-is, but I would send it to peer review because the idea deserves a fair test and the flaws are fixable with appropriate experiments.

Referee Report

4 major / 5 minor

Summary. The paper proposes MergeSAM, an unsupervised change detection method for high-resolution remote sensing images that wraps SAM automatic masks in two strategies: MaskMatching (associating objects across dates via IoU >= 0.75) and MaskSplitting (treating leftover intersected masks as changed parts), followed by per-mask average feature MSE and Otsu thresholding. It evaluates on GZ_CD_data, comparing with CVA, CVA+SAM, and AnyChange, and reports F1 improvements from 24.84 to 31.65 (ViT-B), 25.29 to 31.59 (ViT-L), and 25.83 to 30.13 (ViT-H), with precision increasing and recall substantially decreasing. The method uses no labels and involves no learning, so the reported numbers are not the result of fitting to ground truth.

Significance. If valid, MergeSAM would be a simple and inexpensive unsupervised baseline: it uses off-the-shelf SAM masks and no task-specific training, and it outperforms the SAM-based AnyChange on F1, OA, and Kappa on the tested dataset. The manuscript is honest in describing the method as unsupervised and does not fit any parameter to ground-truth labels; the fixed threshold tau_iou=0.75 and SAM proposal settings are free hyperparameters, but they are not learned from labels, so the circularity concern is low. However, the contribution is currently evidenced by a single dataset, no ablations, and no statistical analysis, so the significance should be regarded as promising rather than established.

major comments (4)

[Section 3.2, Table 1] The claim that MergeSAM produces 'less missed detections' is contradicted by the reported recall: MergeSAM drops recall from 79.98 to 66.18 (ViT-B), 77.76 to 63.64 (ViT-L), and 77.02 to 57.81 (ViT-H) relative to AnyChange. The F1 gain is entirely precision-driven (e.g., precision rises from 14.70 to 20.80 for ViT-B), which is consistent with a more conservative detector rather than better detection of split/merge changes. Please revise the qualitative claim and discuss the precision/recall trade-off.
[Section 2.2, Table 1] No ablation isolates the contributions of MaskMatching and MaskSplitting. Since MergeSAM differs from AnyChange in several ways simultaneously (SAM mask proposal parameters, IoU matching, splitting, per-mask feature averaging with Otsu thresholding), the reported end-to-end F1 gain cannot be attributed to the two named strategies. Please add ablations that remove or replace each strategy (e.g., no MaskSplitting, no MaskMatching, SAM masks replaced by superpixels) and report F1/precision/recall for each configuration.
[Section 3.1] The evaluation is limited to one dataset (GZ_CD_data) with no error bars, no per-image variance, and no statistical test, so the 7% F1 improvement over AnyChange could reflect dataset-specific effects. Please report per-image results with standard deviations or a paired significance test, and ideally validate on at least one additional VHR change detection dataset.
[Section 2.2] The matching step assumes that SAM masks from the two dates are spatially aligned well enough for IoU at a fixed threshold of 0.75 to correspond to the same real-world object. The paper does not describe any co-registration or radiometric normalization step, and GZ_CD_data contains seasonal variations (Section 3.1). Please state the registration status of the input pairs and test the sensitivity of results to tau_iou and to the SAM proposal thresholds, since these free parameters directly control the change map.

minor comments (5)

[Section 2.2] The notation contains garbled or mismatched symbols: e.g., 'ԧ φ', 'Ԃ ֏ φ', and the feature embeddings for the two dates are both written as 'Ԕ φ' (and 'Ԕ φ'); presumably they should be different variables. Please clean up the formula notation.
[Introduction and Conclusion] There are typos: 'deatection' in Section 1 and 'comprehsenive' in the Conclusion.
[Section 3.2] The statement 'exhibits a 7% improvement in the F1 score' should specify whether this is percentage points or relative improvement; Table 1 shows an increase from 24.84 to 31.65 (about 6.8 percentage points, or about 27% relative) for ViT-B.
[Section 3.1, Table 1] The CVA row in Table 1 has no backbone entry; please mark it as '—' or 'N/A' to avoid implying that it uses a SAM backbone.
[References] Reference [14] is titled 'Segment Any Change' while the text calls it 'AnyChange'; please align the naming and check the reference formatting (e.g., [8] and [17]).

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: MergeSAM is an unsupervised heuristic pipeline whose reported gains are empirical comparisons against an external dataset, not quantities defined by its own inputs.

full rationale

MergeSAM's derivation chain is not circular. The method combines off-the-shelf SAM masks with IoU-based MaskMatching and MaskSplitting, then applies per-mask feature averaging, MSE, and Otsu thresholding. Every ingredient is either an external model (SAM), a hand-set threshold (IoU = 0.75, NMS = 0.7, etc.), or a standard statistical operation (Otsu). No parameter is fitted to the GZ_CD_data reference labels, and no equation defines the reported F1, precision, or recall in terms of the method's own thresholds. The final scores are comparisons against external ground truth, so they are not self-confirming. The only substantive weaknesses are experimental, not circular: the paper changes SAM proposal settings, MaskMatching, MaskSplitting, and postprocessing simultaneously relative to AnyChange, and it provides no ablation isolating the two named strategies. Also, the text's claim of 'less missed detections' conflicts with the recall drop in Table 1 (e.g., 79.98 to 66.18 for ViT-B). These are attribution and reporting problems, not cases where a claimed prediction reduces by construction to an input. There are no load-bearing self-citations: AnyChange [14] and SAM [9] are external works with no author overlap, and the GZ_CD_data reference [17] is an independent dataset. Accordingly, the circularity score is 0.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The central claim rests on SAM masks being meaningful in VHR satellite scenes, temporal images being aligned enough for IoU matching, SAM encoder features being stable across dates, and hand-set thresholds being adequate. The paper provides no code, error bars, ablation, or second dataset to validate these assumptions.

free parameters (6)

IoU matching threshold tau_iou = 0.75
Fixed value set in Section 2.2 and repeated in Parameter Setting; no sensitivity analysis or ablation is reported.
SAM point per side = 64
Proposal generation parameter in Section 3.1; hand-set without justification or sensitivity study.
SAM NMS threshold = 0.7
Proposal generation parameter in Section 3.1; hand-set without sensitivity analysis.
SAM predicted IoU threshold = 0.5
Proposal generation parameter in Section 3.1; hand-set without sensitivity analysis.
SAM stability score threshold = 0.8
Proposal generation parameter in Section 3.1; hand-set without sensitivity analysis.
Resize longest side = 1600
Image scaling parameter in Section 3.1; no ablation of its effect on mask quality or change detection.

assumptions (4)

domain assumption SAM's zero-shot segmentation masks are meaningful on high-resolution remote sensing imagery.
Introduced in Section 2.1 and relied on throughout; SAM was trained mainly on natural images, and transfer to VHR satellite scenes is assumed rather than demonstrated.
domain assumption The two temporal images are sufficiently co-registered and radiometrically consistent that IoU between segmentation masks is a valid matching signal.
Section 2.2 assumes spatial alignment for IoU matching, but the paper does not describe any registration or radiometric normalization step.
domain assumption Average SAM-encoder features inside a mask are stable across dates except for true change, so MSE is a valid change measure.
Section 2.2 uses mean feature embeddings and MSE similarity without cross-date feature normalization or calibration.
domain assumption Otsu thresholding on the MSE map provides a valid decision boundary between changed and unchanged pixels.
Section 2.2 applies the Otsu algorithm; the paper does not check whether the MSE distribution in this setting is bimodal enough for Otsu to be appropriate.

how reviews work

0 comments

Cite this review

Pith. "Pith review of MergeSAM: Unsupervised change detection of remote sensing images based on the Segment Anything Model." pith.science (2026). https://pith.science/paper/W5QQ7AQD

@misc{pith2026250722675,
  author       = {Pith},
  title        = {Pith review of: MergeSAM: Unsupervised change detection of remote sensing images based on the Segment Anything Model},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/W5QQ7AQD}},
  note         = {Machine review of arXiv:2507.22675}
}

read the original abstract

Recently, large foundation models trained on vast datasets have demonstrated exceptional capabilities in feature extraction and general feature representation. The ongoing advancements in deep learning-driven large models have shown great promise in accelerating unsupervised change detection methods, thereby enhancing the practical applicability of change detection technologies. Building on this progress, this paper introduces MergeSAM, an innovative unsupervised change detection method for high-resolution remote sensing imagery, based on the Segment Anything Model (SAM). Two novel strategies, MaskMatching and MaskSplitting, are designed to address real-world complexities such as object splitting, merging, and other intricate changes. The proposed method fully leverages SAM's object segmentation capabilities to construct multitemporal masks that capture complex changes, embedding the spatial structure of land cover into the change detection process.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

13 extracted references · 6 canonical work pages

[2]

DeepGlobe 2018: A Challenge to Parse the Earth throug h Satellite Images,

I. Demir et al. , “DeepGlobe 2018: A Challenge to Parse the Earth throug h Satellite Images,” May 17, 2018, arXiv : arXiv:1805.06561. doi: 10.48550/arXiv.1805.06561

arXiv doi:10.48550/arxiv.1805.06561 2018
[3]

Deep learning for change detection in remote sensing: a review,

T. Bai et al. , “Deep learning for change detection in remote sensing: a review,” Geo - spatial Information Science , vol. 0, no. 0, pp. 1 – 27, Jul. 2022, doi: 10.10 80/10095020.2022.2085633

arXiv 2022
[4]

Change Detection From Very - High - Spatial - Resolution Optical Remote Sensing Images: Methods, applications, and future directions,

D. Wen et al. , “Change Detection From Very - High - Spatial - Resolution Optical Remote Sensing Images: Methods, applications, and future directions,” IEEE Geoscience and Remote Sensing Magazine , vol. 9, no. 4, pp. 68 – 101, Dec. 2021

work page 2021
[6]

Deep learning for remote sensing data: a technical tutorial on the state of the art,

L. Zhang, L. Zhang, and B. Du, “Deep learning for remote sensing data: a technical tutorial on the state of the art,” IEEE Geosci. Remote Sens. Mag. , vol. 4, no. 2, pp. 22 – 40, Jun. 2016, doi: 10.1109/MGRS.2016.2540798

work page arXiv 2016
[7]

Artificial intelligence for geoscience: Progress, challenges, and perspectives,

T. Zhao et al. , “Artificial intelligence for geoscience: Progress, challenges, and perspectives,” The Innovation , vol. 5, no. 5, p. 100691, 2 024, doi: 10.1016/j.xinn.2024.100691

arXiv 2024
[8]

HyperSIGMA: Hyperspectral Intelligence Comprehension Foundation Model,

D. Wang et al. , “HyperSIGMA: Hyperspectral Intelligence Comprehension Foundation Model,” arXiv preprint arXiv:2406.11519 ,

arXiv
[9]

Segment anything,

A. Kirillov et al. , “Segment anything,” in Proceedings of the IEEE/CVF Internation al Conference on Computer Vision , 2023, pp. 4015 –

work page 2023
[11]

MeSAM: Multiscale Enhanced Segment Anything Model for Optical Remote Sensing Images,

X. Zhou et al. , “MeSAM: Multiscale Enhanced Segment Anything Model for Optical Remote Sensing Images,” IEEE Transactions on Geoscience and Remote Sensing , pp. 1 – 1, 2024, doi: 10.1109/TGRS.202 4.3398038

work page doi:10.1109/tgrs.202 2024

Show all 13 references

[12]

RSPrompter: Learning to Prompt for Remote Sensing Instance Segmentation Based on Visual Foundation Model,

K. Chen et al. , “RSPrompter: Learning to Prompt for Remote Sensing Instance Segmentation Based on Visual Foundation Model,” IEEE Trans. Geosci. Remote Sensing , vol. 62, pp. 1 – 17, 2024, doi: 10.1109/TGRS.2024.3356074

2024
[13]

Adapting Segment Anything Model for Change Detection in VHR Remote Sensing Images,

L. Ding, K. Zhu, D. Peng, H. Tang, K. Yang, and L. Bruzzone, “Adapting Segment Anything Model for Change Detection in VHR Remote Sensing Images,” IEEE Transactions on Geoscience and Remote Sensing , vol. 62, pp. 1 – 11, 2024, doi: 10.1109/TGRS.2024.3368168. [ 14] Z. Zheng, Y. ...

2024
[14]

Available: http://arxiv.org/abs/2402.01188

[Online]. Available: http://arxiv.org/abs/2402.01188

arXiv
[15]

A Comprehensive Survey on Segment Anything Mod el for Vision and Beyond,

C. Zhang et al. , “A Comprehensive Survey on Segment Anything Mod el for Vision and Beyond,” May 19, 2023, arXiv : arXiv:2305.08196. doi: 10.48550/arXiv.2305.08196

2023 doi arXiv
[2024]

Available: http://arxiv.org/abs/2401.09019

[Online]. Available: http://arxiv.org/abs/2401.09019

arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [2]

DeepGlobe 2018: A Challenge to Parse the Earth throug h Satellite Images,

I. Demir et al. , “DeepGlobe 2018: A Challenge to Parse the Earth throug h Satellite Images,” May 17, 2018, arXiv : arXiv:1805.06561. doi: 10.48550/arXiv.1805.06561

arXiv doi:10.48550/arxiv.1805.06561 2018

[2] [3]

Deep learning for change detection in remote sensing: a review,

T. Bai et al. , “Deep learning for change detection in remote sensing: a review,” Geo - spatial Information Science , vol. 0, no. 0, pp. 1 – 27, Jul. 2022, doi: 10.10 80/10095020.2022.2085633

arXiv 2022

[3] [4]

Change Detection From Very - High - Spatial - Resolution Optical Remote Sensing Images: Methods, applications, and future directions,

D. Wen et al. , “Change Detection From Very - High - Spatial - Resolution Optical Remote Sensing Images: Methods, applications, and future directions,” IEEE Geoscience and Remote Sensing Magazine , vol. 9, no. 4, pp. 68 – 101, Dec. 2021

work page 2021

[4] [6]

Deep learning for remote sensing data: a technical tutorial on the state of the art,

L. Zhang, L. Zhang, and B. Du, “Deep learning for remote sensing data: a technical tutorial on the state of the art,” IEEE Geosci. Remote Sens. Mag. , vol. 4, no. 2, pp. 22 – 40, Jun. 2016, doi: 10.1109/MGRS.2016.2540798

work page arXiv 2016

[5] [7]

Artificial intelligence for geoscience: Progress, challenges, and perspectives,

T. Zhao et al. , “Artificial intelligence for geoscience: Progress, challenges, and perspectives,” The Innovation , vol. 5, no. 5, p. 100691, 2 024, doi: 10.1016/j.xinn.2024.100691

arXiv 2024

[6] [8]

HyperSIGMA: Hyperspectral Intelligence Comprehension Foundation Model,

D. Wang et al. , “HyperSIGMA: Hyperspectral Intelligence Comprehension Foundation Model,” arXiv preprint arXiv:2406.11519 ,

arXiv

[7] [9]

Segment anything,

A. Kirillov et al. , “Segment anything,” in Proceedings of the IEEE/CVF Internation al Conference on Computer Vision , 2023, pp. 4015 –

work page 2023

[8] [11]

MeSAM: Multiscale Enhanced Segment Anything Model for Optical Remote Sensing Images,

X. Zhou et al. , “MeSAM: Multiscale Enhanced Segment Anything Model for Optical Remote Sensing Images,” IEEE Transactions on Geoscience and Remote Sensing , pp. 1 – 1, 2024, doi: 10.1109/TGRS.202 4.3398038

work page doi:10.1109/tgrs.202 2024