Enhancing SAR Object Detection with Self-Supervised Pre-training on Masked Auto-Encoders

Feng Xu; Xinyang Pu

REVIEW 3 major objections 7 minor 8 references

Enhancing SAR Object Detection with Self-Supervised Pre-training on Masked Auto-Encoders

T0 review · 3 major / 7 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Self-supervised pre-training on unlabeled SAR images gives object detection a better starting point than ImageNet pre-training.

desk verdict A clean empirical recipe for SAR MAE pre-training with a modest, consistent gain that needs multiple seeds and a cleaner pre-training/fine-tuning split before the headline number is trusted. read the letter →

arxiv 2501.11249 v1 pith:2PR7QQ5X submitted 2025-01-20 cs.CV

Xinyang Pu , Feng Xu This is my paper · ORCID

classification cs.CV

keywords self-supervisedlearningmaskedauto-encodersimagemodelingSARobjectdetectionSyntheticApertureRadarvisiontransformerpre-trainingSARDet-100k

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a Masked Auto-Encoder pre-trained on unlabeled synthetic aperture radar images provides a better backbone initialization for SAR object detection than the common practice of loading weights pre-trained on ImageNet. The authors argue that the visual gap between natural photographs and SAR imagery limits fine-tuning performance, and that same-domain self-supervised pre-training narrows this gap without requiring any extra annotations. On the SARDet-100k benchmark, the proposed pre-training plus fine-tuning routine raises mean average precision from 53.6 to 54.9 over the ImageNet-initialized baseline and also beats training from scratch. A sympathetic reader would care because this is a cheap, annotation-free way to adapt a generic vision backbone to a domain with scarce labeled data.

What carries the argument

The load-bearing mechanism is Masked Auto-Encoding with an asymmetric Vision Transformer: the input image is cut into non-overlapping patches, 75% are randomly masked, only visible patches go through the ViT encoder, and a lightweight decoder reconstructs masked patch pixel values under mean squared error loss. The high masking ratio forces the encoder to learn semantic structure rather than local continuity. After pre-training, the decoder is discarded and the encoder becomes the plain ViT-Base backbone of ViTDet, which adds a simple Feature Pyramid Network and a Faster R-CNN region proposal and RoI head for multi-scale object detection. The same encoder weights are then fine-tuned with supervision on SARDet-100k.

What would settle it

Repeat the ViTDet fine-tuning protocol on SARDet-100k multiple times with different random seeds for three initializations: ImageNet pre-trained, SARDet-100k MAE pre-trained, and from scratch; if the overlap in mAP distributions across the first two is comparable to or larger than the reported 1.3-point gap, the central claim would not survive.

Watch

Extended reading notes

Core claim

The central claim is that masking most of an SAR image and training a Vision Transformer encoder to reconstruct the missing patches learns latent representations that transfer better to SAR object detection than representations learned on natural images. Concretely, the MAE encoder pre-trained on the unlabeled training split of SARDet-100k, then inserted as the ViTDet backbone with a Faster R-CNN head and simple FPN, reaches 54.9 mAP on the SARDet-100k validation set, versus 53.6 mAP when the same detector is initialized with ImageNet pre-trained ViT-Base weights and 47.2 mAP when trained from scratch. The paper reports that the SAR pre-trained model improves every reported metric: AP50, AP75, and mAP for small, medium, and large objects. The authors interpret this as evidence that pre-training on large-scale data with a distribution consistent with the downstream task is more useful than generic natural-image pre-training for fine-grained SAR detection.

Load-bearing premise

The paper treats the 1.3 mAP advantage of SAR pre-training over ImageNet pre-training as a real effect, but each setting is fine-tuned only once, with no multiple seeds, confidence intervals, or significance tests reported.

Editorial extensions

If this is right

If the central claim holds, SAR object detectors no longer need ImageNet backbone weights as their default initialization; same-domain self-supervised pre-training is a competitive alternative.
The pre-training stage consumes only unlabeled SAR imagery, so the pipeline can be scaled up as more unlabeled SAR data becomes available without new annotation cost.
The reported gains across AP50, AP75, small, medium, and large object mAP imply the benefit is not concentrated in one detection regime.
The gap over random initialization (54.9 versus 47.2 mAP) reinforces the paper's claim that pre-training matters for SAR detection, not just the choice of pre-training data.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The same MAE pre-train then fine-tune recipe is a natural candidate for other SAR tasks with little labeled data, such as semantic segmentation, change detection, or ship classification, though the paper only evaluates object detection.
Mask ratio, reconstruction loss, and fine-tuning schedule are standard MAE choices; varying them on SAR data could reveal whether SAR-specific image statistics call for different settings than natural images.
A direct testable extension is to keep the detector fixed and pre-train the same ViT architecture on different unlabeled SAR corpora to see whether detection mAP tracks the amount or diversity of pre-training data.
Comparing MAE against other self-supervised objectives such as contrastive or joint-embedding methods on the same SAR data would position the reconstruction objective's contribution relative to the domain shift itself.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The headline: the paper applies standard MAE pre-training to SAR images and reports a 1.3 mAP gain (53.6 to 54.9) over ImageNet initialization on SARDet-100k detection. The method is unchanged from He et al.; the novelty is the domain and the benchmark. What it does well: same detector, three initializations, all standard COCO metrics reported, and the improvement holds across every row of Table I. The writing is clear and the dataset is public. That is a legitimate, if modest, contribution.

The main weakness is statistical. The paper reports a single fine-tuning run per condition with no error bars, no multiple seeds, no significance test. A 1.3 mAP gap on a dataset of this size could easily be within run-to-run variance for ViTDet fine-tuning, which is known to be seed-sensitive. This is not a manufactured concern; it is the load-bearing claim of the paper. I would want at least 3–5 seeds and either standard deviations or a simple paired test.

The second soft spot is that the SAR MAE pre-training uses the same SARDet-100k training images that later appear in fine-tuning (Sections III.B and III.C). That conflates domain-specific representation learning with extra exposure to the exact target images. It does not invalidate the paper—MAE is unsupervised, and the comparison with ImageNet pre-training still tells you something—but it weakens the abstract's broader claim about a general SAR pre-trained backbone. The conclusion should be conditioned on this overlap.

Minor points: no code is released, and the citation pattern is fine (SAR-JEPA, FG-MAE, SARDet-100k all cited appropriately). There is no derivation to be circular about; the paper is an empirical comparison.

Who this is for: SAR/remote-sensing detection researchers looking for a practical recipe for domain-specific pre-training. It is not a methodological advance, but it is a useful data point. I would send it to peer review because the question is valid, the experiments are clean in principle, and the benchmark is public. The referee should ask for robustness checks before acceptance.

Recommendation: engage with the work and send it to review, but the authors need to add multiple seeds (and ideally a pre-training on a disjoint SAR set or an ablation that separates the pre-training data) before the headline gain is convincing.

Referee Report

3 major / 7 minor

Summary. The paper proposes using Masked Auto-Encoders (MAE) for self-supervised pre-training on unlabeled SAR images from SARDet-100k, then fine-tuning a ViTDet object detector initialized with the pre-trained ViT-Base encoder. The authors compare three initializations: training from scratch, ImageNet MAE pre-training, and SARDet-100k MAE pre-training. They report that the SAR MAE pre-training improves detection performance on the SARDet-100k validation set from 53.6 to 54.9 mAP over the ImageNet MAE baseline, with consistent gains across AP50, AP75, and small/medium/large object mAP. The central claim is that domain-specific SAR pre-training provides a better backbone initialization for SAR object detection than natural-image pre-training.

Significance. If the reported improvement is robust, the result is a useful empirical contribution: it demonstrates that domain-matched masked image modeling can outperform ImageNet pre-training for SAR object detection, and it provides a pre-trained SAR backbone that practitioners could adopt. The experimental protocol is clean in principle because the detector, training schedule, and evaluation metrics are held fixed across the three initializations. However, the magnitude of the claimed gain is modest (1.3 mAP), the paper reports a single fine-tuning run per condition with no error bars, and the SAR pre-training uses the same images that later appear in the fine-tuning split. These issues currently limit the strength of the empirical support. The method itself is standard MAE with no architectural novelty, and no code or trained weights are released, so the contribution rests entirely on the reliability of the reported comparison.

major comments (3)

[Section III.C.3, Table I] The central claim of a 1.3 mAP improvement (53.6 to 54.9) over ImageNet pre-training rests on a single fine-tuning run per initialization. No multiple seeds, confidence intervals, or significance tests are reported. Because ViTDet fine-tuning is sensitive to random seed, data order, and augmentation, a 1.3 mAP gap may be within run-to-run variance. Please report at least three independent fine-tuning runs per condition (mean ± std) and, if possible, a paired significance test over the validation images. Without this, the conclusion that the proposed method exceeds ImageNet pre-training is not statistically supported.
[Section III.B and III.C] The SAR MAE pre-training uses all SARDet-100k training images (Section III.B.1), and the same images are later used for supervised fine-tuning (Section III.C.1). The ImageNet pre-training did not see these images. Consequently, the comparison conflates domain-matched representation learning with additional exposure to the exact target training images. To support the paper's stated conclusion that converting the pre-training domain from natural scenes to SAR images is beneficial, the authors should either pre-train on unlabeled SAR images disjoint from the fine-tuning split, or hold out a subset of labeled training images from pre-training and fine-tune only on that subset, while reporting the same comparison with ImageNet initialization.
[Section III.C.3] The paper reports only validation-set metrics, even though SARDet-100k provides a test split (Section III.A). Validation mAP is used to make the headline claim, but the abstract and conclusion speak of improved generalization. Reporting test-set mAP, or a cross-dataset evaluation, would directly support the generalization claim and reduce concerns about validation-set overfitting or selection.

minor comments (7)

[Abstract] There are grammatical errors such as 'an self-supervised learning' and 'an self-supervised method'; the manuscript would benefit from a careful copyedit.
[Section II.A] The section title 'Mask Auto-encoder' is inconsistent with the standard term 'Masked Auto-encoders' used elsewhere in the paper; please unify the terminology.
[Figure 1] Figure 1 contains garbled text ('downidentityupup') that appears to be a typesetting artifact; the figure should be regenerated or the caption corrected.
[References] The references are formatted inconsistently (e.g., 'K.;He, X.;Chen, S.;Xie et al.'); please convert them to standard IEEE style.
[Section III.C.3] The paper does not compare against other SAR-domain self-supervised methods such as SAR-JEPA [2] or FG-MAE [3] in the object detection setting; adding at least one such baseline would better contextualize the contribution.
[Section III.B.2] The learning rate is described as 'scaled as 1.5 × 10−4 × 512/256'; please specify the base batch size and clarify that the scaling factor is batch_size/256.
[Section IV] The conclusion states that the method leads to 'significant improvements', but no statistical significance testing was performed; please rephrase to 'consistent improvements' unless significance tests are added.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper reports an empirical pre-training comparison whose objective is independent of the downstream detection metric.

full rationale

The paper's central claim is an empirical comparison, not a derivation: MAE pre-training on SARDet-100k images is followed by supervised fine-tuning, and Table I reports mAP improvements over ImageNet pre-training and training from scratch. The reconstruction objective in Eq. (1) is standard mean squared error on masked patches; it does not incorporate the downstream detection metric, and no parameter is fitted to the mAP values and then renamed as a prediction. The MAE encoder is transferred to ViTDet, the decoder is discarded, and fine-tuning uses the standard Faster R-CNN/ViTDet loss. Citations [1], [5], [6], and [7] are external prior methodological work, while [4] is a benchmark dataset rather than an unverified premise that forces the result. The concern that only one fine-tuning run is reported and that the 1.3 mAP gap may be within run-to-run stochasticity is a statistical robustness issue, not circularity. There is no self-definitional reduction, no uniqueness theorem imported from the authors' prior work, and no ansatz smuggled in via self-citation. Accordingly, no circular steps are identified.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The paper is an empirical application of existing methods, so there are no newly invented entities. The central claim depends on standard MAE and ViTDet hyperparameters, reliable benchmark data, a fair ImageNet control, and the assumption that a single-run 1.3 mAP difference is not noise. The ledger captures these dependencies.

free parameters (3)

Mask ratio r = 0.75
Chosen from the standard MAE recipe [1], not tuned on SAR data; the pre-training setup depends on it.
Pre-training duration = 400 epochs
Follows the MAE default training length; not varied or optimized for SARDet-100k.
Detection fine-tuning schedule = 12 epochs, initial LR 1e-4
Standard ViTDet schedule from [6]; held constant for all baselines, but the central comparison rests on this recipe.

assumptions (4)

domain assumption MAE pre-training representations transfer to object detection fine-tuning
The method's success relies on the transferability of MAE features, established on ImageNet/COCO in [1] and [6], and assumed here for SAR images.
domain assumption SARDet-100k splits and labels are reliable and split leakage-free
The evaluation numbers rest on the correctness of the benchmark [4]; the paper does not audit labels or splits.
domain assumption The ImageNet MAE baseline is a strong, fairly configured control
The paper does not specify which ImageNet MAE checkpoint is used or whether the same fine-tuning hyperparameters were applied; this is required for a clean comparison.
domain assumption Single-run mAP differences are treated as meaningful
No seeds or confidence intervals are reported; the 1.3 mAP gap is small relative to typical detection training variance, so the conclusion assumes stability.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Enhancing SAR Object Detection with Self-Supervised Pre-training on Masked Auto-Encoders." pith.science (2026). https://pith.science/paper/2PR7QQ5X

@misc{pith2026250111249,
  author       = {Pith},
  title        = {Pith review of: Enhancing SAR Object Detection with Self-Supervised Pre-training on Masked Auto-Encoders},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2PR7QQ5X}},
  note         = {Machine review of arXiv:2501.11249}
}

read the original abstract

Supervised fine-tuning methods (SFT) perform great efficiency on artificial intelligence interpretation in SAR images, leveraging the powerful representation knowledge from pre-training models. Due to the lack of domain-specific pre-trained backbones in SAR images, the traditional strategies are loading the foundation pre-train models of natural scenes such as ImageNet, whose characteristics of images are extremely different from SAR images. This may hinder the model performance on downstream tasks when adopting SFT on small-scale annotated SAR data. In this paper, an self-supervised learning (SSL) method of masked image modeling based on Masked Auto-Encoders (MAE) is proposed to learn feature representations of SAR images during the pre-training process and benefit the object detection task in SAR images of SFT. The evaluation experiments on the large-scale SAR object detection benchmark named SARDet-100k verify that the proposed method captures proper latent representations of SAR images and improves the model generalization in downstream tasks by converting the pre-trained domain from natural scenes to SAR images through SSL. The proposed method achieves an improvement of 1.3 mAP on the SARDet-100k benchmark compared to only the SFT strategies.

Figures

Figures reproduced from arXiv: 2501.11249 by the authors.

**Figure 2.** Reconstruction results of the MAE model on the validation set. Row [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** Detection results of the ViTDet pre-trained on SARDet-100k. Row [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

8 extracted references · 7 canonical work pages

[1]

Masked autoencoders are scalable vision learners,

K.;He, X.;Chen, S.;Xie et al., “Masked autoencoders are scalable vision learners,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , June 2022, pp. 16 000–16 009

work page 2022
[2]

Predicting gradient is better: Exploring self-supervised learning for sar atr with a joint- embedding predictive architecture,

W.;Li, W.;Yang, T.;Liu et al. , “Predicting gradient is better: Exploring self-supervised learning for sar atr with a joint- embedding predictive architecture,” ISPRS Journal of Pho- togrammetry and Remote Sensing , vol. 218, pp. 326–338, 2024

work page 2024
[3]

Feature guided masked autoencoder for self-supervised learning in re- mote sensing,

Y .;Wang, H. H.;Hern ´andez, C. M.;Albrecht et al. , “Feature guided masked autoencoder for self-supervised learning in re- mote sensing,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing , vol. 18, pp. 321–336, 2025

work page 2025
[4]

Sardet-100k: Towards open-source benchmark and toolkit for large-scale sar object detection,

Y .;Li, X.;Li, W.;Li et al. , “Sardet-100k: Towards open-source benchmark and toolkit for large-scale sar object detection,” arXiv preprint arXiv:2403.06534, 2024

arXiv 2024
[5]

An image is worth 16x16 words: Transformers for image recognition at scale,

A.;Dosovitskiy, L.;Beyer, A.;Kolesnikov et al. , “An image is worth 16x16 words: Transformers for image recognition at scale,” in International Conference on Learning Representations , 2021

work page 2021
[6]

Exploring plain vision trans- former backbones for object detection,

Y .;Li, H.;Mao, R.;Girshick et al., “Exploring plain vision trans- former backbones for object detection,” in European conference on computer vision . Springer, 2022, pp. 280–296

work page 2022
[7]

Faster r-cnn: Towards real- time object detection with region proposal networks,

S.;Ren, K.;He, R.;Girshick et al. , “Faster r-cnn: Towards real- time object detection with region proposal networks,” IEEE transactions on pattern analysis and machine intelligence , vol. 39, no. 6, pp. 1137–1149, 2016

work page 2016
[8]

Microsoft coco: Com- mon objects in context,

T.-Y .;Lin, M.;Maire, S.;Belongie et al. , “Microsoft coco: Com- mon objects in context,” in Computer Vision – ECCV 2014 , D.;Fleet, T.;Pajdla, B.;Schiele et al. , Eds. Cham: Springer International Publishing, 2014, pp. 740–755

work page 2014

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [1]

Masked autoencoders are scalable vision learners,

K.;He, X.;Chen, S.;Xie et al., “Masked autoencoders are scalable vision learners,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , June 2022, pp. 16 000–16 009

work page 2022

[2] [2]

Predicting gradient is better: Exploring self-supervised learning for sar atr with a joint- embedding predictive architecture,

W.;Li, W.;Yang, T.;Liu et al. , “Predicting gradient is better: Exploring self-supervised learning for sar atr with a joint- embedding predictive architecture,” ISPRS Journal of Pho- togrammetry and Remote Sensing , vol. 218, pp. 326–338, 2024

work page 2024

[3] [3]

Feature guided masked autoencoder for self-supervised learning in re- mote sensing,

Y .;Wang, H. H.;Hern ´andez, C. M.;Albrecht et al. , “Feature guided masked autoencoder for self-supervised learning in re- mote sensing,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing , vol. 18, pp. 321–336, 2025

work page 2025

[4] [4]

Sardet-100k: Towards open-source benchmark and toolkit for large-scale sar object detection,

Y .;Li, X.;Li, W.;Li et al. , “Sardet-100k: Towards open-source benchmark and toolkit for large-scale sar object detection,” arXiv preprint arXiv:2403.06534, 2024

arXiv 2024

[5] [5]

An image is worth 16x16 words: Transformers for image recognition at scale,

A.;Dosovitskiy, L.;Beyer, A.;Kolesnikov et al. , “An image is worth 16x16 words: Transformers for image recognition at scale,” in International Conference on Learning Representations , 2021

work page 2021

[6] [6]

Exploring plain vision trans- former backbones for object detection,

Y .;Li, H.;Mao, R.;Girshick et al., “Exploring plain vision trans- former backbones for object detection,” in European conference on computer vision . Springer, 2022, pp. 280–296

work page 2022

[7] [7]

Faster r-cnn: Towards real- time object detection with region proposal networks,

S.;Ren, K.;He, R.;Girshick et al. , “Faster r-cnn: Towards real- time object detection with region proposal networks,” IEEE transactions on pattern analysis and machine intelligence , vol. 39, no. 6, pp. 1137–1149, 2016

work page 2016

[8] [8]

Microsoft coco: Com- mon objects in context,

T.-Y .;Lin, M.;Maire, S.;Belongie et al. , “Microsoft coco: Com- mon objects in context,” in Computer Vision – ECCV 2014 , D.;Fleet, T.;Pajdla, B.;Schiele et al. , Eds. Cham: Springer International Publishing, 2014, pp. 740–755

work page 2014