{"id":"007a117f-dbdb-4510-ab47-3af4e9f91ea8","arxiv_id":"2505.10931","paper_version":4,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"M4-SAR is a large public optical-SAR fusion benchmark with rotated boxes, and the proposed E2E-OSDet detector reports the top mAP of 61.4 percent on it.","lead":"This paper introduces M4-SAR, a dataset of 112,174 aligned optical-SAR image pairs with about 982,000 labeled objects in six categories, plus a benchmarking toolkit and a new fusion detector. The dataset gives researchers a common evaluation ground for combining weather-robust radar with detailed optical imagery, though the reported 5.7 percent fusion gain does not match the tables.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Label transfer from optical to SAR is asserted, not validated; weak alignment and pseudo-label expansion make every benchmark mAP hostage to unmeasured label noise.","rationale":"The reader's weakest assumption—that labels drawn on optical images can be transferred to SAR and cloudy optical images with enough accuracy for benchmarking—is also the most load-bearing concern I can identify. The paper's central contribution is the dataset itself: 112,174 aligned image pairs and 981,862 labeled instances. If those labels are systematically wrong in the SAR modality, then the benchmark is not a benchmark of optical-SAR fusion detection; it is a benchmark of how well models fit an unvalidated label-transfer procedure. This would affect not only the headline 5.7% fusion-gain claim but also every comparative conclusion in Tables 2-4 and the qualitative claims in Figures 7-8. The paper's own text supports the concern: Section 2.4 asserts reliable transfer without reporting any verification, Section 5.1 admits only coarse geographic alignment, and Figure 3(d) quantifies weak cross-modal similarity. The iterative pseudo-label process adds a second source of unmeasured error, since the initial detector is trained and evaluated on the same small subset and later iterations can amplify systematic misses. Other concerns raised by the reader—the nontraceable 5.7% figure and unequal training budgets for MMRotate baselines—are real but secondary: they affect the strength of the fusion-gain claim, not the integrity of the resource itself. A conditional verdict is therefore appropriate, with the required check being an independent SAR-native annotation study. If that study shows high agreement, the main obstacle to accepting the dataset at face value is removed. If it shows low agreement, the paper would need either label correction or a documented reannotation protocol before the benchmark numbers can be trusted.","tokens_in":24279,"tokens_out":5873,"duration_ms":62591,"concrete_test":"Take a stratified random sample of at least 1,000 M4-SAR SAR patches (at least 100 per category) and have independent SAR-trained annotators label them from SAR imagery alone, following the same six-category oriented-box protocol. Compute the mean oriented IoU and per-category agreement between the released SAR labels and the SAR-native labels. Then retrain or fine-tune E2E-OSDet on the SAR-native labels for the sampled split and compare per-category and overall mAP against Table 2. If mean IoU is below about 0.5, or if rankings shift materially, the label-transfer assumption fails and every benchmark number is suspect. A secondary check should compare labels on a sample of cloudy optical replacements against their cloud-free counterparts to verify that cloud replacement does not introduce additional label error.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The benchmark's validity rests on the claim in Section 2.4 that annotations derived from cloud-free optical images can be 'reliably transferred' to paired SAR images and to cloudy optical replacements. The paper provides no quantitative check of that transfer. Figure 3(d) reports SSIM 0.1256, MSE 0.0662, and MI 0.1729, and the text itself calls the alignment 'weak'; Section 5.1 concedes that alignment is geographic-coordinate-based rather than pixel-level. In addition, the iterative semi-supervised labeling starts from a detector trained and evaluated on the same 5% subset (Section 2.4), so detector errors can propagate into the pseudo-label expansion that produces most of the 981,862 instances. Because every mAP in Table 2, including E2E-OSDet's 61.4%, is computed against these transferred labels, a systematic offset in SAR label placement—plausible at 10-60 m resolution with acquisition gaps of up to 10 days, and especially damaging for small classes such as wind turbines (average area about 96 px in Figure 3(c))—would change all rankings and the claimed 5.7% fusion gain. Selecting static targets mitigates but does not eliminate this risk, since geolocation error and detector bias still affect label coordinates. No annotation-agreement study, localization-error measurement, or SAR-native label verification is reported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript introduces M4-SAR, a paired optical-SAR rotated object detection dataset built from Sentinel-1 and Sentinel-2 data, containing 112,174 image pairs and 981,862 oriented instances over six static coastal infrastructure categories, with multi-resolution (10m/60m), multi-polarization (VV/VH), and multi-scene coverage. It also presents MSRODet, a unified benchmarking toolkit implementing seven existing fusion detectors, and proposes E2E-OSDet, an end-to-end fusion detector combining a Filter Augment Module (FAM), a Cross-modal Mamba Interaction Module (CMIM), and an Area-attention Fusion Module (AFM). On the M4-SAR test set, E2E-OSDet reports 61.4% mAP, and the paper claims a 5.7% mAP improvement from optical-SAR fusion over single-source inputs.","tokens_in":24475,"tokens_out":6655,"duration_ms":58245,"significance":"If the dataset labels are trustworthy, M4-SAR is a valuable community resource: it is substantially larger than existing optical-SAR paired benchmarks (OGSOD-1.0/2.0), adds rotated boxes and six categories, and ships with a standardized, reproducible evaluation toolkit and a strong baseline detector. The authors also provide public code and data. However, the benchmark's scientific value hinges on the validity of transferring optical annotations to SAR and cloudy images, which is asserted rather than demonstrated; the reported cross-modal alignment is weak (SSIM 0.1256, MSE 0.0662, MI 0.1729), so label noise could affect every reported mAP. The comparison protocols also mix training budgets and backbones across single-source baselines, making the headline fusion gain hard to interpret.","major_comments":[{"comment":"The dataset's validity rests on the assertion in §2.4 that annotations derived from cloud-free optical images can be 'reliably transferred' to paired SAR images and to cloudy optical replacements, but no quantitative validation of this transfer is provided. The paper itself reports weak alignment (SSIM 0.1256, MSE 0.0662, MI 0.1729 in Fig. 3(d)) and acquisition time gaps of up to 10 days, and §5.1 concedes that alignment is geographic-coordinate-based rather than pixel-level. Because every mAP in Table 2, including E2E-OSDet's 61.4%, is computed against labels inherited from optical images, a systematic offset in SAR label placement—plausible at 10–60 m resolution, and especially damaging for the smallest class (wind turbines, average area about 96 px in Fig. 3(c))—would change all reported numbers, including the claimed fusion benefit. Please add an annotation-agreement study, a localization-error measurement on SAR, or a SAR-native manual verification subset to quantify label transfer accuracy.","section":"§2.4, Fig. 3(d), §5.1"},{"comment":"The abstract's headline claim that fusing optical and SAR data improves mAP by 5.7% over single-source inputs is not traceable to Table 2. The best optical-only result is YOLO-Master at 56.3% and the best SAR-only result is YOLO-Master at 48.5%, while the best fusion result (E2E-OSDet) is 61.4%, giving gaps of 5.1 and 12.9 points respectively; no pairwise combination yields 5.7. Please specify the exact baseline configuration and protocol underlying the 5.7% figure, or revise the abstract to report the numbers that Table 2 actually supports.","section":"Abstract, Table 2"},{"comment":"The comparison between multi-source fusion methods and the MMRotate single-source baselines is confounded by differing training protocols. All fusion methods are trained from scratch for 300 epochs with a YOLOv11-S backbone and YOLOv8 OBB head (§4.1, §10.1), whereas the MMRotate baselines (R-FCOS, R-ATSS, O-RepPoint, RTMDet, PSC, LSKNet) are trained for 72 epochs with an ImageNet-pretrained ResNet-50 under a different optimizer and head. This makes the conclusion in §4.2 that 'all multi-source fusion methods outperform their corresponding single-modal baselines' unsupported for this subset of Table 2; the only fair single-source references are the YOLO-family rows, which do not share the fusion methods' architectures. Please add matched single-source versions of the fusion methods under identical training budgets, or re-train the MMRotate baselines for 300 epochs from scratch, and report both.","section":"§4.1, §10.1, Table 2"},{"comment":"The semi-supervised annotation procedure contains a circular step: the initial detector is trained and evaluated on the same 5% subset ('with the same data used for both training and evaluation during the initial phase'), and that detector then generates pseudo-labels that are manually corrected and used to expand the labeled set. As described, the initial model's biases can propagate into the majority of the 981,862 instances. Please clarify whether the initial 5% subset was excluded from later training/evaluation rounds, and report agreement statistics between raw pseudo-labels and corrected labels, or a separate validation of label quality on a fully manually annotated sample.","section":"§2.4"}],"minor_comments":[{"comment":"The column header for per-class APs is ambiguous: 'Bri. Har. Oil. Ply. Apr. Win.ALL' should be separated clearly and the 'ALL' column defined as the mean over classes.","section":"Table 2"},{"comment":"The sentence stating that the best two results are highlighted 'in Black and mark' is unclear; please specify the intended highlight style (e.g., bold and underline) so the table is interpretable.","section":"§4.2"},{"comment":"The label example in Fig. 9 shows duplicate coordinate rows and a confusing arrangement of category IDs; please ensure the example label file exactly matches the described format.","section":"Fig. 9"},{"comment":"Equation (9) defines L_total = L_reg + L_dfl + L_cls but does not define L_reg, L_dfl, or L_cls; add explicit formulas or references for each loss term.","section":"§9.2"},{"comment":"Table 1 lists M4-SAR resolution as '10M, 60M' while the text uses '10m, 60m'; use consistent units and notation throughout.","section":"Table 1"},{"comment":"The conclusion claims E2E-OSDet is 'the first fusion framework specifically designed for optical-SAR scenarios,' yet related work cites E2E-MFD [61] and other optical-SAR fusion works; please soften or justify this novelty claim.","section":"Conclusion"}],"recommendation":"major_revision","confidential_remarks":"The dataset and code release are strengths, and the central benchmarking contribution is plausible if the label-transfer concern is addressed. The annotation pipeline's initial training-and-evaluation-on-the-same-5% step is a form of circularity that should be probed in revision; if the authors cannot provide quantitative validation of label transfer or matched training protocols, the benchmark's credibility would be seriously weakened. I would not reject outright because the issues are fixable within a major revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know up front. The dataset is real and fills a genuine gap: no prior optical-SAR benchmark combines this scale (112,174 paired images, ~982k instances), rotated boxes, six categories, dual resolution, and dual polarization, with test sets in both modalities. OGSOD's test sets are SAR-only and the large SAR datasets are single-source. Second, the headline number—5.7% fusion gain—is not traceable in Table 2. E2E-OSDet's 61.4 mAP beats the best optical-only YOLO by 5.1 points and the best SAR-only by 12.9, so the 5.7 as written does not align with any row of the table, and the comparison is confounded: MMRotate single-source baselines get 72 epochs with an ImageNet-pretrained ResNet-50, while fusion models get 300 epochs from scratch with a YOLOv11-S backbone. Fusion winning despite the handicap strengthens the qualitative conclusion, but the magnitude is unestablished.\n\nWhat the paper does well: every one of the nine fusion methods beats every single-source method in Table 2, so the claim that fusion helps is directionally solid and not dependent on the authors' own detector. The ablations in Table 3 show each module contributing, Fig. 5 shows the framework is not fragile to hyperparameter choices, and the deliberate restriction to static targets is the right response to temporal misalignment. The limitations section is honest, including the weak-alignment numbers.\n\nThe load-bearing soft spot is the annotation pipeline. Labels are drawn on cloud-free optical images and transferred to SAR and to cloudy optical replacements via geographic-coordinate alignment that the paper itself rates as weak—SSIM 0.1256, up to 10-day acquisition gaps. No annotation-agreement or localization-error study is reported. The iterative pseudo-labeling also begins from a detector trained and evaluated on the same 5% subset, so detector errors can propagate into most of the 982k instances. The static-target constraint and the manual refinement step in each iteration blunt the worst case, but the transferred SAR labels are simply unvalidated, and for small classes like wind turbines (average ~96 px) a systematic offset would move rankings. This is the most serious concern and it is fixable: a sampled human re-annotation of SAR boxes, with agreement statistics, would largely settle it.\n\nSmaller issues: the training conditions for the single-source YOLOs are not stated as explicitly as for the fusion models; hyperparameters appear selected on test numbers; and there are no error bars anywhere.\n\nBottom line: this paper deserves serious peer review. It is a substantial resource for the optical-SAR detection community, the flaws are addressable rather than structural, and the core claim holds directionally. I would send it out and require the annotation-quality study, an equalized baseline comparison, and an honest statement of where 5.7% comes from before acceptance.","headline":"Genuinely useful new optical-SAR fusion benchmark with a directionally solid result, but the headline 5.7% fusion-gain figure is untraceable and the label-transfer step needs validation before the numbers can be trusted.","tokens_in":25127,"tokens_out":6806,"would_cite":true,"duration_ms":65274,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A new benchmark with 981,862 labeled optical-SAR instances shows fusion beats either sensor alone.","keywords":["optical-SAR fusion","object detection","rotated bounding boxes","remote sensing dataset","multi-source fusion","synthetic aperture radar","Sentinel-1","Sentinel-2"],"falsifier":"Manually re-annotate a random sample of SAR test images—say 1,000 instances—and measure the average IoU between the fresh boxes and the labels transferred from optical images. If the mean IoU is well below 0.5, the ground truth is systematically displaced and the reported mAP values, including the 5.7-point fusion gain, rest on misaligned labels.","tokens_in":23952,"feed_emoji":"🛰️","tokens_out":12436,"duration_ms":106849,"temperature":0.7,"pith_summary":"This paper argues that optical-SAR fusion detection has been blocked by the absence of a large, standardized dataset, and it supplies one. M4-SAR contains 112,174 aligned optical-SAR image pairs and 981,862 rotated-box instances across six coastal-infrastructure categories, drawn from Sentinel-1 and Sentinel-2 imagery at two resolutions. Labels are produced on cloud-free optical images and transferred to the paired radar images and to cloudy optical replacements, keeping annotation costs down. On this benchmark every fusion detector tested beats its single-modality counterpart, and the proposed E2E-OSDet reaches 61.4% mAP; the paper reports a 5.7-point gain from fusing the two modalities. A released toolkit gives future methods a common protocol for comparison.","feed_headline":"Fusing optical and SAR images lifts detection mAP to 61.4 percent","feed_subtitle":"A 112,174-pair rotated-box benchmark gives radar-plus-optical detectors a standardized test bed and a new baseline.","key_machinery":"The machinery is two coupled pieces. First, the dataset pipeline: a semi-supervised optical-assisted annotation loop in which a detector trained on 5% manually labeled cloud-free optical images produces pseudo-labels for the next 5%, humans correct them, and the cycle repeats; because optical and radar images are aligned only by geographic coordinates, the optical boxes are then transferred to the SAR pairs and to cloudy optical replacements. Second, E2E-OSDet, whose three modules each attack the optical-SAR domain gap: the Filter Augment Module projects both modalities through classical filters (HOG, Canny, Haar, gradient, wavelet scattering) to raise their structural similarity; the Cross-modal Mamba Interaction Module interleaves the two feature sequences so that a Mamba scan—a linear-time state-space sequence model—sees corresponding optical and SAR positions as adjacent; and the Area-attention Fusion Module weights salient local regions in the merged features before the oriented-box head.","core_discovery":"The paper's central claim is that a large, instance-level aligned optical-SAR dataset is feasible to build by annotating the easier modality and transferring labels across sensors, and that such a dataset reveals a consistent, exploitable fusion advantage. M4-SAR covers bridges, harbors, oil tanks, playgrounds, airports, and wind turbines at 10 m and 60 m resolution, in VH and VV polarizations, with optical and radar acquisitions up to ten days apart. Across the benchmark, every fusion method beats its single-modality counterpart at the same backbone, with the largest gains in cloudy, snowy, low-light, and low-resolution scenes. E2E-OSDet, which narrows the cross-modal domain gap before fusing features, reports the top mAP of 61.4% on M4-SAR and the best results on the OGSOD-1.0 and OGSOD-2.0 benchmarks, supporting the claim that explicit domain alignment is what makes optical-SAR fusion work.","pith_inferences":["Testable extension the authors do not run: split the optical test images into cloud-free and cloudy subsets and measure the fusion gain on each; the paper predicts the gain concentrates in degraded scenes, and that prediction is directly checkable with the released pairs.","The interleaved-sequence arrangement in CMIM is sensor-agnostic: the same optical-radar-optical-radar ordering could be applied to any poorly aligned multimodal pair, such as visible/infrared or multi-date radar, where forcing corresponding patches to be adjacent may help any sequence model stay aligned.","Because the categories are static coastal infrastructure, the alignment premise is untested on moving targets such as ships; a dynamic-category extension would be the sharpest stress test, since the up-to-ten-day acquisition gap would likely break label transfer there.","The headline 5.7-point fusion gain is not read directly off the main table: the gap to the best optical-only method is 5.1 points and to the same-backbone YOLOv11 optical result is 7.1 points, so pinning the single-source reference to one named method would make the headline number reproducible."],"forward_implications":["Fusion beats single-source detection at fixed architecture: with YOLOv11 as the shared backbone, every fusion method in the benchmark exceeds both the optical-only (54.3 mAP) and SAR-only (44.7 mAP) results, and the gap is largest in degraded scenes.","The dataset's rotated boxes, dense instances (8.75 per image), category imbalance (bridges are 43.6% of instances), and extreme aspect-ratio spread make it a stress test for oriented detection under realistic coastal conditions.","Each of the three E2E-OSDet modules adds measurable gain in the ablation (from 58.0 to 61.4 mAP), and the full model transfers to other optical-SAR benchmarks, so aligning the feature domain before fusion is a generalizable design rather than a dataset-specific trick.","Releasing the dataset and toolkit with a shared backbone and evaluation protocol makes head-to-head comparisons of future optical-SAR detectors possible and reproducible."],"supporting_citations":[{"why":"Supplies the SAR imagery (SLC format, VH and VV polarizations) from which every radar image in M4-SAR is derived.","marker":"[43]"},{"why":"Supplies the Sentinel-2 optical imagery whose RGB bands form the optical side of each pair.","marker":"[7]"},{"why":"The YOLO detector used in the semi-supervised annotation loop to generate pseudo-labels, and the shared backbone-neck for all fusion methods in the benchmark.","marker":"[15]"},{"why":"CSSA is the fusion baseline whose ablation (58.0 to 61.4 mAP) isolates the contribution of each E2E-OSDet module.","marker":"[2]"},{"why":"Mamba is the state-space sequence model that the CMIM module builds on for cross-modal interaction.","marker":"[10]"},{"why":"OGSOD-1.0 is the existing optical-SAR benchmark used to test E2E-OSDet's generalization beyond M4-SAR.","marker":"[48]"},{"why":"OGSOD-2.0 provides the second cross-benchmark transfer test for E2E-OSDet.","marker":"[47]"},{"why":"SARDet-100K is the large SAR-only dataset against which M4-SAR's scale and multi-source design are positioned.","marker":"[24]"}],"fun_headline_variants":["M4-SAR: 112K optical-SAR pairs boost detection by 5.7%","New optical-SAR benchmark: fusion lifts mAP by 5.7%","E2E-OSDet tops M4-SAR with 61.4% mAP","Fusing optical and SAR improves detection by 5.7% on M4-SAR","Optical-SAR fusion benchmark shows consistent 5.7% mAP gain"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that labels drawn on clear optical images remain accurate enough when shifted, using only geographic-coordinate alignment, onto radar images taken up to ten days apart—even though the two modalities register poorly at the pixel level.","fun_headline_variants_meta":{"raw":{"variants":["M4-SAR: 112K optical-SAR pairs boost detection by 5.7%","New optical-SAR benchmark: fusion lifts mAP by 5.7%","E2E-OSDet tops M4-SAR with 61.4% mAP","Fusing optical and SAR improves detection by 5.7% on M4-SAR","Optical-SAR fusion benchmark shows consistent 5.7% mAP gain"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001402,"raw_usage":{"total_tokens":5712,"prompt_tokens":1033,"completion_tokens":4679,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":649,"completion_tokens_details":{"reasoning_tokens":4564}},"tokens_in":649,"tokens_out":4679,"duration_ms":29904,"temperature":1.0,"reasoning_tokens":4564,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T21:01:25.017699+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Manually re-annotate a random sample of SAR test images—say 1,000 instances—and measure the average IoU between the fresh boxes and the labels transferred from optical images. If the mean IoU is well below 0.5, the ground truth is systematically displaced and the reported mAP values, including the 5.7-point fusion gain, rest on misaligned labels.","supporting_citations":[{"cited_title":"RSE120, 9–24 (2012)","cited_arxiv_id":null,"evidence_quote":"Supplies the SAR imagery (SLC format, VH and VV polarizations) from which every radar image in M4-SAR is derived."},{"cited_title":"RSE120, 25–36 (2012)","cited_arxiv_id":null,"evidence_quote":"Supplies the Sentinel-2 optical imagery whose RGB bands form the optical side of each pair."},{"cited_title":"com/ultralytics/ultralytics","cited_arxiv_id":null,"evidence_quote":"The YOLO detector used in the semi-supervised annotation loop to generate pseudo-labels, and the shared backbone-neck for all fusion methods in the benchmark."},{"cited_title":"In: CVPR","cited_arxiv_id":null,"evidence_quote":"CSSA is the fusion baseline whose ablation (58.0 to 61.4 mAP) isolates the contribution of each E2E-OSDet module."},{"cited_title":"In: COLM (2024)","cited_arxiv_id":null,"evidence_quote":"Mamba is the state-space sequence model that the CMIM module builds on for cross-modal interaction."},{"cited_title":"IEEE TGRS61, 1–14 (2023)","cited_arxiv_id":null,"evidence_quote":"OGSOD-1.0 is the existing optical-SAR benchmark used to test E2E-OSDet's generalization beyond M4-SAR."},{"cited_title":"In: ICASSP","cited_arxiv_id":null,"evidence_quote":"OGSOD-2.0 provides the second cross-benchmark transfer test for E2E-OSDet."},{"cited_title":"NeurIPS37, 128430–128461 (2024)","cited_arxiv_id":null,"evidence_quote":"SARDet-100K is the large SAR-only dataset against which M4-SAR's scale and multi-source design are positioned."}],"review_version":1}