{"id":"1fbffc05-b003-4bd0-a4c5-37d19bfb3d37","arxiv_id":"1908.10907","paper_version":3,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"DFPENet-geology claims new state-of-the-art landslide recognition accuracy using RGB imagery, geological morphology filters, and temporal subtraction, but the evaluation is undermined by training/test overlap and an excluded boundary error.","lead":"A deep learning segmentation network, DFPENet, is combined with geological filters and pre/post-earthquake image subtraction to recognize co-seismic landslides, and it is applied to the 2017 Jiuzhaigou and 2018 Hokkaido earthquakes. The paper reports very high accuracy on Jiuzhaigou, but the training data overlap with the test area and the boundary-excluded metric weaken the state-of-the-art claim.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 98.67% Jiuzhaigou result is internally inconsistent with the described post-processing, so the central SOTA claim is unsupported as written.","rationale":"The reader's rejection is sound: the Jiuzhaigou consistency analysis is not a valid cross-scene test if training samples overlap the test region, and excluding boundary error makes the comparison to prior work inequitable. My stress-test found an additional, more concrete problem that reinforces the same verdict: Table 4's fine-tuning row is arithmetically incompatible with the described temporal-subtraction step. Specifically, TP increases and FN decreases by exactly 8,554 pixels between the DFPENet row and the fine-tuning row, while FP drops by 55,608 pixels. Subtraction of pre-event predictions would remove pixels, not convert false negatives into true positives; the only way to obtain such a table is an undocumented operation, most plausibly a correction informed by the same ground truth used for evaluation. This is an internal inconsistency, not a matter of interpretive preference, and it directly undermines the headline 98.67% accuracy. The Vaihingen benchmark is the strongest part of the paper, but the central landslide-recognition claim rests on the Jiuzhaigou evaluation, which is not trustworthy as reported. The Hokkaido transfer demonstration is also not quantitatively evaluated on the final landslide map, so it cannot independently support cross-scene generalization. Given these issues, the appropriate verdict remains REJECT; a clean spatial split, a faithful reproduction of the post-processing pipeline, and boundary-aware metrics could convert this into a conditional accept or unverdictable status, but that would require additional evidence beyond the current manuscript.","tokens_in":21646,"tokens_out":7590,"duration_ms":78109,"concrete_test":"Reproduce Table 4 from the released code and data using only the operations described in Section 3.3: run DFPENet-geology on the post-event image, apply the geologic filter, then subtract the pre-event predictions. Compare the resulting TP/FP/FN counts with the 'Fine tuning results' row. If the reproduction cannot simultaneously add 8,554 TPs and remove 55,608 FPs, the 98.67% figure depends on an undescribed or ground-truth-informed step. As a second check, intersect the training patches, especially the Xiongmaohai-centered samples, with the 53.6 km² consistency-analysis region; any nonzero overlap means the evaluation must be rerun on a strictly disjoint spatial split and the reported mIoU recomputed.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing result is Table 4's 'Fine tuning results' row (mIoU 98.67%), but the described pipeline cannot produce that row. Section 3.3 Phase 2 states that temporal fine-tuning subtracts the pre-event recognition result from the post-event result ('two recognition results are subtracted to remove misidentification areas'). Subtraction can only remove predicted landslide pixels, so relative to the DFPENet row it should leave TP unchanged or lower and FN unchanged or higher. Table 4 instead shows TP rising from 915,698 to 924,252 (+8,554) and FN falling from 20,753 to 12,199 (−8,554), with FP collapsing from 55,853 to 245. No described step—geologic filtering, temporal subtraction, or vectorization—adds true positives. This pattern is consistent with an undocumented correction step that uses the validation ground truth, which would make the evaluation circular. Independent of that arithmetic issue, the training set (Table 3) explicitly includes 'Jiuzhaigou seismic landslides around Xiongmaohai' in the same scenic area used for the consistency analysis, so even a clean reproduction would test memorization rather than cross-scene generalization. The abstract's own caveat that 'landslide boundary error is not considered' further weakens direct comparison with prior full-boundary accuracies.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes DFPENet, an encoder-decoder semantic segmentation network with a dense feature pyramid module and attention/gated feature filtering, and evaluates it on the ISPRS Vaihingen benchmark, reporting 93.06% overall accuracy. It then introduces DFPENet-geology, a four-phase landslide recognition scheme that combines DFPENet predictions with morphological filtering, temporal subtraction, vectorization, and transfer learning, and applies it to the 2017 Jiuzhaigou earthquake (claimed 98.67% pixel-level consistency) and the 2018 Hokkaido earthquake. The authors claim a new state-of-the-art performance in regional landslide identification while noting that landslide boundary error is not considered. Code and trained models are publicly available on GitHub.","tokens_in":21869,"tokens_out":5991,"duration_ms":56700,"significance":"If the Vaihingen result and the transfer-learning workflow are reproducible, the paper provides a useful segmentation backbone and a practical, RGB-only pipeline for rapid landslide mapping; the component ablation and the public release of models are strengths. However, the central cross-scene generalization claim rests on the Jiuzhaigou consistency analysis, which is compromised by training-set overlap with the test region and by an internally inconsistent fine-tuning row in Table 4. The significance of the landslide claim therefore cannot be assessed without a redesigned evaluation.","major_comments":[{"comment":"Phase 2 states that the temporal fine-tuning step subtracts the pre-event recognition result from the post-event result to remove misidentification areas. Subtraction can only eliminate predicted landslide pixels; it cannot turn previously missed landslide pixels into detected ones. In Table 4, however, the 'Fine tuning results' row increases TP from 915,698 to 924,252 (+8,554) and decreases FN from 20,753 to 12,199 (−8,554), while FP collapses from 55,040 to 245. No described step—morphological filtering, temporal subtraction, or vectorization—adds true positives. This pattern is exactly what one would obtain if an undocumented correction using the validation ground truth was applied, which would make the claimed 98.67% mIoU circular. The authors need to explain how the TP increase is produced, or the result cannot be considered valid.","section":"Table 4 and Section 3.3, Phase 2"},{"comment":"The consistency analysis treats the manually interpreted Jiuzhaigou landslide inventory as an external test, but the training set explicitly includes 'Jiuzhaigou seismic landslides around Xiongmaohai' (Table 3), and Section 4.1.1 states that 'the area centered on Xiongmaohai was also added to the training set (Fig. 6)', where Fig. 6 is the geographic location of the research region. If Xiongmaohai lies inside the 53.6 km² Jiuzhaigou Scenic Area test region, the model has seen samples from the test scene during training, and the reported 98.67% is a memorization measure rather than evidence for cross-scene generalization. The validation set also includes Jiuzhaigou seismic landslides, compounding the concern.","section":"Table 3 and Section 4.1.1"},{"comment":"The abstract and conclusions claim a new state-of-the-art performance in regional landslide identification while 'not considering the landslide boundary error', yet the comparison to previous methods (Ma et al. 85%, Liu and Wu 97.40%, CDMRF correctness > 0.75) is made without stating whether those methods applied the same boundary treatment. The per-event morphological thresholds are also adapted to each site (minimum area 37.5 m² for Jiuzhaigou and 75 m² for Hokkaido), and no sensitivity analysis is provided for these thresholds or for the length-width ratio threshold. The SOTA claim is therefore not commensurable with the cited baselines as reported.","section":"Abstract and Section 4.1.2"}],"minor_comments":[{"comment":"The abbreviation for Attention Gate Mechanism is introduced as AGM but the text often uses ACM; please make the abbreviations consistent throughout, including in Fig. 1.","section":"Section 3.2.2"},{"comment":"The inline formulas are badly garbled by typesetting; the operator symbols and indexing are unclear. Please provide a clean typeset version.","section":"Equations (1)-(3)"},{"comment":"The dataset description says 'randomly sample the 600×600 patches from the original 33 images' before defining the train/test split; clarify that patches for the test set are sampled only from the 17 test images.","section":"Section 3.2.5"},{"comment":"The pre-trained dataset is described as using '16 images from the training set and 4 images from the validation set', but no validation split has been defined for the 16 training images; please clarify.","section":"Section 3.2.5"},{"comment":"The GPU is reported as 'Tesla K80' in Section 3.2.4 and as 'Tesla P80' in Sections 3.2.5 and 4.1.2; please correct the inconsistency.","section":"Section 3.2.4 and Section 4.1.2"},{"comment":"The text refers to Fig. 10(a)-(f) for the Hokkaido pre/post images and final results, but Fig. 10 is also used earlier for the training-set-size curves; renumber the figures.","section":"Section 4.2.2"}],"recommendation":"reject","confidential_remarks":"The strength of the paper lies in the Vaihingen benchmark evaluation and the public code release; the landslide application is the main advertised contribution, and the reported headline number is not supported by the described pipeline. If the authors can provide a clean re-evaluation with non-overlapping training and test regions and a consistent account of the post-processing steps, a resubmission could be considered; as written, the central claim should not enter the literature."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. The Vaihingen segmentation contribution is real: DFPENet+PD reports 93.06% OA and 81.06% mIoU, above HUSTW5, with an ablation showing each added module helps. The landslide headline, 98.67% consistency accuracy, is not supported by the evaluation as written; the numbers in Table 4 cannot be produced by the described pipeline.\n\nWhat is new is the integration: DFPENet assembles known components (ResNet-101 with dilated convolutions, ASPP, attention gates, gated convolutions, dense top-down pyramid), and the four-phase landslide workflow adds morphological filtering, temporal subtraction, vectorization, and transfer learning. That is a legitimate systems contribution, and the authors post code and trained models on GitHub, which makes the Vaihingen part checkable.\n\nThe soft spots are serious. Temporal subtraction can only remove predicted landslide pixels; it cannot turn a false negative into a true positive. Yet the 'Fine tuning results' row shows TP rising by 8,554 and FN falling by exactly 8,554, while FP drops from 55,040 to 245. That pattern is consistent with an undocumented correction using ground truth, not with subtraction. Separately, Table 3 puts 'Jiuzhaigou seismic landslides around Xiongmaohai' in the training database while the test region is the Jiuzhaigou Scenic Area, so even a clean re-run would test same-event memorization rather than cross-scene generalization. The abstract's own caveat that 'landslide boundary error is not considered' weakens direct comparison with prior full-boundary accuracies. The Hokkaido transfer is only evaluated on a 30-image validation set; the final map has no quantitative accuracy. One smaller loose end: the Vaihingen pre-training set is said to use 16 training images plus 4 validation images, but no validation split was defined earlier — if those 4 came from the 17 test images, the 93.06% is contaminated. Naming the four tile IDs would settle it.\n\nWho this is for: remote sensing and geohazards readers working on deep-learning landslide mapping. The Vaihingen result deserves a serious referee; the landslide SOTA claim needs clean spatial splits, boundary-aware metrics, and an independent Hokkaido inventory before it can be trusted. I would send it out with an expectation of major revision, and I would not cite the 98.67% number as it stands.","headline":"Solid Vaihingen segmentation work, but the 98.67% landslide result is internally inconsistent with the described post-processing and likely reflects same-event training overlap.","tokens_in":22452,"tokens_out":5352,"would_cite":false,"duration_ms":54095,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Deep learning scheme recognizes and segments co-seismic landslides from RGB imagery alone, with reported 98.67% pixel consistency in Jiuzhaigou and successful transfer to Hokkaido.","keywords":["co-seismic landslide recognition","semantic segmentation","DFPENet","feature pyramid network","transfer learning","temporal change detection","geologic feature fusion","remote sensing"],"falsifier":"A reader could settle this by checking whether any training image, especially the Xiongmaohai-centred Jiuzhaigou samples listed in Table 3, geographically overlaps the 53.6 km² Jiuzhaigou Scenic Area test region; if overlap exists, recompute the consistency after removing those training samples. A second check is to run the same fine-tuning procedure on a held-out earthquake where no landslides from that event appear in training, and compare the resulting mIoU with the reported 98.67%.","tokens_in":1427,"feed_emoji":"🌋","tokens_out":1501,"duration_ms":51948,"temperature":0.7,"pith_summary":"This paper claims that a single deep segmentation network, DFPENet, combined with landslide shape rules and the difference between pre- and post-earthquake images, can recognize and outline co-seismic landslides from RGB imagery alone with accuracy high enough for emergency response. The authors argue the scheme is cross-scene: a model trained on Chinese landslides, transferred with a small fine-tuning set, also mapped the 2018 Hokkaido landslides. On the ISPRS Vaihingen benchmark the core network reports 93.06% overall accuracy, and consistency with a manually interpreted Jiuzhaigou inventory reaches 98.67% on pixels and 89.95% by landslide count. The stated aim is a rapid, general, RGB-only pipeline for post-earthquake landslide mapping, while explicitly setting aside the landslide boundary error.","feed_headline":"Deep network maps quake landslides at 98.67 percent accuracy","feed_subtitle":"RGB-only scheme adds landslide shape rules and pre/post subtraction to map quake landslides across regions.","key_machinery":"The load-bearing mechanism is a two-stage correction loop: the segmentation network outputs candidate landslide polygons, and the geology module filters them by minimum area (at least four times the spatial resolution) and length-width ratio thresholds, then the pre-event segmentation is subtracted from the post-event segmentation to delete unchanged false positives. The network itself is carried by a dense top-down feature pyramid fed by a ResNet-101 encoder with dilated convolution and atrous spatial pyramid pooling; the feature-filter equations combine gated feature maps so high-level semantic features and low-level detail propagate without redundancy.","core_discovery":"The paper's central discovery is a pipeline that treats co-seismic landslide recognition as a semantic segmentation task plus a set of cheap, interpretable correction steps. DFPENet is an encoder-decoder with a ResNet-101 dilated backbone, a feature-filter stage using attention gates and gated convolutions, and a dense top-down feature pyramid; it captures small landslides that generic baselines miss. The geology module removes non-landslide regions by minimum-area and length-width ratio thresholds; temporal resolution is added by subtracting the model's pre-earthquake segmentation from its post-earthquake segmentation. The authors report that this raises pixel-level consistency from 92.28% to 98.67% on Jiuzhaigou and, after fine-tuning on 100 patches, transfers to Hokkaido with 77.21% mIoU. Boundary error is explicitly excluded from the headline accuracy.","pith_inferences":["A testable extension is to apply the same geology-plus-temporal pipeline to a multi-event benchmark with held-out earthquake events, measuring per-landslide boundary IoU rather than pixel accuracy; the paper's own caveat that boundary error is not considered suggests boundary-aware metrics would be the first place where the 98.67% number would drop.","The length-width and minimum-area filters are prior knowledge encoded as explicit rules; one could train a second network to predict them end-to-end and compare sample efficiency against the hand-tuned thresholds.","The cross-scene claim is phrased as RGB-only, but both test sites are mountainous and relatively sparsely vegetated; a stronger test would apply the transferred model to urbanized or densely vegetated earthquake regions, where the temporal subtraction could erroneously remove genuine landslides."],"forward_implications":["If the scheme works as reported, earthquake response teams can produce editable landslide maps from RGB pre/post imagery within minutes of a scene being available.","The 93.06% Vaihingen score indicates the same network is a competitive general remote-sensing segmenter, not just a landslide-specific model.","Transfer learning from Jiuzhaigou to Hokkaido with 100 patches suggests a trained model can be adapted to new earthquake regions with only small labeled sets, provided at least 20-40% of such a set is available.","The temporal subtraction step alone contributes a 6.39% pixel-accuracy gain, implying that unchanged terrain and roads are the main false-positive source in single-date segmentation."],"supporting_citations":[{"why":"Provides the Feature Pyramid Networks (FPN) concept that DFPENet's dense top-down pyramid extends.","marker":"Lin et al., 2016"},{"why":"Supplies the dilated convolution technique used in the encoder to preserve spatial resolution.","marker":"Yu and Koltun, 2015"},{"why":"Supplies the ResNet-101 backbone that DFPENet initializes for bottom-up feature extraction.","marker":"He et al., 2016"},{"why":"Supplies the DeepLabv3+ baseline and ASPP module that DFPENet adapts; also the reference architecture compared in Table 2.","marker":"Chen et al., 2018b"},{"why":"Provides the minimum-area and length-width ratio thresholds for landslide morphological filtering in the geology module.","marker":"Ma et al., 2016"},{"why":"Supplies the Jiuzhaigou landslide inventory used as reference for the consistency evaluation.","marker":"Fan et al., 2018"},{"why":"Offers the HUSTW5 result on Vaihingen that DFPENet outperforms in the benchmark table.","marker":"Sun, 2018"},{"why":"Provides the 97.40% deep auto-encoder landslide recognition baseline the authors compare against.","marker":"Liu and Wu, 2016"},{"why":"Supplies the CDMRF landslide mapping correctness values used as a comparison in the discussion.","marker":"Lu et al., 2019b"}],"fun_headline_variants":["Quake landslide AI hits 98.67% pixel consistency with RGB only","Pre/post subtraction boosts quake landslide mapping to 98.67%","RGB-only scheme maps quake landslides, transfers across zones","AI and geology rules map quake landslides at 98.67% consistency","Transfer learning extends quake landslide AI to new regions"],"cache_read_input_tokens":24576,"weakest_assumption_plain":"The headline accuracy assumes the manually interpreted Jiuzhaigou inventory is an independent test, even though the training set deliberately includes landslides from the same 2017 Jiuzhaigou event and an area centered on Xiongmaohai near the test region; if those samples overlap the test scene, the 98.67% number measures scene memorization, not generalization.","fun_headline_variants_meta":{"raw":{"variants":["Quake landslide AI hits 98.67% pixel consistency with RGB only","Pre/post subtraction boosts quake landslide mapping to 98.67%","RGB-only scheme maps quake landslides, transfers across zones","AI and geology rules map quake landslides at 98.67% consistency","Transfer learning extends quake landslide AI to new regions"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001591,"raw_usage":{"total_tokens":6350,"prompt_tokens":959,"completion_tokens":5391,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":575,"completion_tokens_details":{"reasoning_tokens":5300}},"tokens_in":575,"tokens_out":5391,"duration_ms":40355,"temperature":1.0,"reasoning_tokens":5300,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T10:31:02.062306+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A reader could settle this by checking whether any training image, especially the Xiongmaohai-centred Jiuzhaigou samples listed in Table 3, geographically overlaps the 53.6 km² Jiuzhaigou Scenic Area test region; if overlap exists, recompute the consistency after removing those training samples. A second check is to run the same fine-tuning procedure on a held-out earthquake where no landslides from that event appear in training, and compare the resulting mIoU with the reported 98.67%.","supporting_citations":[],"review_version":1}