{"id":"2ad3d311-2ee9-4b25-96c0-06e793c5c95d","arxiv_id":"2508.03920","paper_version":1,"verdict":"CONDITIONAL","confidence":"LOW","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"On Moon and Mars imagery, YOLO yields the most balanced crater detection, while ResNet-50 excels at large craters, under a two-stage CNN-based framework.","lead":"Researchers compared three deep learning models, YOLO, ResNet-50 and a simple CNN, for spotting impact craters in satellite images of the Moon and Mars, using a two-stage pipeline that first classifies image patches and then localizes craters. The practical result: YOLO delivers the most balanced precision and recall, while ResNet-50 is best for large craters.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The YOLO-vs-ResNet comparison is not yet shown to be independent of the training crater catalog; label incompleteness for small craters and possible train/test overlap could produce the reported 'balanced' vs 'large-crater' result as an artifact.","rationale":"The reader's weakest assumption—ground-truth completeness—is the same structural dependency I identify: if the catalog under-labels small craters, precision/recall comparisons are confounded with label density. My stress-test sharpens it by noting the further requirement that the test set be independent of the training catalog and by making the leakage path concrete in the two-stage pipeline. Since the reader already assigned CONDITIONAL with low confidence and no code/data are available, this concern does not change the verdict; it reinforces it. The central claim is plausible but currently rests on an empirical comparison whose data premise is unaudited. The concrete test is the single check that would settle it: an external, diameter-stratified evaluation on held-out tiles.","tokens_in":13688,"tokens_out":6043,"duration_ms":73802,"concrete_test":"Hold out five 1024×1024 LRO NAC and five MRO CTX tiles from regions not used in training, spanning a documented range of crater diameters. Have at least two annotators independently mark every crater ≥5 px in diameter and adjudicate disagreements. Run the trained YOLO and ResNet-50 pipelines on these tiles with no further training, fix the IoU threshold at 0.5, and set confidence thresholds by maximizing F1 on a validation subset of the original catalog. Report precision, recall, F1, and PR-AUC in diameter bins (5–10, 10–20, 20–50, >50 px) separately for Moon and Mars. If the ResNet-50 large-crater advantage and the YOLO 'balance' do not survive this independent, diameter-stratified evaluation, the central claim is an artifact.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is an empirical comparison: YOLO gives the most balanced detection, ResNet-50 gives high precision on large craters. For such a comparison to be meaningful, the precision/recall tables (the per-class 'precision/recall/f1-score/support' reports and detection tables visible in the manuscript) must be computed on a test population independent of training labels, with a fixed matching criterion and with completeness that does not vary by crater size. The paper does not establish this; the legible portions do not document train/test splits, catalog provenance, or an IoU/threshold protocol. The two-stage design creates a concrete leakage path: stage-1 classifiers and the stage-2 YOLO detector appear to be trained from the same crater catalog, so reported accuracy is circular with respect to that catalog. Since crater catalogs are typically incomplete for small craters, a detector that finds real but unlabeled small craters is penalized as false-positive, while a large-crater specialist benefits from denser labeling of large craters. The 'balanced' YOLO outcome could also depend on a single confidence threshold. This is precisely the assumption on which the abstract's conclusion rests, and it is currently unsupported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes a two-stage deep-learning framework for crater detection and identification on the Moon and Mars. Stage one applies three classifiers—described as a simple classic CNN, ResNet-50, and YOLO—to identify craters, and stage two uses YOLO for localization. The authors report per-class precision/recall/F1/support results and detection tables for selected regions, concluding that YOLO gives the most balanced crater detection performance while ResNet-50 excels at large craters with high precision. The central claim is empirical and benchmark-oriented. Unfortunately, the supplied full text is largely unreadable because of character-encoding corruption (mojibake), so most methodological details—dataset splits, ground-truth catalog, thresholds, training setup, and region selection—cannot be verified from the manuscript as provided.","tokens_in":13899,"tokens_out":4821,"duration_ms":63498,"significance":"If the empirical comparison is correctly executed, the paper would provide a useful applied benchmark comparing standard deep-learning detectors for planetary crater mapping. The claimed trade-off—YOLO balancing precision and recall while ResNet-50 favors large-crater precision—is clear, internally consistent, and in principle falsifiable from the reported tables. The contribution is modest rather than architecturally novel, since YOLO and ResNet are established models. No code, data, or machine-checked artifacts are supplied in the visible text, so the value rests entirely on the credibility and completeness of the evaluation protocol, which is currently not documented in a verifiable way.","major_comments":[{"comment":"The body of the manuscript is corrupted by character-encoding errors (mojibake). I cannot read the methods, dataset description, evaluation protocol, or discussion. Only the abstract and fragmentary tables are legible. This prevents verification of every load-bearing claim in the paper. A clean, legible manuscript is a prerequisite for any further technical review; the current file cannot be evaluated as a scientific article.","section":"Entire manuscript (as supplied)"},{"comment":"These tables are the evidence for the abstract's central trade-off, but the manuscript does not document the train/test split, the source and version of the ground-truth crater catalog, or whether stage-1 classifiers and the stage-2 YOLO detector were trained on labels from the same catalog. If small craters are underlabeled in the catalog, YOLO's false positives can include real but unlabeled craters, while ResNet-50's large-crater precision can simply reflect denser labeling of large craters. This is a concrete leakage path. Please report held-out geographic regions, catalog provenance, and a completeness analysis by crater size.","section":"Per-class precision/recall/F1 tables (tables with headers 'precision recall f1-score support')"},{"comment":"The detection evaluation lacks a stated protocol. The manuscript does not specify the IoU matching threshold, the confidence threshold for YOLO, non-maximum-suppression settings, or the rule for counting a predicted box as a true positive. All precision and recall values are threshold-dependent, so the claim that YOLO is 'most balanced' and ResNet-50 has 'high precision on large craters' is not identified under a fixed, reproducible criterion. Please report the full protocol or provide precision-recall curves.","section":"Detection results tables (Section 4, 'Crater detection and identification')"},{"comment":"The selected regions on the Moon and Mars are not specified with coordinates, image sources, resolutions, or lighting conditions. Without a documented selection protocol, the reported numbers cannot be reproduced and cannot be interpreted outside the specific chosen tiles. The paper should give region identifiers and a rationale for why these regions are representative.","section":"'Summary report with remote sensing data' (Section 5)"}],"minor_comments":[{"comment":"The abstract describes YOLO and ResNet as 'novel models,' which is misleading since they are established architectures. Suggest using 'modern deep-learning models' or similar.","section":"Abstract"},{"comment":"The phrase 'different types of craters' is used without defining the type taxonomy. Please specify the classes used in the per-class tables.","section":"Introduction / Section 1"},{"comment":"Figure captions and axis labels are not readable in the supplied text. In the resubmission, ensure all figures are legible and include scale bars and coordinate information where relevant.","section":"Figures"},{"comment":"The reference list is garbled and incomplete in the provided file. A complete, correctly formatted bibliography is required for review.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The provided file appears to be a corrupted rendering of the manuscript; this should be returned to the authors for a clean resubmission before any substantive review. If the clean version is unchanged in content, the decisive technical issues will be the undocumented evaluation protocol: catalog provenance, train/test separation, matching thresholds, and region selection. I do not regard the novelty as a barrier; the contribution is a benchmark-style comparison, but the current manuscript does not establish the reliability of its own numbers."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Honest take: this is a standard-architecture benchmark paper with a plausible headline result, but right now it's unverifiable. The supplied full text is mostly decoding garbage; I could read the abstract and fragments of tables, not the methods. So the verdict has to be conditional on getting a clean version and the underlying data.\n\nWhat's genuinely there: the two-stage design—classifier first, then YOLO for localization—is a reasonable way to combine identification and bounding boxes. Running the same comparison on Moon and Mars regions is a practical service to the crater-detection community. The abstract's claims (YOLO most balanced, ResNet-50 strong on large craters) are internally consistent and the kind of thing benchmark tables can support. Using standard models is fine; calling them 'novel' is not.\n\nThe soft spots are about evidence, not method. No code or data release is apparent. No train/test split, catalog provenance, IoU threshold, or confidence threshold is documented in what I can read. The stress-test concern lands: if the ground-truth catalog under-labels small craters, a detector that finds real but unlabeled small craters gets penalized, while a large-crater specialist looks better. That could produce the reported pattern without any genuine skill difference. This is a missing-documentation problem, not a demonstrated flaw, but it's exactly what a referee needs to check.\n\nAlso, no error bars or significance tests are visible for the precision/recall tables, so I can't tell whether the YOLO-vs-ResNet difference is real or threshold luck.\n\nWho this is for: someone working on automated crater detection who wants a quick comparison of off-the-shelf models. It's not a methods paper and won't change the field.\n\nMy recommendation: send it to peer review, but with a strong request for code, data, and full experimental documentation, and require the authors to fix the corrupt text before it's reviewable. The claims are addressable; the paper is not fatally flawed.","headline":"Plausible but unverifiable benchmark comparison of standard crater-detection models; the full text is garbled and the evaluation dependencies are undocumented.","tokens_in":14471,"tokens_out":2422,"would_cite":false,"duration_ms":30123,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A two-stage deep-learning framework detects and identifies impact craters on the Moon and Mars, with YOLO the most balanced and ResNet-50 the most precise on large craters.","keywords":["crater detection","deep learning","YOLO","ResNet-50","CNN","Moon","Mars","remote sensing"],"falsifier":"Take a set of regions where an independent, more complete crater catalog exists, run the same two-stage pipeline, and compare results against both the original and the independent labels. If YOLO's balanced precision-recall and ResNet-50's large-crater edge shrink or reverse when the reference labels change, the central comparison is an artifact of the original catalog.","tokens_in":13480,"feed_emoji":"🛰️","tokens_out":4343,"duration_ms":49263,"temperature":0.7,"pith_summary":"This paper tries to establish a practical division of labor for automated crater mapping on the Moon and Mars: YOLO gives the most balanced trade-off between finding craters and avoiding false alarms, while ResNet-50 is the most reliable when the goal is to identify large craters with high precision. The authors build a two-stage framework in which the first stage identifies craters using a classic CNN, ResNet-50, or YOLO, and the second stage uses YOLO-based detection to localize them. They evaluate on selected regions of Mars and the Moon using remote-sensing data and report per-region results. If the comparison holds, planetary researchers get a concrete model-selection rule: use YOLO for general crater surveys and ResNet-50 for large-crater cataloging.","feed_headline":"YOLO balances crater detection; ResNet-50 nails large craters","feed_subtitle":"A two-stage deep-learning pipeline identifies and locates impact craters on the Moon and Mars from remote-sensing imagery.","key_machinery":"The central mechanism is the two-stage pipeline: Stage 1 classifies image patches as containing craters using a classic CNN, ResNet-50, or YOLO; Stage 2 uses YOLO-based object detection to draw bounding boxes around identified craters. The argument turns on comparing YOLO and ResNet-50 within the same framework, so the reported differences in precision and recall are attributed to the identification model rather than to different localization machinery.","core_discovery":"The paper's central claim is that a two-stage deep-learning framework can detect and identify impact craters on selected regions of the Moon and Mars with a useful model-dependent trade-off: YOLO achieves the most balanced precision-recall performance across craters, while ResNet-50 achieves the highest precision on large craters. The first stage performs crater identification using a classic CNN, ResNet-50, and YOLO; the second stage uses YOLO-based detection to localize the craters identified. The authors evaluate on remote-sensing data for selected regions and report per-region results, including a summary that connects detected craters to remote-sensing context.","pith_inferences":["Because the paper does not document how the selected regions were chosen, I would not assume the same balance holds on all terrains; a direct extension is to test the same pipeline across highlands, maria, polar terrain, and varied lighting.","If the ResNet-50 large-crater advantage generalizes, it has a natural downstream use in crater size-frequency dating, where large craters dominate age estimates; the paper itself stops at detection and identification.","The reliance on existing crater catalogs suggests the safest practical use is candidate generation for human verification rather than a final ground-truth catalog.","The same two-stage structure could be pointed at other airless bodies such as Mercury or Ceres, though crater morphology differs; a testable extension is to check whether the same balance-versus-precision split appears there."],"forward_implications":["Planetary researchers can choose YOLO when a balanced crater survey is needed and ResNet-50 when large craters must be captured with few false positives.","The two-stage design separates identification from localization, so either stage can be improved or replaced independently.","The per-region summary format can be reproduced for other selected areas on the Moon and Mars, making the framework a reusable mapping tool.","Automated screening of this kind can reduce the manual effort of crater counting in routine planetary mapping work."],"supporting_citations":[],"fun_headline_variants":["Moon and Mars craters: YOLO balances, ResNet-50 excels big","Two-stage deep learning maps craters on Moon, Mars","YOLO wins balanced crater ID, ResNet-50 for large ones","AI crater detector: YOLO for all, ResNet for big","Crater ID on Moon, Mars: YOLO balanced, ResNet-50 precise"],"cache_read_input_tokens":2816,"weakest_assumption_plain":"The evaluation assumes the ground-truth crater catalog for the selected Moon and Mars regions is complete and correct; if it misses many small or degraded craters, the reported precision-recall balance and the large-crater advantage reflect label gaps rather than genuine detection skill.","fun_headline_variants_meta":{"raw":{"variants":["Moon and Mars craters: YOLO balances, ResNet-50 excels big","Two-stage deep learning maps craters on Moon, Mars","YOLO wins balanced crater ID, ResNet-50 for large ones","AI crater detector: YOLO for all, ResNet for big","Crater ID on Moon, Mars: YOLO balanced, ResNet-50 precise"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000634,"raw_usage":{"total_tokens":2755,"prompt_tokens":731,"completion_tokens":2024,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":475,"completion_tokens_details":{"reasoning_tokens":1933}},"tokens_in":475,"tokens_out":2024,"duration_ms":15322,"temperature":1.0,"reasoning_tokens":1933,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T01:03:09.092546+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a set of regions where an independent, more complete crater catalog exists, run the same two-stage pipeline, and compare results against both the original and the independent labels. If YOLO's balanced precision-recall and ResNet-50's large-crater edge shrink or reverse when the reference labels change, the central comparison is an artifact of the original catalog.","supporting_citations":[],"review_version":1}