{"id":"c3b54c88-41bc-4de6-94fa-8b0947176b32","arxiv_id":"2501.08584","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"A Siamese CNN trained on hand-labeled Qaidam-Mars image pairs returns small feature distances for four known analogue pairs, but the validation is circular and no code or data are released.","lead":"Researchers trained a Siamese convolutional neural network to compare surface images of Mars and Earth, then used it to score how closely four Qaidam Basin landforms match four Martian landforms. The method is aimed at helping to pick Mars landing sites by finding terrestrial analogue terrains, especially evaporite and polygonal structures that could preserve signs of past life.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported matching distances are not a predictive test: the four correspondences in §5.3 are the same manual correspondences used to build training labels in §5.1, and the train/test split allows near-duplicate augmented images in both sets.","rationale":"The reader's verdict identifies the central weakness correctly: dataset construction in Section 5.1 uses the four manually asserted correspondences as training labels, and Section 5.3 reports those same correspondences as the model's 'findings.' This is an internal evidential circularity, not a disagreement with geological consensus, and it makes the headline numbers (0.11, 0.26, 0.38, 0.05) incapable of supporting the landing-site selection claim. A leave-one-correspondence-out split is the minimal experiment that would break the circularity: it would show whether the learned distance transfers to a Qaidam–Mars pair never used in training. Additionally, the random 9:1 split on augmented pairs from only four regions allows near-duplicate images to appear in both training and test, so the 99.82% accuracy is not a credible generalization measure. Because the paper reports no such transfer test, no baseline comparison, and no error bars, the central claim is not supported by the presented evidence. The reader's REJECT verdict remains appropriate; no adjustment is needed.","tokens_in":14091,"tokens_out":4083,"duration_ms":38886,"concrete_test":"Perform a leave-one-correspondence-out evaluation. Train the Siamese model on pairs built from only three of the four Qaidam–Mars correspondences (e.g., A–A', B–B', C–C'), withholding all images and labels of region D and D'. Then rank the four Mars regions against the Qaidam D-region images using the learned Euclidean distance; check whether D–D' is ranked first. Repeat for each held-out correspondence. If ranking accuracy drops to chance, the §5.3 matches are training artifacts rather than transferable geomorphological similarity.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim—that the CNN similarity metric can select priority Mars landing sites by finding geomorphological analogues in the Qaidam Basin—rests on the minimum distances of §5.3 (A–A' 0.11, B–B' 0.26, C–C' 0.38, D–D' 0.05) and on the 99.82% test accuracy. Those numbers are not evidence of predictive transfer. In §5.1, training pairs are constructed by taking one image from each Qaidam region and pairing it with 'all images from the corresponding research area on Mars' as similar, and with images from different Mars research areas as dissimilar. The 'corresponding' Mars areas are exactly A', B', C', D'—the same regions later reported as matched in §5.3. The network is thus trained to reduce the distance for these four manual correspondences and then evaluated on those same correspondences. The 9:1 split is applied to pairs drawn from only four positive region-pairs after 10× augmentation, so the same source images (via augmented copies) occur in both train and test; the 99.82% accuracy can reflect near-duplicate recognition. No held-out Mars region, unlabelled terrain, or geographical split is tested, so the landing-site selection claim is unsupported by the reported experiment.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a Siamese convolutional neural network trained with a contrastive loss to learn a similarity metric between terrestrial and Martian surface images, and applies it to four hand-selected geomorphic regions in the Qaidam Basin and four corresponding regions on Mars. The authors report a 99.82% test-set accuracy and minimum Euclidean distances for the pairs A-A', B-B', C-C', and D-D', concluding that the model identifies the same analogue correspondences that geologists selected and can therefore be used to select priority Mars landing sites. The geological background on evaporites, halite, and fluid inclusions as biosignature repositories is appropriate for an astrobiology-motivated analogue study, but the supporting experiment is severely compromised by the way training labels are constructed and by the absence of any held-out evaluation on new terrain.","tokens_in":14344,"tokens_out":4202,"duration_ms":42197,"significance":"If the method were shown to transfer to unseen Martian terrains, it would offer a useful quantitative complement to expert geological judgement for planetary analogue site selection. The paper is, to the best of my knowledge, novel in applying deep metric learning to Mars-Earth geomorphological matching, and the underlying geological motivation (evaporites and halite fluid inclusions as preservation windows) is well referenced. The authors also make the standard choice of a Siamese architecture and contrastive loss, following Hadsell et al. (2006). However, the experimental validation is not independent of the label construction: the four positive correspondences used to build training pairs are the same four correspondences later reported as the minimally distant matches, and the train/test splitting procedure allows augmented copies of identical source images to appear in both sets. As a result, the reported accuracy and distance values do not provide evidence for the paper's central claim that the model can select landing sites by discovering analogues in unlabelled Mars imagery.","major_comments":[{"comment":"The experimental evaluation is circular. In §5.1, similar pairs are defined by taking one image from each Qaidam region and pairing it with 'all images from the corresponding research area on Mars' as similar, and with images from the other three Mars regions as dissimilar. The 'corresponding' Mars areas are exactly the four regions later reported in §5.3 to have the smallest distances (A-A', B-B', C-C', D-D'). The network is therefore trained to minimize distances for these four manual correspondences and then evaluated on those same correspondences. The reported minimum distances 0.11, 0.26, 0.38, and 0.05 are not a test of whether the model has discovered the same analogue pairings independently; they are a measure of how well the model reproduces the training labels. This undermines the landing-site selection claim.","section":"§5.1 and §5.3"},{"comment":"The 9:1 random split of labeled pairs does not prevent data leakage. The images are augmented by a factor of 10 through rotation, blurring, scaling, and grayscaling, and the pairs are built from these augmented images. A random split of pairs, rather than a split by source image identity or by geographic sub-region, means that augmented copies of the same original image can appear in both training and test sets. The reported 99.82% test accuracy can therefore reflect near-duplicate recognition rather than generalization to new terrain. To validate the method, the authors would need to evaluate on geographically disjoint areas or on Mars regions not used during training.","section":"§5.1"},{"comment":"The paper's central claim that the model can select priority landing sites on Mars is not supported by the experiment. All four Mars regions (A', B', C', D') were pre-selected by the authors as known or suspected analogues based on prior geological literature, and the experiment only asks whether the model reproduces those four correspondences. No test is conducted on unlabelled Mars terrains, no candidate set is ranked beyond the four pre-chosen regions, and no external validation against expert judgement on a held-out regional set is provided. The conclusion 'By identifying different landforms, priority landing sites on Mars can be selected' therefore extends beyond what the experiments demonstrate.","section":"Abstract and §5.3"},{"comment":"The equations defining the distance function and the contrastive loss are missing. In §4.2 the text states that the Euclidean distance 'can be defined as:' followed by an empty reference '(1)', and in §4.3 the loss function is referenced as equations (2), (3), and (4) without any of the formulas being displayed. Since these definitions are the core of the method and are needed to assess the correctness of the training procedure, the manuscript must show the explicit equations for D_W, the contrastive loss, and its partial losses L_S and L_D.","section":"§4.2 and §4.3"}],"minor_comments":[{"comment":"There are numerous typographical and grammatical errors, including 'Qadam' for 'Qaidam', 'Accrodingly', 'geomorpholigcal', 'analaogue', 'cystals', and inconsistent punctuation around citations. These should be corrected.","section":"Throughout"},{"comment":"The coordinates for region D' are given as '(341.61°E, 50.70°N)', which reverses the latitude/longitude order used for the other regions (e.g., A'=10.461°S, 176.446°E). Please standardize the coordinate format.","section":"§5.3"},{"comment":"The phrase 'Their differences are all less than 0.5, indicating a very high similarity' introduces an arbitrary threshold of 0.5 with no justification or calibration. Please justify the threshold or present the full distributions of distances rather than only minima.","section":"§5.3"},{"comment":"The paper reports a single final loss value of 0.0088 and a test accuracy of 99.82%, but provides no training curves, validation loss, or error analysis. Adding convergence diagnostics and a confusion matrix or per-class accuracy would help the reader assess the training process.","section":"§5.2"},{"comment":"The dataset description is incomplete: the number of distinct images per region, the number of pairs generated, and the exact division of images among the four regions are not reported. This information is essential for reproducibility.","section":"§4.1"},{"comment":"The manuscript does not state whether code or trained model weights will be made available. Given the reproducibility standard in deep learning research, a data/code availability statement should be added.","section":"Experimental Methods"}],"recommendation":"reject","confidential_remarks":"The experimental design is circular: the positive correspondences used to build the training labels in §5.1 are the same as the 'discovered' matches in §5.3, and the random pair split with heavy augmentation permits near-duplicate leakage. Because the only quantitative evidence supporting the landing-site selection claim is invalid, the paper cannot be accepted without a fundamentally new evaluation that tests generalization to held-out Mars regions and independent ground-truth correspondences. The missing equations and extensive language issues further reduce the manuscript's readiness. I recommend rejection, although a substantially revised version with a proper validation strategy could be reconsidered as a new submission."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a useful idea wrapped in a circular experiment. The new bit is applying a Siamese CNN with contrastive loss to Mars–Earth landform matching, and the geological background on Qaidam Basin evaporites as a Mars analogue is competent and well referenced. But the headline matching distances are not evidence of predictive ability, because the evaluation is built from the same four manual correspondences used to construct the training labels.\n\nWhat it does well: the paper makes a genuine case for why quantitative, repeatable analogue selection would help landing-site screening, and the descriptions of halite, fluid inclusions, and the Qaidam Basin literature are solid. It is also transparent about the dataset construction, which makes the flaw easy to trace.\n\nWhere it falls down: in §5.1 the training pairs are built by pairing one Qaidam image from each region A–D with all images from the corresponding Mars region A'–D' as similar. Section 5.3 then reports that exactly A–A', B–B', C–C', D–D' come out as closest. That is the network reproducing its own labels. The 9:1 train/test split is applied after 10× augmentation, so augmented copies of the same source image can appear in both sets; 99.82% accuracy is consistent with near-duplicate recognition rather than geomorphological understanding. There is no held-out Mars region, no baseline, no error bars, and no code or data to assess how much of the result is image-level memorization.\n\nThis is a fixable design problem, not a broken concept. The proper test would train on some areas and evaluate on held-out pairs that never contributed to labels, with a spatial split that keeps augmented versions of a source image in one fold, plus simple baselines like pixel-level distance or a generic pretrained CNN embedding. Until that is done, the landing-site selection claim is unsupported.\n\nThe paper deserves a serious referee, because the application is timely and the flaw is instructive, but my own verdict is that the experimental core needs to be redesigned before the claims can stand. It is a good reading-group example of circular validation in learned similarity.","headline":"A useful idea wrapped in a circular experiment: the reported matches reproduce the same manual correspondences used to build the training labels, so the landing-site claim is unsupported as presented.","tokens_in":14938,"tokens_out":2934,"would_cite":false,"duration_ms":28578,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A Siamese CNN trained on Mars and Qaidam Basin images learns a quantitative geomorphological similarity metric, recovers the four known analogue pairings, and uses the resulting distances to propose priority landing sites.","keywords":["Mars landing sites","Qaidam Basin","evaporites","Siamese network","contrastive loss","geomorphological similarity","deep learning","planetary analogues"],"falsifier":"Take the trained network and feed it Martian terrain images from regions that were not among the four training targets. If its closest Qaidam Basin matches do not match an independent classification (for example, expert geomorphological labels or hyperspectral evidence of evaporites), the landing-site-selection claim fails. A faster check is label permutation: retrain with the Mars–Qaidam pairings randomly shuffled; if accuracy stays near 99%, the metric is not learning the intended analogue structure.","tokens_in":13831,"feed_emoji":"🪐","tokens_out":7998,"duration_ms":74851,"temperature":0.7,"pith_summary":"This paper proposes using a Siamese convolutional neural network trained with a contrastive loss to turn visual similarity between Mars and Qaidam Basin surface images into a quantitative number. The network embeds each 100×100 image patch into a five-dimensional feature vector, and the Euclidean distance between two vectors is the terrain-similarity score. On four geomorphic classes, the closest matches recovered are exactly the pairings a geologist would choose: yardangs, linear dunes, barchan chains, and polygonal salt crusts, with distances 0.11, 0.26, 0.38, and 0.05. The paper's aim is to make Mars landing-site selection an automatic, repeatable ranking procedure, with priority going to evaporite polygonal terrains where halite fluid inclusions could preserve signs of life.","feed_headline":"CNN pairs Mars landforms with Qaidam Basin analogues","feed_subtitle":"Four matching regions score below 0.5, flagging evaporite polygons as priority landing spots.","key_machinery":"The mechanism is a Siamese network: two identical convolutional networks that share the same weights and map an input image to a five-dimensional feature vector, with similarity defined as the Euclidean distance $D_W(X_1,X_2)=\\|G_W(X_1)-G_W(X_2)\\|_2$. Training uses a contrastive loss (the DrLIM formulation from Hadsell et al., 2006, applied to pairs labeled similar or dissimilar by the researchers): the loss shrinks the distance for same-location pairs and, for different-location pairs, increases it up to a margin $m$. This learned embedding is what carries the argument, because after training, matching a new Mars image to a Qaidam image is just feature extraction plus a distance lookup, with no retraining.","core_discovery":"The central claim is that a weight-sharing Siamese CNN with a contrastive loss can learn a globally coherent nonlinear mapping in which images from the same landform type are pulled close together and images from different types are pushed apart, so that a new terrain image can be matched to its best Earth analogue by nearest-neighbour distance alone. The paper reports that the trained model reaches a test-set accuracy of 99.82% and that the four minimum distances between the Qaidam sub-regions and the four Martian target regions are all below 0.5, with the polygonal-structure pairing D-D' the closest at 0.05. In the authors' words, 'By identifying different landforms, priority landing sites on Mars can be selected.' The application-level corollary is that evaporite-rich polygonal terrain in the Qaidam Basin is the best Earth model for deciding where to land a Mars rover in search of preserved biosignatures.","pith_inferences":["The reported 99.82% accuracy is measured on random splits of pairs built from the same four labelled Mars–Qaidam correspondences, so I read it as evidence that the network reproduces the manual labels rather than proof that it can discover new analogues; a held-out-region test would separate the two.","Because the embedding is five-dimensional and the inputs are 100×100 optical patches, the similarity is likely dominated by texture and pattern statistics; folding in mineralogical or topographic channels would test whether the same metric tracks geological process, not just visual resemblance.","Tiling all of Mars through the same embedding would yield a global distance-to-Qaidam map, turning the four point matches into a planet-wide similarity surface that could be inspected for unanticipated analogues."],"forward_implications":["Landing-site preselection becomes a quantitative ranking: a future mission can tile a candidate region into patches, embed them, and rank them by distance to terrestrial analogue images.","The metric can serve as a validation tool for geologist-chosen analogues, since it recovered the A-A', B-B', C-C', and D-D' pairings with distances below 0.5.","Polygonal salt-crust terrains are singled out as the highest-priority targets, because halite and gypsum can carry fluid inclusions that may preserve biological signatures over geological time.","The same Siamese pipeline transfers to other analogue deserts (for example, Atacama or Antarctic dry valleys) and to other planetary bodies, provided paired training images exist."],"supporting_citations":[{"why":"Supplies the DrLIM method for learning an invariant mapping from neighborhood relationships, which the network uses to embed images.","marker":"Hadsell et al., 2006"},{"why":"Introduces the Siamese architecture with shared weights that the paper adopts for comparing image pairs.","marker":"Bromley et al., 1993"},{"why":"Provides the discriminative contrastive-loss formulation for similarity metric learning.","marker":"Chopra et al., 2005"},{"why":"Establishes the western Qaidam Basin as a Martian environmental analogue and is a source for the landform comparisons and Mars imagery.","marker":"Anglés and Li, 2017"},{"why":"Documents Qaidam Basin as a Mars-analogue site and supplies the geomorphic feature set used to define the four study regions.","marker":"Xiao et al., 2017"},{"why":"Provides the Earth–Mars evaporite comparison and the basis for treating polygonal salt-crust structures as priority astrobiology targets.","marker":"Zheng et al., 2013"},{"why":"Grounds the convolutional-network feature extraction that maps raw images to low-dimensional vectors.","marker":"LeCun et al., 2015"},{"why":"Justifies the use of small 3×3 convolution kernels in the embedding network.","marker":"Simonyan and Zisserman, 2014"}],"fun_headline_variants":["CNN matches Mars terrain to Qaidam evaporite analogs","Deep Siamese CNN picks Mars landing sites from Qaidam analogues","Evaporite polygons in Qaidam Basin flagged as Mars landing targets","CNN's Qaidam Basin similarity maps Mars landing priorities","Siamese CNN links Mars polygons to Qaidam salt flats"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the four manually chosen Mars–Qaidam pairings used to label the training data are the right ground truth, so the network is only learning to reproduce those pairings rather than to recognise geomorphological similarity that would transfer to unlabelled Mars regions.","fun_headline_variants_meta":{"raw":{"variants":["CNN matches Mars terrain to Qaidam evaporite analogs","Deep Siamese CNN picks Mars landing sites from Qaidam analogues","Evaporite polygons in Qaidam Basin flagged as Mars landing targets","CNN's Qaidam Basin similarity maps Mars landing priorities","Siamese CNN links Mars polygons to Qaidam salt flats"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000961,"raw_usage":{"total_tokens":4095,"prompt_tokens":950,"completion_tokens":3145,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":566,"completion_tokens_details":{"reasoning_tokens":3058}},"tokens_in":566,"tokens_out":3145,"duration_ms":20069,"temperature":1.0,"reasoning_tokens":3058,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T22:52:13.699585+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the trained network and feed it Martian terrain images from regions that were not among the four training targets. If its closest Qaidam Basin matches do not match an independent classification (for example, expert geomorphological labels or hyperspectral evidence of evaporites), the landing-site-selection claim fails. A faster check is label permutation: retrain with the Mars–Qaidam pairings randomly shuffled; if accuracy stays near 99%, the metric is not learning the intended analogue structure.","supporting_citations":[],"review_version":1}