{"id":"ee2f0ad0-4454-40db-bdd1-73ee9af3a753","arxiv_id":"2608.09276","paper_version":1,"verdict":"ACCEPT","confidence":"HIGH","novelty_score":7.0,"correctness_risk":"low","formal_verification":"none","parameter_count":4,"one_line_summary":"SelenoVLM generates raster-verifiable lunar geological descriptions, yet its numeric ages default to the global mare prior unless the published age is injected through retrieval.","lead":"A robot geologist for the Moon reads maps and writes descriptions that check against the data, but its age estimates simply repeat the average age unless the published age is handed to it through a search step. This shows which tasks an AI geologist should do with vision and which it should do by citing the literature.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Free-running tile-swap grounding is not measured; the central 'tile-grounded prose' claim rests on a teacher-forced flip rate that may be dominated by the deterministic opener facts.","rationale":"I considered the reader's weakest assumption—that the Hiesinger/USGS/Neukum references could be systematically wrong. That is a legitimate external-validity caveat, but it is not load-bearing for the architectural claims: the paper's central conclusions are about causal dependence of generations on the tile and about where numeric ages are recovered (vision vs retrieval), not about the truth of the published chronologies. The open-book transfer is explicitly reported as transfer fidelity, and the closed-book failure is relative to the same reference set; a uniform shift in the references would not change the qualitative findings. The concern I raise instead is internal: the measurement of the paper's main positive claim. The tile-swap flip rate is teacher-forced and likely weighted toward the deterministic opener facts, while the only free-running intervention is vision ablation. Without a free-running tile-swap generation comparison, the statement that the interpretive prose is tile-grounded goes beyond the reported evidence. The proposed test is directly feasible from the released code and would either confirm the headline claim or require it to be narrowed. Because this is a measurement gap rather than a demonstrated failure, the appropriate verdict is acceptance conditional on the free-running tile-swap analysis. The age-related negative result (Sec. 1.4) and the open-book transfer (Sec. 1.5) are supported by multiple independent controls and are not affected by this concern.","tokens_in":31304,"tokens_out":24765,"duration_ms":259475,"concrete_test":"Use the released checkpoint to generate free-running responses for the 500 test tiles under two conditions: the tile's own visual tokens and a randomly paired other tile's visual tokens, with identical prompts and decoding. Compute the mean token-set Jaccard distance and ROUGE-L between the paired generations, and compare with (a) the reported vision-ablated Jaccard distance (0.824) and (b) an inter-tile baseline (distance between two different tiles' own-input generations). Stratify the comparison by whether the target opener string is unique or duplicated. If the swapped-tile distance is near the original-vs-itself floor (≈0) or far below the vision-ablated distance, the prose is not tile-grounded and the headline claim should be narrowed to the opener facts; if it approaches the inter-tile baseline, the prose-grounding claim is confirmed.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper's central causal-grounding metric, f_swap (Eq. 1), is computed teacher-forced: it measures next-token argmax flips on the target tokens. The supervision targets open with four deterministic raster-derived facts (albedo, relief, roughness, 415/750 ratio) that are the most tile-specific content in the target. These opener tokens are likely to dominate f_swap, and the teacher-forced protocol supplies the ground-truth opener prefix when scoring later positions, so a model that reads vision only to emit the opener facts and then generates generic prose conditioned on that prefix could still show a high f_swap. The only generation-level intervention reported is vision ablation (Jaccard distance 0.824); the corresponding generation-level tile-swap comparison is missing. Free-running metrics such as unit naming, age coverage, and stated-fact fidelity confirm that specific slots depend on the tile, but they do not establish that the interpretive prose—the sentences about stratigraphy and formation history—is tile-grounded. The multi-turn dialogue is a single illustrative example. The claim 'tile-grounded geological prose' therefore rests on an indirect measurement. A further weakness: only 44% of intra-mare targets are distinct at the served granularity, so the design condition 'cannot be reproduced without reading the tile' is only partially satisfied; the coarsening ablation shows the opener granularity matters, but it does not identify how much of the observed f_swap is attributable to the opener.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents SelenoVLM, a two-stage vision-language system for lunar mare stratigraphy. Stage A is a multimodal masked autoencoder pretrained on co-registered WAC imagery, GLD100 elevation, Clementine ratio composites, the USGS geologic map, and weak text captions. Stage B freezes that encoder and bridges it into a Qwen2.5-7B language model with a Perceiver resampler, gated cross-attention, and LoRA, supervised by distilled paragraph targets that open with deterministic raster-derived facts (albedo, relief, roughness, 415/750 ratio) and, on mare tiles, an age sentence. The paper reports held-out causal-grounding flip rates, stated-fact fidelity, stratigraphically gated age emission, and a closed-book negative result in which stated numeric ages track the global mare prior rather than local evidence, supported by probes that localize the dating signal up to the resampler bottleneck. In the open-book regime, the same generation channel transfers retrieved published ages faithfully and causally. The central claim is a division of labor: site evidence should enter through vision and be verifiable against the rasters, while quantitative chronology should be retrieved from the literature.","tokens_in":31574,"tokens_out":7875,"duration_ms":86435,"significance":"If the claims hold, the paper makes a valuable methodological contribution to grounded scientific generation. Its strengths include a spatially disjoint block split, cluster bootstrapping, constant-prior baselines for age recovery, leave-one-region-out controls, a probe battery with shuffle controls, explicit labeling of the open-book agreement as circular transfer fidelity, and a clear negative result on closed-book numeric age dating that is unusual and informative. The paper is also unusually explicit that perplexity and ROUGE on distilled targets are only fit diagnostics on circular supervision. The released code and weights, together with the detailed target-construction protocol, make the claims checkable. The main weakness is that the positive 'tile-grounded prose' claim rests on a teacher-forced flip-rate metric over all target positions rather than on a free-running generation-level tile-swap comparison; this is addressable and does not undermine the negative results.","major_comments":[{"comment":"The central causal-grounding statistic f_swap is computed teacher-forced over all target positions, including the deterministic opener-fact tokens that are almost guaranteed to flip under a tile swap. The manuscript does not report the flip rate restricted to the interpretive prose positions, and the only generation-level intervention is vision ablation (Jaccard distance 0.824), not a generation-level tile swap. As written, the evidence does not rule out a model that reads the tile only to emit the opener facts and then produces generic prose conditioned on that prefix. Since the Discussion's positive claim is specifically about 'tile-grounded geological prose,' please report a per-position breakdown of f_swap (opener-fact tokens versus the remaining target tokens) or, preferably, a free-running generation-level tile-swap metric, for example token-set Jaccard distance between generations conditioned on one tile's visual tokens and another tile's visual tokens with no teacher forcing. This is directly computable from the existing pipeline and would make the central claim directly measured rather than indirect.","section":"Sec. 1.1 and Sec. 3.8, Eq. (1)"},{"comment":"The design premise that supervision targets 'cannot be reproduced without reading the tile' is only partially realized: at the served granularity, only 44% of intra-mare targets are distinct, and ordinal wording alone leaves only 35% distinct. This means a substantial fraction of targets are shared across tiles, so high f_swap on those tiles cannot be attributed to tile-specific grounding; the model could be reproducing a memorized target. Please report the core grounding metrics separately for target-distinct versus target-duplicate tiles, or otherwise quantify how much of the observed f_swap comes from the 44% unique targets. This would also sharpen the interpretation of the coarsening ablation in Extended Data Table 3, where the distinctness fraction collapses to 13%.","section":"Sec. 3.5, target construction"},{"comment":"The 'probe-generation gap' is used to localize the numeric-dating failure to the language pathway, but the generation slope (-0.054) is measured on the 93-tile polygon-accurate set while the probe slope (0.143) is measured on a separate 20,956-tile leave-unit-out set. These sets differ in tile composition and in ground-truth convention, so the gap could partly reflect set differences rather than the generation channel. The broader localization is supported by the in-loop age head result, but the specific quantitative comparison should be made on a common evaluation set, or the text should state explicitly why the cross-set comparison remains valid.","section":"Sec. 1.4, Extended Data Tables 5 and 6"}],"minor_comments":[{"comment":"The served checkpoint is described as selected at the 'knee' of the causal-grounding versus target-adherence trade-off, but no formal criterion is given. A reproducible rule, such as maximizing tile-swap flip subject to a minimum fact-fidelity threshold, would strengthen the claim that checkpoint selection is principled rather than discretionary.","section":"Sec. 3.4, checkpoint selection"},{"comment":"The qualitative chat examples are selected as best-of-N candidates and this is disclosed, but a randomly sampled or greedy decode would help calibrate typical free-running behavior and avoid the impression of cherry-picking.","section":"Extended Data Fig. 1 and Supplementary Fig. S4"},{"comment":"The phrase 'accurate describe' in the abstract and 'deployed system states correct published ages' in Sec. 1.5 could be read as claiming geophysical correctness. Since the evaluation is against the published crater-count record, consider tightening the wording to 'agrees with the published chronology' to keep the claim coextensive with the measurement.","section":"Abstract and Sec. 1.5"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is publishable in substance: the negative closed-book age result, the probe battery, and the circularity labeling are exemplary. My recommendation is driven by one missing control and one reporting gap concerning the central positive claim. If the authors supply a generation-level tile-swap metric and a per-segment flip-rate breakdown, and clarify the probe-generation comparison set, I would expect to support acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe paper is worth your attention for one result: it measures, rather than asserts, where a machine geologist should get its numbers. Under closed book, the model's stated ages cluster at the global mare prior and the apparent skill vanishes under leave-one-region-out. Open book, the same generation channel follows injected record ages with a causal perturbation slope of 0.711. That dissociation is the real contribution, and it's backed by constant-prior baselines, spatially disjoint splits, cluster bootstrap, and a probe battery localizing the signal loss to the language pathway. The target-design experiment (deterministic raster opener vs coarsened bins) is also a clean demonstration that supervision granularity controls causal grounding.\n\nThe stress-test note about f_swap is right to poke at it. The tile-swap flip rate is teacher-forced and computed over target positions; since every target opens with deterministic raster facts, those opener tokens are the most tile-specific part and can dominate the flip rate. The paper doesn't report f_swap separately for opener vs prose positions, and the generation-level intervention is vision removal, not a swap to another tile. So 'tile-grounded geological prose' is a bit stronger than the direct evidence. The unit naming (85.8%) and stated-fact fidelity show the model reads the tile for specific slots, and the coarsening ablation shows the opener design drives the grounding signal, but the interpretive sentences about stratigraphy and formation history are not quantitatively shown to depend on tile identity. That's a genuine soft spot, though not a fatal one.\n\nAlso minor: the checkpoint knee is not formally specified, and the figure examples are selected best cases (the paper says so). The reliance on published chronologies as ground truth is inherent to the validation design; if Hiesinger's ages were systematically wrong, the transfer fidelity would be fidelity to a flawed record. That's a premise, not a flaw.\n\nOverall, the core claim—numeric ages should be retrieved, not generated from vision—holds up. The prose-grounding claim needs one more experiment (generation-level tile swap, and position-split f_swap) before I'd take it at full strength. Send it to review; the authors have the discipline to address that.\n\nFor your reading group: maybe, if you work on VLM evaluation or scientific grounding. I'd cite it for the evaluation protocol.","headline":"Careful empirical paper dissociating numeric age retrieval from visual grounding; the central claim holds, but the prose-grounding evidence is slightly weaker than the authors' wording.","tokens_in":32102,"tokens_out":4803,"would_cite":true,"duration_ms":48744,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A lunar geology model produces verifiably grounded prose from maps, yet its numeric ages track the global mare prior until the published chronology is retrieved.","keywords":["multimodal vision-language models","lunar mare stratigraphy","grounded generation","crater chronology","retrieval-augmented generation","causal grounding","masked autoencoder","lunar geology"],"falsifier":"Train the same frozen vision tower and bridge with all age sentences removed from the training targets, then evaluate closed-book stated ages against polygon-accurate ground truth under a leave-one-region-out split; if the model still beats the constant-prior baseline, the claim that numeric ages cannot be extracted through the generation channel would be falsified.","tokens_in":31103,"feed_emoji":"🌙","tokens_out":7312,"duration_ms":68784,"temperature":0.7,"pith_summary":"This paper tries to build a machine 'geologist' that writes grounded geological interpretations of lunar terrain from co-registered topographic, spectral, and geologic maps, and to measure where that grounding actually holds. Its central claim is that a vision-language model can produce text that depends on the observed tile rather than on memorized generalities, provided the training targets contain numeric facts that cannot be reproduced without reading that tile. The same target design exposes a sharp limit: stated numeric ages collapse to the global mare age prior even though the visual representation carries a usable dating signal. Adding an open-book retrieval step lets the identical generation channel transfer published ages faithfully and causally, which yields the paper's main architectural conclusion: site evidence must be read from local data, while quantitative historical context must be cited from the scientific record.","feed_headline":"Moon-geology machine grounds descriptions, cites ages","feed_subtitle":"Descriptions check against the data; ages default to the global mare prior until the published record is retrieved and cited.","key_machinery":"The load-bearing object is the supervision target: each training paragraph opens with four deterministic facts computed from the tile's own rasters at fixed quantization, followed on mare tiles by an age sentence gated on the mapped geology, under the design rule that a target must not be reproducible without reading the tile. Causal grounding is measured by the tile-swap flip rate, the fraction of next-token decisions that change when the tile's visual tokens are replaced by another tile's tokens; this distinguishes a model that reads the site from one that uses vision only as an on/off cue. The open-book condition adds a retriever over the published mare-basalt chronologies and map-unit descriptions, whose injected record can be perturbed to prove that the transferred age is causally driven by the retrieved text.","core_discovery":"The paper shows that the supervision target, not the architecture, decides whether a vision-language system is verifiably grounded. With a frozen self-supervised vision tower and a standard bridge into a language model, prose that opens with deterministic raster-derived facts (albedo, elevation relief, roughness, and the Clementine 415/750 ratio) makes the generated text change when the input tile is swapped and makes the stated facts verifiable against the rasters. The same discipline demonstrates the precise negation: closed-book numeric ages cluster at the global mare prior, a leave-one-region-out control attributes apparent skill to memorized regional structure, and probe experiments localize the loss to language-side extraction rather than to representation. In the open-book regime, when a retrieved record supplies the published age, the same model transfers that value with a causal perturbation slope of 0.711. The claim is a division of labour: descriptive and stratigraphic interpretation can be grounded in vision, while numeric chronology must be retrieved and cited.","pith_inferences":["If the external chronologies used as ground truth are themselves systematically biased, both the supervision targets and the open-book retrieval inherit that bias; the paper's separation of visual evidence from numeric context would still hold, but the 'published age' being transferred could be wrong.","The same target-design principle could be tested on other visually homogeneous planetary surfaces, where a similar prior-tracking failure for quantitative values is plausible unless retrieval is integrated.","The probe evidence points toward a decoupled regression head on the frozen representation, aggregating over mapped units, as a design that might recover ages where the language channel fails; the paper identifies this direction but does not build it."],"forward_implications":["A paragraph-producing geological interpretation system can be made verifiable without a new architecture by putting tile-specific numbers in the supervision targets and scoring with swap and removal ablations.","The same system is ready to carry descriptive and stratigraphic content, because its stated facts check against the input rasters and its unit-naming and highland-silence behaviour follow the stratigraphic model.","Numeric ages should not be generated from vision in this architecture; they should be supplied by a retrieval layer and cited, with deployed accuracy bounded by retrieval quality rather than by the model.","The evaluation discipline transfers to any vision-language system over visually homogeneous scenes: validation loss on distilled targets is not a grounding signal, and checkpoint selection must trade target adherence against causal grounding."],"supporting_citations":[{"why":"Supplies the frozen multimodal masked autoencoder vision tower that the language bridge reads in stage B.","marker":"[Sander et al., 2026]"},{"why":"Provides the 4M-style any-to-any masked modeling architecture used for stage-A pre-training over the co-registered lunar modalities.","marker":"[Mizrahi et al., 2023]"},{"why":"Supplies the gated cross-attention and Perceiver-resampler bridge design that connects the frozen vision tower to the frozen language model.","marker":"[Alayrac et al., 2022]"},{"why":"Provides the published mare-basalt model ages and stratigraphy that form the age supervision targets and the open-book retrieval corpus.","marker":"[Hiesinger et al., 2011, 2023]"},{"why":"Defines the lunar crater-chronology function used for the chronology-aligned target variant and the epoch conventions.","marker":"[Neukum et al., 2001]"},{"why":"Supplies the USGS Unified Geologic Map unit labels and polygons used for age gating, unit naming, and polygon-accurate ground truth.","marker":"[Fortezzo et al., 2020]"},{"why":"Supplies the global crater database whose counts enter the chronology variant and the probe ceiling estimate.","marker":"[Robbins, 2019]"},{"why":"Establishes the retrieval-augmented generation paradigm that the open-book regime applies to the published chronology.","marker":"[Lewis et al., 2020]"}],"fun_headline_variants":["Moon geology AI: descriptions grounded, ages from retrieved records","Vision-language geologist: local data for descriptions, citations for ages","AI lunar stratigraphy: visual grounding works, ages need lookup","Supervision target, not architecture, grounds lunar geology AI"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The argument assumes the external chronologies used for supervision and evaluation—the published crater-count model ages, the geologic-map unit labels, and the lunar chronology calibration curve—are correct readings of lunar history; if those are systematically wrong, both the training targets and the open-book transfer measurements inherit that error.","fun_headline_variants_meta":{"raw":{"variants":["Moon geology AI: descriptions grounded, ages from retrieved records","Vision-language geologist: local data for descriptions, citations for ages","AI lunar stratigraphy: visual grounding works, ages need lookup","Supervision target, not architecture, grounds lunar geology AI"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000534,"raw_usage":{"total_tokens":2539,"prompt_tokens":888,"completion_tokens":1651,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":504,"completion_tokens_details":{"reasoning_tokens":1581}},"tokens_in":504,"tokens_out":1651,"duration_ms":12143,"temperature":1.0,"reasoning_tokens":1581,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T20:18:14.318912+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the same frozen vision tower and bridge with all age sentences removed from the training targets, then evaluate closed-book stated ages against polygon-accurate ground truth under a leave-one-region-out split; if the model still beats the constant-prior baseline, the claim that numeric ages cannot be extracted through the generation channel would be falsified.","supporting_citations":[],"review_version":1}