{"id":"10b0f28a-0bb4-4a20-8c2f-066b5451701d","arxiv_id":"2607.19370","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A 4.3-million-parameter HEALPix-aware U-Net trained on EMARS reanalysis forecasts Mars temperature and winds stably for 25 hours at ~110 km resolution.","lead":"A compact neural network trained on Mars weather reanalysis can forecast temperature and winds one hour ahead and stay stable for a full Martian day. The model uses a sphere-aware grid that avoids map distortions, and runs hundreds of times faster than traditional physics-based simulations.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Missing persistence/climatology baselines and unspecified split protocol make the reported skill metrics uninterpretable as evidence of learned atmospheric dynamics.","rationale":"The reader's weakest assumption focused on reanalysis bias, which affects external validity. I argue the more load-bearing concern is internal validity: the paper reports absolute MSE and RMSE without comparison to persistence or climatology, and it does not specify the temporal structure of the train/validation split. If the split is random, the model can interpolate between nearly identical adjacent hours in the reanalysis, making the low validation error an artifact of memorization. Even if the split is temporal, the lack of a persistence baseline means there is no evidence the model has learned dynamics rather than the trivial autocorrelation of the atmosphere. This is directly relevant to the central claim of capturing synoptic-scale dynamics. The paper itself acknowledges the missing persistence baseline (Section 3.2) and the uncontrolled architecture comparison (Section 3.1), so these are known limitations. Nevertheless, they are severe enough that the current evidence cannot substantiate the headline claim. I therefore maintain the reader's CONDITIONAL verdict, but for a different primary reason. The concrete test I propose (adding baselines and a temporal split) would settle the issue.","tokens_in":10077,"tokens_out":6004,"duration_ms":65529,"concrete_test":"Compute persistence (x_{t+1}=x_t) and climatology (monthly/hourly per-pixel mean) baselines on the same validation set and compare RMSE/MSE to the model's 1.58e-5 validation MSE. Also retrain the HEALPix U-Net with a strict temporal split (e.g., train MY24-31, validate MY32-33) under identical settings. The central claim is confirmed only if the model's t+1h MSE is at least ~20% lower than persistence and the temporal-split MSE is comparable to the reported value (not inflated by an order of magnitude).","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that the model learned a dynamical propagation operator (Section 3.2) is not supported because no persistence or climatology baseline is reported. Section 3.2 explicitly defers this comparison to future work. For an atmosphere with high temporal autocorrelation, a persistence forecast may achieve RMSE close to the reported 0.004 at t+1h; without that baseline, the absolute MSE cannot distinguish skill from trivial self-similarity. Additionally, Section 2.3 gives only an '80/20 train/validation split' without stating whether it is temporal or random. If random, neighboring hours in train and validation sets are nearly identical, allowing the model to memorize rather than learn dynamics. These two omissions undermine both the absolute accuracy and the architecture-comparison claims.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents Aircast-Mars, a 4.3M-parameter HEALPix-aware 2D U-Net trained on EMARS v1.0 hourly temperature, zonal wind, and meridional wind fields regridded to Nside=64 (~110 km). The model is trained to predict the next hourly state. The manuscript reports a best validation MSE of 1.58e-5, stable autoregressive rollouts for one Martian sol (RMSE growing from ~0.004 at t+1h to ~0.031 at t+25h), an order-of-magnitude improvement over a 30M-parameter 3D U-Net baseline, and a 0.5-second per-step GPU inference time. The central claim is that geometry-respecting HEALPix-aware convolutions capture synoptic-scale Martian atmospheric dynamics with high efficiency.","tokens_in":10244,"tokens_out":2935,"duration_ms":32604,"significance":"If the claims are substantiated, Aircast-Mars would be a useful fast surrogate for Mars weather emulation, with plausible applications to mission planning, ensemble generation, and onboard forecasting. The paper is honest about several limitations, including normalized units, reanalysis bias, and the absence of aerosol channels. However, the headline skill claims are currently not supported by controlled comparisons or standard baselines. The 3D U-Net comparison is explicitly uncontrolled, no persistence or climatology baseline is reported, and the data-split protocol is not specified. The code and data are not yet available, so the numerical results cannot be independently verified from the manuscript alone.","major_comments":[{"comment":"The central 'geometry matters' claim rests on an uncontrolled comparison. The 3D U-Net was trained at Nside=8 with 20 epochs and lr=1e-3, while the HEALPix U-Net ran at Nside=64 with 50 epochs and lr=1e-4. The HEALPix U-Net at Nside=16 achieves only 1.4e-4, identical to the 3D U-Net, so the reported order-of-magnitude gain could be due to resolution, training length, or optimizer settings rather than HEALPix-aware padding. The manuscript acknowledges this in Section 3.1, but Section 5 still lists 'Geometry matters' as a key finding. This overstates the evidence. A controlled comparison (matching resolution, epochs, and learning rate, or ablating inter-face padding) is needed before the architecture claim can be accepted.","section":"Section 3.1, Table 1"},{"comment":"The data-split protocol is underspecified. The text says only '80/20 train/validation split' without stating whether it is temporal or random. If the split is random, then neighboring hourly states from the same sols are likely in both training and validation sets, allowing the network to memorize rather than learn dynamics. This would inflate the reported validation MSE and undermine both the absolute accuracy and the rollout results. The authors must specify the split criterion (e.g., contiguous blocks, by Mars year, or by sol) and justify that no temporal leakage occurs.","section":"Section 2.3"},{"comment":"The claim that the network has 'learned a meaningful dynamical propagation operator rather than a trivial identity mapping' is not supported because no persistence or climatology baseline is provided. In fact, the text states that such comparisons are 'deferred to future work.' For an atmosphere with high hourly temporal correlation, a persistence forecast could plausibly achieve RMSE near 0.004 in normalized units. Without that baseline, the absolute RMSE cannot distinguish learned dynamics from self-similarity. This is a load-bearing omission for the central claim and should be addressed before the paper can be considered complete.","section":"Section 3.2"},{"comment":"The rollout stability result is based on 'a single initial condition.' The RMSE growth curve in Figure 2 is therefore anecdotal; a different start date, season, or dust condition could yield substantially different behavior. The paper should present statistics over multiple initial conditions (e.g., 10–20 cases across Mars seasons and years), with mean and spread of RMSE as a function of lead time. Without this, the monotonic growth and no-divergence conclusion cannot be generalized.","section":"Section 3.3"}],"minor_comments":[{"comment":"The abstract and Section 2.2.1 attribute the 'DLESyM framework' to reference [1], but reference [1] is Watt-Meyer et al., which describes the ACE model, not DLESyM. Please clarify the correct citation or define DLESyM explicitly.","section":"References"},{"comment":"The figure caption says '24 forecast hours' while the text (Sections 3.3 and 5) says 25 hours (one Martian sol). Please align these numbers.","section":"Figure 2 caption"},{"comment":"The physical-unit translation states that wind components have a 'typical range of ~350 m/s.' Given that Mars wind speeds are generally reported up to ~30 m/s, this range seems implausibly large and needs justification (e.g., if it is the full reanalysis range including extreme events, that should be stated).","section":"Section 4.2"},{"comment":"There are several typographical artifacts (e.g., 'T able 1', 'F old/Unfold', 'C ¸ic ¸ek') that should be corrected in a final version.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is a promising proof-of-concept, and the authors are transparent about several weaknesses. However, the abstract and conclusions assert stronger claims than the currently controlled evidence supports. The uncontrolled baseline comparison, missing persistence/climatology baselines, and unspecified split protocol are not merely presentation issues; they directly affect the validity of the headline results. I would also note that code availability is promised only 'upon publication,' which for a fully computational paper makes the reported numbers difficult to verify during review. I recommend major revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper is an honestly written proof-of-concept that adapts a HEALPix-aware DLESyM-style U-Net to Mars reanalysis. The result — stable 25-hour rollouts of T, U, V at ~110 km with a 4.3M-parameter model — is genuinely new for Mars. But the paper does not yet demonstrate what it claims: that geometry-aware convolution is the reason for the skill, or that the model has actually learned a dynamical propagation operator. The missing persistence/climatology baselines and unspecified split protocol are not minor omissions; they are load-bearing.\n\nWhat is new and good: This is the first application of this architecture family to Mars, and the model is compact and fast. The paper is clear about its limitations: it explicitly notes the 3D U-Net comparison is not controlled (different resolution, epochs, learning rate), defers baselines to future work, and concedes EMARS diurnal-cycle bias and dust omission. That honesty is worth something. The rollout stability and monotonic RMSE growth are interesting, and the physical-unit error translation is helpful.\n\nWhere it falls short: The central claims 'geometry matters' and 'learned a meaningful dynamical propagation operator' are unsupported as stated. With no persistence or climatology baseline, a one-hour RMSE of 0.004 in normalized units cannot be interpreted; Mars hourly autocorrelation is high, and a persistence forecast could easily be in that range. The 3D U-Net baseline is problematic in two respects: it is trained at Nside=8 vs. 64, and its 'third dimension' over the 12 HEALPix faces is not a physically meaningful spatial dimension. The split is given as '80/20' without saying whether it is temporal or random; if random, consecutive hours are nearly identical, which invites memorization. The rollout evaluation appears to be on a single initial condition and only for temperature, which limits any claim about general stability. Code is not yet available, so none of the numbers can be checked.\n\nProportion: These are addressable gaps, not fatal contradictions. The paper's own text acknowledges several of them. But the abstract and conclusions overstate what has been shown. A revised version with persistence/climatology baselines, a controlled or at least clearly delimited architecture comparison, a described split, and multi-initial-condition rollout statistics would be a solid contribution.\n\nWho this is for: planetary mission planners and ML researchers interested in domain transfer. It deserves a serious referee, but with the expectation that the authors will have to do the baseline work or drastically soften the claims.\n\nRecommendation: send to peer review, with a request for baseline comparison and split details. I would not cite the current numbers as evidence of skill.","headline":"A promising but under-validated first Mars weather emulator using HEALPix-aware convolutions; the core skill claim is plausible but rests on missing baselines and an uncontrolled comparison.","tokens_in":10802,"tokens_out":4874,"would_cite":true,"duration_ms":46911,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A 4.3-million-parameter neural network can forecast Mars weather for a full Martian sol, reaching a one-hour validation RMSE of about 0.004 in normalized units and a 25-hour RMSE of about 0.031 without divergence.","keywords":["Mars weather forecasting","deep learning emulator","HEALPix convolution","planetary atmosphere","autoregressive rollout","EMARS reanalysis","U-Net","foundation model"],"falsifier":"A direct comparison of Aircast-Mars one-hour and 25-hour forecasts against a persistence baseline on the same hold-out set, or against independent surface meteorological measurements over the same dates, would settle the claim: the model must beat persistence at short lead times and stay within reanalysis-level error on independent observations.","tokens_in":9944,"feed_emoji":"🔴","tokens_out":5950,"duration_ms":62161,"temperature":0.7,"pith_summary":"Deep learning has produced fast weather emulators for Earth, but Mars is a different regime: a thin CO2 atmosphere, no oceans, and rapid thermal tides. Aircast-Mars adapts an equal-area spherical-convolution architecture to the Martian atmosphere, training on the EMARS reanalysis to predict hourly temperature and winds. The paper's central claim is that a compact, geometry-respecting network with only 4.3 million parameters captures synoptic-scale Martian dynamics: it reaches a validation MSE of 1.58e-5, and when unrolled autoregressively it stays stable for 25 hours with RMSE growing from about 0.004 to 0.031. This matters because a 0.5-second single-GPU forecast turns thousand-member ensembles and onboard forecasting into practical options for mission support. The authors are careful to note that skill is measured against reanalysis, not independent observations, and that the diurnal cycle may be under-represented.","feed_headline":"Small AI model forecasts Mars weather for a full sol","feed_subtitle":"The 4.3M-parameter network beats a 30M-parameter baseline by 10x and stays stable for 25 hours.","key_machinery":"The load-bearing mechanism is the HEALPix inter-face padding scheme. HEALPix is a spherical pixelization with 12 equal-area curvilinear faces; before each 3x3 convolution, the model folds the faces into a batch dimension and pads each face by borrowing strips from its neighbors according to the HEALPix adjacency graph, applying the correct rotations for polar faces and a 50/50 blend at tri-face corners. This removes the artificial discontinuities that zero- or reflect-padding would create at face boundaries. The rest of the network uses residual convolutional blocks with capped GELU activations, the cap preventing unbounded intermediate values that would destabilize long autoregressive rollo","core_discovery":"The central discovery is that a 2D U-Net operating on the 12-face HEALPix sphere, with inter-face padding that carries neighboring face values across convolution boundaries, learns a skillful one-hour propagation operator for Mars temperature and winds. On the EMARS v1.0 reanalysis (28 levels, three variables), it achieves a best validation MSE of 1.58e-5 in normalized units, corresponding to roughly 0.59 K at t+1h and 4.59 K at t+25h for temperature. Recursive rollout for 25 hours produces monotonically growing but non-divergent RMSE that saturates toward the model's learned climatology after about 12 hours. Against a 3D U-Net baseline that treats the sphere's faces as an unconnected third","pith_inferences":["The claimed superiority over the 3D U-Net is not from a controlled ablation: the baseline trained at lower resolution, for fewer epochs, and with a different learning rate. A fair comparison holding resolution, epochs, and optimizer constant is needed to isolate the contribution of inter-face padding; until then, the order-of-magnitude claim should be read as an indication, not a measurement.","Because EMARS assimilates temperature retrievals from a limited set of spacecraft overpass local times, the reanalysis's diurnal cycle may be smoother than Mars's actual 60-80 K surface swings. If so, the model's stable 25-hour rollout could be partly a consequence of that smoothing, and validation against in-situ rover measurements at fixed local times would reveal it.","The t+1h error of ~0.004 is not yet benchmarked against a persistence forecast. The paper states that comparison is deferred; if persistence matches or beats the model at one hour, the learned operator may be closer to an interpolation than to dynamics.","A strong stress test for the same architecture would be a global dust storm, where radiative heating changes the thermal profile within hours; the current model has no dust channel, so its next natural testbed is whether adding dust optical depth as a channel preserves rollout stability or requires an explicit regime-switching mechanism."],"forward_implications":["A single GPU can produce a 25-hour Mars weather rollout in about 12 seconds, enabling thousands of perturbed initial conditions for landing-risk assessment in minutes rather than weeks.","The order-of-magnitude gap over the 3D U-Net suggests that other geometry-agnostic deep emulators for planetary atmospheres could be improved by respecting the target topology.","The compact 17MB checkpoint opens the door to near-real-time forecasting on mission hardware.","The stable saturation of RMSE after ~12 hours implies the model is a reliable short-range propagator but not a mid-range climate simulator, setting an expectation for future recurrent or multi-step architectures.","The same three-variable, single-step training recipe can be extended with dust and surface-pressure channels, which the paper identifies as the critical next step."],"fun_headline_variants":["AI forecasts Mars weather a full sol in 0.5s","Tiny 4.3M-parameter AI beats big 3D U-Net on Mars","Geometry-aware AI predicts Mars weather for 25 hours","HEALPix AI: fast Mars weather forecasting model","Mars weather AI: 0.5s per forecast, stable for a sol"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"All skill numbers are measured against the EMARS v1.0 reanalysis, which assimilates sparse spacecraft temperature retrievals and may under-represent the true diurnal cycle; if EMARS carries model or assimilation artifacts, the reported MSE/RMSE values overstate real-world forecast skill.","fun_headline_variants_meta":{"raw":{"variants":["AI forecasts Mars weather a full sol in 0.5s","Tiny 4.3M-parameter AI beats big 3D U-Net on Mars","Geometry-aware AI predicts Mars weather for 25 hours","HEALPix AI: fast Mars weather forecasting model","Mars weather AI: 0.5s per forecast, stable for a sol"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000238,"raw_usage":{"total_tokens":1438,"prompt_tokens":927,"completion_tokens":511,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":671,"completion_tokens_details":{"reasoning_tokens":415}},"tokens_in":671,"tokens_out":511,"duration_ms":5912,"temperature":1.0,"reasoning_tokens":415,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T11:12:30.818322+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A direct comparison of Aircast-Mars one-hour and 25-hour forecasts against a persistence baseline on the same hold-out set, or against independent surface meteorological measurements over the same dates, would settle the claim: the model must beat persistence at short lead times and stay within reanalysis-level error on independent observations.","supporting_citations":[],"review_version":1}