REVIEW 4 major objections 4 minor 29 references
A 4.3-million-parameter neural network can forecast Mars weather for a full Martian sol, reaching a one-hour validation RMSE of about 0.004 in normalized units and a 25-hour RMSE of about 0.031 without divergence.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-02 11:12 UTC pith:AURURXZQ
load-bearing objection A promising but under-validated first Mars weather emulator using HEALPix-aware convolutions; the core skill claim is plausible but rests on missing baselines and an uncontrolled comparison. the 4 major comments →
Aircast-Mars: A Mars Foundation Model for Global Weather Forecasting with HEALPix-Aware Convolutions
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The central discovery is that a 2D U-Net operating on the 12-face HEALPix sphere, with inter-face padding that carries neighboring face values across convolution boundaries, learns a skillful one-hour propagation operator for Mars temperature and winds. On the EMARS v1.0 reanalysis (28 levels, three variables), it achieves a best validation MSE of 1.58e-5 in normalized units, corresponding to roughly 0.59 K at t+1h and 4.59 K at t+25h for temperature. Recursive rollout for 25 hours produces monotonically growing but non-divergent RMSE that saturates toward the model's learned climatology after about 12 hours. Against a 3D U-Net baseline that treats the sphere's faces as an unconnected third
What carries the argument
The load-bearing mechanism is the HEALPix inter-face padding scheme. HEALPix is a spherical pixelization with 12 equal-area curvilinear faces; before each 3x3 convolution, the model folds the faces into a batch dimension and pads each face by borrowing strips from its neighbors according to the HEALPix adjacency graph, applying the correct rotations for polar faces and a 50/50 blend at tri-face corners. This removes the artificial discontinuities that zero- or reflect-padding would create at face boundaries. The rest of the network uses residual convolutional blocks with capped GELU activations, the cap preventing unbounded intermediate values that would destabilize long autoregressive rollo
Load-bearing premise
All skill numbers are measured against the EMARS v1.0 reanalysis, which assimilates sparse spacecraft temperature retrievals and may under-represent the true diurnal cycle; if EMARS carries model or assimilation artifacts, the reported MSE/RMSE values overstate real-world forecast skill.
What would settle it
A direct comparison of Aircast-Mars one-hour and 25-hour forecasts against a persistence baseline on the same hold-out set, or against independent surface meteorological measurements over the same dates, would settle the claim: the model must beat persistence at short lead times and stay within reanalysis-level error on independent observations.
If this is right
- A single GPU can produce a 25-hour Mars weather rollout in about 12 seconds, enabling thousands of perturbed initial conditions for landing-risk assessment in minutes rather than weeks.
- The order-of-magnitude gap over the 3D U-Net suggests that other geometry-agnostic deep emulators for planetary atmospheres could be improved by respecting the target topology.
- The compact 17MB checkpoint opens the door to near-real-time forecasting on mission hardware.
- The stable saturation of RMSE after ~12 hours implies the model is a reliable short-range propagator but not a mid-range climate simulator, setting an expectation for future recurrent or multi-step architectures.
- The same three-variable, single-step training recipe can be extended with dust and surface-pressure channels, which the paper identifies as the critical next step.
Where Pith is reading between the lines
- The claimed superiority over the 3D U-Net is not from a controlled ablation: the baseline trained at lower resolution, for fewer epochs, and with a different learning rate. A fair comparison holding resolution, epochs, and optimizer constant is needed to isolate the contribution of inter-face padding; until then, the order-of-magnitude claim should be read as an indication, not a measurement.
- Because EMARS assimilates temperature retrievals from a limited set of spacecraft overpass local times, the reanalysis's diurnal cycle may be smoother than Mars's actual 60-80 K surface swings. If so, the model's stable 25-hour rollout could be partly a consequence of that smoothing, and validation against in-situ rover measurements at fixed local times would reveal it.
- The t+1h error of ~0.004 is not yet benchmarked against a persistence forecast. The paper states that comparison is deferred; if persistence matches or beats the model at one hour, the learned operator may be closer to an interpolation than to dynamics.
- A strong stress test for the same architecture would be a global dust storm, where radiative heating changes the thermal profile within hours; the current model has no dust channel, so its next natural testbed is whether adding dust optical depth as a channel preserves rollout stability or requires an explicit regime-switching mechanism.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents Aircast-Mars, a 4.3M-parameter HEALPix-aware 2D U-Net trained on EMARS v1.0 hourly temperature, zonal wind, and meridional wind fields regridded to Nside=64 (~110 km). The model is trained to predict the next hourly state. The manuscript reports a best validation MSE of 1.58e-5, stable autoregressive rollouts for one Martian sol (RMSE growing from ~0.004 at t+1h to ~0.031 at t+25h), an order-of-magnitude improvement over a 30M-parameter 3D U-Net baseline, and a 0.5-second per-step GPU inference time. The central claim is that geometry-respecting HEALPix-aware convolutions capture synoptic-scale Martian atmospheric dynamics with high efficiency.
Significance. If the claims are substantiated, Aircast-Mars would be a useful fast surrogate for Mars weather emulation, with plausible applications to mission planning, ensemble generation, and onboard forecasting. The paper is honest about several limitations, including normalized units, reanalysis bias, and the absence of aerosol channels. However, the headline skill claims are currently not supported by controlled comparisons or standard baselines. The 3D U-Net comparison is explicitly uncontrolled, no persistence or climatology baseline is reported, and the data-split protocol is not specified. The code and data are not yet available, so the numerical results cannot be independently verified from the manuscript alone.
major comments (4)
- [Section 3.1, Table 1] The central 'geometry matters' claim rests on an uncontrolled comparison. The 3D U-Net was trained at Nside=8 with 20 epochs and lr=1e-3, while the HEALPix U-Net ran at Nside=64 with 50 epochs and lr=1e-4. The HEALPix U-Net at Nside=16 achieves only 1.4e-4, identical to the 3D U-Net, so the reported order-of-magnitude gain could be due to resolution, training length, or optimizer settings rather than HEALPix-aware padding. The manuscript acknowledges this in Section 3.1, but Section 5 still lists 'Geometry matters' as a key finding. This overstates the evidence. A controlled comparison (matching resolution, epochs, and learning rate, or ablating inter-face padding) is needed before the architecture claim can be accepted.
- [Section 2.3] The data-split protocol is underspecified. The text says only '80/20 train/validation split' without stating whether it is temporal or random. If the split is random, then neighboring hourly states from the same sols are likely in both training and validation sets, allowing the network to memorize rather than learn dynamics. This would inflate the reported validation MSE and undermine both the absolute accuracy and the rollout results. The authors must specify the split criterion (e.g., contiguous blocks, by Mars year, or by sol) and justify that no temporal leakage occurs.
- [Section 3.2] The claim that the network has 'learned a meaningful dynamical propagation operator rather than a trivial identity mapping' is not supported because no persistence or climatology baseline is provided. In fact, the text states that such comparisons are 'deferred to future work.' For an atmosphere with high hourly temporal correlation, a persistence forecast could plausibly achieve RMSE near 0.004 in normalized units. Without that baseline, the absolute RMSE cannot distinguish learned dynamics from self-similarity. This is a load-bearing omission for the central claim and should be addressed before the paper can be considered complete.
- [Section 3.3] The rollout stability result is based on 'a single initial condition.' The RMSE growth curve in Figure 2 is therefore anecdotal; a different start date, season, or dust condition could yield substantially different behavior. The paper should present statistics over multiple initial conditions (e.g., 10–20 cases across Mars seasons and years), with mean and spread of RMSE as a function of lead time. Without this, the monotonic growth and no-divergence conclusion cannot be generalized.
minor comments (4)
- [References] The abstract and Section 2.2.1 attribute the 'DLESyM framework' to reference [1], but reference [1] is Watt-Meyer et al., which describes the ACE model, not DLESyM. Please clarify the correct citation or define DLESyM explicitly.
- [Figure 2 caption] The figure caption says '24 forecast hours' while the text (Sections 3.3 and 5) says 25 hours (one Martian sol). Please align these numbers.
- [Section 4.2] The physical-unit translation states that wind components have a 'typical range of ~350 m/s.' Given that Mars wind speeds are generally reported up to ~30 m/s, this range seems implausibly large and needs justification (e.g., if it is the full reanalysis range including extreme events, that should be stated).
- [General] There are several typographical artifacts (e.g., 'T able 1', 'F old/Unfold', 'C ¸ic ¸ek') that should be corrected in a final version.
Circularity Check
No circularity: the model is trained on EMARS and evaluated on held-out EMARS states; no fitted parameter is renamed as a prediction and no load-bearing self-citation chain exists.
full rationale
The derivation chain is a standard supervised-learning setup: EMARS reanalysis fields are preprocessed (staggered-grid interpolation, normalization, HEALPix regridding), a HEALPix-aware 2D U-Net is trained to minimize MSE against the next-hour EMARS state, and skill is then reported on a held-out portion of the same reanalysis using both one-step and autoregressive rollouts. The target quantity (the t+1h state) is not baked into the model definition or into any fitted parameter; it is the supervised label, and the reported MSE/RMSE are measured on validation data that were not used for training. No equation in the paper reduces to an identity or to a fit renamed as a prediction. The DLESyM/ACE-inspired architecture is attributed to an external reference [1], and the authors of the present paper do not overlap with that reference's authors; there is no uniqueness theorem imported from prior work by the same authors. The comparison to the 3D U-Net is explicitly acknowledged as not strictly controlled (Section 3.1), which weakens the architecture claim but is a statement about experimental design, not circularity. The absence of a persistence or climatology baseline is also explicitly acknowledged and deferred to future work (Section 3.2); this is a missing baseline, not a circular step. The reanalysis-bias caveat (Section 4.2) and the aerosol limitation similarly limit external validity without making the derivation self-referential. The central scientific content — that a 4.3M-parameter geometry-aware network achieves the reported validation MSE and stable 25-hour rollout on held-out EMARS data — stands independently of its inputs; it could in principle be falsified by a different training/evaluation protocol. Therefore, no significant circularity is present and the score is 0.
Axiom & Free-Parameter Ledger
free parameters (5)
- Neural network weights (4.3M) =
Trained to minimize MSE on EMARS
- Min-max normalization statistics =
Global per-variable min/max from EMARS
- Architecture hyperparameters =
Widths 64/128/256, 3 encoder levels, 2x2 pooling, 3x3 kernels, Capped GELU cap (value not stated)
- Training hyperparameters =
lr 1e-4, batch size 1, 50 epochs, 80/20 split (random vs chronological not stated)
- HEALPix corner blending ratio =
50/50
axioms (6)
- domain assumption EMARS v1.0 reanalysis is an accurate representation of Martian atmospheric state at hourly resolution
- domain assumption Temperature, zonal wind, and meridional wind are sufficient prognostic variables for short-term Mars forecasting
- domain assumption Bilinear regridding from lat-lon to HEALPix preserves the dynamics needed for forecasting
- domain assumption Min-max normalization to [0,1] and MSE loss produce a meaningful learning target
- domain assumption Standard supervised training on hourly samples learns a useful dynamical operator rather than interpolation or autocorrelation structure
- standard math HEALPix inter-face padding as implemented correctly realizes spherical adjacency
Cite this review
Pith. "Pith review of Aircast-Mars: A Mars Foundation Model for Global Weather Forecasting with HEALPix-Aware Convolutions." pith.science (2026). https://pith.science/paper/AURURXZQ
@misc{pith2026260719370,
author = {Pith},
title = {Pith review of: Aircast-Mars: A Mars Foundation Model for Global Weather Forecasting with HEALPix-Aware Convolutions},
year = {2026},
howpublished = {\url{https://pith.science/paper/AURURXZQ}},
note = {Machine review of arXiv:2607.19370}
}
read the original abstract
Foundation models for planetary atmospheres promise fast, lightweight surrogates of expensive general circulation models (GCMs) for mission planning and scientific inquiry. Here we present Aircast-Mars, a deep-learning weather prediction system for Mars trained on the Ensemble Mars Atmosphere Reanalysis System (EMARS) v1.0. We regrid temperature, zonal wind, and meridional wind fields across 28 vertical levels onto a hierarchical equal-area isolatitude pixelization (HEALPix) mesh at Nside = 64 (~110 km resolution) and train a HEALPix-aware 2D U-Net inspired by the DLESyM architecture to predict the next hourly atmospheric state. The model employs custom inter-face padding that respects the topology of the 12-face HEALPix sphere and modern ConvNeXt residual blocks with capped Gaussian Error Linear Unit (GELU) activations. While containing 4.3 million trainable parameters, a compact size compared to terrestrial weather foundation models, the network achieves a best validation Mean Squared Error (MSE) of 1.58e-5 in normalized units. Recursive autoregressive rollouts remain stable and physically coherent for 25 hours (one Martian sol), with Root Mean Square Error (RMSE) growing monotonically from ~0.004 at t + 1 h to ~0.031 at t + 25 h without divergence. Compared to a baseline 3D U-Net, the HEALPix-aware architecture reduces validation loss by more than an order of magnitude while using fewer parameters. The model generates a one-hour forecast in approximately 0.5 seconds on a single GPU, offering several orders-of-magnitude speedup over traditional numerical GCMs. These results demonstrate that parsimonious, geometry-respecting neural architectures can capture synoptic-scale Martian atmospheric dynamics and provide a foundation for planetary-scale weather forecasting.
Figures
Reference graph
Works this paper leans on
-
[1]
Watt-Meyer, O., Dresdner, G., McGibbon, J., et al. (2024). ACE: A fast, skillful learned global atmospheric model for climate predic- tion.arXiv preprint arXiv:2310.02074
Pith/arXiv arXiv 2024
-
[2]
Hendrycks, D. and Gimpel, K. (2016). Gaus- sian error linear units (GELUs).arXiv preprint arXiv:1606.08415
Pith/arXiv arXiv 2016
-
[3]
Liu, Z., Mao, H., Wu, C.-Y., Feichtenhofer, C., Darrell, T., and Xie, S. (2022). A Con- vNet for the 2020s. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11976–11986
2022
-
[4]
Ronneberger, O., Fischer, P., and Brox, T. (2015). U-Net: Convolutional networks for biomedical image segmentation. InInterna- tional Conference on Medical Image Comput- ing and Computer-Assisted Intervention, pp. 234–241. Springer
2015
-
[5]
S., Brox, T., and Ronneberger, O
C ¸i¸ cek,¨O., Abdulkadir, A., Lienkamp, S. S., Brox, T., and Ronneberger, O. (2016). 3D U-Net: Learning dense volumetric segmenta- tion from sparse annotation. InInternational Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 424–432. Springer
2016
-
[6]
L., Lewis, S
Read, P. L., Lewis, S. R., and Mulholland, D. P. (2015). The physics of Martian weather and climate: A review.Reports on Progress in Physics, 78(12):125901
2015
-
[7]
M., Clancy, R
Haberle, R. M., Clancy, R. T., Forget, F., Smith, M. D., and Zurek, R. W., eds. (2017). The Atmosphere and Climate of Mars. Cam- bridge University Press
2017
-
[8]
J., Gillespie, H
Greybush, S. J., Gillespie, H. E., and Wil- son, R. J. (2019). Transient eddies in the TES/MCS Ensemble Mars Atmosphere Re- analysis System (EMARS).Icarus, 317:158– 181
2019
-
[9]
J., Wilson, R
Greybush, S. J., Wilson, R. J., Hoffman, R. N., Hoffman, M. J., Miyoshi, T., Ide, K., McConnochie, T., and Kalnay, E. (2012). Ensemble Kalman filter data assimilation of Thermal Emission Spectrometer temperature retrievals into a Mars GCM.Journal of Geo- physical Research: Planets, 117(E11)
2012
-
[10]
R., Moreno, R
Karlbauer, M., Cresswell-Clay, N., Durran, D. R., Moreno, R. A., Kurth, T., Bonev, B., Brenowitz, N., and Butz, M. V. (2024). Ad- vancing parsimonious deep learning weather prediction using the HEALPix mesh.Jour- nal of Advances in Modeling Earth Systems, 16(8):e2023MS004021. Preprint 9
2024
-
[11]
M., Hivon, E., Banday, A
G´ orski, K. M., Hivon, E., Banday, A. J., Wan- delt, B. D., Hansen, F. K., Reinecke, M., and Bartelmann, M. (2005). HEALPix: A frame- work for high-resolution discretization and fast analysis of data distributed on the sphere. The Astrophysical Journal, 622(2):759
2005
-
[12]
Bi, K., Xie, L., Zhang, H., Chen, X., Gu, X., and Tian, Q. (2023). Pangu-Weather: A 3D high-resolution model for fast and accurate global weather forecast.Nature, 619(7970):533–538
2023
-
[13]
Lam, R., Sanchez-Gonzalez, A., Willson, M., et al. (2023). GraphCast: Learning skill- ful medium-range global weather forecasting. Science, 382(6677):1416–1421
2023
-
[14]
Pathak, J., Subramanian, S., Harrington, P., et al. (2022). FourCastNet: A global data- driven high-resolution weather forecasting model.arXiv preprint arXiv:2202.11214
Pith/arXiv arXiv 2022
-
[15]
Montabone, L., Forget, F., Millour, E., et al. (2015). Eight-year climatology of dust optical depth on Mars.Icarus, 251:65–95
2015
-
[16]
A., Murphy, J
Kahre, M. A., Murphy, J. R., Newman, C. E., Wilson, R. J., Cantor, B. A., Lemmon, M. T., and Wolff, M. J. (2017). The Mars dust cycle. In Haberle, R. M. et al., eds.,The Atmosphere and Climate of Mars, pp. 295–337. Cambridge University Press
2017
-
[17]
Bonev, B., Kurth, T., Grossman, C., et al. (2023). Spherical Fourier neural operators: Learning stable dynamics on the sphere. In International Conference on Machine Learn- ing, pp. 2806–2823. PMLR
2023
-
[18]
Paszke, A., Gross, S., Massa, F., et al. (2019). PyTorch: An imperative style, high- performance deep learning library. InAd- vances in Neural Information Processing Sys- tems, vol. 32
2019
-
[19]
Kingma, D. P. and Ba, J. (2015). Adam: A method for stochastic optimization. InPro- ceedings of the 3rd International Conference on Learning Representations (ICLR)
2015
-
[20]
Battalio, J. M. and Wang, H. (2021). The Mars dust activity database (MDAD): A com- prehensive statistical study of dust storm se- quences.Icarus, 354:114059
2021
-
[21]
E., Bertrand, T., Battalio, J
Newman, C. E., Bertrand, T., Battalio, J. M., et al. (2021). Multi-model meteorological and aeolian predictions for Mars 2020 and the Jezero crater region.Space Science Reviews, 217(1):20
2021
-
[22]
Kochkov, D., Yuval, J., Langmore, I., et al. (2024). Neural general circulation models for weather and climate.Nature, 632(8027):1060– 1066
2024
-
[23]
and Raftery, A
Gneiting, T. and Raftery, A. E. (2007). Strictly proper scoring rules, prediction, and estimation.Journal of the American Statisti- cal Association, 102(477):359–378
2007
-
[24]
Keisler, R. (2022). Forecasting global weather with graph neural networks.arXiv preprint arXiv:2202.07575
Pith/arXiv arXiv 2022
-
[25]
Leovy, C. B. and Zurek, R. W. (1979). Ther- mal tides and Martian dust storms: Direct evidence for coupling.Journal of Geophysical Research: Solid Earth, 84(B6):2956–2968
1979
-
[26]
L., Rosenbaum, J
Wang, J. L., Rosenbaum, J. J., Prasad, A. N., Raad, R. R., Putman, E. J., Harrington, A. D., et al. (2025). Potential health im- pacts, treatments, and countermeasures of Martian dust on future human space explo- ration.GeoHealth, 9:e2024GH001213
2025
-
[27]
S., Kerschmann, R
Levine, J. S., Kerschmann, R. L., and Win- terhalter, D. (Eds.). (2018).Dust in the At- mosphere of Mars and its Impact on Human Exploration. Cambridge Scholars Publishing
2018
-
[28]
Hapgood, M. (2019). The Impact of Space Weather on Human Missions to Mars: The Need for Good Engineering and Good Fore- casts. In: Szocik, K. (Eds)The Human Fac- tor in a Mission to Mars. Space and Society. Springer, Cham
2019
-
[29]
Hersbach, H., Bell, B., Berrisford, P., et al. (2020). The ERA5 global reanalysis.Quar- terly Journal of the Royal Meteorological So- ciety, 146(730):1999–2049
2020
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.