{"id":"54fafb03-dc8c-465e-90c2-634eb4e2ff83","arxiv_id":"2411.16339","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Solaris, a transformer with roughly 100 million parameters trained on 13 years of SDO solar images, forecasts the Sun's atmosphere 12 hours ahead and fine-tunes to a data-poor wavelength better than training from scratch.","lead":"Researchers trained a large AI model, Solaris, on 13 years of images of the Sun's atmosphere and showed it can predict the Sun's appearance 12 hours later. The model also adapted quickly to a rarely observed light wavelength, a hint that such models could help space weather forecasting.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 1700 Å transfer result lacks a disclosed test split and a converged from-scratch baseline, so the central 'transfer beats from-scratch' claim is not yet established.","rationale":"The reader's weakest assumption is exactly the concern I independently identify: the 1700 Å fine-tuning comparison is the single quantitative pillar of the central claim, and it is not a controlled experiment as reported. The undisclosed split and the short from-scratch training budget directly threaten the inference from 'achieves lower RMSE after 25 steps' to 'generalizes by transferring learned dynamics to a new wavelength.' I agree with the conditional verdict: the architecture and dataset release are concrete contributions, and the pretraining qualitative results (Figures 2-3 and 6-9) are consistent with a working model, so rejecting the paper outright would be too strong. But the headline claim needs the fine-tuning comparison to be made rigorous before acceptance. I did not find an additional load-bearing concern beyond the reader's: the RMSE curves in Figure 4 and the qualitative multi-wavelength forecasts support the pretraining claims, and the parameter-count inconsistency (109M in the abstract versus 117M for SolarisS in Table 1) is a reporting error, not a flaw in the core argument. The proposed check is the minimal experiment that would settle the concern.","tokens_in":8172,"tokens_out":1627,"duration_ms":13180,"concrete_test":"Re-run the Section 4.2 experiment with full disclosure: (1) split the 987 1700 Å samples by timestamp into train (2010-2022), validation (2023H1), and test (2023H2), and verify no test timestamp appears in fine-tuning or in the pretraining input frames; (2) train the from-scratch baseline to convergence, e.g., 7750 steps or until validation RMSE plateaus, with at least 3 random seeds for both the fine-tuned and from-scratch models; (3) report test-set RMSE with standard errors. If the fine-tuned model still beats the converged from-scratch baseline on a non-overlapping 2023 test set with non-overlapping error bars, the transfer claim stands.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper's central claim rests on Section 4.2: fine-tuning SolarisS for 25 steps on 1700 Å outperforms an identical model trained from scratch for 775 steps. Three omissions make this comparison uninterpretable as evidence of transfer. First, the split of the 987 available 1700 Å samples across the stated training (2010-2022), validation (Jan-Jun 2023), and test (Jul-Dec 2023) periods is not disclosed. Since the pretraining and fine-tuning sets are both drawn from the same 2010-2022 interval, test frames from the 2023 period are chronologically clean, but fine-tuning frames and pretraining frames may share nearby solar states or identical input frames if sample selection is not timestamp-disjoint. If any test pair overlaps with fine-tuning data, the result is inflated. Second, the from-scratch baseline is trained for only 775 steps (one-tenth of the pretraining budget of 7750 steps). With an effective batch size of 32 and roughly 9000 training samples, 775 steps is approximately 2.75 epochs; Figure 5 shows the from-scratch RMSE still declining at the end of training, so the comparison likely measures training speed, not asymptotic skill. Third, no error bars or repeated-seed statistics are reported, so the claimed margin cannot be distinguished from noise. The single quantitative result supporting the headline generalization claim is therefore not robustly quantified.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents Solaris, a 3D Swin Transformer-based foundation model for forecasting the Sun's atmosphere. Solaris is pre-trained on 13 years of SDO/AIA synoptic images at eight wavelengths with a 12-hour forecasting objective, and the authors report multi-wavelength forecast RMSEs and qualitative predictions. The central claim is that fine-tuning the pre-trained SolarisS model for only 25 steps on the 1700 Å wavelength, which was not used in pre-training, outperforms an identical model trained from scratch for 775 steps. The authors argue this demonstrates transfer from well-sampled wavelengths to a data-poor, unseen wavelength and positions Solaris as a generalizable solar foundation model.","tokens_in":8407,"tokens_out":7109,"duration_ms":64727,"significance":"If the transfer claim is established, the paper would provide a valuable demonstration of foundation-model transfer in heliophysics, with practical implications for forecasting at sparsely observed wavelengths and for completing missing archive data. The paper also contributes a publicly released machine-learning dataset covering a full solar cycle, which is a useful community resource. The transfer setup, where the target wavelength is held out from pre-training and evaluation is on a temporally future period, is a sensible and well-posed experimental design. However, the current evaluation does not quantitatively support the headline claims: the fine-tuning result lacks split disclosure, error bars, and a converged baseline, and the main forecasting results are presented as raw RMSE without persistence or skill baselines.","major_comments":[{"comment":"The central transfer claim is not quantitatively established because the fine-tuning data split is undisclosed. The paper states that 987 samples are available for 1700 Å, but it does not report how many fall in the 2010-2022 training period, the Jan-Jun 2023 validation period, and the Jul-Dec 2023 test period, nor how many test frames are evaluated. Without this information, and without repeated-seed statistics or error bars, the sentence 'Finetuning SolarisS for only 25 training steps ... outperformed an identical model trained from scratch for 775 steps' cannot be assessed for statistical significance.","section":"4.2"},{"comment":"The from-scratch baseline may not be converged. At the stated effective batch size of 32, 775 steps corresponds to roughly 25 epochs if all 987 samples are available for training, and Figure 5 shows the from-scratch RMSE still declining at the final step. The paper does not report the fine-tuning hyperparameters (learning rate, schedule, loss weights) beyond saying the configuration is 'the same', nor does it provide final RMSE values. A longer training run, ideally with early stopping or a convergence criterion, is needed to distinguish a training-speed advantage from a genuine transfer advantage.","section":"4.2, Figure 5"},{"comment":"The quantitative forecasting results are reported as raw-intensity RMSE with no baseline. No persistence forecast, autoregressive baseline, or scale-invariant skill metric is provided, so the reader cannot tell whether the RMSE values represent skillful prediction or merely a model that reproduces slowly evolving solar features. Because intensity scales differ by orders of magnitude across the eight wavelengths, comparing raw RMSE across rows is not meaningful; normalized error metrics or skill scores should be added.","section":"Table 2"},{"comment":"The claim that accurate predictions of emerging regions imply 'an internal representation of the Sun's global state' is supported only by visual inspection. A quantitative test is needed, such as evaluating forecast error as a function of heliographic longitude or limb distance, or comparing predicted far-side emergence against HMI magnetogram evolution. Without such an analysis, the statement over-interprets the figures.","section":"4.1, Figures 2-3"}],"minor_comments":[{"comment":"The text and Table 2 use 355 Å while Figures 2 and 3 use 335 Å; the standard AIA passband is 335 Å. Please correct the typo throughout.","section":"4.1, Table 2, Figures 2-3"},{"comment":"Figures 6-9 are not referenced in the text and appear after the references; either cite them in Section 4.1 or remove them.","section":"Figures 6-9"},{"comment":"Figure 1 and the encoder description mention EVE and HMI inputs, but the experiments use only AIA data; clarify that EVE and HMI are planned extensions rather than part of the current model.","section":"Figure 1, Section 3.1"},{"comment":"The transform equation applies min(x, 2.5) and max(x, 10^-3) but the choice of these constants is not justified, and the behavior for x = 0 should be stated explicitly.","section":"3.4"},{"comment":"The weighted MAE loss weights are not specified; report how the per-wavelength weights are chosen.","section":"4.1"}],"recommendation":"major_revision","confidential_remarks":"For the editor: the manuscript is a workshop-style contribution with a worthwhile dataset and a well-posed transfer experiment, but the headline claim rests on an under-specified evaluation. The requested revisions (data split disclosure, converged baseline, baselines and error bars) are feasible within the manuscript's scope. I see no ethical or scope issue; the 'first foundation model' claim is acceptable if the transfer result is substantiated."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"I read Solaris with some interest. It is a straightforward extension of Aurora to SDO/AIA imagery, and the public dataset is the real contribution. The authors assembled a full solar cycle of 12-hour, 512x512 AIA images across eight wavelengths, preprocessed them consistently, and released them. That is genuinely useful for solar physics and ML alike. The architecture is clearly described, the chronological train/val/test split for pretraining is honest, and the qualitative forecasts look plausible.\n\nThe evaluation is where it gets thin. Table 2 reports RMSE on raw intensity scales with no persistence baseline, no error bars, and no comparison to existing solar forecasting models. The claim that Solaris has an 'internal representation of the Sun's global state' is inferred from a few images, not measured. The abstract says 109M parameters; Table 1 says 117M for the same configuration. Minor, but sloppy.\n\nThe bigger issue is the 1700 Å fine-tuning experiment, which carries the headline claim: 25 steps of fine-tuning beats 775 steps from scratch. That could be a real transfer effect, but the paper does not disclose how the 987 available samples were split across train, validation, and test periods. The from-scratch baseline is trained for only 775 steps and Figure 5 shows its RMSE still declining, so the comparison may measure training speed rather than asymptotic skill. No repeated-seed statistics are reported, so the margin could be noise. None of this is fatal — the pipeline is described well enough to re-implement, and the dataset is out — but the central evidence for cross-wavelength generalization is not yet robust.\n\nThe paper is worth a serious referee: it introduces a new resource and a plausible model, and the weaknesses are fixable with baselines, split disclosure, and uncertainty quantification. I would send it to review, but with a request for those revisions before acceptance.","headline":"Plausible first solar foundation model with a useful public dataset, but the transfer claim is under-evidenced: no baselines, no error bars, and an undisclosed 1700 Å split.","tokens_in":8992,"tokens_out":2623,"would_cite":true,"duration_ms":25277,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A 109-million-parameter transformer pre-trained on 13 years of multi-wavelength Sun images forecasts the solar atmosphere and transfers to an unseen wavelength.","keywords":["solar atmosphere forecasting","foundation model","Swin Transformer","transfer learning","multi-wavelength imaging","space weather","Solar Dynamics Observatory","pre-training"],"falsifier":"Repeat the 1700 Å experiment with a from-scratch model trained for the same 7750 steps as pre-training, with the 987 samples split by time so that no fine-tuning frame shares a date with the test set, and with the exact split published; if the from-scratch model reaches the fine-tuned RMSE, the transfer advantage is not established.","tokens_in":7919,"feed_emoji":"☀️","tokens_out":5833,"duration_ms":50705,"temperature":0.7,"pith_summary":"The paper introduces Solaris, which it calls the first foundation model trained to forecast the Sun's atmosphere. The model is pre-trained on 13 years of full-disk, multi-wavelength images from the Solar Dynamics Observatory, spanning a complete solar cycle, and learns to predict the solar state 12 hours ahead from two prior states. The central evidence for the foundation-model claim is a transfer test: fine-tuning Solaris for 25 steps on the data-poor 1700 Å wavelength, which was never seen in pre-training, outperforms an identical model trained from scratch for 775 steps. If this holds, a single pre-trained model could capture generalizable solar dynamics and be adapted cheaply to wavelengths with scarce observations.","feed_headline":"First solar foundation model forecasts the Sun's atmosphere","feed_subtitle":"Fine-tuned on 25 steps of sparse 1700 Å data, it beats a scratch model trained 775 steps.","key_machinery":"The architecture follows the Aurora atmospheric model: a perceiver-based encoder tokenizes each wavelength image into patches, uses cross-attention to aggregate the variable number of wavelengths into a fixed set of latent vectors, and adds Fourier spatial and temporal embeddings; a 3D Swin Transformer U-Net processor with shifted-window self-attention evolves this representation through time; a perceiver-based decoder projects back to pixel space for any requested wavelength subset. A per-wavelength scaling factor normalizes intensities into [0,2], and a learnable blend of the raw value and its logarithm balances sensitivity to bright and faint features. Pre-training used two 12-hour-separated input states at eight wavelengths to forecast the next state, with a weighted MAE loss, AdamW, and 7750 steps.","core_discovery":"Solaris's central claim is that pre-training on eight well-sampled extreme-ultraviolet wavelengths gives the model a general representation of solar atmospheric dynamics that transfers to a wavelength it never saw. The concrete result is that fine-tuned SolarisS, a 117M-parameter 3D Swin Transformer, reaches lower RMSE on the 1700 Å forecasting task after 25 fine-tuning steps than an identical architecture trained from scratch for 775 steps. The paper also reports that the pre-trained model forecasts emerging active regions rotating into view, which suggests an internal representation of the Sun's global state rather than mere image interpolation. The model predicts all eight pre-training wavelengths with consistent error despite large intensity differences, and the larger SolarisS configuration beats the smaller SolarisT.","pith_inferences":["The 25-versus-775-step comparison likely measures sample efficiency more than final accuracy; a longer from-scratch run could narrow the gap, so the paper's strongest evidence is the speed of convergence, not necessarily the asymptotic skill.","A direct testable extension is to fine-tune Solaris on a synthetic wavelength generated by corrupting one of the pre-training channels, such as adding noise or masking, and compare against a scratch model on the same synthetic data; this would control for the unknown 1700 Å data distribution.","The claimed internal representation of the Sun's far side could be probed by checking whether the model's latent representation predicts the magnetogram of a region once it rotates onto the visible disk; a null result would suggest the emerging-region forecasts come from temporal persistence of rotating features.","If transfer is real, the same approach should work for other sparsely observed AIA wavelengths or for EVE spectral irradiance, providing a cheap way to extend a foundation model without pre-training on new data."],"forward_implications":["A pre-trained Solaris can be adapted to a new solar wavelength with only a handful of training steps, making forecasting feasible for wavelengths with sparse archival data.","The 1700 Å experiment suggests that missing wavelength channels in observational archives could be filled by fine-tuning rather than by collecting new data.","Consistent performance across eight wavelengths indicates that the normalization scheme prevents bright wavelengths from dominating training, a precondition for multi-wavelength foundation models.","The apparent ability to forecast emerging regions implies the model encodes a global, not just disk-facing, representation of the Sun.","Larger SolarisS outperforms SolarisT, suggesting further gains from scaling model size and input resolution."],"supporting_citations":[{"why":"Supplies the Aurora architecture that Solaris adapts for solar forecasting.","marker":"[3]"},{"why":"Provides the perceiver cross-attention mechanism used to aggregate variable wavelengths into fixed latent tokens.","marker":"[8]"},{"why":"Defines the Swin Transformer with shifted windows used in the processor.","marker":"[12]"},{"why":"Contributes the U-Net-like transformer structure for the processor's multi-scale encoding.","marker":"[5]"},{"why":"Supplies the preprocessing pipeline (degradation correction, exposure normalization, centering, resizing) for the SDO/AIA data.","marker":"[6]"},{"why":"Establishes the Vision Transformer backbone that the processor builds on.","marker":"[2]"}],"fun_headline_variants":["Solar foundation model beats scratch on unseen wavelength","Sun's first foundation model forecasts unseen wavelength","Solaris: pre-trained on Sun, fine-tunes to new view","25 steps beat 775: Sun model transfers across wavelengths","Solar foundation model generalizes to a ninth wavelength"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"That the 1700 Å fine-tuning comparison is a fair and uncontaminated test of transfer: the 987 samples are split so test frames never appear in fine-tuning or pre-training, and the from-scratch baseline receives enough training steps to converge.","fun_headline_variants_meta":{"raw":{"variants":["Solar foundation model beats scratch on unseen wavelength","Sun's first foundation model forecasts unseen wavelength","Solaris: pre-trained on Sun, fine-tunes to new view","25 steps beat 775: Sun model transfers across wavelengths","Solar foundation model generalizes to a ninth wavelength"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000216,"raw_usage":{"total_tokens":1375,"prompt_tokens":833,"completion_tokens":542,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":449,"completion_tokens_details":{"reasoning_tokens":466}},"tokens_in":449,"tokens_out":542,"duration_ms":5451,"temperature":1.0,"reasoning_tokens":466,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T13:17:16.776320+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Repeat the 1700 Å experiment with a from-scratch model trained for the same 7750 steps as pre-training, with the 987 samples split by time so that no fine-tuning frame shares a date with the test set, and with the exact split published; if the from-scratch model reaches the fine-tuned RMSE, the transfer advantage is not established.","supporting_citations":[{"cited_title":"A machine- learning data set prepared from the nasa solar dynamics observatory mission","cited_arxiv_id":null,"evidence_quote":"Supplies the preprocessing pipeline (degradation correction, exposure normalization, centering, resizing) for the SDO/AIA data."}],"review_version":1}