{"id":"f8087558-f6b1-443a-8505-79934ad645ee","arxiv_id":"2501.14684","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"A single-event retrospective study claims AI could have foreseen the May 2024 superstorm, but the evidence is undermined by a false positive, an autoregressive SYM-H forecast, and a misquoted CME error.","lead":"This paper applies four previously published AI models to the May 2024 solar superstorm and reports that they could have predicted the flares, the CME arrival, and the geomagnetic storm. The headline minute-level CME accuracy is contradicted by the paper's own table, and the storm 'forecast' uses the index it aims to predict.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The geomagnetic storm 'prediction' in §4.4 feeds SYM-H into an LSTM that forecasts SYM-H one hour ahead, making it a persistence nowcast; the claim that AI predicted the storm is therefore unsupported.","rationale":"We agree with the reader's overall REJECT verdict and high correctness risk. The most load-bearing flaw is not the drag-based-model single-body assumption (the reader's weakest_assumption), but the SYM-H leakage in §4.4. The model inputs the very index it is trained to forecast; consequently, its 'prediction' of the storm's onset and recovery is a one-hour persistence/nowcast, not an independent forecast. This directly falsifies the abstract's claim that AI 'could have predicted' the geomagnetic storm. Even if the CME travel-time model were exactly correct, the chain would still fail at its last link. We therefore recommend REJECT. The DBM issue is real but secondary: the loss functions in Eqs. (1)–(2) assume a single rigid CME, while the May 8 CME cannibalized earlier ejections; however, the model could in principle learn effective drag parameters for interacting events. The SYM-H leakage is a methodological error with no comparable defense.","tokens_in":11163,"tokens_out":7759,"duration_ms":66813,"concrete_test":"Retrain the Section 4.4 LSTM on the same 2005–2023 Wind dataset with all SYM-H components removed from the input feature vector. Evaluate it on the May 8–11, 2024 interval, and compare the first alarm time and TSS against: (i) the published model, and (ii) a persistence baseline that predicts SYM-H(t+1)=SYM-H(t). If the no-SYM-H model issues no alarm before the SYM-H crossing of −50 nT, or if persistence achieves equal TSS, the geomagnetic prediction is not a genuine forecast.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4.4 states the LSTM input is a 24-hour time series of solar-wind features 'and the SYM-H index', and the output is the probability that SYM-H drops below −50 nT in the next hour. Because SYM-H is both an input and the target, the model is not forecasting the storm from upstream drivers; it is extrapolating an already ongoing disturbance. In the May 2024 event, the input window contains the initial sharp SYM-H descent, so an alarm issued at the onset is a nowcast, not a pre-storm warning. The paper's claim to have predicted the geomagnetic storm (Section 3, Figure 3 bottom) with one-hour lead is thus unsubstantiated. A persistence baseline that simply carries SYM-H forward one hour would likely match the reported TSS. This is a data-leakage flaw in the final, decisive link of the 'entire chain' claim, independent of the drag-based-model concerns.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents a retrospective case study of three AI-based tools applied to the May 2024 superstorm chain: a Vision Transformer classifies the morphological evolution of NOAA AR 13644 from HMI magnetograms, a video-based CNN-LSTM forecasts M- and X-class flares from 24-hour magnetogram sequences, a physics-driven neural network with a drag-based model predicts the CME travel time from coronal and solar-wind inputs, and an LSTM ingesting 24 hours of in-situ solar-wind and SYM-H data estimates the probability of SYM-H dropping below -50 nT in the next hour. The central claim is that AI could have predicted the entire chain of events, with the CME travel time predicted to 44.21 +/- 3.20 h versus an observed 44.23 h and the geomagnetic storm onset and recovery 'predicted' one hour in advance. The authors conclude that AI offers unprecedented accuracy and outperforms traditional methods, but the supporting evidence is uneven and the headline claims overreach the analysis.","tokens_in":11348,"tokens_out":5779,"duration_ms":46416,"significance":"If the central claim were fully supported, the paper would be a valuable demonstration of end-to-end AI-based space weather forecasting. The physics-informed loss function for the CME travel-time model and the ensemble-uncertainty approach are thoughtful, and the flare-forecasting architecture is clearly described. However, the geomagnetic 'forecast' is an autoregressive nowcast of the target variable, the CME accuracy rests on a single event with a misquoted error, and no quantitative skill scores or baseline comparisons are given for the flare link. The paper is a useful retrospective case study, but it does not substantiate the strong predictive claims in the abstract and title.","major_comments":[{"comment":"The geomagnetic storm 'prediction' is largely circular because the LSTM input is a 24-hour time series that explicitly includes the SYM-H index, and the output is the probability that SYM-H drops below -50 nT in the next hour. When the storm onset is already present in the input window, the model is an autoregressive nowcast of the target quantity rather than a forecast from solar wind drivers. The claim in §2.5 that AI 'accurately predict[s] not only the onset but also the whole recovery phase 1 hour in advance' is therefore unsupported without a persistence baseline or an input set excluding SYM-H.","section":"§2.5 and §4.4"},{"comment":"The abstract's 'unprecedented accuracy ... uncertainty as small as one minute' and the Discussion's 'error margin of less than one minute' contradict the reported values. Table 1 gives a predicted travel time of 44.21 +/- 3.20 h against an observed 44.23 h; the difference is 0.02 h (72 s), which is not less than one minute, and the stated uncertainty is +/- 3.20 h, not one minute. The 72 s agreement is a single point with no propagated uncertainties from the cone-model and solar-wind inputs, so it cannot support 'unprecedented accuracy'.","section":"Abstract; §3 Discussion; Table 1"},{"comment":"The physics-driven CME model encodes a single-body drag-based equation with ambient solar wind inputs, yet the paper itself describes the May 8 CME as cannibalizing several earlier ejections into a merged cloud (§2.4 and Discussion). The loss functions in Eqs. (1)-(2) contain no term for CME-CME interaction or for the merged body's effective mass and cross-section. The close travel-time agreement for this one interacting event is therefore not evidence that the model can predict interacting CMEs generally, and the paper should either include interacting-event cases in validation or temper the generalization claim.","section":"§2.4, §4.3, Eqs. (1)-(2)"},{"comment":"The flare forecasting result contains an acknowledged false positive (the alarm in the window between 2027-05-06 and 2024-05-07, which should read 2024-05-06/07), and no skill scores, confidence intervals, or comparison with a persistent or climatological baseline are provided for this event. The statement that the models 'outperformed traditional methods' (Abstract) is therefore not demonstrated for the flare link of the chain.","section":"§2.3, Fig. 3 top panel"}],"minor_comments":[{"comment":"The date '2027-05-06' is a typo and should read '2024-05-06'.","section":"§2.3"},{"comment":"The phrase 'situ measurements' should be 'in-situ measurements'.","section":"§2.5"},{"comment":"The ViT classification is applied to already recorded magnetograms; the authors should clarify that this is a retrospective classification rather than a forecast, and provide quantitative agreement with the visual classification.","section":"§2.2"},{"comment":"The units row 't h' is unclear; use '[h]' and define 't0' and 'tf' clearly.","section":"Table 1"},{"comment":"The notation 'LC' in Eq. (1) is inconsistent with the narrative; define it explicitly.","section":"§4.3"},{"comment":"No data or code availability statement is provided, which limits reproducibility for this case study.","section":"General"}],"recommendation":"reject","confidential_remarks":"The paper would be more appropriately framed as a retrospective case study of AI methods, with the geomagnetic component described as a nowcast demonstration. As submitted, the abstract and title overstate the results, and the 'less than one minute' error is factually incorrect. The geomagnetic circularity is a load-bearing flaw that a revision cannot fix without changing the paper's central claim, so I recommend rejection rather than major revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague, quick take: this is a retrospective application of the authors' own previously published AI models to the May 2024 superstorm. The standout result is the CME travel time: predicted 44.21 ± 3.20 hours against an observed 44.23 hours. That single-point agreement is remarkable and makes the paper worth a look. But the headline claim that AI “could have predicted all space weather events” does not survive close reading.\n\nFirst, the good. The physics-driven ensemble for CME travel time is a legitimate extension of earlier work, and reporting it for this extreme event is a useful data point. The paper is also transparent in places: it admits the flare model produced a false positive, and it acknowledges that the May 8 CME cannibalized earlier ejections. That honesty is appreciated.\n\nThe problems are in the claims. The abstract promises “uncertainty as small as one minute” and the discussion says “error margin of less than one minute.” The actual error is 0.02 hours, which is 72 seconds—not less than a minute—and the ensemble uncertainty is 3.20 hours. That is a misstatement, plain and simple.\n\nThe bigger issue is the geomagnetic storm prediction. The LSTM in Section 4.4 takes a 24-hour series that includes SYM-H and predicts the probability that SYM-H drops below −50 nT in the next hour. That is an autoregressive nowcast, not a forecast from solar-wind drivers. The paper gives no persistence baseline, so the claim of accurate onset and recovery prediction is unsubstantiated. This is the load-bearing flaw for the “entire chain” narrative.\n\nThe DBM concern is real but secondary. The model assumes a single rigid CME in ambient solar wind, while the event involved CME-CME merging. The paper acknowledges the interaction but does not model it, so the impressive travel-time accuracy may not transfer to other interacting cases. No code or data are provided, and there are no comparisons against operational baselines beyond vague statements.\n\nOn balance, this is a modest case study with an inflated title. The CME result deserves attention, and the pipeline integration is new for this specific event. But the geomagnetic “forecast” is essentially persistence, the one-minute claim is wrong, and the generalizability is unproven. A serious referee could push the authors to fix these issues, so I would not desk-reject it.\n\nMy recommendation: send it to peer review with a request for major revision. Require toning down the claims, adding a persistence baseline for SYM-H, correcting the one-minute misquote, and discussing the single-body DBM limitation explicitly. Then it could be a useful case study.","headline":"An interesting single-event case study with one genuinely impressive CME travel-time result, but the claim to have predicted the entire chain of May 2024 space weather is undercut by the SYM-H nowcast setup and a misquoted uncertainty.","tokens_in":11942,"tokens_out":2424,"would_cite":false,"duration_ms":22495,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a chain of AI models, trained on data from before the event, could have predicted every major space-weather event of the May 2024 superstorm, from the X-class flares to the CME's 44.21-hour arrival and the G5…","keywords":["space weather forecasting","May 2024 geomagnetic superstorm","coronal mass ejection travel time","solar flare prediction","physics-driven neural network","Vision Transformer","SYM-H geomagnetic index","drag-based model"],"falsifier":"Take the same ensemble-training protocol from Section 4.3 and apply it to a set of historical CMEs known to have merged or interacted in transit, where the observed arrival times are recorded; if the median absolute error of predicted travel times is far larger than in the May 2024 case or grows with the number of interacting ejections, the single-body drag assumption is the limiting factor.","tokens_in":10924,"feed_emoji":"🌞","tokens_out":8349,"duration_ms":69908,"temperature":0.7,"pith_summary":"The paper argues that a suite of AI models, applied retrospectively to the May 2024 solar superstorm, could have forecast the whole event chain: the active region's evolution, the M- and X-class flares, the arrival of the merged coronal mass ejection, and the G5 geomagnetic storm. Its headline result is a predicted CME travel time of $44.21 \\pm 3.20$ hours against a measured $44.23$ hours, an agreement far tighter than the roughly 12-hour uncertainty of traditional forecasts. The authors present this single extreme event as a case study showing that physics-informed and data-driven machine learning can serve as an operational early-warning tool for critical infrastructure. If the claim holds, the lesson is that the ingredients for warning of extreme space weather already exist and only need to be run in real time.","feed_headline":"AI predicted May 2024 superstorm's full chain of events","feed_subtitle":"Retrospective forecasts matched CME arrival to within a minute and flagged the G5 storm an hour ahead.","key_machinery":"The load-bearing object is the drag-based equation (DBM), a kinematic model in which a coronal mass ejection is treated as a single body decelerated by a drag proportional to the square of its speed relative to the solar wind. The paper embeds the analytical solution of that equation in the loss functions of a two-network cascade: the first network estimates the drag parameter, the second predicts travel time, and an ensemble of retrained networks provides the uncertainty. Around this core are three trained deep learners: a Vision Transformer (a deep image-classification network based on self-attention) for active-region classification, a parallel-CNN plus LSTM video network for flare forecasting, and an LSTM (a recurrent network for sequences) fed by in-situ solar wind features such as field magnitude and $B_z$, speed and $V_x$, temperature, helicity, and energy terms for geomagnetic-impact prediction.","core_discovery":"The central claim is that each link in the May 2024 chain, from flaring in NOAA active region 13644 to the Earth-directed CME that cannibalized several earlier ejections to the resulting G5 storm, could have been predicted by AI tools developed before the event. On magnetogram cutouts, a Vision Transformer tracked AR13664's morphology and assigned it to the $\\beta$-gamma class. A video-based CNN plus LSTM model issued alarms for M-class and X-class flares from May 5 to May 13, with one false positive. The physics-driven ensemble predicted the merged CME's travel time as $44.21 \\pm 3.20$ hours versus the observed $44.23$ hours, and an LSTM driven by solar wind and SYM-H features forecast the storm's onset and recovery phase one hour ahead. The authors conclude that this accuracy supports AI as both an operational forecasting tool and a reverse-engineering probe for CME interaction physics.","pith_inferences":["Because the travel-time claim rests on one extreme event, the minute-level accuracy should not be read as a calibrated uncertainty until the ensemble is re-run over a catalog of historical interacting CMEs; that is a test the paper does not perform.","The single-body drag loss treats the merged cloud as one body with a prescribed ambient wind, so the method's success here may reflect the specific May 8 merger rather than a general rule; applying it to other cannibalizing CMEs would separate the two.","A natural extension would be to couple the predicted CME arrival and magnetic-field orientation to the storm forecaster, replacing observed in-situ inputs with AI-predicted ones and yielding a fully predictive rather than retrospective pipeline.","The authors' reverse-engineering suggestion implies that per-event drag parameters inferred by the network could be compared with MHD simulations of merging ejections, turning the AI into a hypothesis tester for CME interaction physics."],"forward_implications":["The same pipeline, run in real time, would have alerted forecasters to a severe Earth-directed storm several days before the May 10 shock.","CME arrival-time uncertainty could shrink from the standard roughly 12 hours to minutes for events where the single-body drag description holds.","The physics-driven network can be used to estimate effective drag parameters from observed travel times, giving a data-driven probe of CME-CME merging and solar wind coupling.","One-hour-ahead SYM-H forecasts of onset and recovery would widen the window for protective action on power grids and satellites.","The chain approach ties flare forecasts to CME and storm predictions, making each stage's output directly actionable for the next."],"supporting_citations":[{"why":"Supplies the physics-driven neural-network ensemble, including the drag-based loss, that the paper applies to the May 8 CME.","marker":"Guastavino et al. 2023a"},{"why":"Supplies the video-based CNN+LSTM flare forecaster that produced the M/X alarms.","marker":"Guastavino et al. 2022a"},{"why":"Supplies the LSTM geomagnetic-impact model, trained on Wind data, that predicted SYM-H below -50 nT onset and recovery.","marker":"Guastavino et al. 2024"},{"why":"Supplies the Vision Transformer training pipeline used to classify AR13664's morphology.","marker":"Legnaro et al. 2024"},{"why":"Provides the drag-based equation whose analytical solution is encoded in the CME travel-time loss.","marker":"Vršnak et al. 2013"},{"why":"Provides the score-oriented loss that optimizes the True Skill Statistic for the flare and storm networks.","marker":"Marchetti et al. 2022"},{"why":"Provides the Enlil MHD simulation identifying the May 8 CME's cannibalization of earlier ejections.","marker":"Odstrcil 2003"},{"why":"Provides the cone model used to infer CME speed, trajectory, and angular width from LASCO images.","marker":"Zhao et al. 2002"}],"fun_headline_variants":["AI predicted May 2024 superstorm chain to within a minute","AI forecast May 2024 storm, CME arrival exact to a minute","AI nailed May 2024 superstorm: flares, CME, storm all predicted","AI predicted May 2024 superstorm with minute-level CME forecast","AI predicted full May 2024 storm chain, including exact CME timing"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that one rigid cloud, slowed only by a prescribed solar-wind drag, can stand in for the merged ejection that hit Earth on May 10; if the merging of several clouds breaks that single-body picture, the minute-level travel-time prediction would not transfer to other interacting CMEs.","fun_headline_variants_meta":{"raw":{"variants":["AI predicted May 2024 superstorm chain to within a minute","AI forecast May 2024 storm, CME arrival exact to a minute","AI nailed May 2024 superstorm: flares, CME, storm all predicted","AI predicted May 2024 superstorm with minute-level CME forecast","AI predicted full May 2024 storm chain, including exact CME timing"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000862,"raw_usage":{"total_tokens":3753,"prompt_tokens":972,"completion_tokens":2781,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":588,"completion_tokens_details":{"reasoning_tokens":2681}},"tokens_in":588,"tokens_out":2781,"duration_ms":17010,"temperature":1.0,"reasoning_tokens":2681,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T14:54:55.152406+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the same ensemble-training protocol from Section 4.3 and apply it to a set of historical CMEs known to have merged or interacted in transit, where the observed arrival times are recorded; if the median absolute error of predicted travel times is far larger than in the May 2024 case or grows with the number of interacting ejections, the single-body drag assumption is the limiting factor.","supporting_citations":[{"cited_title":"2024, Astrophys","cited_arxiv_id":null,"evidence_quote":"Supplies the LSTM geomagnetic-impact model, trained on Wind data, that predicted SYM-H below -50 nT onset and recovery."},{"cited_title":"Deep Learning for Active Region Classification: A Systematic Study from Convolutional Neural Networks to Vision Transformers","cited_arxiv_id":"2410.17816","evidence_quote":"Supplies the Vision Transformer training pipeline used to classify AR13664's morphology."},{"cited_title":"2022, Pattern Recognition, 132, 108913","cited_arxiv_id":null,"evidence_quote":"Provides the score-oriented loss that optimizes the True Skill Statistic for the flare and storm networks."},{"cited_title":"2003, Adv","cited_arxiv_id":null,"evidence_quote":"Provides the Enlil MHD simulation identifying the May 8 CME's cannibalization of earlier ejections."},{"cited_title":"P., Plunkett, S","cited_arxiv_id":null,"evidence_quote":"Provides the cone model used to infer CME speed, trajectory, and angular width from LASCO images."}],"review_version":1}