Pith. sign in

REVIEW 3 major objections 6 minor 18 references

SEP-PRISM Data fuses five solar-observation families into a reproducible 24-hour-ahead forecasting benchmark spanning 39 years.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 21:10 UTC pith:FTHKNUY2

load-bearing objection Valuable SEP forecasting benchmark, but pre-2010 rows leak future information through the alignment regressions—fix that before treating it as a forecasting dataset. the 3 major comments →

arxiv 2607.16160 v1 pith:FTHKNUY2 submitted 2026-07-17 astro-ph.SR

SEP-PRISM Data: A multi-source dataset for solar energetic particle forecasting

Yian Yu , Yang Chen , Lulu Zhao , Kathryn Whitman , Ward Manchester , Tamas Gombosi This is my paper
classification astro-ph.SR
keywords solar energetic particlesspace weather forecastingdataset curationSHARP/SMARP alignmentCME cataloguesGOES proton fluxmachine learning benchmarkmulti-source data fusion
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper introduces SEP-PRISM Data, a curated multi-source dataset for 24-hour-ahead forecasting of operational solar energetic particle (SEP) events, defined by >10 pfu proton flux in the GOES >10 MeV channel. It spans 3 February 1986 to 10 September 2025 and contains 14,464 daily labeled samples, including 650 positive operational SEP cases. The authors' central claim is that by aligning magnetic-field and CME records from different eras into extended archives (SMHARP and CDAWDONKI), they provide a unified, reproducible benchmark that overcomes the historical coverage gaps that have hampered data-driven SEP forecasting. If correct, this gives the community a common testbed for model comparison, feature analysis, and operational validation across multiple solar cycles.

Core claim

SEP-PRISM Data provides a supervised, window-based dataset for 24-hour-ahead SEP forecasting, with predictors summarized as minimum, mean, and maximum over fixed non-overlapping 24-hour windows and targets defined over the subsequent 24 hours. To extend coverage, SHARP and SMARP magnetic parameters were aligned into SMHARP (back to 1996) using regression models fitted on the 2010 overlap period, and DONKI and CDAW CME records were aligned into CDAWDONKI (back to 1996) using models fitted on 2010-2025. The dataset fuses flare records, magnetic parameters, CME catalogues, GOES soft X-ray flux, and proton flux into a single table with consistent naming and missing-value rules, and it explicitly

What carries the argument

The central mechanism is the two alignment pipelines: SMHARP, which maps SOHO/MDI SMARP parameters to SDO/HMI SHARP equivalents via stepwise-AIC linear regression on the 2010 overlap, and CDAWDONKI, which maps CDAW CME attributes to DONKI-like descriptors using nearest-event matching and regression on 2010-2025. These extend modern high-quality magnetic and CME predictors back to 1996, and the rolling-window min/mean/max summarization converts heterogeneous cadences into one supervised learning format.

Load-bearing premise

The entire pre-2010 extension rests on regression models fitted during the 2010 and 2010-2025 overlaps being valid when applied backward to 1996-2010; the validation tables only test those models on the same overlap periods, not on the extrapolation range.

What would settle it

Fit the SMARP-to-SHARP regression on a temporal split within 2010 (e.g., first half) and evaluate on the second half; then apply it to SMARP data from solar cycle 23 (1996-2008) and check whether reconstructed free-energy and shear parameters correlate with the occurrence of subsequent SEP events. A sharp drop in correlation outside the overlap would falsify the backward extrapolation.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Reproducible benchmarking: any SEP forecasting model can be trained and evaluated on the same 14,464 daily samples, making reported skill scores comparable across studies.
  • Long-horizon feature analysis: with magnetic and CME predictors extended to 1996, models can learn relationships across three solar cycles instead of only the SHARP/DONKI era.
  • Operational alignment: the primary binary target matches the NOAA operational criterion (>10 pfu), so models trained here map directly to operational decision thresholds.
  • Multi-task capacity: auxiliary regression targets such as future peak proton flux, XRSB flux, and CME speed enable multi-task learning and probabilistic calibration in one framework.
  • Live updates: the included notebook can refresh raw downloads and generate 1-hour-step windows for near-real-time forecasting experiments.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the backward extrapolation holds, the dataset effectively creates a modern magnetic-field record for 1996-2010, a period with strong SEP activity; this could substantially increase the number of usable training events for rare-event models.
  • The 4.5% positive-event rate means naive classification accuracy will be misleading; the dataset's design invites precision-recall and event-based verification, but it does not itself prescribe those metrics.
  • The 1-hour overlapping file contains strongly autocorrelated samples; users will need blocked or gap-aware cross-validation, which the paper flags but does not fully specify.
  • Because the alignment models are linear and fitted on overlap periods, a non-stationary solar-cycle dependence in magnetic configurations could bias reconstructed features; a simple diagnostic is to compare reconstructed free-energy proxies between cycles against known flare-productivity relationships.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. This paper introduces SEP-PRISM Data, a curated multi-source dataset for 24-hour-ahead forecasting of operational solar energetic particle (SEP) events, spanning 3 February 1986 to 10 September 2025. The dataset fuses CLEAR SEP benchmark labels, HEK flare records, SHARP/SMARP active-region magnetic parameters, DONKI/CDAW CME catalogues, GOES XRSB and >10 MeV proton flux into a common rolling-window schema with 14,464 non-overlapping 24-hour samples (650 positive operational SEP windows). To extend limited temporal coverage, the authors align SHARP and SMARP into an SMHARP archive and DONKI and CDAW into a CDAWDONKI archive using regression models fit on their overlap periods, then apply the models to pre-2010 data. The paper describes the source data, preprocessing, alignment, validation, and usage notes, and provides code and data on Zenodo.

Significance. The dataset addresses a real community need: a reproducible, harmonized benchmark for data-driven SEP forecasting. The pipeline is detailed, the three-tier structure with provenance flags (From_SMARP, From_CDAW) is good practice, and the release of code and processed tables is commendable. If the temporal-leakage issue in the reconstructed features were resolved, this would be a valuable resource. As it stands, however, the central claim that the analyzed tables are a valid forecasting benchmark is compromised by the use of future-fitted alignments for pre-2010 rows and for CDAWDONKI rows throughout. The paper is therefore not acceptable in its current form, but the problem is addressable by releasing a causally valid version or substantially narrowing the benchmark claims.

major comments (3)
  1. [Methods, 'Alignment of magnetic records' and 'Alignment of CME records'; Usage Notes] The alignment procedures create a temporal information leak in the analyzed benchmark tables. The SMARP-to-SHARP regression is fit on the 2010 overlap only and then applied to 1996-2010 SMARP records; the CDAW-to-DONKI regression is fit on 2010-2025 and applied to pre-2010 CDAW events (and, for CDAWDONKI features, to events throughout the record). Consequently, every pre-2010 window in rolling_combinded_seq_24hours.csv contains SMHARP and CDAWDONKI features that are functions of data from after the window's observation time. If a user trains on post-2010 data and tests on pre-2010 rows, or uses a random split, information from the future is available to the test set, inflating skill estimates. The same issue affects post-2010 CDAWDONKI features because the regression is fit on the full 2010-2025 overlap and a row at time t uses coefficients estimated from times > t. The Usage Notes discl
  2. [Technical Validation, Tables 10 and 11] The validation of the two alignments is performed entirely on the same overlap period used to fit the regression models. Table 10 evaluates SMARP-derived estimates against native SHARP during 2010; Table 11 evaluates CDAW-derived estimates against DONKI during 2010-2025. These checks confirm in-sample calibration fidelity but do not test the extrapolation that is the entire purpose of the alignments: applying the fitted relationships to 1996-2010 data. The reader is asked to accept that linear coefficients learned in the modern overlap are valid for earlier solar cycles and for older instrument configurations (SOHO/MDI vs SDO/HMI, and earlier LASCO epochs). As a concrete test, the authors should split the overlap into fitting and holdout periods (e.g., fit on 2010-2015, evaluate on 2016-2025) and report COR/RMSE for the holdout. Without such a temporal holdout or an independent check aga
  3. [Data Records, Table 9; Technical Validation, Table 11] Several features retained in the analyzed tables are themselves regression outputs with only moderate reconstruction quality. Table 11 reports post-alignment COR of 0.4899 for lon_deg and 0.4560 for halfAngle_deg; latitude is 0.0821 and was excluded. These noise levels matter because the dataset is explicitly intended to support feature analysis and model development. Using noisy reconstructed features as predictors without discussion of measurement error can attenuate effect sizes and distort feature importance rankings in downstream analyses. The paper should quantify the expected impact (e.g., by comparing models trained on native vs reconstructed features during the overlap period, or by reporting error bars for feature importance) or at minimum issue a strong caveat in Usage Notes.
minor comments (6)
  1. [Background] Typo: 'activate-region' should be 'active-region'.
  2. [Methods, GOES X-ray and proton flux] The text refers to 'GPES particle-flux product'; this should be 'GOES'.
  3. [Data Records, Analyzed data records] Typo: 'wherase' should be 'whereas'.
  4. [Methods, Alignment of CME records] Spacing: 'PoororV ery Poor' should read 'Poor or Very Poor'.
  5. [Table 10] Several entries use nonstandard notation such as '1.7706+21' and '4.4258+1'; use scientific notation (e.g., 1.77e+21, 4.43e+1) for clarity.
  6. [Figure 1] The figure is adapted from the companion paper's Supplementary Information. Please confirm that this is permissible and that appropriate attribution is maintained in the figure caption.

Circularity Check

0 steps flagged

No significant circularity: SEP labels come from the external CLEAR benchmark; the SMARP/SHARP and CDAW/DONKI alignments are input-side reconstructions, not target predictions.

full rationale

The dataset's primary prediction target is externally defined: 'SEP event labels were taken from the CLEAR SEP benchmark dataset' (Methods, Source data preprocessing and alignment), and Future_OSEP_label is constructed directly from that catalogue (Table 9). The SMARP-to-SHARP and CDAW-to-DONKI regression alignments are feature-side surrogates, not definitions of the SEP target, so no prediction target reduces to a fitted value. The in-sample nature of Tables 10 and 11 — the SMARP-SHARP regression is fit on the 2010 overlap and then evaluated on that same overlap, and similarly for DONKI-CDAW on 2010-2025 — is a real validation weakness and should be read as reporting fit quality rather than independent confirmation of the pre-2010 extrapolation. This is a correctness/leakage concern, not a tautology. The companion-work self-citations (refs. 10, 12) are explicitly described as 'an additional, non-primary indication' and are not load-bearing for the dataset's construction. The Usage Notes also honestly disclose the main limitation: 'pre-2010 magnetic predictors and pre-2010 DONKI-like CME descriptors are statistically reconstructed surrogate variables rather than native observations.' No claimed derivation is equivalent to its own input by construction, so circularity is minimal.

Axiom & Free-Parameter Ledger

6 free parameters · 6 axioms · 2 invented entities

The central dataset is assembled from public catalogs, but its historical depth is bought with regression-fitted surrogate variables. The SEP labels themselves are external, so the target side is not circular; the input side contains fitted free parameters and domain assumptions whose validity outside the overlap periods is untested.

free parameters (6)
  • SMARP to SHARP linear regression coefficients (per feature) = not tabulated; fitted on 2010 overlap
    These coefficients generate SHARP-equivalent magnetic parameters for pre-2010 SMARP records, forming the SMHARP archive.
  • CDAW to DONKI regression coefficients (speed, longitude, half-angle) = not tabulated; fitted on 2010-2025 overlap
    These coefficients generate DONKI-like CME attributes for pre-2010 CDAW records, forming the CDAWDONKI archive.
  • SHARP-SMARP spatial matching threshold = 5 degrees
    Used to link SMARP and SHARP observations by bounding-box or flux-weighted center when NOAA active-region matching is ambiguous.
  • CDAW-DONKI matching cutoff = 24 hours
    CDAW events without a DONKI event within 24 hours are excluded from the alignment set, affecting which events are used to fit the model.
  • kNN imputation neighborhood size = k = 5
    Used to impute missing or nonphysical CME speeds and energies before fitting the alignment model.
  • Default flare log-strength floor for empty windows = log10(1e-10)
    Assigned when no flares occur in a window so that logarithmic summary features remain finite.
axioms (6)
  • domain assumption NOAA operational SEP definition (proton flux > 10 pfu in the >10 MeV GOES channel) is the correct target
    The primary binary label is defined by this external convention; the dataset inherits its operational relevance from this choice.
  • domain assumption CLEAR SEP benchmark event list is complete and correctly labeled
    All SEP event start and end times are taken from this external catalogue; any label errors propagate into every downstream target.
  • ad hoc to paper Linear regression relationships fitted in the 2010 overlap remain valid when applied to 1996-2010 SMARP data
    SMHARP's pre-2010 extension has no native SHARP ground truth; this extrapolation is the central statistical risk of the resource.
  • domain assumption Nearest-in-time matching within 24 hours identifies the same CME in CDAW and DONKI
    Time-only matching may pair unrelated events, and the regression cannot correct event-association errors.
  • domain assumption GOES satellite stitching using primary/secondary intervals yields a homogeneous flux record
    X-ray and proton time series are merged across GOES satellites; residual cross-calibration offsets are not characterized.
  • domain assumption Missing values are ignorable after nearest-neighbor or kNN imputation
    Window summaries inherit imputed values; if missingness is informative, derived summary statistics will be biased.
invented entities (2)
  • SMHARP archive (SHARP-equivalent magnetic parameters reconstructed from SMARP) no independent evidence
    purpose: Extend active-region magnetic predictors from 2010 back to 1996
    Pre-2010 SHARP values do not exist; overlap-period validation is in-sample and does not independently verify the extrapolated era.
  • CDAWDONKI archive (DONKI-like CME attributes reconstructed from CDAW) no independent evidence
    purpose: Extend DONKI-like CME predictors from 2010 back to 1996
    Longitude and half-angle reconstruction correlations are about 0.46-0.49, latitude was excluded, and no pre-2010 validation exists.

pith-pipeline@v1.3.0-alltime-deepseek · 11298 in / 15584 out tokens · 135430 ms · 2026-08-01T21:10:14.587581+00:00 · methodology

0 comments
read the original abstract

Solar energetic particle (SEP) event forecasting often involves integrating heterogeneous observations that differ in cadence, temporal coverage, format, and historical availability, posing challenges for reproducible analysis of data-driven approaches. This paper presents SEP-PRISM Data, a curated multi-source dataset designed for 24-hour ahead forecasting of operational SEP events, defined by proton flux exceeding 10 pfu in the GOES > 10 MeV channel. SEP-PRISM Data integrates flare records, active-region magnetic field parameters, coronal mass ejection (CME) catalogue data, GOES soft X-ray flux, and historical proton flux into a common window-based representation spanning 3 February 1986 to 10 September 2025. To improve temporal coverage and cross-source consistency, SHARP and SMARP magnetic products were aligned into a unified SMHARP archive, and CME records from DONKI and CDAW were aligned into a unified CDAWDONKI event set. Predictor variables were summarized over fixed non-overlapping 24-hour historical windows using minimum, mean, and maximum statistics and paired with targets defined over the subsequent 24-hour window, forming a supervised learning dataset. The resulting SEP-PRISM Data contains 14,464 labeled samples, including 650 positive operational SEP cases, and is intended to support reproducible benchmarking, model development, feature analysis, and future studies of space weather forecasting.

Figures

Figures reproduced from arXiv: 2607.16160 by Kathryn Whitman, Lulu Zhao, Tamas Gombosi, Ward Manchester, Yang Chen, Yian Yu.

Figure 1
Figure 1. Figure 1: Timeline of multi-source data used in this study from 3 February 1986 to 10 September 2025 (adapted from a figure originally published in the Supplementary Information of SEPNET-PRISM Model10). The rows show GOES X-ray flux (XRSB), proton flux (> 10 MeV), flares, CDAW/DONKI CME events, SHARP/SMARP parameters, and SEP events. Shaded bars indicate data availability and the start time and end times of events … view at source ↗
Figure 2
Figure 2. Figure 2: Comparison of SMARP observations, SHARP observations, and regression-based fitted SHARP values for three shared parameters over the period from 1 June 2010 to 10 July 2010. DONKI-CDAW CME fusion validation The DONKI-CDAW alignment was assessed over 3 April 2010 to 9 September 2025 using COR and RMSE computed from temporally matched event pairs ( [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

18 extracted references · 4 canonical work pages

  1. [1]

    P.et al.The economic impact of space weather: Where do we stand?Risk Analysis37, 206–218, DOI: 10.1111/risa.12765 (2017)

    Eastwood, J. P.et al.The economic impact of space weather: Where do we stand?Risk Analysis37, 206–218, DOI: 10.1111/risa.12765 (2017)

    doi:10.1111/risa.12765 2017

  2. [2]

    InTriennial Earth-Sun Summit(2024)

    Whitman, K.et al.A multi-year effort to forward the validation of solar energetic particle models. InTriennial Earth-Sun Summit(2024). 3.Reames, D. Solar energetic particle variations.Adv. Space Res.34, 381–390, DOI: 10.1016/j.asr.2003.02.046 (2004)

    work page doi:10.1016/j.asr.2003.02.046 2024

  3. [4]

    Y ., De Angelis, G

    Kim, M.-H. Y ., De Angelis, G. & Cucinotta, F. A. Probabilistic assessment of radiation risk for astronauts in space missions. Acta Astronaut.68, 747–759, DOI: 10.1016/j.actaastro.2010.08.035 (2011)

    work page doi:10.1016/j.actaastro.2010.08.035 2010

  4. [5]

    & Giacalone, J

    Desai, M. & Giacalone, J. Large gradual solar energetic particle events.Living Rev. Sol. Phys.13, 3, DOI: 10.1007/ s41116-016-0002-5 (2016)

    2016

  5. [6]

    & Dalla, S

    Klein, K.-L. & Dalla, S. Acceleration and propagation of solar energetic particles.Space Sci. Rev.212, 1107–1136, DOI: 10.1007/s11214-017-0382-4 (2017)

    doi:10.1007/s11214-017-0382-4 2017

  6. [7]

    Space Res.72, 5161–5242, DOI: 10.1016/j.asr.2022.08.006 (2023)

    Whitman, K.et al.Review of solar energetic particle prediction models.Adv. Space Res.72, 5161–5242, DOI: 10.1016/j.asr.2022.08.006 (2023)

    doi:10.1016/j.asr.2022.08.006 2022

  7. [8]

    2505.10390

    Sadykov, V .et al.Operational and exploration requirements and research capabilities for SEP environment monitoring and forecasting (2025). 2505.10390

    Pith/arXiv arXiv 2025

  8. [9]

    2606.19539

    Kasapis, S.et al.Review of machine learning models for solar energetic particle prediction.arXiv preprint arXiv:2606.19539(2026). 2606.19539

    Pith/arXiv arXiv 2026

  9. [10]

    2606.14440

    Yu, Y .et al.Realtime forecasting of solar energetic particle event and proton flux using multi-source solar observations and multi-task deep learning (2026). 2606.14440. 11.NOAA Space Weather Prediction Center. Goes proton flux. https://www.swpc.noaa.gov/products/goes-proton-flux

    Pith/arXiv arXiv 2026

  10. [12]

    Yu, Y .et al.Solar energetic particle forecasting with multi-task deep learning: SEPNET.J. Geophys. Res. Mach. Learn. Comput.3, e2026JH001247, DOI: 10.1029/2026JH001247 (2026). 13.Whitman, K. FetchSEP: CLEAR benchmark dataset version 1.0 (2025). Version CLEAR_Benchmark_v1.0, software

    work page doi:10.1029/2026jh001247 2026

  11. [14]

    G., Wright, P

    Bobra, M. G., Wright, P. J., Sun, X. & Turmon, M. J. SMARPs and SHARPs: Two solar cycles of active region data.The Astrophys. J. Suppl. Ser.256, 26, DOI: 10.3847/1538-4365/ac1f1d (2021)

    work page doi:10.3847/1538-4365/ac1f1d 2021

  12. [15]

    Space Weather Database of Notifications, Knowledge, Information (DONKI) (2026)

    NASA Community Coordinated Modeling Center. Space Weather Database of Notifications, Knowledge, Information (DONKI) (2026). Space weather event database. 16.Gopalswamy, N.et al.The SOHO LASCO CME catalog – version 2 (2025). 2407.04165

    Pith/arXiv arXiv 2026

  13. [17]

    S.et al.The SunPy project: Open source development and status of the version 1.0 core package.The Astrophys

    Community, T. S.et al.The SunPy project: Open source development and status of the version 1.0 core package.The Astrophys. J.890, 68, DOI: 10.3847/1538-4357/ab4f7a (2020). 18.Lockheed Martin Solar and Astrophysics Laboratory. Heliophysics events knowledgebase (HEK) (2026). Database

    doi:10.3847/1538-4357/ab4f7a 2020

  14. [19]

    H.et al.The helioseismic and magnetic imager (HMI) investigation for the solar dynamics observatory (SDO)

    Scherrer, P. H.et al.The helioseismic and magnetic imager (HMI) investigation for the solar dynamics observatory (SDO). Sol. Phys.275, 207–227, DOI: 10.1007/s11207-011-9834-2 (2012). 11/12

    doi:10.1007/s11207-011-9834-2 2012

  15. [20]

    H.et al.The solar oscillations investigation-Michelson Doppler imager.Sol

    Scherrer, P. H.et al.The solar oscillations investigation-Michelson Doppler imager.Sol. Phys.162, 129–188, DOI: 10.1007/BF00733429 (1995)

    doi:10.1007/bf00733429 1995

  16. [21]

    G., Choudhary, N., Amezcua, A

    Glogowski, K., Bobra, M. G., Choudhary, N., Amezcua, A. B. & Mumford, S. J. drms: A Python package for accessing HMI and AIA data.J. Open Source Softw.4, 1614, DOI: 10.21105/joss.01614 (2019)

    doi:10.21105/joss.01614 2019

  17. [22]

    Goes x-ray flux

    NOAA Space Weather Prediction Center. Goes x-ray flux. https://www.swpc.noaa.gov/products/goes-x-ray-flux. Numerical data available via https://services.swpc.noaa.gov/json/goes/

  18. [23]

    A new look at the statistical model identification.IEEE Transactions on Autom

    Akaike, H. A new look at the statistical model identification.IEEE Transactions on Autom. Control.19, 716–723 (1974). 12/12

    1974