Beyond Mean Solar Wind Conditions: Turbulence-Aware Forecasting of the AE Index

Cara L. Waters; Christopher H. K. Chen; Mathew J. Owens

REVIEW 2 major objections 8 minor 2 references

Explicit solar-wind turbulence descriptors—fluctuation amplitude, intermittency, and Alfvénic structure—improve short-horizon forecasts of the AE index beyond mean conditions, with stable economic value at extreme thresholds.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 11:10 UTC pith:7IMT4KJ3

load-bearing objection Solid empirical comparison showing turbulence descriptors add modest AE forecast skill, but the headline cost/loss robustness at extreme thresholds is likely over-interpreted from a sparse two-year test tail. the 2 major comments →

arxiv 2606.16518 v1 pith:7IMT4KJ3 submitted 2026-06-15 physics.space-ph

Beyond Mean Solar Wind Conditions: Turbulence-Aware Forecasting of the AE Index

Cara L. Waters , Christopher H. K. Chen , Mathew J. Owens This is my paper

classification physics.space-ph

keywords AE indexsolar wind turbulencespace weather forecastingauroral electrojetgradient-boosted treesIMF Bz fluctuationscost-loss analysisnorthward IMF

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper sets out to show that explicit solar wind turbulence information—fluctuation amplitude, intermittency, and Alfvénic structure—improves short-timescale forecasts of the AE index beyond what mean solar wind parameters alone can deliver. By comparing a baseline gradient-boosted tree model against a turbulence-augmented version over the same 2010–2024 data, it finds systematic gains that are small in global averages but concentrated where baseline models struggle: during northward IMF and at high AE levels. The turbulence-aware model keeps similar skill across 60–90 minute horizons, whereas the baseline has a sharp peak at 75 minutes, and it reduces false alarms for extreme events. A cost-loss analysis shows the turbulence-aware forecast holds a roughly constant zero-value threshold across AE event severities, indicating stable decision-relevant value where the baseline becomes less useful as thresholds rise. A reader should care because it identifies a physically interpretable, low-complexity way to push space-weather forecasting closer to the information limit of upstream solar wind measurements.

Core claim

On the paper's own terms, the discovery is that turbulence descriptors carry complementary, scale-dependent information about geoeffectiveness. Two one-minute-cadence models were trained on near-Earth solar wind and IMF measurements with strictly causal time lags: a baseline using rolling means of density, velocity, field magnitude and components, and a turbulence-aware model that adds RMS fluctuation amplitudes, higher-order moments, and the normalized cross helicity, residual energy, and magnetic compressibility over 5–60 minute windows. The turbulence-aware model improves over the baseline and over persistence at every lead time tested, passes the 0.8 correlation mark at 60 minutes, and—u

What carries the argument

The load-bearing element is the turbulence descriptor set: fluctuations δB and δV computed relative to a rolling-mean background, summarized by RMS amplitudes, skewness and kurtosis, and by three dimensionless MHD turbulence parameters—the normalized cross helicity σc = 2⟨δv·δb⟩/⟨|δv|²+|δb|²⟩, residual energy σr = (⟨|δv|²⟩−⟨|δb|²⟩)/(⟨|δv|²⟩+⟨|δb|²⟩), and magnetic compressibility CB = ⟨δ|B|²⟩/⟨δBx²+δBy²+δBz²⟩—computed over 5–60 minute windows and appended to the mean-parameter feature set. A specially constructed noise-control model, with these features replaced by statistically matched random surrogates, isolates their contribution: it performs like the baseline, showing the improvement is i

Load-bearing premise

The operational cost-loss claim assumes the 2023–2024 test set contains enough minutes above the 800–1200 nT thresholds to fix the zero-value intercept; with a two-year test window dominated by quiet time, the reported constant intercept could be sampling noise rather than a stable property of the turbulence-aware model.

What would settle it

Compute the zero-value cost-loss intercept for each threshold from block-bootstrapped samples of the 2023–2024 test set (block resampling to preserve autocorrelation), and check whether the turbulence-model intercept is constant to within the bootstrap uncertainty; if the intercept declines with threshold once sampling error is accounted for, the economic-stability claim is refuted.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

Operational AE forecasts built from upstream measurements can include turbulence descriptors as cheap, interpretable additions without changing model architecture.
A single turbulence-aware model can serve users with widely different cost-loss preferences across event severities, since its positive-value range does not collapse at high AE thresholds.
The 60–90 minute plateau of skill suggests that fluctuation information extends the useful forecast horizon by roughly 15–30 minutes relative to mean-parameter models.
Improvements under northward IMF imply turbulence inputs matter most when steady dayside reconnection is suppressed—relevant for predicting activity during quiet or weakly driven intervals.
The persistence of improvements across test events indicates upstream solar wind fluctuation structure is a stable and generally applicable predictor, not a tuning artifact.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

Because the same turbulence diagnostics are physically grounded in MHD, a similar descriptor set should transfer to other electrojet indices (AL, SML) and perhaps to radiation-belt electron flux, where fluctuation-driven transport matters.
The paper leaves open whether a probabilistic formulation would preserve the constant zero-value cost-loss threshold; one testable extension is to replace the deterministic threshold forecast with calibrated probabilities and re-run the cost-loss analysis on a longer test period.
The sharpest untested implication is that turbulence descriptors may act as proxies for unresolved solar wind structure (sub-ion scales, stream interfaces); comparing with higher-cadence measurements upstream would separate intrinsic turbulence effects from sampling artifacts.
Since tree models systematically under-predict the most extreme AE spikes, combining turbulence features with regression methods designed for imbalanced extremes could close the residual gap above the 99th percentile that this paper documents.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

Solid empirical comparison showing turbulence descriptors add modest AE forecast skill, but the headline cost/loss robustness at extreme thresholds is likely over-interpreted from a sparse two-year test tail.

read the letter

Dear colleague,

The paper is a careful XGBoost comparison for AE forecasting: a baseline using mean solar wind/IMF parameters plus AE history, versus a model that adds turbulence diagnostics (RMS fluctuations, skewness, kurtosis, cross helicity, residual energy, compressibility), with a noise-surrogate control. The design is solid: chronological split, causal lags, persistence baseline, and the surrogate control isolates the effect of adding features rather than just more inputs. That last piece is a genuinely useful addition to the AE-ML literature.

The central claim, that turbulence descriptors add complementary skill beyond mean parameters, is plausible and mostly supported. The gains are modest in global metrics (0.4–2% MAE) but concentrated where baseline models are weakest: northward IMF and elevated AE. The SHAP analysis is broadly consistent with known coupling physics, and the finding that short-timescale Bz fluctuations matter most under northward IMF is interesting.

Now the soft spots. No confidence intervals or significance tests accompany the reported gains, so we don't know if the improvement is stable or driven by a few storms. The skill plateau at 60–90 min for the turbulence model is presented as a headline result, but hyperparameters are only shown for the 60-min models; whether other horizons were tuned separately or used fixed settings is not stated, which makes the plateau comparison hard to interpret.

The bigger issue is the cost/loss claim in the abstract: "maintains an approximately constant zero-value cost-loss threshold across all event levels." The test set is only 2023–2024, and at AE thresholds of 800–1200 nT the number of qualifying minutes is tiny—the paper itself notes the test interval contains the two largest AE spikes of the whole period. Figure 7c is essentially a small-sample contingency table; one missed or false-alarm storm can shift the intercept substantially. No event counts, confidence intervals, or sensitivity analysis are given. The constant intercept could be sampling noise or a degenerate regime, not a stable economic property. This doesn't invalidate the average-error improvement, but this operational claim is load-bearing and it is not demonstrated.

Who is this for? Researchers working on solar wind–magnetosphere coupling and operational AE/Kp forecasting will get value from the feature comparison and the surrogate-control methodology. It deserves a serious referee—the empirical design is above average—but the revision should add significance testing and rework or soften the cost/loss conclusion at extreme thresholds.

Recommendation: send to review, but ask for bootstrap confidence intervals or event-count reporting for the cost/loss analysis, and clarification on hyperparameter tuning across horizons.

Referee Report

2 major / 8 minor

Summary. The paper tests whether adding solar wind turbulence descriptors (fluctuation amplitudes, skewness/kurtosis, normalized cross helicity, residual energy, magnetic compressibility) to an XGBoost AE forecast model improves skill beyond a baseline using mean solar wind and IMF parameters plus AE history. Data are 1-min OMNI and AE from 2010 to 2024, with a chronological train/test split (2010–2022 train, 2023–2024 test) and strictly causal feature lags. Three models are compared: baseline, turbulence-aware, and a noise surrogate. Reported results include correlation >0.8 at 60 min, sustained turbulence-model skill from 60 to 90 min, modest MAE improvements concentrated at high AE and northward IMF, and a cost/loss analysis suggesting the turbulence model's zero-value cost/loss threshold is roughly constant across AE thresholds 800–1200 nT while the baseline's decreases. The manuscript concludes that turbulence provides complementary, decision-relevant information beyond mean solar wind parameters.

Significance. If the results hold, the paper is a useful contribution to operational space weather forecasting by showing that simple, physically interpretable turbulence descriptors add forecast value beyond mean solar wind parameters. The experimental design has notable strengths: a chronological split, causal feature lags, a persistence baseline, a noise-surrogate control, SHAP-based physical interpretation, and publicly available code. The main limitations are the short test window and the absence of uncertainty quantification, which currently leave the extreme-event cost/loss claim under-supported.

major comments (2)

[Section 6, Figure 7, Eq. (12)] The abstract's claim that the turbulence-aware model 'maintains an approximately constant zero-value cost-loss threshold across all event levels' is load-bearing but rests on very few extreme minutes. The test period is only two years (Jan 2023–Dec 2024), and the manuscript itself notes (Section 4, Figure 4) that it contains the two largest AE spikes of the whole record. At thresholds of 800–1200 nT, the number of qualifying minutes is small, and the zero-value intercept of V(r) in Eq. (12) is a contingency-table statistic that can shift substantially with one storm. No event counts, confidence intervals, or bootstraps are provided, and the limitations paragraph in Section 7 does not mention this sparse-sample issue. I request the manuscript report the number of event minutes at each threshold, add bootstrap or permutation confidence intervals for the intercepts, and test sensitivity to
[Section 4, Figures 2 and 3] The central claim of 'consistent improvement' over the baseline and persistence is based on MAE differences of order 0.4–2%, with no quantification of sampling uncertainty. The binned comparisons (e.g., northward Bz in Figure 2b and AE-percentile bands in Figure 3b) likely have small sample sizes in the high-activity bins, and the reported differences may be within noise. The noise-surrogate model is a good control for feature count but does not address sampling variability. Please provide confidence intervals (e.g., block bootstrap over the test period) or formal tests for the key comparisons, including the claimed 60–90 min skill plateau of the turbulence-aware model in Figure 2a.

minor comments (8)

[Section 3.3, hyperparameter paragraph] In the description of hyperparameters, the text says 'Finally, γ is the L2 regularisation term on the leaf weights,' but the L2 term is λ in Table 1 and in the preceding sentence. This is likely a typo.
[Section 8, first paragraph] The phrase 'physically motivated measured of variability' should be 'physically motivated measures of variability'.
[Section 6, first paragraph] The phrase 'the span of r over which the curve is positive indicates the breath of users' should be 'breadth of users'.
[Section 2, data preprocessing] State explicitly whether 'forward-interpolated' for gaps of up to three minutes is a forward-fill using past values only, and confirm that no future information enters the features at time t.
[Figure 2a caption] The caption does not describe the markers for the noise model (squares, triangles, circles are listed but not assigned); also the text alternates between 'noise model' and 'noise-control model'. Please make terminology and figure legend consistent.
[Equations (1)–(3)] Equation (3) normalizes B by the local Alfvén speed, but the units of B and n are not specified in the equation. Please state the unit conversions used in the implementation to avoid ambiguity.
[References] The SHAP references appear twice with slightly different author formatting (Lundberg & Lee 2017a/b). Please consolidate into a single citation.
[Section 3.5, Eq. (8)] The climatological event rate o in Eq. (8) is computed from the same test period used in Eqs. (10)–(11). Clarify whether the test-period frequency is the intended climatology or whether a long-term (training) climatology should be used for the climatological decision in Eq. (10).

Circularity Check

0 steps flagged

No significant circularity: the turbulence-aware AE comparison is an independently evaluated empirical test, not a derivation that reduces to its inputs.

full rationale

The central claim—that adding solar wind turbulence descriptors to mean solar wind predictors improves short-horizon AE forecasts—is tested on a chronological held-out test set (Jan 2010–Dec 2022 training; Jan 2023–Dec 2024 test, Section 2), with strictly causal rolling-window features (Section 3.1) and a noise-surrogate control that performs like the baseline model (Sections 3.2 and 4). The turbulence features (Eqs. 1–4) are constructed solely from solar wind/IMF fluctuations and are not defined in terms of the AE target, so the comparison is not self-definitional. The cost/loss analysis (Eqs. 6–12, Section 3.5) is a standard evaluation (Richardson 2011; Owens and Riley 2017) applied to independent test-set predictions; the Figure 7 zero-value intercept comparison is a performance statistic, not a fitted parameter renamed as a prediction. The self-citations present (C. H. K. Chen, 2016, for sigma_c/sigma_r; Waters, 2026, for code) are definitional/reproducibility references and do not carry the central result. The manuscript itself flags the real limitation for the extreme-event cost/loss claim—'tree ensembles cannot extrapolate beyond the range of target values present in the training data' (Section 3.2) and the test interval contains the two largest AE spikes of the whole period (Section 4, Figure 4)—so the approximately constant zero-value intercept at 800–1200 nT is a small-sample/robustness concern, not circularity.

Axiom & Free-Parameter Ledger

2 free parameters · 4 axioms · 0 invented entities

The central claim depends mainly on feature/window design and test-period event sampling; no new physical entities are introduced. The free parameters are the ML hyperparameters and hand-chosen feature-window lengths; no physical constants were fitted.

free parameters (2)

XGBoost hyperparameters (60-min models) = base: learning_rate=0.0182, max_depth=4, subsample=0.423, colsample_bytree=0.830, min_child_weight=0.152, gamma=3.66, la
Tuned with Optuna on a validation subset; only the 60-min horizon values are reported, leaving cross-horizon transferability an assumption.
Feature rolling-window lengths = Bulk windows 5-120 min; turbulence windows 5-60 min (e.g., 5, 10, 30, 60, 120 and 5, 15, 30, 45, 60 min shown in Figures
Chosen by hand rather than optimized; the conclusion that turbulence adds skill at 60-90 min horizons may be sensitive to these timescales.

axioms (4)

domain assumption OMNI phase-front propagation to the bow shock has timing uncertainty small relative to fluctuation windows and lead times.
Stated in Section 2: 'This uncertainty is small compared with the timescales over which the fluctuations we examine evolve and the forecast lead time'.
domain assumption The 2023-2024 test interval contains enough extreme AE samples to support cost-loss claims at 800-1200 nT thresholds.
The cost-loss analysis in Section 6 uses only this test period; high-threshold events are rare, so stable zero-value intercepts are assumed rather than demonstrated with uncertainty.
domain assumption AE(t) is known in real time and can be used as a model input in operational forecasting.
AE history is the dominant predictor (Section 5.1); if AE at time t is not available to forecasters, the practical lead time is reduced.
domain assumption The 2010-2022 training distribution encompasses the range of solar wind driver states seen in 2023-2024, including the May 2024 storm.
Tree ensembles cannot extrapolate beyond the training target range (stated in Section 3.2); the extreme-event improvements depend on the test events being at least partially represented in training.

pith-pipeline@v1.3.0-alltime-deepseek · 19880 in / 15247 out tokens · 156066 ms · 2026-08-02T11:10:01.516455+00:00 · methodology

0 comments

read the original abstract

The auroral electrojet (AE) index is a key indicator of high latitude geomagnetic activity and is widely used in operational space weather monitoring, yet forecasting AE from upstream solar wind conditions remains challenging due to nonlinear coupling, internal magnetospheric dynamics, and multiscale variability. We test whether incorporating solar wind turbulence improves short timescale AE forecasts beyond models based only on mean solar wind and interplanetary magnetic field parameters. Two gradient boosted decision tree (XGBoost) models are developed using near-Earth solar wind observations: a baseline model using standard mean parameters and a turbulence-aware model that additionally includes measures of fluctuation amplitude, intermittency, and Alfvenic structure. Both models achieve peak performance at short lead times, with correlations exceeding 0.8 at 60 minutes. However, while the baseline model exhibits a clear skill peak at 75 minutes, the turbulence-aware model maintains comparable skill across 60-90 minute horizons, indicating reduced degradation with lead time. The turbulence-aware model also provides consistent improvements over both the baseline and persistence and, critically, improves forecast robustness for high-impact events. Cost-loss analysis shows that, for the baseline model, economic value decreases systematically with increasing AE threshold and the range of cost-loss ratios yielding positive value narrows. In contrast, the turbulence-aware model maintains an approximately constant zero-value cost-loss threshold across all event levels, indicating stable economic usefulness even for extreme AE conditions. This demonstrates that turbulence provides complementary, scale-dependent information beyond mean solar wind parameters, improving both forecast performance and decision-relevant value for operational space weather applications.

Figures

Figures reproduced from arXiv: 2606.16518 by Cara L. Waters, Christopher H. K. Chen, Mathew J. Owens.

**Figure 1.** Figure 1: An overview of the period of OMNI data and AE index used, with (a) the AE index over the entire period, with the training period highlighted in blue, the test period highlighted in red, and the maximum of solar cycle 24 and start of solar cycle 25 indicated by vertical lines, (b) histograms of AE index over both test periods, with means and means plus three standard deviations indicated, and (c) the perc… view at source ↗

**Figure 2.** Figure 2: (a) Comparison of the performance of each of the base, turbulence and noise models for each of the chosen forecast horizons, with skill vs persistence shown in blue and correlation coefficient in red, with triangular markers for the turbulence-aware model, squares for the noise model, and circles for the base model. The range of horizons giving high quality predictions of AE index are shown by a grey shad… view at source ↗

**Figure 3.** Figure 3: (a) Observed vs predicted AE index by the base model (blue) and turbulence model (red), binned in predicted AE index, with AEobs = AEpred shown as a grey dashed line. The associated 10th and 90th percentiles are shown for each model as a confidence interval. (b) Average MAE for both the base (blue) and turbulence (red) models for bands of AE index, defined by the 0th, 50th, 75th, 90th, 95th, 99th and 100t… view at source ↗

**Figure 4.** Figure 4: Two example space weather events with (a, e) observed AE index in grey, predicted (by the base model) AE index in blue, and predicted (by the turbulence model) AE index in red, (b, f) zGSM component of magnetic field Bz,GSM in nT in purple and RMS fluctuations in this component δBz,rms in yellow, (c, g) xGSE component of solar wind velocity Vx,GSE (km/s) in green, and (d, h) proton number density n (cm−3 )… view at source ↗

**Figure 5.** Figure 5: (a, b, d, e) Mean of the magnitudes of the SHAP values in the turbulence-aware model for each of the features shown as a heatmap, organised by physical variable and window W, for (a) bulk parameters under southward IMF, (b) bulk parameters under northward IMF, (d) turbulence parameters under southward IMF, and (e) turbulence parameters under northward IMF. (c) shows the difference in these values (southwa… view at source ↗

**Figure 6.** Figure 6: Median values of SHAP value for bins of (a) z component of magnetic field Bz,GSM, (b) RMS fluctuations of Bz δBz,rms, (c) skew of δBz, (d) x component of velocity Vx,GSE, (e) proton number density n, (f) magnetic field strength |B|, (g) RMS fluctuations of field strength δ|B|rms, (h) magnetic compressibility CB, and (i) residual energy σr. Bulk parameters (a, d, e, f) are shown windowed over 5, 10, 30, 60 … view at source ↗

**Figure 7.** Figure 7: Potential economic value V against cost/loss ratio r for the base model in (a) and turbulence model in (b), for a range of thresholds of AE between 800 and 1200 nT. (c) The xintercept of each of the curves plotted against the threshold AE index for the turbulence model (red circles) and the base model (blue squares). bulence model is largely invariant, whereas the intercept for the base model decreases st… view at source ↗

discussion (0)

Reference graph

Works this paper leans on

2 extracted references · 1 canonical work pages

[130]

doi: 10.1029/2025JA034571 Axford, W. I. (1964). Viscous interaction between the solar wind and the earth’s magnetosphere.Planetary and Space Science,12, 45–53. doi: 10.1016/0032 -0633(64)90067-4 Axford, W. I., & Hines, C. O. (1961). A unifying theory of high-latitude geophysical phenomena and geomagnetic storms.Canadian Journal of Physics,39, 1433–

work page doi:10.1029/2025ja034571 1964
[1464]

F., Baker, D

doi: 0.1139/p61-172 Bargatze, L. F., Baker, D. N., McPherron, R. L., & Hones, E. W. (1985). Magne- tospheric impulse response for many levels of geomagnetic activity.Journal of Geophysical Research,90(A7), 6387–6394. doi: 10.1029/JA090iA07p06387 Beedle, J. M. H., Genestreti, K. J., Shuster, J. R., Rice, R. C., Fuselier, S. A., Phan, T. D., . . . Torbert, ...

arXiv 1985

[1] [130]

doi: 10.1029/2025JA034571 Axford, W. I. (1964). Viscous interaction between the solar wind and the earth’s magnetosphere.Planetary and Space Science,12, 45–53. doi: 10.1016/0032 -0633(64)90067-4 Axford, W. I., & Hines, C. O. (1961). A unifying theory of high-latitude geophysical phenomena and geomagnetic storms.Canadian Journal of Physics,39, 1433–

work page doi:10.1029/2025ja034571 1964

[2] [1464]

F., Baker, D

doi: 0.1139/p61-172 Bargatze, L. F., Baker, D. N., McPherron, R. L., & Hones, E. W. (1985). Magne- tospheric impulse response for many levels of geomagnetic activity.Journal of Geophysical Research,90(A7), 6387–6394. doi: 10.1029/JA090iA07p06387 Beedle, J. M. H., Genestreti, K. J., Shuster, J. R., Rice, R. C., Fuselier, S. A., Phan, T. D., . . . Torbert, ...

arXiv 1985