Pith. sign in

REVIEW 3 major objections 4 minor 8 references

Predictive Analytics of Air Alerts in the Russian-Ukrainian War

T0 review · 3 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Air alerts in a Ukrainian region can be predicted 5 to 15 minutes ahead from neighbor regions' alert status and the calendar.

desk verdict A plausible geospatial claim undercut by target leakage in the feature construction; the EDA is fine, but the accuracy numbers reflect persistence, not forecasting. read the letter →

arxiv 2411.14625 v1 pith:5F5EMSUC submitted 2024-11-21 cs.LG cs.AIcs.CY

Demian Pavlyshenko , Bohdan Pavlyshenko This is my paper · ORCID
classification cs.LGcs.AIcs.CY
keywords airalertsUkrainewarpredictiveanalyticsrandomforestgeospatialcorrelationtimeseriesclassificationearlywarning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that air alerts in Ukrainian regions are geographically correlated, and that these correlations can be exploited to forecast whether a region will be under alert within the next 5 or 15 minutes. Using minute-level alert data from March 2022 through November 2024, the authors train separate random-forest classifiers for four regions, with features drawn from all regions' alert duration counters plus calendar variables. The reported feature-importance rankings show that neighboring regions' alert statuses contribute most to predictions for a given region, with hour, day of week, month, and days-since-start also contributing. If this is right, civilians and emergency services could gain a few minutes of advance notice before an alert reaches their own region.

What carries the argument

The central object is the minute-by-minute binary alert time series for each Ukrainian region, encoded as a cumulative alert-duration feature that is zero during quiet periods and equals the current alert's length during an alert. Four separate random-forest classifiers are trained, one per target region, using these cumulative features for all regions plus month, day of week, hour, and a days-since-start counter; each classifier's target is a binary indicator of whether the target region will be under alert within the next 5 minutes (or 15 minutes in a second setup). The random forest's feature-importance output is the mechanism used to identify which neighboring regions matter most for each target. This setup lets the authors test whether geospatial correlation in alert time series carries predictive information beyond the target region's own current state.

What would settle it

Re-run the same 5-minute-ahead experiment with the target region's own cumulative alert-duration feature removed, or shifted forward by the full prediction horizon, and compare accuracy against a baseline that simply predicts the current alert status will persist for another 5 minutes. If the neighboring-region features add little beyond that persistence baseline, then the paper's core claim about geospatial predictive power is unsupported.

Watch

Extended reading notes

Core claim

The central claim is that the binary alert status of a Ukrainian region can be predicted from the current alert statuses of adjacent regions, with seasonality and elapsed time as secondary signals. For each of four target regions, a random-forest model was trained on minute-by-minute cumulative alert-duration features for all regions and on calendar features, predicting a target of whether an alert would be active within the next 5 or 15 minutes. The authors report that neighboring-region features dominate the feature-importance rankings, that ROC and accuracy scores on a held-out test period support feasibility, and that the importance of the days-since-start feature indicates alert patterns change over time.

Load-bearing premise

The reported skill is assumed to reflect forecasting, not the persistence of already-running alerts, because the target region's own cumulative alert-duration feature may leak the current status into the next-5-minute target.

Editorial extensions

If this is right

  • If the geospatial pattern holds outside the training window, regional alert status can be forecast 5 to 15 minutes ahead using only current alert states of adjacent regions and the calendar.
  • The method could be packaged into an early-warning tool that gives people extra minutes to reach shelter before an alert is officially declared in their own region.
  • The reported dominance of neighboring-region features implies that a regional alert network behaves like a propagating wave, so a model trained on one set of regions may transfer to nearby regions with limited retraining.
  • Because alert patterns change over time, any operational model would need periodic retraining to track evolving strike behavior.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An ablation study that removes each neighboring region's features one at a time would show which geographic links carry the predictive signal, rather than relying on random-forest feature-importance rankings alone.
  • The same 5- and 15-minute prediction framing could transfer to other spatially propagating hazards, such as severe weather warnings or earthquake early warning, where nearby sensors provide advance information.
  • A stricter evaluation with all features lagged by at least the full prediction horizon would be needed to separate true forecasting skill from persistence, and would likely lower the currently reported accuracy figures.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper analyzes minute-level air-alert data for Ukrainian regions from March 2022 to November 2024. It presents exploratory correlation heatmaps and trains a Random Forest classifier for four regions (Lvivska, Vinnytsia, Kyivska, Kharkivska) to predict whether an alert will occur within the next 5 or 15 minutes. Features include current cumulative alert durations for regions, calendar features (month, day of week, hour), and a day-count feature. The authors report high accuracy and AUC values and conclude that neighboring regions' alert features are the most important predictors. The paper includes an appendix with code for training and validation and extension to a 15-minute horizon.

Significance. If the conclusions were supported, the paper would provide a practical demonstration that geospatial correlations in air alerts can be exploited for short-term early warning, which would be of genuine value in a conflict setting. The underlying public dataset and the authors' attempt to share code are positive aspects, and the exploratory correlation analysis is a reasonable first step. However, the current analysis does not establish the main claim because the feature set includes an unlagged measure of the target region's current alert status and no persistence baseline or ablation is reported. The paper's contribution is therefore currently at the level of a hypothesis-generating case study rather than a validated predictive model.

major comments (3)
  1. [Section 3, Appendix A.2] The cumulative alert-duration feature for the target region is not lagged and acts as a near-persistence proxy. The feature is defined as 0 when no alert is ongoing and as the current alert duration otherwise. The target is 1 when an alert starts within the next 5 minutes or when an ongoing alert lasts at least 5 more minutes. Consequently, a model can achieve high accuracy by predicting 1 whenever the target region's own cumulative-duration feature is positive and 0 otherwise, especially since most alerts last longer than 5 minutes. The paper does not report this trivial baseline, nor does it ablate the target region's own feature. Without such an ablation, the feature-importance plots (Figures 11-22 and 23-34) cannot support the central conclusion that neighboring regions' features are the most important predictors. Please re-run the analysis with lagged features only, or at least remove the target region's own current-status feature and compare against a persistence baseline.
  2. [Section 3] The evaluation consists of a single chronological split at 2024-07-01, with no confidence intervals, no repeated runs, and no comparison against simple baselines (e.g., always predict current status, majority class, logistic regression with the same features). Random Forest also has hyperparameters (n_estimators, max_depth, etc.) whose sensitivity is not examined. The reported accuracy and AUC values are therefore not enough to establish that the model has genuine forecasting skill. Please add error bars over multiple seeds or bootstraps, and include at least a persistence baseline and a no-skill baseline.
  3. [Section 3, Appendix A.2] The exact feature set and target construction are not fully specified. The text mentions both lagged binary features and cumulative duration features, but it is not stated which of these are used in the final model, nor whether all regions' cumulative features are included. The code in A.2 only shows the fitting and evaluation; it does not show how the target variable is derived from the raw alert series or how the feature matrix is constructed. This makes the experiments irreproducible and prevents an independent check of the leakage concern. Please provide the full data-preparation code and a data dictionary listing all features and the precise target definition.
minor comments (4)
  1. [Figure 25 caption] The caption reads 'Accuracy scores for Lvivska oblas'; 'oblas' should be 'oblast'.
  2. [Figures 29 and 30] Both figures carry the title 'Features importance for Kyivska oblast (15 minute time horizon of the target variable)'; one of them should be the ROC curve for that region.
  3. [References] The paper does not state the access date or version for the GitHub dataset [1]; please add this information for reproducibility.
  4. [Section 3] The 'ndays' feature is described as reflecting that 'air alert pattern changes with time', but the paper does not discuss whether this represents a real temporal trend or a data artifact; a brief discussion would help.

Circularity Check

1 steps flagged · score 6.0 of 10

Unlagged own-region cumulative duration feature makes reported near-term accuracy a persistence restatement; geospatial prediction claim is only partially supported.

  1. self definitional [Section 3 ('Training and validation of machine learning model'), target and cumulative-feature definitions; Appendix A.2 code]
    "To build a predictive model, we can use lagged values for binary regions features. So, we can face with a large number of lagged values. We tried another approach based on the features with cumulative values for alert duration. When there is no air alert, the value for this feature is 0. When the air alert starts in a region, then this feature is equal to the current duration of the alert. When the alert in the region is over, then the feature will have the value 0. ... In the case when the alert is ongoing, it means that the alert will last for at least 5 minutes."

    The target is defined so that any ongoing alert makes the next-5-minute target equal to 1: 'value 1 means that the alert will take place within next 5 minutes' and 'In the case when the alert is ongoing, it means that the alert will last for at least 5 minutes.' The cumulative alert-duration feature is, by the quoted definition, exactly the current alert duration: nonzero while an alert is ongoing and 0 otherwise. For the target region, a nonzero value of this feature therefore logically forces target=1 at the same timestamp. The Appendix code (A.2) constructs X and y from the same alert_ts frame with no shift/lag operation, and no code path drops the target region's own cumulative feature.

full rationale

The paper's derivation chain contains one load-bearing circular step. The own-region cumulative alert-duration feature is not lagged, and the target is defined to include ongoing alerts, so part of the reported predictive performance reduces to persistence of the current alert state rather than to a true 5- to 15-minute forecast. No self-citation chain is involved: the dataset is external and no prior author results are invoked. The circularity is partial because the model also uses calendar features and neighboring-region features, which could carry independent predictive signal; however, the reported accuracy and AUC figures cannot be interpreted as evidence for the geospatial-propagation conclusion without a baseline such as 'always predict current status' or an ablation that removes the target region's own cumulative feature. The paper provides neither, so the central empirical claim is partially circular. Score 6 reflects that one or more predictions reduce by construction while the neighbor-feature part retains some independent content.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central predictive claim rests on external dataset accuracy, a feature set that mixes contemporaneous regional states with calendar variables, and hand-chosen model settings. The most consequential assumption is that current cumulative alert duration features are legitimate predictors; because the target includes ongoing alerts, this assumption creates a shortcut that inflates apparent model skill.

free parameters (4)
  • n_estimators = 500
    Number of trees in the Random Forest, chosen by the authors in training; not tuned and affects variance of reported accuracy.
  • random_state = 15
    Single fixed seed for the classifier; no repeated-seed averaging or confidence intervals are provided.
  • train_test_split_date = 2024-07-01
    Temporal split point chosen by hand with no sensitivity analysis; model performance could vary with this date.
  • time_horizon_minutes = 5 and 15
    Target horizon in the definition of the label; both values are design choices, not data-fitted quantities.
assumptions (4)
  • domain assumption The external air alert dataset [1] is complete and accurate at one-minute resolution.
    All analysis and labels derive from this dataset; the paper does not validate against official sources.
  • domain assumption Region-level alerts can be predicted from contemporaneous alert features of all regions plus calendar variables.
    The predictive model assumes that current alert state of regions and time features are sufficient for next-5-minute prediction.
  • domain assumption Random Forest with hand-chosen hyperparameters is an appropriate model for this binary time-series classification.
    No comparison to other model classes or temporal cross-validation is provided.
  • domain assumption The target definition, 1 if an alert occurs within the next 5 minutes including ongoing alerts, is a valid operationalization of prediction.
    This definition conflates ongoing alerts with newly starting alerts, which drives the leakage issue.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Predictive Analytics of Air Alerts in the Russian-Ukrainian War." pith.science (2026). https://pith.science/paper/5F5EMSUC

@misc{pith2026241114625,
  author       = {Pith},
  title        = {Pith review of: Predictive Analytics of Air Alerts in the Russian-Ukrainian War},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5F5EMSUC}},
  note         = {Machine review of arXiv:2411.14625}
}
read the original abstract

The paper considers exploratory data analysis and approaches in predictive analytics for air alerts during the Russian-Ukrainian war which broke out on Feb 24, 2022. The results illustrate that alerts in regions correlate with one another and have geospatial patterns which make it feasible to build a predictive model which predicts alerts that are expected to take place in a certain region within a specified time period. The obtained results show that the alert status in a particular region is highly dependable on the features of its adjacent regions. Seasonality features like hours, days of a week and months are also crucial in predicting the target variable. Some regions highly rely on the time feature which equals to a number of days from the initial date of the dataset. From this, we can deduce that the air alert pattern changes throughout the time.

Figures

Figures reproduced from arXiv: 2411.14625 by the authors.

Figure 1
Figure 1. Heatmap for the total duration of air alerts in Ukrainian regions (minutes) [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Heatmap for regions’ total duration of air alerts which occurred simultaneously [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Time series for daily median for alert durations by regions (minutes) [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (31 more)
Figure 4
Figure 4. Figure 4: Boxplots for the duration of alerts by regions (minutes) [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: Boxplots for the total duration of air alerts per day by regions (minutes) [PITH_FULL_IMAGE:figures/full_fig_p004_5.png]
Figure 6
Figure 6. Figure 6: Correlation heatmap of time series of binary alert features for Ukrainian regions [PITH_FULL_IMAGE:figures/full_fig_p005_6.png]
Figure 7
Figure 7. Figure 7: Alert time feature for Lvivska oblast (minutes) [PITH_FULL_IMAGE:figures/full_fig_p006_7.png]
Figure 8
Figure 8. Figure 8: Alert time feature for Vinnytska oblast (minutes) [PITH_FULL_IMAGE:figures/full_fig_p006_8.png]
Figure 9
Figure 9. Figure 9: Alert time feature for Kyivska oblast (minutes) [PITH_FULL_IMAGE:figures/full_fig_p006_9.png]
Figure 10
Figure 10. Figure 10: Alert time feature for Kharkivska oblast (minutes) [PITH_FULL_IMAGE:figures/full_fig_p006_10.png]
Figure 11
Figure 11. Figure 11: Features importance for Lvivska oblast (5 minute time horizon of target vari [PITH_FULL_IMAGE:figures/full_fig_p007_11.png]
Figure 12
Figure 12. Figure 12: ROC curve for Lvivska oblast (5 minute time horizon of target variable) [PITH_FULL_IMAGE:figures/full_fig_p008_12.png]
Figure 13
Figure 13. Figure 13: Accuracy scores for Lvivska oblast (5 minute time horizon of target variable) [PITH_FULL_IMAGE:figures/full_fig_p008_13.png]
Figure 14
Figure 14. Figure 14: Features importance for Vinnytska oblast (5 minute time horizon of target [PITH_FULL_IMAGE:figures/full_fig_p009_14.png]
Figure 15
Figure 15. Figure 15: ROC curve for Vinnytska oblast (5 minute time horizon of target variable) [PITH_FULL_IMAGE:figures/full_fig_p009_15.png]
Figure 16
Figure 16. Figure 16: Accuracy scores for Vinnytska oblast (5 minute time horizon of target variable) [PITH_FULL_IMAGE:figures/full_fig_p010_16.png]
Figure 17
Figure 17. Figure 17: Features importance for Kyivska oblast (5 minute time horizon of target vari [PITH_FULL_IMAGE:figures/full_fig_p010_17.png]
Figure 18
Figure 18. Figure 18: ROC curve for Kyivska oblast (5 minute time horizon of target variable) [PITH_FULL_IMAGE:figures/full_fig_p011_18.png]
Figure 19
Figure 19. Figure 19: Accuracy scores for Kyivska oblast (5 minute time horizon of target variable) [PITH_FULL_IMAGE:figures/full_fig_p011_19.png]
Figure 20
Figure 20. Figure 20: Features importance for Kharkivska oblast (5 minute time horizon of target [PITH_FULL_IMAGE:figures/full_fig_p012_20.png]
Figure 21
Figure 21. Figure 21: ROC curve for Kharkivska oblast (5 minute time horizon of target variable) [PITH_FULL_IMAGE:figures/full_fig_p012_21.png]
Figure 22
Figure 22. Figure 22: Accuracy scores for Kharkivska oblast (5 minute time horizon of target variable) [PITH_FULL_IMAGE:figures/full_fig_p013_22.png]
Figure 23
Figure 23. Figure 23: Features importance for Lvivska oblast (15 minute time horizon of the target [PITH_FULL_IMAGE:figures/full_fig_p014_23.png]
Figure 24
Figure 24. Figure 24: ROC curve for Lvivska oblast (15 minute time horizon of the target variable) [PITH_FULL_IMAGE:figures/full_fig_p015_24.png]
Figure 25
Figure 25. Figure 25: Accuracy scores for Lvivska oblas (15 minute time horizon of the target variable) [PITH_FULL_IMAGE:figures/full_fig_p015_25.png]
Figure 26
Figure 26. Figure 26: Features importance for Vinnytska oblast (15 minute time horizon of the target [PITH_FULL_IMAGE:figures/full_fig_p016_26.png]
Figure 27
Figure 27. Figure 27: ROC curve for Vinnytska oblast (15 minute time horizon of the target variable) [PITH_FULL_IMAGE:figures/full_fig_p016_27.png]
Figure 28
Figure 28. Figure 28: Accuracy scores for Vinnytska oblast (15 minute time horizon of the target [PITH_FULL_IMAGE:figures/full_fig_p017_28.png]
Figure 29
Figure 29. Figure 29: Features importance for Kyivska oblast (15 minute time horizon of the target [PITH_FULL_IMAGE:figures/full_fig_p017_29.png]
Figure 30
Figure 30. Figure 30: Features importance for Kyivska oblast (15 minute time horizon of the target [PITH_FULL_IMAGE:figures/full_fig_p018_30.png]
Figure 31
Figure 31. Figure 31: Accuracy scores for Kyivska oblast (15 minute time horizon of the target [PITH_FULL_IMAGE:figures/full_fig_p018_31.png]
Figure 32
Figure 32. Figure 32: Features importance for Kharkivska oblast (15 minute time horizon of the [PITH_FULL_IMAGE:figures/full_fig_p019_32.png]
Figure 33
Figure 33. Figure 33: ROC curve for Kharkivska oblast (15 minute time horizon of the target variable) [PITH_FULL_IMAGE:figures/full_fig_p019_33.png]
Figure 34
Figure 34. Figure 34: Accuracy scores for Kharkivska oblast (15 minute time horizon of the target [PITH_FULL_IMAGE:figures/full_fig_p020_34.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

8 extracted references · 8 canonical work pages

  1. [1]

    Air Raid Datasets

    Vadym Klymenko. Air Raid Datasets. URL: https://github.com/Vadimkin/ ukrainian-air-raid-sirens-dataset

    work page

  2. [2]

    Public response to govern- ment alerts saves lives during russian invasion of ukraine

    David Van Dijcke, Austin L Wright, and Mark Polyak. Public response to govern- ment alerts saves lives during russian invasion of ukraine. Proceedings of the National Academy of Sciences, 120(18):e2220160120, 2023

    work page 2023

  3. [3]

    URL: https://air-alarms.in.ua/en

    Air-Alarms.In.Ua. URL: https://air-alarms.in.ua/en

    work page

  4. [4]

    URL: https://stfalconcom.medium

    An App for Air Alert Notifications in Ukraine. URL: https://stfalconcom.medium. com/an-app-for-air-alert-notifications-in-ukraine-505d895b19ea

    work page

  5. [5]

    URL: https://github.com/dr-mod/ air-raid-monitor

    Air Raid Monitor for Raspberry Pi. URL: https://github.com/dr-mod/ air-raid-monitor

    work page

  6. [6]

    URL: https://www.kaggle.com/code/ dimakyn/visualizing-air-alert-in-ukraine-war

    Visualizing Air Alert in Ukraine War. URL: https://www.kaggle.com/code/ dimakyn/visualizing-air-alert-in-ukraine-war

    work page

  7. [7]

    Visualizing Air Alerts During the Ukraine War

    Dima Kynal. Visualizing Air Alerts During the Ukraine War. URL: https://medium.com/@dimakynal/visualizing-air-alerts-during-the-ukraine- war-315c7f0d5371. 13

    work page

  8. [8]

    "" Training and validation machine learning model for prediction of alert duration

    Danylo Halaiko. Building air alarm notification telegram bot. URL: https://d9nich. medium.com/building-air-alarm-notification-telegram-bot-116de4ec2607 . A Appendix A.1 Forecasting alerts with time horizon 15 minutes Figures 23–34 show features importance, ROC curve and accuracy scores on the test dataset for the regions under investigations in case of 15...

    work page

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.