{"id":"bf3b824a-0ad0-4032-82d6-7d80bc3a80b5","arxiv_id":"2411.14625","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"Random Forest models trained on Ukrainian air alert data indicate that adjacent regions' alert status and time features predict alerts in Lviv, Vinnytsia, Kyiv, and Kharkiv within 5 or 15 minutes.","lead":"This paper applies machine learning to minute-level air raid alert records from Ukraine and reports that neighboring regions' alert status is the strongest predictor of near-term alerts. The practical significance is that, if valid, a system could warn adjacent regions minutes before an alert reaches them during the ongoing war.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Target-region cumulative alert-duration feature is not lagged, so reported accuracy likely measures persistence of ongoing alerts rather than forecast skill; retraining without that feature settles it.","rationale":"The paper's stated contribution is empirical evidence that air-alert status can be predicted 5–15 minutes ahead using neighboring-region and calendar features. The Appendix code shows no lagging of the cumulative duration features, and the target is forward-looking; for an ongoing alert, the target region's own cumulative feature exactly encodes current status, which by construction implies the target is 1 for the next 5 minutes. This is not a minor implementation detail but a direct restatement of the target, so the reported accuracy, AUC, and feature importances cannot be interpreted as forecasting skill. The qualitative EDA claim about geospatial correlation is plausible and independently reasonable, but the quantitative model evidence is the load-bearing support for 'predictive analytics,' and it is broken by this leakage. No code, data, baselines, or error bars are provided to allow a reader to verify otherwise. A concrete ablation retraining without the target region's own cumulative feature would settle the question; in the meantime, the submitted evidence does not support the central claim, so the verdict should remain REJECT.","tokens_in":3961,"tokens_out":3649,"duration_ms":38883,"concrete_test":"Using the authors' `alert_ts` data and the Appendix A.2 function, reconstruct the 5-minute and 15-minute targets for one region, e.g., Kharkivska. Retrain the Random Forest on the pre-2024-07-01 split with the same features except (a) drop the target region's own cumulative duration feature, and (b) lag every feature by the forecasting horizon (5 or 15 minutes) so that no current-status information is used. Compare test accuracy, AUC, and feature importances with the reported figures. If accuracy falls to the majority-class baseline or AUC approaches 0.5, the reported performance is leakage-driven; if accuracy is preserved, the neighboring-region claim survives.","verdict_should_be":"REJECT","load_bearing_attack":"The central claim—that neighboring regions' features are the most important predictors of a region's alert status 5–15 minutes ahead—rests on the model performance reported in Section 3. In the Appendix code (A.2), features are taken directly from `alert_ts` with no `shift`/lag operation before fitting, and the target is defined as the forward-looking indicator that an alert occurs within the next 5 minutes. The cumulative alert-duration feature for the target region is described in Section 3 as 0 when no alert is ongoing and equal to the current alert duration when an alert is ongoing. Consequently, at any minute t, if that feature is positive, an alert is ongoing at t, and by the paper's own definition the target for the next 5 minutes is 1. The model can therefore achieve high accuracy and AUC simply by copying the target region's current alert status forward, without learning any cross-region propagation. The feature-importance plots do not rescue the claim because the leakage feature is present in every row, and any contribution from neighboring regions could be marginal once persistence is removed. The paper provides no baselines (e.g., 'always predict current status' or 'predict 1 for all ongoing alerts') and no lagged-feature ablation, so the reported quantitative evidence cannot distinguish geospatial forecasting from trivial persistence.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper analyzes minute-level air-alert data for Ukrainian regions from March 2022 to November 2024. It presents exploratory correlation heatmaps and trains a Random Forest classifier for four regions (Lvivska, Vinnytsia, Kyivska, Kharkivska) to predict whether an alert will occur within the next 5 or 15 minutes. Features include current cumulative alert durations for regions, calendar features (month, day of week, hour), and a day-count feature. The authors report high accuracy and AUC values and conclude that neighboring regions' alert features are the most important predictors. The paper includes an appendix with code for training and validation and extension to a 15-minute horizon.","tokens_in":4188,"tokens_out":8544,"duration_ms":79431,"significance":"If the conclusions were supported, the paper would provide a practical demonstration that geospatial correlations in air alerts can be exploited for short-term early warning, which would be of genuine value in a conflict setting. The underlying public dataset and the authors' attempt to share code are positive aspects, and the exploratory correlation analysis is a reasonable first step. However, the current analysis does not establish the main claim because the feature set includes an unlagged measure of the target region's current alert status and no persistence baseline or ablation is reported. The paper's contribution is therefore currently at the level of a hypothesis-generating case study rather than a validated predictive model.","major_comments":[{"comment":"The cumulative alert-duration feature for the target region is not lagged and acts as a near-persistence proxy. The feature is defined as 0 when no alert is ongoing and as the current alert duration otherwise. The target is 1 when an alert starts within the next 5 minutes or when an ongoing alert lasts at least 5 more minutes. Consequently, a model can achieve high accuracy by predicting 1 whenever the target region's own cumulative-duration feature is positive and 0 otherwise, especially since most alerts last longer than 5 minutes. The paper does not report this trivial baseline, nor does it ablate the target region's own feature. Without such an ablation, the feature-importance plots (Figures 11-22 and 23-34) cannot support the central conclusion that neighboring regions' features are the most important predictors. Please re-run the analysis with lagged features only, or at least remove the target region's own current-status feature and compare against a persistence baseline.","section":"Section 3, Appendix A.2"},{"comment":"The evaluation consists of a single chronological split at 2024-07-01, with no confidence intervals, no repeated runs, and no comparison against simple baselines (e.g., always predict current status, majority class, logistic regression with the same features). Random Forest also has hyperparameters (n_estimators, max_depth, etc.) whose sensitivity is not examined. The reported accuracy and AUC values are therefore not enough to establish that the model has genuine forecasting skill. Please add error bars over multiple seeds or bootstraps, and include at least a persistence baseline and a no-skill baseline.","section":"Section 3"},{"comment":"The exact feature set and target construction are not fully specified. The text mentions both lagged binary features and cumulative duration features, but it is not stated which of these are used in the final model, nor whether all regions' cumulative features are included. The code in A.2 only shows the fitting and evaluation; it does not show how the target variable is derived from the raw alert series or how the feature matrix is constructed. This makes the experiments irreproducible and prevents an independent check of the leakage concern. Please provide the full data-preparation code and a data dictionary listing all features and the precise target definition.","section":"Section 3, Appendix A.2"}],"minor_comments":[{"comment":"The caption reads 'Accuracy scores for Lvivska oblas'; 'oblas' should be 'oblast'.","section":"Figure 25 caption"},{"comment":"Both figures carry the title 'Features importance for Kyivska oblast (15 minute time horizon of the target variable)'; one of them should be the ROC curve for that region.","section":"Figures 29 and 30"},{"comment":"The paper does not state the access date or version for the GitHub dataset [1]; please add this information for reproducibility.","section":"References"},{"comment":"The 'ndays' feature is described as reflecting that 'air alert pattern changes with time', but the paper does not discuss whether this represents a real temporal trend or a data artifact; a brief discussion would help.","section":"Section 3"}],"recommendation":"major_revision","confidential_remarks":"The paper's main conclusion is not supported as written because the reported model uses the target region's current alert status as a feature and no persistence baseline or ablation is provided. This issue is fixable: the authors should re-run with lagged features, add baselines, report uncertainty, and provide full feature-engineering code. The topic is timely and the public dataset is appropriate, so a careful revision could make a modest contribution. I recommend major revision rather than rejection because the central idea is defensible and the methodological gap can be addressed within the manuscript's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is a short empirical study of air alert prediction in Ukraine. The descriptive part—correlation heatmaps and simultaneous-alert maps—is fine and shows real spatial structure. The new bit is the claim, backed by random-forest feature importance, that neighboring regions' current alert status is the main predictor of a target region's alert status 5–15 minutes out. That is a plausible and potentially useful finding, and the appendix code makes the method reproducible.\n\nThe problem is in the modeling section. The features include the target region's own cumulative alert duration, and the code in A.2 does not lag it. The target is 1 whenever an alert occurs in the next 5 minutes, which is automatically true if an alert is ongoing at time t. So the model can achieve high accuracy by copying current status forward. The reported accuracy and AUC are therefore not evidence of forecasting skill, and the feature-importance plots are hard to interpret because the leaky feature is present in every row. The paper also has no baselines (e.g., always predict the current status), no lagged-feature ablation, and no error bars. These omissions matter because the central claim is about predictive power.\n\nI don't think this is deliberate. The article reads as a straightforward exploratory report, and the authors even mention lagged binary features before switching to cumulative duration. But they appear not to have noticed that the cumulative feature for the target region is equivalent to the label for ongoing alerts. A simple fix—shift the cumulative features by the horizon, or exclude the target region's own current state—would settle it. Without that, the numbers don't support the conclusion.\n\nWho gets value: someone working on early-warning systems or spatial time-series ML could use this as a cautionary example. The EDA figures are of some independent interest. But as a research contribution, it needs major revision.\n\nRecommendation: send it to peer review—the question is real and the data are public—but the reviewers should require a lagged-feature ablation and baseline comparisons before it can be accepted.","headline":"A plausible geospatial claim undercut by target leakage in the feature construction; the EDA is fine, but the accuracy numbers reflect persistence, not forecasting.","tokens_in":4699,"tokens_out":2620,"would_cite":false,"duration_ms":24301,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Air alerts in a Ukrainian region can be predicted 5 to 15 minutes ahead from neighbor regions' alert status and the calendar.","keywords":["air alerts","Ukraine war","predictive analytics","random forest","geospatial correlation","time series classification","early warning"],"falsifier":"Re-run the same 5-minute-ahead experiment with the target region's own cumulative alert-duration feature removed, or shifted forward by the full prediction horizon, and compare accuracy against a baseline that simply predicts the current alert status will persist for another 5 minutes. If the neighboring-region features add little beyond that persistence baseline, then the paper's core claim about geospatial predictive power is unsupported.","tokens_in":3737,"feed_emoji":"🚨","tokens_out":7534,"duration_ms":68648,"temperature":0.7,"pith_summary":"The paper tries to establish that air alerts in Ukrainian regions are geographically correlated, and that these correlations can be exploited to forecast whether a region will be under alert within the next 5 or 15 minutes. Using minute-level alert data from March 2022 through November 2024, the authors train separate random-forest classifiers for four regions, with features drawn from all regions' alert duration counters plus calendar variables. The reported feature-importance rankings show that neighboring regions' alert statuses contribute most to predictions for a given region, with hour, day of week, month, and days-since-start also contributing. If this is right, civilians and emergency services could gain a few minutes of advance notice before an alert reaches their own region.","feed_headline":"Neighboring regions forecast Ukrainian air alerts 5-15 minutes ahead","feed_subtitle":"A wartime alert dataset shows geospatial patterns that make short-term prediction feasible.","key_machinery":"The central object is the minute-by-minute binary alert time series for each Ukrainian region, encoded as a cumulative alert-duration feature that is zero during quiet periods and equals the current alert's length during an alert. Four separate random-forest classifiers are trained, one per target region, using these cumulative features for all regions plus month, day of week, hour, and a days-since-start counter; each classifier's target is a binary indicator of whether the target region will be under alert within the next 5 minutes (or 15 minutes in a second setup). The random forest's feature-importance output is the mechanism used to identify which neighboring regions matter most for each target. This setup lets the authors test whether geospatial correlation in alert time series carries predictive information beyond the target region's own current state.","core_discovery":"The central claim is that the binary alert status of a Ukrainian region can be predicted from the current alert statuses of adjacent regions, with seasonality and elapsed time as secondary signals. For each of four target regions, a random-forest model was trained on minute-by-minute cumulative alert-duration features for all regions and on calendar features, predicting a target of whether an alert would be active within the next 5 or 15 minutes. The authors report that neighboring-region features dominate the feature-importance rankings, that ROC and accuracy scores on a held-out test period support feasibility, and that the importance of the days-since-start feature indicates alert patterns change over time.","pith_inferences":["An ablation study that removes each neighboring region's features one at a time would show which geographic links carry the predictive signal, rather than relying on random-forest feature-importance rankings alone.","The same 5- and 15-minute prediction framing could transfer to other spatially propagating hazards, such as severe weather warnings or earthquake early warning, where nearby sensors provide advance information.","A stricter evaluation with all features lagged by at least the full prediction horizon would be needed to separate true forecasting skill from persistence, and would likely lower the currently reported accuracy figures."],"forward_implications":["If the geospatial pattern holds outside the training window, regional alert status can be forecast 5 to 15 minutes ahead using only current alert states of adjacent regions and the calendar.","The method could be packaged into an early-warning tool that gives people extra minutes to reach shelter before an alert is officially declared in their own region.","The reported dominance of neighboring-region features implies that a regional alert network behaves like a propagating wave, so a model trained on one set of regions may transfer to nearby regions with limited retraining.","Because alert patterns change over time, any operational model would need periodic retraining to track evolving strike behavior."],"supporting_citations":[{"why":"Supplies the historical air-alert dataset for all regions that is used for exploratory analysis and for training and testing every model.","marker":"[1]"}],"fun_headline_variants":["Adjacent regions predict Ukraine air alerts 5-15 min","Neighbor regions key to forecasting Ukraine air alerts","Geospatial patterns forecast Ukrainian air alerts minutes ahead","Predicting Ukraine air alerts via neighboring regions"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported skill is assumed to reflect forecasting, not the persistence of already-running alerts, because the target region's own cumulative alert-duration feature may leak the current status into the next-5-minute target.","fun_headline_variants_meta":{"raw":{"variants":["Adjacent regions predict Ukraine air alerts 5-15 min","Neighbor regions key to forecasting Ukraine air alerts","Geospatial patterns forecast Ukrainian air alerts minutes ahead","Predicting Ukraine air alerts via neighboring regions"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001132,"raw_usage":{"total_tokens":4631,"prompt_tokens":801,"completion_tokens":3830,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":417,"completion_tokens_details":{"reasoning_tokens":3769}},"tokens_in":417,"tokens_out":3830,"duration_ms":24248,"temperature":1.0,"reasoning_tokens":3769,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T15:04:33.808053+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the same 5-minute-ahead experiment with the target region's own cumulative alert-duration feature removed, or shifted forward by the full prediction horizon, and compare accuracy against a baseline that simply predicts the current alert status will persist for another 5 minutes. If the neighboring-region features add little beyond that persistence baseline, then the paper's core claim about geospatial predictive power is unsupported.","supporting_citations":[{"cited_title":"Air Raid Datasets","cited_arxiv_id":null,"evidence_quote":"Supplies the historical air-alert dataset for all regions that is used for exploratory analysis and for training and testing every model."}],"review_version":1}