Pith. sign in

REVIEW 3 major objections 4 minor 19 references

Integrating Spatiotemporal Features in LSTM for Spatially Informed COVID-19 Hospitalization Forecasting

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Pith's one-line read A parallel-stream LSTM with a social-connectivity spatiotemporal feature forecasts 28-day COVID-19 hospitalizations more accurately than COVID-19 Forecast Hub ensembles, with per-state gains of 27 to 69 during Omicron.

desk verdict Solid, reusable SPH feature and a well-executed internal ablation; the headline Hub comparison is on a possibly mismatched target and needs a matched-target re-run before the outperformance claim is taken seriously. read the letter →

arxiv 2506.05752 v2 pith:YSXGVRLQ submitted 2025-06-06 cs.LG cs.AI

classification cs.LGcs.AI
keywords COVID-19hospitalizationforecastingLSTMspatiotemporalfeatureSocialConnectednessIndexspatialspilloverquantileregressionmulti-horizonensemble
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a deep-learning model can forecast state-level COVID-19 hospitalizations up to 28 days ahead more accurately than the official forecasting ensembles, if it is given a feature that summarizes how hospitalizations in socially connected states are moving. The feature, called Social Proximity to Hospitalizations (SPH), is a weighted average of other states' hospitalization rates using Facebook friendship-connection weights, and it is intended to capture interstate transmission. The model, SLSTM, is a pair of stacked LSTM networks that read 7-day and 28-day input windows in parallel and fuse them with a learnable weight, then combine 7-, 14-, and 28-day output models into a 15-member ensemble. On the $\Delta$ and Omicron surges, the paper reports average per-state improvements over the COVID-19 Forecast Hub ensembles that grow with horizon, reaching 69 hospitalizations per state at the 28-day horizon during Omicron. The claim matters because hospitalization forecasts are used for staffing and resource allocation, and the Omicron surge was precisely the period when existing models struggled most.

What carries the argument

The load-bearing object is SPH, the Social Proximity to Hospitalizations feature, defined as the row-normalized Social Connectedness Index-weighted sum of hospitalization rates in connected states. It converts a static matrix of interstate friendship strengths into a daily time series that carries spatial spillover into each state's LSTM input. The other pieces are the parallel-stream architecture, with two stacked LSTM networks processing 7-day and 28-day windows fused by a learnable scalar weight, the multi-horizon ensemble of 15 members with staggered 7-day, 14-day, and 28-day output horizons aggregated by median, and quantile loss with 23 probability levels for probabilistic forecasts. Together these let the model balance short- and long-term temporal dependence, smooth over stochastic training runs, and emit prediction intervals evaluated by the Weighted Interval Score.

What would settle it

Re-score the stored COVIDhub-4 week ensemble forecasts on the same 7-day smoothed state-level hospitalization series used for SLSTM, matching the smoothing window and evaluation weeks, then compare MAE and WIS to SLSTM's reported numbers; if the ensemble's smoothed-target error meets or beats SLSTM's, the claimed margin disappears.

Watch

Extended reading notes

Core claim

The paper's central claim is that spatiotemporal information derived from social connectivity can be turned into a scalar feature that materially improves deep-learning hospitalization forecasts. SPH is computed as a row-normalized, Social Connectedness Index-weighted sum of hospitalization rates in all connected states, so each state's model input carries a signal of what is happening in states whose populations interact with it. The paper argues that adding SPH to a parallel-stream LSTM, called SLSTM, reduces average 28-day MAE by roughly 363 hospitalizations per state compared with the same model without SPH, reduces forecast variance across ensemble members, and beats the COVID-19 Forecast Hub baseline and both Hub ensemble models on MAE, MAPE, RMSE, and WIS during both the $\Delta$ and Omicron evaluation windows. The paper also claims that combining 7-day, 14-day, and 28-day output models by taking the median across 15 members improves on a single 28-day direct model by about 167 hospitalizations per state on average.

Load-bearing premise

The claim assumes the comparison models were scored on the same smoothed 7-day hospitalization series that SLSTM was trained and evaluated on, whereas the paper states only that it chose smoothed hospitalizations as its evaluation target and does not say the Hub forecasts were re-scored on that same series.

Editorial extensions

If this is right

  • With SPH, the model improves average 28-day MAE by about 363 hospitalizations per state versus the same model without SPH, and the spatial model's variance across ensemble members is consistently lower.
  • Using a multi-horizon ensemble instead of a single 28-day direct output improves average MAE by 167 hospitalizations per state over the Omicron evaluation period.
  • SLSTM outperforms the COVID-19 Forecast Hub ensembles on both Delta and Omicron, with the Omicron advantage growing from roughly 15 hospitalizations per state on day 2 to about 64 per state on day 23.
  • SLSTM's 95% prediction intervals contain 90.2% of actual values, versus 68.6% and 70.2% for the two Hub ensembles, and its WIS is lowest in both waves.
  • The same framework is presented as transferable to influenza-style forecasting hubs and to other multi-step time-series problems such as air quality, weather, and energy demand.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The comparison to Hub ensembles is only fully fair if those ensembles are re-scored on the same 7-day smoothed hospitalization series; the paper does not state that this was done, so the reported margins may partly reflect a smoothing advantage.
  • Because the Social Connectedness Index is static and freely available, SPH could be built with other connectivity proxies such as migration or cell-phone mobility where Facebook data are unavailable; a direct test would be to permute the connectivity weights and see whether the gains survive.
  • The multi-horizon median ensembling idea likely generalizes to any high-noise multi-step forecasting task, independent of epidemiology.
  • A more direct test of the mechanism would compare SPH against an equally informative non-spatial feature, like a lagged national average, to check whether spatial structure rather than extra information is what drives the improvement.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes SLSTM, a parallel-stream LSTM architecture for forecasting daily state-level COVID-19 hospitalizations in the United States. Two LSTM branches process 7-day and 28-day histories and are fused with a learnable weight; the model also uses Social Proximity to Hospitalizations (SPH), a spatial feature built from Meta's Social Connectedness Index and hospitalization rates in socially connected states. A multi-horizon ensemble combines 7-, 14-, and 28-day output members. The model is evaluated on Delta and Omicron waves against COVIDhub baselines and ensembles, reporting consistent improvements, with the Omicron headline claim of 27, 42, 54, and 69 hospitalizations per state at 7-, 14-, 21-, and 28-day horizons. The authors also report an ablation showing SPH improves performance and reduces ensemble variance.

Significance. The manuscript makes a concrete, falsifiable claim: a spatially informed LSTM with SPH improves hospitalization forecasts over the COVID-19 Forecast Hub ensembles, and the ablation is internally sound. The open-source code release and the explicit evaluation on two surge periods are strengths. The circularity concern sometimes raised about SPH does not apply: SPH at time t uses observed hospitalization rates in connected states while the target is future hospitalizations in the focal state. If the head-to-head comparison is confirmed on a common target scale, the paper would be a useful contribution to operational epidemic forecasting and to the use of social-connectivity data in spatial epidemiology.

major comments (3)
  1. [§3.5, §3.6, Tables 3 and C3] The headline comparison against the COVIDhub ensembles may be evaluated on different target series. Section 3.5 states: "we selected smoothed hospitalizations as the evaluation target instead of raw values," and the model is trained on 7-day rolling averages. The Hub ensembles, however, were designed and operationally evaluated on raw daily incident hospitalizations. The paper does not state whether the Hub forecast trajectories were transformed to the smoothed scale (for example, by applying the same 7-day trailing average) before computing MAE and WIS in Tables 3 and C3, and Equation (7) does not specify whether y_i,t is smoothed or raw. Since smoothing reduces volatility, scoring SLSTM on a smoothed target while scoring Hub models on raw data would inflate the apparent advantage. The authors must either confirm that all baselines were re-evaluated on the same smoothed target or rerun the comparison on a common target series.
  2. [§4.3, Tables 3 and C3] The superiority claims rest on average errors over only 10 Omicron and 15 Delta forecast dates, but no uncertainty quantification is provided. The 27–69 hospitalization differences are reported as point estimates without confidence intervals or paired significance tests. Because these averages are taken over a small number of weekly forecasts, the authors should add a paired test across forecast dates (e.g., Wilcoxon signed-rank or Diebold-Mariano) or bootstrap confidence intervals to establish whether the improvements are statistically reliable rather than driven by one or two forecast weeks.
  3. [§3.5] The paper does not specify whether the 7-day rolling average applied to cases, hospitalizations, and the evaluation target is trailing or centered. If the average is centered, the smoothed target at horizon k would incorporate future observations relative to the forecast date, which would constitute lookahead leakage. Please state explicitly that a trailing (past-7-day) average is used, or correct the procedure accordingly.
minor comments (4)
  1. [Abstract and §4.3] The abstract says "surpasses the ensemble by 27, 42, 54, and 69 hospitalizations," but Table 3 reports two ensemble models and the specific numbers correspond to the COVIDhub-4 week ensemble. Please specify which ensemble is meant in the abstract.
  2. [§3.4] The text refers to "four subgroups of such 7-day forecast output models," but the description indicates four individual 7-day models with different seeds, not subgroups. Please rephrase for clarity.
  3. [Equation (4), §3.1] Please clarify whether the set C in the SPH denominator excludes the focal state i, and whether the row-normalization is computed over all other 50 states or over a subset of connected states.
  4. [§3.1 and §3.5] The paper does not explicitly describe the inverse transform from the MinMax-scaled rates back to hospitalization counts used in the reported MAE and WIS tables. Since the code is released, this is probably available, but an explicit equation or a pointer to the relevant code block would aid reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: SPH is a spatial lag of current hospitalization rates, the target is future hospitalizations, and the Hub comparison concern is an evaluation-target mismatch, not circularity.

full rationale

The paper's central prediction chain is not circular. The SPH feature is defined in Eq. (4) as a row-normalized SCI-weighted sum of current hospitalization rates in connected states at time t, while the forecast target is incident hospitalizations at horizons t+1 through t+28. The feature is therefore a spatial lag of the input series, not a restatement of the target, and no equation in the paper defines the target in terms of SPH or vice versa. The learnable fusion weight W, the ensemble member architecture, and the quantile loss are ordinary model-fitting components rather than fitted parameters relabeled as predictions. The reported 27-to-69 hospitalization improvements come from a cross-model comparison against COVIDhub baselines; the paper states in Section 3.5 that it 'selected smoothed hospitalizations as the evaluation target instead of raw values,' which raises a legitimate apples-to-oranges comparability concern if the Hub forecasts were scored on raw data. That is an evaluation-fairness issue, not a circularity issue, because it does not make the model's output equal to its input by construction. The internal SPH ablation compares models on the same target and is likewise free of circularity. Self-citations to Lucas et al. (2023), Vahedi et al. (2021), and Kuchler et al. (2022) motivate the spatial feature concept, but the SPH contribution is tested by an ablation on held-out evaluation periods rather than being asserted only through those citations. No uniqueness theorem, imported ansatz, or renaming of a known result is load-bearing. Accordingly, the appropriate circularity score is 0.

Assumptions & free parameters 6 free parameters · 4 assumptions · 1 invented entities

The model relies on several hand-chosen hyperparameters (lag pair, layer sizes, learning rate, smoothing window) and on domain assumptions about SCI validity, rate-based inputs, and the smoothed evaluation target. SPH is a new derived feature with independent evidence from the ablation study.

free parameters (6)
  • Learnable fusion weight W = optimized, initialized to 1.0
    Balances short-term and long-term LSTM branches; learned via backpropagation.
  • Input lag pair = 7 and 28 days
    Chosen via preliminary tests in Section 3.2; pairs such as 7 and 14, 7 and 21, or 14 and 28 performed worse.
  • LSTM layer sizes = 256, 128, 128, 128, dense 64
    Architecture hyperparameters, not justified from first principles.
  • Learning rate and batch size = 0.0008 and 64
    Adam optimizer settings specified in Section 3.2.
  • Smoothing window = 7-day rolling average
    Applied to cases and hospitalizations and also used as the evaluation target in Section 3.5.
  • Validation state selection = three states with lowest, median, and highest hospitalization rates
    Spatial cross-validation criterion for early stopping; changes each forecast date.
assumptions (4)
  • domain assumption SCI is a valid proxy for interstate human interaction and disease transmission.
    Prior studies (Kuchler et al. 2022; Bailey et al. 2018) support the correlation with COVID spread and mobility; assumed stable over the study period.
  • ad hoc to paper Smoothed (7-day average) hospitalizations are a more reliable evaluation target than raw daily reports.
    Section 3.5: chosen because raw reports have weekend and holiday artifacts; this is the paper's evaluation target, not an external benchmark.
  • domain assumption Hospitalization rates per 10,000 population are preferable to raw counts as model inputs.
    Section 3.1: exploratory experiments showed better performance with rates; not a first-principles result.
  • domain assumption Static annual SCI remains valid during the Delta and Omicron surges.
    SPH weights are fixed over time; any temporal change in social connectedness is not captured by the model.
invented entities (1)
  • Social Proximity to Hospitalizations (SPH) independent evidence
    purpose: Weighted sum of hospitalization rates in socially connected states, used as a spatiotemporal feature.
    Defined in Eq. (4) from public SCI and hospitalization data; its predictive power is tested via ablation in Table 1. It is a derived variable, not a physical entity.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Integrating Spatiotemporal Features in LSTM for Spatially Informed COVID-19 Hospitalization Forecasting." pith.science (2026). https://pith.science/paper/YSXGVRLQ

@misc{pith2026250605752,
  author       = {Pith},
  title        = {Pith review of: Integrating Spatiotemporal Features in LSTM for Spatially Informed COVID-19 Hospitalization Forecasting},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YSXGVRLQ}},
  note         = {Machine review of arXiv:2506.05752}
}
read the original abstract

The COVID-19 pandemic's severe impact highlighted the need for accurate and timely hospitalization forecasting to support effective healthcare planning. However, most forecasting models struggled, particularly during variant surges, when they were most needed. This study introduces a novel parallel-stream Long Short-Term Memory (LSTM) framework to forecast daily state-level incident hospitalizations in the United States. Our framework incorporates a spatiotemporal feature, Social Proximity to Hospitalizations (SPH), derived from Meta's Social Connectedness Index, to improve forecasts. SPH serves as a proxy for interstate population interaction, capturing transmission dynamics across space and time. Our architecture captures both short- and long-term temporal dependencies, and a multi-horizon ensembling strategy balances forecasting consistency and error. An evaluation against the COVID-19 Forecast Hub ensemble models during the Delta and Omicron surges reveals the superiority of our model. On average, our model surpasses the ensemble by 27, 42, 54, and 69 hospitalizations per state at the 7-, 14-, 21-, and 28-day horizons, respectively, during the Omicron surge. Data-ablation experiments confirm SPH's predictive power, highlighting its effectiveness in enhancing forecasting models. This research not only advances hospitalization forecasting but also underscores the significance of spatiotemporal features, such as SPH, in modeling the complex dynamics of infectious disease spread.

Figures

Figures reproduced from arXiv: 2506.05752 by the authors.

Figure 1
Figure 1. Histogram and Skewness Index of Raw Incident Hospitalizations and Hospitalization Rates by March 12, 2022, for all 51 states (around our evaluation period of the Omicron surge). Rates are less skewed, allowing for a better capture of spatial spillovers, independent of population size. 3.2. Model Architecture and Hyperparameters We developed a parallel stacked Long Short-Term Memory (LSTM) network with spa￾tial featu… view at source ↗
Figure 2
Figure 2. Architecture of SLSTM, i.e., one of the parallel network members of our multi-horizon ensemble. The learnable parameter (weight) denoted in the green box balances information learned from the 7- and 28- day input branches and is learned for each ensemble member and forecast date separately. We added early stopping to avoid overfitting. The model utilizes Quantile Loss as the loss function (described in Section 3.3) … view at source ↗
Figure 3
Figure 3. Ensembling Strategy. For each time-step t in the output, there are 15 predictions for each output quantile, using 7-day, 14-day, and 28-day output sequences to balance between consistency and error optimiza￾tion per sequence. Similarly, two staggered subgroups with seven members each are trained to forecast 14-day sequences, with seven members predicting t+ 1, . . . , t+ 14, and seven members predicting t + 15, . . … view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Raw and smoothed daily U.S. COVID-19 cases (top) and hospitalizations (bottom). Purple and red bands mark the Delta (15 weeks, June 21 - September 27, 2021) and Omicron (10 weeks, December 6 - February 7, 2022) evaluation windows; each band extends 28 days beyond its f…
Figure 5
Figure 5. Figure 5: Average MAE over the 10-week evaluation period for models with SPH, without SPH, and with both SPH and SPC spatial features. and has better generalizability [PITH_FULL_IMAGE:figures/full_fig_p017_5.png]
Figure 6
Figure 6. Figure 6: Average MAE over the 10-week evaluation period for models using multi-horizon ensemble strategy and using 28-Day direct ensemble. The superior performance of the SLSTM (Multi-horizon Ensemble) can be at￾tributed to its ability to reduce error over segments, as well as …
Figure 7
Figure 7. Figure 7: Average prediction MAE and WIS over 28-day forecast horizon during the Delta (15 weekly forecasts, June 21–September 27, 2021) and Omicron (10 weekly forecasts, December 6, 2021–February 7, 2022) waves [PITH_FULL_IMAGE:figures/full_fig_p019_7.png]
Figure 8
Figure 8. Figure 8: Spatial distribution of hospitalization rates and normalized prediction errors of three baseline models and SLSTM on 2nd, 9 th, 16th, and 23rd days forecasted on January 3, 2022 [PITH_FULL_IMAGE:figures/full_fig_p020_8.png]
Figure 9
Figure 9. Figure 9: COVID-19 hospitalization forecasts of SLSTM, COVIDhub-baseline, COVIDhub-4 week ensemble, and COVIDhub-trained ensemble. Blue shadow represents 95% prediction intervals of SLSTM a perennial challenge in time-series forecasting, incorporating more dynamic features has t…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

19 extracted references · 15 canonical work pages

  1. [5]

    arXiv preprint arXiv:2007.03113

    Examining covid-19 forecasting using spatio-temporal graph neural networks. arXiv preprint arXiv:2007.03113 . Kendall, D.G.,

  2. [7]

    PLoS computational biology, 20 (5), e1011200

    Challenges of covid-19 case forecasting in the us, 2020–2021. PLoS computational biology, 20 (5), e1011200. Lucas, B., Vahedi, B., and Karimzadeh, M.,

  3. [8]

    Migration patterns, friendship networks, and the diaspora: the potential of Facebook Social Connectedness Index to anticipate displacement patterns induced by Russia invasion of Ukraine in the European Union

    Migration patterns, friendship networks, and the diaspora: the poten- tial of facebook social connectedness index to anticipate displacement patterns induced by russia invasion of ukraine in the european union. arXiv preprint arXiv:2209.01833 . Monod, M., et al. ,

  4. [10]

    medRxiv, 2024–01

    Unifying human infectious disease models and real-time awareness of population-and subpopulation-level intervention effectiveness. medRxiv, 2024–01. Shahid, F., Zameer, A., and Muneeb, M.,

  5. [11]

    Fast and Accurate Forecasting of COVID-19 Deaths Using the SIkJ$\alpha$ Model

    Fast and accurate forecasting of covid-19 deaths using the sikjα model. arXiv preprint arXiv:2007.05180 . Tanaka, H.,et al.,

  6. [14]

    International Journal of Medical Informatics , 164, 104804

    Biases in using social media data for public health surveillance: A scoping review. International Journal of Medical Informatics , 164, 104804. 29 Appendix A. SLSTM Model Architecture and Pseudocode Algorithm 1Short- and Long-Term LSTM Fusion for Quantile Prediction Require: Xshort: short-term input sequence (e.g., 7 days) Require: Xlong: long-term input ...

  7. [15]

    31 Appendix C

    for models with SPH, without SPH, and with both SPH and SPC spatial features. 31 Appendix C. Comparative Analysis of F orecasting Results Figure C2. Average MAPE, and RMSE over the 10-week Omicron evaluation period (December 6, 2021 – February 7,

  8. [17]

    waves. T able C1.Average MAE, MAPE, and RMSE of state-level predicted hospitalization incidence over 10 forecast dates (per week) in the Omicron evaluation period (December 6, 2021 – February 7, 2022). SLSTM (with SPH) SLSTM (without SPH) SLSTM (with SPH and SPC) Day MAE MAPE RMSE MAE MAPE RMSE MAE MAPE RMSE t+1 13.22 5.44 23.60 11.25 5.16 20.26 13.22 5.3...

Show all 19 references
  1. [18]

    wave. SLSTM (w/ SPH) COVIDhub-baseline COVIDhub-4 wk ensemble COVIDhub-trained ensemble Day MAE MAPE RMSE MAE MAPE RMSE MAE MAPE RMSE MAE MAPE RMSE t+1 5 .35 5 .71 10 .65 15 .31 28 .93 31 .25 15 .17 15 .22 32 .52 19 .24 17 .63 39 .45 t+2 7 .61 7 .87 15 .18 17 .73 29 .67 35 .85...

  2. [19]

    wave. SLSTM (w/ SPH) COVIDhub-baseline COVIDhub-4 wk ensemble COVIDhub-trained ensemble Day MAE MAPE RMSE MAE MAPE RMSE MAE MAPE RMSE MAE MAPE RMSE t+1 13.22 5.44 23.60 28.17 12.84 52.91 22.05 9.15 38.96 24.38 9.64 43.94 t+2 13.81 5.82 24.65 35.49 14.97 64.40 28.06 11.11 48.03...

  3. [2005]

    Center for spatially integrated social science, 1963,

    Exploring spatial data with geodatm: a workbook. Center for spatially integrated social science, 1963,

  4. [2014]

    arXiv preprint arXiv:1412.6980

    Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. 27 Kocherginsky, M., He, X., and Mu, Y.,

  5. [2017]

    arXiv preprint arXiv:1711.11053

    A multi-horizon quantile recurrent forecaster. arXiv preprint arXiv:1711.11053. Zhang, H., et al. ,

  6. [2019]

    BMC Public Health, 19, 1–9

    The epidemic characteristics and spatial autocorrelation analysis of hand, foot and mouth disease from 2010 to 2015 in shantou, guangdong, china. BMC Public Health, 19, 1–9. Zhao, Y., et al.,

  7. [2020]

    Data in brief , 29, 105340

    Application of the arima model on the covid-2019 epidemic dataset. Data in brief , 29, 105340. Borchering, R.K.,

  8. [2021]

    and Omicron (10 weekly forecasts, December 6, 2021 – February 7,

  9. [2022]

    medRxiv, 2022–08

    Covid-19 modeling: A review. medRxiv, 2022–08. Centers for Disease Control and Prevention,

  10. [2023]

    arXiv preprint arXiv:2303.10993

    A survey on oversmoothing in graph neural networks. arXiv preprint arXiv:2303.10993 . Scharf, L.G., et al.,

  11. [2024]

    influenza surveillance report

    Fluview: Weekly u.s. influenza surveillance report. https://www.cdc.gov/flu/weekly/index.htm. Accessed: 2024-09-05. Changruenngam, S., Bicout, D.J., and Modchang, C.,

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.