Pith. sign in

REVIEW 3 major objections 4 minor 18 references

Impact of COVID-19 on The Bullwhip Effect Across U.S. Industries

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Pith's one-line read COVID-19 amplified the bullwhip effect in 31% of U.S. industries

desk verdict Useful industry-level map of where bullwhip appeared during COVID, but the causal claim rests on an unvalidated forecast-as-counterfactual and needs a serious revision. read the letter →

arxiv 2506.06368 v1 pith:5BOMNE2P submitted 2025-06-04 econ.GN q-fin.ECstat.ML

classification econ.GNq-fin.ECstat.ML
keywords BullwhipeffectCOVID-19SupplychainmanagementLSTMforecastingIndustry-levelanalysisAmplificationratioU.S.CensusdataPanel
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that the COVID-19 pandemic intensified the Bullwhip Effect—the amplification of demand variability as orders move from retailers to wholesalers to manufacturers—across U.S. industry-level data. Using monthly Census data for 77 industries over 1992–2023, the authors build a counterfactual baseline with LSTM forecasts trained on pre-pandemic data, then compare forecasted variance-amplification ratios to actual ones for 2020–2023. They report that 31.2% of industries show actual bullwhip in the pandemic period where the forecast baseline predicted none, with manufacturers the most affected. The intended payoff is a way to identify which industries are structurally vulnerable to external shocks and to forecast bullwhip risk before a crisis.

What carries the argument

The load-bearing object is the amplification ratio, $\mathrm{Var}(\text{production})/\mathrm{Var}(\text{demand})$, computed on log-differenced monthly series, where production for a stage is inferred from the identity $Y_t = S_t + (I_t - I_{t-1})$, shipments plus inventory change. A ratio above 1 indicates bullwhip, and a ratio below 1 indicates variance smoothing. The argument runs on the comparison between actual ratios and ratios computed from LSTM forecasts; the LSTM forecast, trained on 1992–2015 and validated on 2016–2019, serves as the "no-COVID" baseline, and industries are classified into four zones depending on whether forecast and actual agree or disagree. The false-negative zone—forecast below 1, actual above 1—is the paper's operational definition of COVID-amplified bullwhip.

What would settle it

Look at LSTM forecast errors in the pre-COVID holdout period 2016–2019: if the model systematically under-predicts variance even before the pandemic, then the 2020–2023 false-negative count is not a clean COVID signal. A concrete check is to compute amplification ratios from forecast residuals for non-pandemic periods and see whether a comparable share of industries would be misclassified as false negatives; if so, the claimed pandemic amplification is partially an artifact of forecast bias.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central discovery is that the COVID-19 shock did not merely amplify demand variability uniformly; it shifted a substantial number of industries across the bullwhip threshold in ways a no-COVID forecast could not anticipate. For 2020–2023 the authors calculate the amplification ratio $\mathrm{Var}(\text{production})/\mathrm{Var}(\text{demand})$ from actual data and from LSTM-forecast data treated as the counterfactual without COVID. Comparing the two, they find 24 of 77 industries (31.2%) in the "false negative" zone—actual bullwhip present but forecast absent—and only 2 false positives. Broken down by stage, 18 of these 24 are manufacturers, and 58.8% of all manufacturing industries exhibit bullwhip in the actual period, consistent with the idea that upstream positions absorb the most amplified demand signal.

Load-bearing premise

The load-bearing premise is that an LSTM trained on 1992–2015 data and validated on 2016–2019 gives a valid picture of what 2020–2023 demand and inventory would have been if COVID-19 had never happened; if the forecast's variance is biased low for unrelated reasons, the false-negative counts overstate the pandemic's effect.

Editorial extensions

If this is right

  • COVID-19 raised bullwhip incidence above what pre-pandemic patterns would predict, with 24 U.S. industries crossing the amplification threshold that a no-COVID forecast missed.
  • Manufacturers bear most of the pandemic-induced bullwhip: 18 of the 24 false-negative industries are manufacturers, and 58.8% of manufacturing industries show actual bullwhip.
  • Wholesalers are more exposed than retailers, consistent with the bullwhip logic that intermediate and upstream stages face both demand and supply volatility.
  • Industry-specific demand and supply shocks—plant outbreaks, shutdowns, panic buying, and logistics breakdowns—are the concrete triggers that turn an otherwise predictable industry into a bullwhip case.
  • Forecasting the amplification ratio during stable periods can flag bullwhip-prone industries before a shock, giving supply chain managers a proactive screening tool.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the counterfactual reading depends on the LSTM forecast's variance being unbiased in 2020–2023; if the model under-predicts variance for reasons unrelated to the pandemic, the 31.2% false-negative rate could overstate COVID's causal contribution.
  • Editorial inference: the four-zone classification could be turned into an early-warning score, treating false-negative industries as a training set for features like upstream position, supply shock exposure, and inventory intensity.
  • Editorial inference: a natural testable extension is to repeat the forecast-baseline comparison on a later non-pandemic window, such as 2024–2026, and check whether the false-negative rate falls back toward pre-COVID levels.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper studies the bullwhip effect (BWE) across 77 U.S. manufacturing, wholesale, and retail industries using monthly Census data from 1992 to 2023. It forecasts demand and inventory series with SARIMA, Prophet, RNN, and LSTM models, selects LSTM based on MAPE, and then compares the actual amplification ratio for 2020–2023 with the ratio computed from LSTM forecasts. The forecasts are interpreted as a no-COVID counterfactual. Based on the comparison, the paper reports that 31.2% of industries (24 of 77) show a false-negative pattern, i.e., actual bullwhip where forecasts predicted none, and concludes that COVID-19 significantly amplified the BWE, especially among manufacturers.

Significance. If the counterfactual interpretation were valid, the paper would provide a useful industry-level map of BWE during COVID-19 and a forecasting-based screening approach. Strengths include the use of public Census data, the transparent variance-ratio definition in Eq. (12), a multi-model forecasting comparison, and a literature-grounded discussion of demand and supply shocks in Section 4.3. However, the central inference currently rests on an unvalidated assumption that LSTM forecasts represent no-COVID conditions, and the paper provides no statistical test of its headline claim. The significance of the reported results therefore remains conditional until the forecasting baseline is shown to be a valid counterfactual.

major comments (3)
  1. [§3, §4.2] The central claim that COVID-19 amplified the bullwhip effect rests on treating the LSTM forecast for 2020–2023 as a no-COVID counterfactual, but this assumption is never validated. Section 3.1 splits the test data into 2016–2019 and 2020–2023, yet the only model selection evidence presented is MAPE in Table 1. MAPE rewards point forecast accuracy and does not guarantee that the forecast preserves the variance of the series, which is the quantity entering Eq. (12). Without a demonstration that forecasted amplification ratios track actual ratios in the stable 2016–2019 period, the 31.2% false-negative rate in Table 2 could be an artifact of variance compression or other forecast error structure rather than a pandemic-induced bullwhip effect. This is a load-bearing gap that must be addressed.
  2. [§4.2, §5] The paper claims in Section 5 that the BWE increase has been 'empirically demonstrated' and the abstract states that COVID-19 'significantly amplified' the BWE, but no statistical test or confidence interval supports this claim. Table 2 reports counts of industries in four zones; even if the counterfactual were valid, a formal comparison (e.g., a paired bootstrap of amplification ratios, a permutation test of the false-negative rate, or a regression with industry fixed effects) would be needed to support the word 'significantly'. As presented, the analysis is purely descriptive and the conclusion overstates what the design can identify.
  3. [§3.1, §4.1] The pre-COVID period 2016–2019 is used only to compare MAPE, not to validate the bullwhip measurement itself. The paper should report, for the pre-COVID test period, both the actual and the LSTM-forecasted amplification ratios per industry, along with the distribution of their differences. This is the minimal diagnostic needed to separate ordinary forecast error from COVID-specific effects, and it can be computed from data already used in the manuscript.
minor comments (4)
  1. [Table 2] The manufacturer false-positive entry is listed as '1 (0.02%)' but 1 out of 51 manufacturer industries is approximately 1.96%; please correct the percentage and verify all cell counts and totals for consistency.
  2. [§3.1, §3.2] There is an internal inconsistency in the training window: Section 3.1 states 'training (1992–2015) and test (2016–2023)', while Section 3.2 states 'Monthly data from 1992 to 2016 was used to train forecasting models'. Please clarify the exact split.
  3. [§3.2.4, §4.1] The LSTM architecture is not specified beyond equations (4)–(9); for reproducibility, please report the number of hidden layers, hidden units, learning rate, batch size, and the tuning procedure referenced in Figure 1.
  4. [§5] The claim that this study is 'the first to employ industry-level data to forecast the BWE' should be qualified, as prior industry-level empirical studies (e.g., Cachon et al. 2007) already use industry-level data; the novelty statement should focus on the forecasting-based counterfactual approach rather than industry-level data per se.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the LSTM forecasts are out-of-sample and the bullwhip comparison is not fitted to the actual amplification ratios.

full rationale

The paper's derivation chain is not circular. Demand and inventory series from Census M3/MWTS/MARTS are split into training (1992-2015) and test (2016-2023) periods; four forecasting models are benchmarked on the 2016-2019 pre-COVID period using MAPE, and LSTM is selected as the best performer. The 2020-2023 LSTM forecasts are genuinely out-of-sample, so the forecasted amplification ratios used in Figure 5 and Table 2 are not fitted to the actual 2020-2023 ratios they are compared against. Equation 12 is applied symmetrically to forecasted and actual series, and no parameter is estimated from the actual amplification ratios. The conclusion that COVID-19 amplified the bullwhip effect depends on the identifying assumption that the forecasted series represents the no-COVID counterfactual; this assumption may be threatened by variance compression in point forecasts optimized for MAPE, but that is a statistical validity concern, not a definitional or self-referential circularity. The paper does not rely on load-bearing self-citations: Equation 10 follows Cachon et al. (2007) and Equation 12 cites external BWE metrics literature. Accordingly, no circular step can be exhibited from the text.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central inference rests on one strong untested domain assumption (forecast-as-counterfactual), plus standard accounting identities and aggregation choices. No new entities are introduced.

free parameters (3)
  • LSTM / RNN hyperparameters (hidden layers, learning rate, batch size) = not reported
    The authors tuned these and report only that they were specified; the chosen values affect the forecasted counterfactual baseline and hence the bullwhip classification.
  • SARIMA orders (p,d,q)(P,D,Q,m) = not reported
    Orders were tuned per series; no values or selection criterion details are given in the text.
  • Prophet settings = defaults
    Default settings were used; this choice influences the forecast baseline but is not fitted to BWE.
assumptions (4)
  • domain assumption LSTM forecasts for 2020-2023 represent the no-COVID counterfactual
    Section 3 states forecasted values served as baseline to represent conditions without COVID; no validation that forecast variance matches historical no-COVID variance.
  • domain assumption Production can be inferred from shipments plus inventory changes (Y_it = S_it + I_it - I_it-1)
    Equation 10, following Cachon et al. (2007); standard proxy but an identity in accounting data, not a causal model.
  • standard math Variance ratio of log-differenced production to demand measures bullwhip
    Equations 11-12, standard BWE metric from Disney and Towill (2003); reasonable but not the only metric.
  • domain assumption NAICS industry-level aggregation is sufficient to observe bullwhip
    Section 3.1 uses M3, MWTS, MARTS reports; aggregation can mix products and mask or create bullwhip.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Impact of COVID-19 on The Bullwhip Effect Across U.S. Industries." pith.science (2026). https://pith.science/paper/5BOMNE2P

@misc{pith2026250606368,
  author       = {Pith},
  title        = {Pith review of: Impact of COVID-19 on The Bullwhip Effect Across U.S. Industries},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5BOMNE2P}},
  note         = {Machine review of arXiv:2506.06368}
}
read the original abstract

The Bullwhip Effect, describing the amplification of demand variability up the supply chain, poses significant challenges in Supply Chain Management. This study examines how the COVID-19 pandemic intensified the Bullwhip Effect across U.S. industries, using extensive industry-level data. By focusing on the manufacturing, retailer, and wholesaler sectors, the research explores how external shocks exacerbate this phenomenon. Employing both traditional and advanced empirical techniques, the analysis reveals that COVID-19 significantly amplified the Bullwhip Effect, with industries displaying varied responses to the same external shock. These differences suggest that supply chain structures play a critical role in either mitigating or intensifying the effect. By analyzing the dynamics during the pandemic, this study provides valuable insights into managing supply chains under global disruptions and highlights the importance of tailoring strategies to industry-specific characteristics.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

18 extracted references · 18 canonical work pages

  1. [1]

    The Published Journal Article of Impact of COVID-19 on The Bullwhip Effect Across U.S

    1 This is a pre-copyedited, author-produced version of an article accepted for publication in International Journal of Industrial Engineering following peer review. The Published Journal Article of Impact of COVID-19 on The Bullwhip Effect Across U.S. Industries is available online at https://doi.org/10.23055/ijietap.2025.32.3.10423. IMPACT OF COVID-19 ON...

  2. [2]

    These reactions, influenced by perceived shortages and fears of ongoing crises, have led to substantial increases in demand for essential goods (Loxton et al

    INTRODUCTION The COVID-19 pandemic has significantly altered consumer behaviors, notably through panic buying and stockpiling. These reactions, influenced by perceived shortages and fears of ongoing crises, have led to substantial increases in demand for essential goods (Loxton et al. 2020, Yuen et al. 2020), and put considerable pressure on supply chains...

  3. [3]

    LITERATURE REVIEW 2.1 The Bullwhip Effect: Demand Shocks and Supply Disruptions The primary catalyst for the BWE under pandemic conditions is the inherent uncertainty in demand, which becomes particularly problematic in environments affected by frequent demand shocks. Such environments lead to less predictable demand patterns, which are challenging to for...

  4. [4]

    Since BWE exists regardless of COVID-19, distinguishing between endogenous BWEs and those induced by external shocks of COVID-19 remains challenging

    RESEARCH METHODOLOGY This research is propelled by a dual motive: to seek empirical evidence of the BWE amidst the pandemic by examining industry-level panel data and to assess the ability of time series analysis to forecast the BWE in terms of amplification ratio. Since BWE exists regardless of COVID-19, distinguishing between endogenous BWEs and those i...

  5. [5]

    Forecasts from all models were aggregated into a forecast pool, and their accuracy was evaluated using metrics such as Mean Absolute Percentage Error (MAPE)

    For Prophet, the default settings were used to allow the model to automatically identify changepoints, growth trends, and seasonalities. Forecasts from all models were aggregated into a forecast pool, and their accuracy was evaluated using metrics such as Mean Absolute Percentage Error (MAPE). The best-performing method was selected based on benchmarking ...

  6. [7]

    3.2.3 Recurrent Neural Network (RNN) RNN, is an artificial neural network that recognizes patterns in data sequences

    𝑦/=𝑔/+𝑠/+ℎ/+𝜖* (2) where, - 𝑦/ is the predicted value at time 𝑡, - 𝑔/ is the trend component, modeling non-periodic changes over time, - 𝑠/ is the seasonality component, capturing periodic changes, - ℎ/ is the effects of holidays or special events, - 𝜖* is the error term at time 𝑡. 3.2.3 Recurrent Neural Network (RNN) RNN, is an artificial neural network ...

  7. [10]

    RESULTS AND DISCUSSION 4.1 Forecast Method Performance Comparison Table 1 shows the comparative analysis of forecasting models across two distinct periods, shown in 2016-2019 (pre-COVID-19) and 2020-2023(during and post-COVID-19), respectively, with the average MAPE and MAPE variances of 77 industries both for Demand and Inventory series. Table 1 Forecast...

  8. [11]

    Sample plots of demand series and corresponding predictions of forecast methods are given for each stage to illustrate the comparison between predicted and actual patterns

    The classification in Table 3 reflects these findings: upward arrows (↑) indicate 12 increased demand or supply levels, downward arrows (↓) signify reduced demand or supply, and horizontal arrows (⟷) represent industries with relatively stable demand or supply conditions. Sample plots of demand series and corresponding predictions of forecast methods are ...

Show all 18 references
  1. [12]

    (Disney and Towill 2003, Kim and Springer 2008, Cannella et al. 2013). The variances of demand and production were computed based on monthly values to calculate the amplification ratio. 𝐴𝑚𝑝𝑙𝑖𝑓𝑖𝑐𝑎𝑡𝑖𝑜𝑛 𝑟𝑎𝑡𝑖𝑜= 𝑉𝑎𝑟'43(56*!37𝑉𝑎𝑟)8&97( (12) 8 By the definition of the BWE, values gre...

  2. [13]

    However, supply chains were challenged by manufacturing and international logistics disruptions, highlighting the need for supply chain visibility and flexibility (Nikolopoulos et al. 2021). 14 Figure 7 Demand Series for Household Appliances and Electrical and Electronic Goods...

  3. [14]

    The authors declare that no funding was received for the conduct of this study and preparation of this article

    DISCLOSURE OF INTEREST The authors confirm that there are no relevant financial or non-financial competing interests to report. The authors declare that no funding was received for the conduct of this study and preparation of this article. ACKNOWLEDGMENT This work was supporte...

  4. [18]

    International Journal of Industrial Engineering: Theory, Applications and Practice, 31 (3)

    A Systematic Analysis of Supply Chain Risk Management Literature: 2012-2021. International Journal of Industrial Engineering: Theory, Applications and Practice, 31 (3). Zighan, S.,

  5. [1981]

    Brookings Papers on Economic Activity, 1981 (2), 443–520

    Retail Inventory Behavior and Business Fluctuations. Brookings Papers on Economic Activity, 1981 (2), 443–520. Blinder, A.S. and Maccini, L.J.,

  6. [2010]

    Journal of Supply Chain Management

    Inventory management and the bullwhip effect during the 2007-2009 recession: Evidence from the manufacturing sector. Journal of Supply Chain Management. Döpke, J., Fritsche, U., and Müller, K.,

  7. [2019]

    The aim was to identify the models that consistently deliver the most reliable and accurate prediction in the first period

    and COVID-19 (2020 – 2023). The aim was to identify the models that consistently deliver the most reliable and accurate prediction in the first period. Since the demand and supply shocks during the COVID-19 period can increase the errors as expected, the 2016-2019 period is co...

  8. [2021]

    study details, the imposition of stay-at-home orders and the closure of key institutional buyers like schools and restaurants shifted consumption patterns from dining out to home consumption, substantially increasing demand for Dairy Products. Demand for Material Handling Equi...

  9. [2023]

    Monthly data from 1992 to 2016 was used to train forecasting models, and 2016 - 2023 data was used to evaluate their performance and confirm their predictions

    The methodologies summarized in Figure 1, integrated into our analysis, include traditional and advanced forecasting models. Monthly data from 1992 to 2016 was used to train forecasting models, and 2016 - 2023 data was used to evaluate their performance and confirm their predi...

  10. [2024]

    Business & Industry Time Series / Trend Charts [online]. Available from: https://www.census.gov/econ_timeseries/?programCode=RESSALES&startYear=1963&endYear=2024&categories[]=ASOLD&dataType=TOTAL&geoLevel=US&adjusted=1&notAdjusted=0&errorData=0 [Accessed 2 May 2024]. Verhoef, ...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.