REVIEW 4 major objections 6 minor 28 references
A Study of Data-driven Methods for Inventory Optimization
T0 review · 4 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read This paper claims that a Deep Q-Network reinforcement-learning agent is the most promising of three data-driven methods for supermarket inventory optimization across lost-sales, dual-sourcing, and multi-echelon models.
desk verdict Self-contradictory comparison: DRL training curves fail while the Conclusion calls DRL most promising, and the biased reward design undermines the ranking. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing machinery is the custom simulation environment (`LostSalesInventoryEnv`) plus the DQN agent that learns inside it. The environment is built on a private supermarket dataset with hand-set costs (10 per unit ordered, 1 per day held, 50 per stockout, 5 per price reduction), a three-action space (order stock, do nothing, reduce prices), and a step function that assumes an order of 50 units and sets estimated lost sales to zero whenever the 'order stock' action is taken. The DQN agent uses two hidden layers of 64 ReLU neurons with dropout, experience replay, and epsilon-greedy exploration to produce the reward curves. The time-series forecaster and the tree regressors supply the demand forecasts that the other two comparison arms are built on.
What would settle it
Re-run the lost-sales experiment and log cumulative reward per episode across all 100 episodes, then compare the trained DQN against a simple order-up-to policy using the same cost parameters; if the DQN does not beat that policy on total cost, the central claim fails. The fastest check is to inspect the step() function: when 'order stock' is chosen, estimated lost sales is assumed to be 0, so setting that value to the true demand shortfall and re-training would decide whether the DRL advantage is real.
Extended reading notes
Core claim
On its own terms, the paper's central claim is that a DQN agent trained in a custom supermarket inventory simulator achieves the best balance of lost-sales reduction, cost control, and multi-echelon coordination among time-series forecasting, Random Forest/Gradient Boosting, and deep reinforcement learning. In the lost-sales model the authors call DRL 'the most promising'; in the multi-echelon model they say DRL demonstrates 'the highest efficiency'; and in dual sourcing they credit all three methods with improving the regular/express sourcing trade-off. The paper also reports, in its discussion sections, that after parameter tuning the DQN agent produced smoother but still negative, flat reward curves and that the agent may not be learning an optimal policy or was unable to learn a stable policy. That internal report is part of the paper's own evidence and leaves the headline conclusion resting on the simulator's design rather than on observed improvement in rewards.
Load-bearing premise
The DRL results stand on the simulator's hand-set costs and on the rule that placing an order erases estimated lost sales, and the private dataset means none of the comparisons can be reproduced or audited.
Editorial extensions
If this is right
- If the ranking is correct, a supermarket manager facing frequent stockouts should prefer a DQN-based replenishment policy over time-series or Random Forest in settings that match the simulated cost structure.
- The near-perfect R2 scores reported for Random Forest and Gradient Boosting in the lost-sales and dual-sourcing models would imply that item, supplier, and date features carry almost all information needed to forecast lost sales and replenishment quantities.
- The multi-echelon conclusion would imply that reinforcement learning can coordinate ordering decisions across warehouses, distribution centers, and stores more efficiently than static forecasting rules.
- Because the DQN reward curves are flat even under the paper's own conclusion, the claim implies that stable, non-improving rewards should be read as evidence of a learned policy rather than as a sign of learning failure.
Reading between the lines
- A direct test of the paper's ranking would be to remove the step-function shortcut that sets estimated lost sales to zero whenever stock is ordered; if the DQN advantage shrinks or reverses, the headline result is an artifact of that simulator rule.
- The very high R2 values for the tree regressors suggest the target or a near-duplicate feature may be leaking into training; a temporal train/test split would settle whether the forecast advantage survives out-of-sample.
- Because the dataset is private and the environment is bespoke, an independent replication on a public inventory benchmark (such as the beer-game setting the paper cites) would show whether DQN's edge transfers beyond this simulator.
- One practical consequence the authors leave implicit is that the same comparison pipeline could be re-run with actual margin, holding, and stockout costs from a real store to turn the qualitative ranking into a dollar-denominated decision rule.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper compares three data-driven methodologies—Time Series (Prophet), Random Forest/Gradient Boosting, and Deep Reinforcement Learning (DQN)—applied to three inventory models (lost sales, dual-sourcing, and multi-echelon) in a supermarket context. For each model the authors report forecast metrics, training curves, and qualitative comparisons, and they conclude that all three methods are effective while DRL is the most promising (lost-sales) and shows the highest efficiency (multi-echelon). The study relies on a private, undisclosed supermarket dataset and hand-coded simulation environments, with no theoretical derivation and no reproduction package.
Significance. If the central evaluative claim were supported, the paper would provide practical guidance for choosing among forecasting and learning methods for supermarket inventory management. The paper has the merit of covering three inventory models and three algorithm families, and the Prophet, Random Forest, and Gradient Boosting forecasting pipelines are described in enough detail to be reimplemented. However, the central claim is contradicted by the paper's own reported evidence: the DQN training curves are flat or declining and negative, and the text repeatedly states that the agent did not learn an optimal or stable policy. In addition, the DRL environment's reward design and the absence of any baseline comparison make the claimed ranking of methods unsupported. The private dataset and arbitrary cost parameters further limit the significance of the empirical comparison.
major comments (4)
- [Section 6 vs. Sections 3.4, 3.5, 4.5, 5.4] The conclusion states that DRL exhibits the most promising results in the lost-sales model and the highest efficiency in the multi-echelon model, but this claim is contradicted by the paper's own evidence. Section 3.4 states that the DQN is not successful in learning the optimal inventory management policy; Section 3.5 reports no obvious upward trend for the total rewards after parameter tuning; Section 4.5 states that the agent is unable to learn in a stable policy; and Section 5.4 concedes that an upward and increasing trend in the total rewards is expected for cost optimization but was not observed. Figures 10, 24, and 37 show total rewards that are negative and flat (or declining) over episodes. A flat or declining reward trajectory cannot support most promising or highest efficiency unless DRL is benchmarked against a competent baseline in the same cost environment, which is not reported. This internal contradiction is load-bearing for the paper's central conclusion.
- [Section 3.3.1, step() function] The DRL environment's reward function encodes the policy it is supposed to evaluate. When the agent takes the order stock action, the step() function assumes order quantity and estimated lost sales as 50 units and 0 respectively, meaning that the stockout cost term (defined later in the same subsection) is identically zero for exactly the action the agent is incentivized to choose. This makes the reward signal a direct function of the agent's action rather than a neutral measure of inventory performance, so the claim that DRL minimizes lost sales is circular with respect to the environment definition. The cost parameters (order cost 10, holding cost 1, stockout cost 50, price reduction cost 5) are asserted without justification or sensitivity analysis, and no comparison to a base-stock or other standard policy is provided to calibrate the reward scale.
- [Sections 3.3.2, 4.3.2, 5.3.2] The DRL results are not compared to any control policy or to the forecasting methods on the same cost metric. The total-reward values are reported in arbitrary units (e.g., approximately -726,000 to -729,000 in Figure 24), while the Time Series and Random Forest sections report MAPE, MSE, and R2, which are not directly comparable. Without a common baseline (such as a myopic order-up-to policy evaluated in the same simulation environment), the relative ranking of DRL against TS and RF is unsupported. Sections 4.4 and 5.4 offer only qualitative statements about DRL's ability to learn and adapt, which cannot establish the degree of efficiency claimed in the conclusion.
- [Sections 3.1.2, 4.1.2, 5.2.2 (Tables 1, 3, 4, 9, 11)] The empirical evidence base is weak and partially inconsistent. The Prophet cross-validation tables report high MAPE values (e.g., 0.20-0.69 in Table 1, 0.43-1.08 in Table 4, 0.43-0.45 in Table 9), indicating substantial forecast error, yet the conclusion claims these methods minimize lost sales and optimize inventory levels. The Random Forest and Gradient Boosting R2 values in the lost-sales model (0.9999 in Table 3) are implausibly high for real sales data and suggest possible target leakage or an effectively trivial prediction task; this possibility is not discussed. In the multi-echelon model the same methods achieve only R2 around 0.69 and 0.66 (Table 11), which is a more credible range but requires a very different interpretation of their accuracy. The paper does not reconcile these discrepancies, and the private dataset is not described or released, so none of the numerical comparisons can be audited or reproduced.
minor comments (6)
- [Section 4.2.2 (Table 7)] The entry 0..9999428256116194 for the Gradient Boosting R2 score contains a typographical double decimal point; it should read 0.9999428256116194.
- [Section 5.4] The text refers to DLR graphs where DRL is intended; this typo appears in the paragraph beginning with the sentence about optimizing inventory level in all echelons.
- [Sections 3.1.2, 4.1.2, 5.1.2] The descriptions of Figures 1-5, 14-19, and 28-32 are lengthy and repetitive; the x-axis and y-axis meanings are stated multiple times within each subsection. Condensing these descriptions would improve readability.
- [Section 2.2.1] The reference to Liashenko and Yakymchuk in the text has no entry in the reference list with a complete year or venue; a full citation should be added.
- [Section 3.3.1] The phrase the number of epsilon is greater than epsilon min should be reworded to the current epsilon value is greater than epsilon_min for clarity.
- [Abstract and Section 1] Several sentences are grammatically incomplete or truncated, e.g., The main purpose is to analyse efficient methods for the data-driven. and particularly effective inventory management solutions.This research is designed. A careful editorial pass is needed.
Circularity Check
The conclusion that DRL is 'most promising' in the lost-sales model is built into the simulator: the step() function sets estimated lost sales to zero for the 'order stock' action, so minimizing lost sales is an assumption of the reward design rather than a learned result.
-
self definitional
[Section 3.3.1 (Deep Reinforcement Learning Code Design, step() description) and Section 6 (Conclusion).]
"If the “order stock” action is taken, the order quantity and the estimated lost sales would be assumed as 50 units and 0 respectively. ... The estimated lost sales cost is multiplied by the current price per unit to determine the lost sales cost. Moreover, the reward is lowered by a defined stockout cost if the quantity sold is less than the total of the estimated lost sales and the current estimated demand."
The reward function defines away the very quantity the paper later claims DRL minimizes: taking the 'order stock' action sets estimated lost sales to 0, which zeroes the lost-sales cost term in the reward. The Conclusion's claim that DRL exhibits 'the most promising results' at minimizing lost sales therefore reflects this hand-coded simulator assumption, not an empirically learned policy. The paper's own Sections 3.4 and 3.5 state that the DQN 'is not successful in learning the optimal inventory management policy' and that after tuning 'there is no obvious upward trend for the total rewards,' confirming the praise is not supported by observed learning behavior.
full rationale
This is an empirical study with no theoretical derivation chain or uniqueness theorem, so the main circularity risk is in the evaluation design. One self-definitional step is present: the DRL environment's reward function sets estimated lost sales to zero whenever the agent chooses the 'order stock' action, and the lost-sales cost is computed as estimated lost sales times price; the Conclusion's verdict that DRL minimizes lost sales is therefore true by construction of the reward rather than by agent performance. The paper also repeatedly concedes that the DQN did not learn a stable or optimal policy and that total rewards showed no upward trend, which makes the 'most promising' verdict an internal contradiction, though that is a correctness concern rather than circularity. No self-citation chain, fitted-parameter-as-prediction, or renamed-known-result was found; the Time Series and Random Forest sections report standard holdout metrics with no evidence that the test target was used to construct the features. The score of 4 reflects one definitional circularity in the central DRL claim without additional circular steps.
Assumptions & free parameters
free parameters (7)
- Prophet hyperparameters (changepoint_prior_scale, seasonality_prior_scale, horizon, seasonality toggles) =
changepoint_prior_scale=0.1 (lost sales) / 0.05 (dual-sourcing, multi-echelon); seasonality_prior_scale=10…
- DQN environment cost parameters =
order cost=10, holding cost=1, stockout cost=50, price reduction cost=5; dual-sourcing order costs 10/15…
- DQN episode and network hyperparameters =
episodes=100; batch size=10; learning rate=0.001 (later 0.005); epsilon decay=0.995 (later 0.885); epsilon min=0.01…
- DRL environment structural constants =
order quantity=50 units; estimated lost sales=0 for order action; initial inventory=100; lead times=5 and 10 days…
- Random Forest / Gradient Boosting hyperparameter search ranges =
n_estimators 50-200; max_depth 2-10; learning rate 0.01-0.1; min_samples_split 2-20; min_samples_leaf 1-20
- Random Forest feature set =
Lost-sales model: Supplier ID Encoded, Item ID Encoded, Shelf-life, Replenishment Year, Quantity Sold x Price…
- Order decision threshold =
minimum order quantity threshold = 10 units
assumptions (5)
- domain assumption The undisclosed supermarket dataset is accurate and representative.
- ad hoc to paper The DRL environment's action semantics are a valid model of supermarket inventory.
- domain assumption Demand in the dual-sourcing model is normally distributed with mean mu and variance sigma squared.
- domain assumption DQN can learn a useful policy in 100 episodes with the chosen hyperparameters.
- domain assumption The RF/GB near-perfect R2 values are genuine and not caused by data leakage.
Cite this review
Pith. "Pith review of A Study of Data-driven Methods for Inventory Optimization." pith.science (2026). https://pith.science/paper/Y5QU36KK
@misc{pith2026250508673,
author = {Pith},
title = {Pith review of: A Study of Data-driven Methods for Inventory Optimization},
year = {2026},
howpublished = {\url{https://pith.science/paper/Y5QU36KK}},
note = {Machine review of arXiv:2505.08673}
}
read the original abstract
This paper shows a comprehensive analysis of three algorithms (Time Series, Random Forest (RF) and Deep Reinforcement Learning) into three inventory models (the Lost Sales, Dual-Sourcing and Multi-Echelon Inventory Model). These methodologies are applied in the supermarket context. The main purpose is to analyse efficient methods for the data-driven. Their possibility, potential and current challenges are taken into consideration in this report. By comparing the results in each model, the effectiveness of each algorithm is evaluated based on several key performance indicators, including forecast accuracy, adaptability to market changes, and overall impact on inventory costs and customer satisfaction levels. The data visualization tools and statistical metrics are the indicators for the comparisons and show some obvious trends and patterns that can guide decision-making in inventory management. These tools enable managers to not only track the performance of different algorithms in real-time but also to drill down into specific data points to understand the underlying causes of inventory fluctuations. This level of detail is crucial for pinpointing inefficiencies and areas for improvement within the supply chain.
Figures
Reference graph
Works this paper leans on
-
[1]
Neural inventory control in networks via hindsight differentiable policy optimization
Matias Alvo, Daniel Russo, and Yash Kanoria. Neural inventory control in networks via hindsight differentiable policy optimization. arXiv preprint arXiv:2306.11246, 2023
arXiv 2023
-
[2]
Deep reinforcement learning: A brief survey
Kai Arulkumaran, Marc Peter Deisenroth, Miles Brundage, and Anil Anthony Bharath. Deep reinforcement learning: A brief survey. IEEE Signal Processing Magazine, 34 0 (6): 0 26--38, 2017
2017
-
[3]
Lost-sales inventory theory: A review
Marco Bijvank and Iris FA Vis. Lost-sales inventory theory: A review. European Journal of Operational Research, 215 0 (1): 0 1--13, 2011
work page 2011
-
[4]
Ali Cheaitou, Christian van Delft, Zied Jemai, and Yves Dallery. Optimal policy structure characterization for a two-period dual-sourcing inventory control model with forecast updating. International Journal of Production Economics, 157: 0 238--249, 2014
work page 2014
-
[5]
Learning to order for inventory systems with lost sales and uncertain supplies
Boxiao Chen, Jiashuo Jiang, Jiawei Zhang, and Zhengyuan Zhou. Learning to order for inventory systems with lost sales and uncertain supplies. Management Science, 70 0 (12): 0 8631--8646, 2024
work page 2024
-
[6]
Strategy, planning, and operation
Sunil Chopra and Peter Meindl. Strategy, planning, and operation. Supply Chain Management, 15 0 (5): 0 71--85, 2001
work page 2001
-
[7]
Katrine T Ejlerskov, Stephen J Sharp, Martine Stead, Ashley J Adamson, Martin White, and Jean Adams. Supermarket policies on less-healthy food at checkouts: natural experimental evaluation using interrupted time series analyses of purchases. PLoS medicine, 15 0 (12): 0 e1002712, 2018
work page 2018
-
[8]
Centralized planning models for multi-echelon inventory systems under uncertainty
Awi Federgruen. Centralized planning models for multi-echelon inventory systems under uncertainty. Handbooks in operations research and management science, 4: 0 133--173, 1993
work page 1993
Show all 28 references
-
[9]
The new science of retailing: how analytics are transforming the supply chain and improving performance
Marshall Fisher and Ananth Raman. The new science of retailing: how analytics are transforming the supply chain and improving performance. Harvard Business Review Press, 2010
2010
-
[10]
Analysis of a dual sourcing inventory model with normal unit demand and erlang mixture lead times
Duncan KH Fong, Virginia M Gempesaw, and J Keith Ord. Analysis of a dual sourcing inventory model with normal unit demand and erlang mixture lead times. European Journal of Operational Research, 120 0 (1): 0 97--107, 2000
2000
-
[11]
Deep reinforcement learning in inventory management
Kevin Geevers. Deep reinforcement learning in inventory management. Master's thesis, University of Twente, 2020
2020
-
[12]
Variable selection using random forests
Robin Genuer, Jean-Michel Poggi, and Christine Tuleau-Malot. Variable selection using random forests. Pattern recognition letters, 31 0 (14): 0 2225--2236, 2010
2010
-
[13]
Can deep reinforcement learning improve inventory management? performance on lost sales, dual-sourcing, and multi-echelon problems
Joren Gijsbrechts, Robert N Boute, Jan A Van Mieghem, and Dennis J Zhang. Can deep reinforcement learning improve inventory management? performance on lost sales, dual-sourcing, and multi-echelon problems. Manufacturing & Service Operations Management, 24 0 (3): 0 1349--1368, 2022
2022
-
[14]
Forecasting the forecastability quotient for inventory management
Arthur V Hill, Weiyong Zhang, and Gerald F Burch. Forecasting the forecastability quotient for inventory management. International Journal of Forecasting, 31 0 (3): 0 651--663, 2015
2015
-
[15]
The application of time-series forecasting models in grocery retail industry
O Liashenko and B Yakymchuk. The application of time-series forecasting models in grocery retail industry
-
[16]
Reinforcement learning with intrinsically motivated feedback graph for lost-sales inventory control
Zifan Liu, Xinran Li, Shibo Chen, Gen Li, Jiashuo Jiang, and Jun Zhang. Reinforcement learning with intrinsically motivated feedback graph for lost-sales inventory control. arXiv preprint arXiv:2406.18351, 2024
2024 arXiv
-
[17]
Deep inventory management
Dhruv Madeka, Kari Torkkola, Carson Eisenach, Anna Luo, Dean P Foster, and Sham M Kakade. Deep inventory management. arXiv preprint arXiv:2210.03137, 2022
2022 arXiv
-
[18]
Applied time series analysis: A practical guide to modeling and forecasting
Terence C Mills. Applied time series analysis: A practical guide to modeling and forecasting. Academic press, 2019
2019
-
[19]
Human-level control through deep reinforcement learning
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, et al. Human-level control through deep reinforcement learning. nature, 518 0 (7540): 0 529--533, 2015
2015
-
[20]
Asynchronous methods for deep reinforcement learning
Volodymyr Mnih, Adria Puigdomenech Badia, Mehdi Mirza, Alex Graves, Timothy Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. Asynchronous methods for deep reinforcement learning. In International conference on machine learning, pages 1928--1937. PmLR, 2016
1928
-
[21]
A data-driven approach to predict the success of bank telemarketing
S \'e rgio Moro, Paulo Cortez, and Paulo Rita. A data-driven approach to predict the success of bank telemarketing. Decision Support Systems, 62: 0 22--31, 2014
2014
-
[22]
A deep q-network for the beer game: Deep reinforcement learning for inventory optimization
Afshin Oroojlooyjadid, MohammadReza Nazari, Lawrence V Snyder, and Martin Tak \'a c . A deep q-network for the beer game: Deep reinforcement learning for inventory optimization. Manufacturing & Service Operations Management, 24 0 (1): 0 285--304, 2022
2022
-
[23]
How many trees in a random forest? In Machine Learning and Data Mining in Pattern Recognition: 8th International Conference, MLDM 2012, Berlin, Germany, July 13-20, 2012
Thais Mayumi Oshiro, Pedro Santoro Perez, and Jos \'e Augusto Baranauskas. How many trees in a random forest? In Machine Learning and Data Mining in Pattern Recognition: 8th International Conference, MLDM 2012, Berlin, Germany, July 13-20, 2012. Proceedings 8, pages 154--168. ...
2012
-
[24]
Approximate Dynamic Programming: Solving the curses of dimensionality, volume 703
Warren B Powell. Approximate Dynamic Programming: Solving the curses of dimensionality, volume 703. John Wiley & Sons, 2007
2007
-
[25]
Mastering the game of go with deep neural networks and tree search
David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, et al. Mastering the game of go with deep neural networks and tree search. nature, 529 0 (7587): 0 484-...
2016
-
[26]
Improving organizational decision-making using random forests and big data analytics
P Sruthi, Krishna Kant Dixit, Bantu Mahesh, R Padmavathy, Muntather Almusawi, et al. Improving organizational decision-making using random forests and big data analytics. In 2024 International Conference on IoT, Communication and Automation Technology (ICICAT), pages 1392--139...
2024
-
[27]
Missforest—non-parametric missing value imputation for mixed-type data
Daniel J Stekhoven and Peter B \"u hlmann. Missforest—non-parametric missing value imputation for mixed-type data. Bioinformatics, 28 0 (1): 0 112--118, 2012
2012
-
[28]
Mining data with random forests: A survey and results of new tests
Antanas Verikas, Adas Gelzinis, and Marija Bacauskiene. Mining data with random forests: A survey and results of new tests. Pattern recognition, 44 0 (2): 0 330--349, 2011
2011
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.