{"id":"042fd781-ff09-4919-9ad0-4f47d25b9480","arxiv_id":"2505.08673","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":2.0,"correctness_risk":"high","formal_verification":"none","parameter_count":7,"one_line_summary":"A comparative study of Prophet, Random Forest/Gradient Boosting, and DQN on three supermarket inventory models, whose favorable DRL conclusion is contradicted by its own training curves.","lead":"This report applies time series, random forest, and deep reinforcement learning methods to supermarket inventory problems and compares them on lost sales, dual-sourcing, and multi-echelon models. The comparison is intended to help retail managers choose an inventory forecasting and replenishment method, but the paper's own results do not support its headline conclusion that deep reinforcement learning performs best.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"DQN shows flat/negative reward trajectories and is described as not learning an optimal policy, so the conclusion that DRL is 'most promising'/'highest efficiency' is contradicted by the paper's own evidence.","rationale":"The reader's strongest_claim identifies the contradiction between the Conclusion and the DQN training curves, which I agree is the most load-bearing issue. The reader's weakest_assumption focuses on the unvalidated DRL environment and arbitrary cost parameters, which is a real concern but secondary. I partially agree because the step() function's zeroing of lost sales on the order action is a concrete technical flaw that undermines the reward signal, but the decisive problem is internal: the paper's own text and figures say DQN did not learn a good policy, while the Conclusion ranks DRL first. This is not merely a disagreement with the field's consensus; it is a failure of the paper's argument to support its own headline claim. The proposed test is specific and would settle the question by replacing the unbenchmarked, contradictory reward trajectories with a direct comparison to a classical inventory policy under corrected lost-sales accounting. Because the central claim is already refuted by the paper's own evidence, the reader's REJECT verdict stands unchanged.","tokens_in":22196,"tokens_out":3783,"duration_ms":36066,"concrete_test":"Re-run the lost-sales DRL experiment after patching step() so that ordering does not automatically zero out estimated lost sales and does not force the order quantity to 50, then evaluate the trained DQN on held-out demand paths against an optimized base-stock or (s,Q) policy using the same cost parameters. If DQN's average discounted cost is not below the baseline, the 'most promising' claim fails; if it is below, the flat reward curves may be an artifact of reward scaling or hyperparameters rather than a fundamental failure of the method.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central evaluative claim in the Conclusion is that DRL exhibits 'the most promising results' in the lost-sales model and 'the highest efficiency' in the multi-echelon model. The only DRL evidence reported is total-reward trajectories that are flat or declining and negative: lost-sales rewards plateau in a negative range (Figure 10), dual-sourcing rewards are around -726,000 to -729,000 with no upward movement (Figure 24), and multi-echelon rewards stay near zero after the initial episodes (Figure 37). The paper itself states in Section 3.4 that the DQN 'is not successful in learning the optimal inventory management policy'; Section 3.5 says that after parameter tuning 'there is no obvious upward trend for the total rewards'; Section 4.5 says the agent 'is unable to learn in a stable policy'; and Section 5.4 concedes that an upward cost-optimization trend was expected but not observed. A flat or declining negative reward trajectory cannot support 'most promising' or 'highest efficiency' unless DRL is benchmarked against a competent baseline policy in the same cost environment, and no such baseline comparison is reported. Additionally, the DRL environment in Section 3.3.1 sets the estimated lost sales to 0 whenever the agent takes the 'order stock' action, which removes stockout cost from exactly the action the agent is incentivized to choose; this makes even the raw reward numbers an unreliable basis for ranking methods. The paper's conclusion is therefore internally contradicted by its own reported results.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper compares three data-driven methodologies—Time Series (Prophet), Random Forest/Gradient Boosting, and Deep Reinforcement Learning (DQN)—applied to three inventory models (lost sales, dual-sourcing, and multi-echelon) in a supermarket context. For each model the authors report forecast metrics, training curves, and qualitative comparisons, and they conclude that all three methods are effective while DRL is the most promising (lost-sales) and shows the highest efficiency (multi-echelon). The study relies on a private, undisclosed supermarket dataset and hand-coded simulation environments, with no theoretical derivation and no reproduction package.","tokens_in":1916,"tokens_out":2185,"duration_ms":37480,"significance":"If the central evaluative claim were supported, the paper would provide practical guidance for choosing among forecasting and learning methods for supermarket inventory management. The paper has the merit of covering three inventory models and three algorithm families, and the Prophet, Random Forest, and Gradient Boosting forecasting pipelines are described in enough detail to be reimplemented. However, the central claim is contradicted by the paper's own reported evidence: the DQN training curves are flat or declining and negative, and the text repeatedly states that the agent did not learn an optimal or stable policy. In addition, the DRL environment's reward design and the absence of any baseline comparison make the claimed ranking of methods unsupported. The private dataset and arbitrary cost parameters further limit the significance of the empirical comparison.","major_comments":[{"comment":"The conclusion states that DRL exhibits the most promising results in the lost-sales model and the highest efficiency in the multi-echelon model, but this claim is contradicted by the paper's own evidence. Section 3.4 states that the DQN is not successful in learning the optimal inventory management policy; Section 3.5 reports no obvious upward trend for the total rewards after parameter tuning; Section 4.5 states that the agent is unable to learn in a stable policy; and Section 5.4 concedes that an upward and increasing trend in the total rewards is expected for cost optimization but was not observed. Figures 10, 24, and 37 show total rewards that are negative and flat (or declining) over episodes. A flat or declining reward trajectory cannot support most promising or highest efficiency unless DRL is benchmarked against a competent baseline in the same cost environment, which is not reported. This internal contradiction is load-bearing for the paper's central conclusion.","section":"Section 6 vs. Sections 3.4, 3.5, 4.5, 5.4"},{"comment":"The DRL environment's reward function encodes the policy it is supposed to evaluate. When the agent takes the order stock action, the step() function assumes order quantity and estimated lost sales as 50 units and 0 respectively, meaning that the stockout cost term (defined later in the same subsection) is identically zero for exactly the action the agent is incentivized to choose. This makes the reward signal a direct function of the agent's action rather than a neutral measure of inventory performance, so the claim that DRL minimizes lost sales is circular with respect to the environment definition. The cost parameters (order cost 10, holding cost 1, stockout cost 50, price reduction cost 5) are asserted without justification or sensitivity analysis, and no comparison to a base-stock or other standard policy is provided to calibrate the reward scale.","section":"Section 3.3.1, step() function"},{"comment":"The DRL results are not compared to any control policy or to the forecasting methods on the same cost metric. The total-reward values are reported in arbitrary units (e.g., approximately -726,000 to -729,000 in Figure 24), while the Time Series and Random Forest sections report MAPE, MSE, and R2, which are not directly comparable. Without a common baseline (such as a myopic order-up-to policy evaluated in the same simulation environment), the relative ranking of DRL against TS and RF is unsupported. Sections 4.4 and 5.4 offer only qualitative statements about DRL's ability to learn and adapt, which cannot establish the degree of efficiency claimed in the conclusion.","section":"Sections 3.3.2, 4.3.2, 5.3.2"},{"comment":"The empirical evidence base is weak and partially inconsistent. The Prophet cross-validation tables report high MAPE values (e.g., 0.20-0.69 in Table 1, 0.43-1.08 in Table 4, 0.43-0.45 in Table 9), indicating substantial forecast error, yet the conclusion claims these methods minimize lost sales and optimize inventory levels. The Random Forest and Gradient Boosting R2 values in the lost-sales model (0.9999 in Table 3) are implausibly high for real sales data and suggest possible target leakage or an effectively trivial prediction task; this possibility is not discussed. In the multi-echelon model the same methods achieve only R2 around 0.69 and 0.66 (Table 11), which is a more credible range but requires a very different interpretation of their accuracy. The paper does not reconcile these discrepancies, and the private dataset is not described or released, so none of the numerical comparisons can be audited or reproduced.","section":"Sections 3.1.2, 4.1.2, 5.2.2 (Tables 1, 3, 4, 9, 11)"}],"minor_comments":[{"comment":"The entry 0..9999428256116194 for the Gradient Boosting R2 score contains a typographical double decimal point; it should read 0.9999428256116194.","section":"Section 4.2.2 (Table 7)"},{"comment":"The text refers to DLR graphs where DRL is intended; this typo appears in the paragraph beginning with the sentence about optimizing inventory level in all echelons.","section":"Section 5.4"},{"comment":"The descriptions of Figures 1-5, 14-19, and 28-32 are lengthy and repetitive; the x-axis and y-axis meanings are stated multiple times within each subsection. Condensing these descriptions would improve readability.","section":"Sections 3.1.2, 4.1.2, 5.1.2"},{"comment":"The reference to Liashenko and Yakymchuk in the text has no entry in the reference list with a complete year or venue; a full citation should be added.","section":"Section 2.2.1"},{"comment":"The phrase the number of epsilon is greater than epsilon min should be reworded to the current epsilon value is greater than epsilon_min for clarity.","section":"Section 3.3.1"},{"comment":"Several sentences are grammatically incomplete or truncated, e.g., The main purpose is to analyse efficient methods for the data-driven. and particularly effective inventory management solutions.This research is designed. A careful editorial pass is needed.","section":"Abstract and Section 1"}],"recommendation":"reject","confidential_remarks":"The manuscript appears to be an undergraduate project report rather than a journal-standard research paper. The most significant issue is internal inconsistency: the conclusion praises DRL while the paper's own results and text state that the DQN did not learn a good policy. The data and code are not available, and the cost parameters and environment design are not validated against any real or standard benchmark, so the empirical claims are not reproducible. Given that the central evaluative claim is contradicted by the paper's own evidence and the underlying data cannot be audited, I do not see a feasible scope of revision that would make the paper publishable in its current form."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The short version: this is a careful, methodical application of three off-the-shelf methods (Prophet, Random Forest/Gradient Boosting, DQN) to three standard inventory models on a private Hong Kong supermarket dataset. The writing is detailed about code and parameters, and the paper's own discussion sections openly admit the DRL agents did not learn a good policy. But the Conclusion then claims DRL is 'most promising' and 'highest efficiency,' directly contradicting the reported flat and negative reward curves. That contradiction is load-bearing, so the paper cannot stand as a comparison.\n\nWhat is genuinely useful: the authors document their environment construction, including cost parameters and action definitions, thoroughly enough that a reader can see exactly what was simulated. The time-series cross-validation tables are honest about horizon-dependent degradation, and the RF/GB results are reported with residuals and learning curves. That is real work, and it would be a decent technical appendix for a course project.\n\nThe soft spots are serious, not minor. First, the conclusion does not follow from the evidence. The DQN reward curves plateau at negative values with no upward trend, and Sections 3.5, 4.5, and 5.4 say the agent did not learn an optimal or stable policy. Calling that 'most promising' is unsupported. Second, the DRL environment in Section 3.3.1 zeroes estimated lost sales whenever the agent chooses 'order stock,' which removes stockout cost from exactly the action being rewarded; that alone makes the reward signal a poor basis for ranking methods. Third, the RF/GB R² values near 0.9999 are unexplained and suspicious, likely leakage or a near-trivial target, and without the dataset or code nobody can check. The private dataset and the absence of any baseline policy comparison, even a simple base-stock heuristic, make the numbers unverifiable.\n\nWho is this for? A reader who wants a concrete walk-through of setting up Prophet, RF/GB, and DQN for three inventory models, with honest limitations in the body, might skim it. It is not a contribution to the research literature: Gijsbrechts et al. (2022), which the paper itself cites, already benchmarks deep RL on exactly these three model classes. I would not bring it to a reading group or cite it.\n\nRecommendation: desk reject. The central evaluative claim is internally contradicted by the paper's own results, and the environment design flaw plus missing data and code make the numbers unverifiable. If the authors fixed the conclusion and added a base-stock baseline plus released code and data, it might become a modest empirical study, but as is it does not deserve referee time.","headline":"Self-contradictory comparison: DRL training curves fail while the Conclusion calls DRL most promising, and the biased reward design undermines the ranking.","tokens_in":23115,"tokens_out":3608,"would_cite":false,"duration_ms":31272,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a Deep Q-Network reinforcement-learning agent is the most promising of three data-driven methods for supermarket inventory optimization across lost-sales, dual-sourcing, and multi-echelon models.","keywords":["inventory optimization","deep reinforcement learning","DQN","time series forecasting","random forest","lost sales model","dual sourcing","multi-echelon inventory"],"falsifier":"Re-run the lost-sales experiment and log cumulative reward per episode across all 100 episodes, then compare the trained DQN against a simple order-up-to policy using the same cost parameters; if the DQN does not beat that policy on total cost, the central claim fails. The fastest check is to inspect the step() function: when 'order stock' is chosen, estimated lost sales is assumed to be 0, so setting that value to the true demand shortfall and re-training would decide whether the DRL advantage is real.","tokens_in":21900,"feed_emoji":"🛒","tokens_out":12697,"duration_ms":110709,"temperature":0.7,"pith_summary":"This paper tries to establish which of three data-driven methods—time-series forecasting, Random Forest with Gradient Boosting as a companion, and a Deep Q-Network (DQN) reinforcement-learning agent—should be trusted with supermarket inventory decisions. Its conclusion is that all three methods can reduce lost sales and coordinate stock across supply-chain levels, and that DQN is the most promising in the lost-sales model and the most efficient in the multi-echelon model. If that ranking were correct, it would give retailers a concrete reason to favor reinforcement-learning systems over conventional forecasting. The paper's own discussion sections record that the DQN reward curves plateaued at large negative values and that the agent did not learn an optimal policy, so the reported advantage sits in tension with the training evidence.","feed_headline":"Paper: DQN wins supermarket inventory tests","feed_subtitle":"Deep reinforcement learning is called the most promising of the three methods tested on supermarket inventory models.","key_machinery":"The load-bearing machinery is the custom simulation environment (`LostSalesInventoryEnv`) plus the DQN agent that learns inside it. The environment is built on a private supermarket dataset with hand-set costs (10 per unit ordered, 1 per day held, 50 per stockout, 5 per price reduction), a three-action space (order stock, do nothing, reduce prices), and a step function that assumes an order of 50 units and sets estimated lost sales to zero whenever the 'order stock' action is taken. The DQN agent uses two hidden layers of 64 ReLU neurons with dropout, experience replay, and epsilon-greedy exploration to produce the reward curves. The time-series forecaster and the tree regressors supply the demand forecasts that the other two comparison arms are built on.","core_discovery":"On its own terms, the paper's central claim is that a DQN agent trained in a custom supermarket inventory simulator achieves the best balance of lost-sales reduction, cost control, and multi-echelon coordination among time-series forecasting, Random Forest/Gradient Boosting, and deep reinforcement learning. In the lost-sales model the authors call DRL 'the most promising'; in the multi-echelon model they say DRL demonstrates 'the highest efficiency'; and in dual sourcing they credit all three methods with improving the regular/express sourcing trade-off. The paper also reports, in its discussion sections, that after parameter tuning the DQN agent produced smoother but still negative, flat reward curves and that the agent may not be learning an optimal policy or was unable to learn a stable policy. That internal report is part of the paper's own evidence and leaves the headline conclusion resting on the simulator's design rather than on observed improvement in rewards.","pith_inferences":["A direct test of the paper's ranking would be to remove the step-function shortcut that sets estimated lost sales to zero whenever stock is ordered; if the DQN advantage shrinks or reverses, the headline result is an artifact of that simulator rule.","The very high R2 values for the tree regressors suggest the target or a near-duplicate feature may be leaking into training; a temporal train/test split would settle whether the forecast advantage survives out-of-sample.","Because the dataset is private and the environment is bespoke, an independent replication on a public inventory benchmark (such as the beer-game setting the paper cites) would show whether DQN's edge transfers beyond this simulator.","One practical consequence the authors leave implicit is that the same comparison pipeline could be re-run with actual margin, holding, and stockout costs from a real store to turn the qualitative ranking into a dollar-denominated decision rule."],"forward_implications":["If the ranking is correct, a supermarket manager facing frequent stockouts should prefer a DQN-based replenishment policy over time-series or Random Forest in settings that match the simulated cost structure.","The near-perfect R2 scores reported for Random Forest and Gradient Boosting in the lost-sales and dual-sourcing models would imply that item, supplier, and date features carry almost all information needed to forecast lost sales and replenishment quantities.","The multi-echelon conclusion would imply that reinforcement learning can coordinate ordering decisions across warehouses, distribution centers, and stores more efficiently than static forecasting rules.","Because the DQN reward curves are flat even under the paper's own conclusion, the claim implies that stable, non-improving rewards should be read as evidence of a learned policy rather than as a sign of learning failure."],"supporting_citations":[{"why":"Provides the Deep Q-Network algorithm that the paper adapts to inventory control.","marker":"[Mnih et al., 2015]"},{"why":"Shows DQN playing the beer game and establishes the inventory-management use case the paper extends.","marker":"[Oroojlooyjadid et al., 2022]"},{"why":"Benchmarks deep reinforcement learning on lost-sales, dual-sourcing, and multi-echelon problems, the same three models this paper compares.","marker":"[Gijsbrechts et al., 2022]"},{"why":"Defines the lost-sales inventory model and stockout assumptions built into the simulation environment.","marker":"[Bijvank and Vis, 2011]"},{"why":"Supplies the dual-sourcing model with normal demand and Erlang lead times used to frame supplier decisions.","marker":"[Fong et al., 2000]"},{"why":"Sets out centralized planning for multi-echelon inventory systems, the basis of that model's coordination objective.","marker":"[Federgruen, 1993]"},{"why":"Motivates data-driven learning to order in lost-sales systems with uncertain supplies.","marker":"[Chen et al., 2024]"}],"fun_headline_variants":["DQN tops supermarket inventory methods in simulated tests","Deep reinforcement learning leads stock models, with caveats","Supermarket inventory: DQN best but agent may underlearn","Data-driven stock control: DQN shines, yet reward curve flat","Simulated retail stock: DQN promising, stability unproven"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The DRL results stand on the simulator's hand-set costs and on the rule that placing an order erases estimated lost sales, and the private dataset means none of the comparisons can be reproduced or audited.","fun_headline_variants_meta":{"raw":{"variants":["DQN tops supermarket inventory methods in simulated tests","Deep reinforcement learning leads stock models, with caveats","Supermarket inventory: DQN best but agent may underlearn","Data-driven stock control: DQN shines, yet reward curve flat","Simulated retail stock: DQN promising, stability unproven"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000617,"raw_usage":{"total_tokens":2838,"prompt_tokens":895,"completion_tokens":1943,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":511,"completion_tokens_details":{"reasoning_tokens":1857}},"tokens_in":511,"tokens_out":1943,"duration_ms":14214,"temperature":1.0,"reasoning_tokens":1857,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T21:48:46.016565+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the lost-sales experiment and log cumulative reward per episode across all 100 episodes, then compare the trained DQN against a simple order-up-to policy using the same cost parameters; if the DQN does not beat that policy on total cost, the central claim fails. The fastest check is to inspect the step() function: when 'order stock' is chosen, estimated lost sales is assumed to be 0, so setting that value to the true demand shortfall and re-training would decide whether the DRL advantage is real.","supporting_citations":[{"cited_title":"A deep q-network for the beer game: Deep reinforcement learning for inventory optimization","cited_arxiv_id":null,"evidence_quote":"Shows DQN playing the beer game and establishes the inventory-management use case the paper extends."},{"cited_title":"Can deep reinforcement learning improve inventory management? performance on lost sales, dual-sourcing, and multi-echelon problems","cited_arxiv_id":null,"evidence_quote":"Benchmarks deep reinforcement learning on lost-sales, dual-sourcing, and multi-echelon problems, the same three models this paper compares."},{"cited_title":"Lost-sales inventory theory: A review","cited_arxiv_id":null,"evidence_quote":"Defines the lost-sales inventory model and stockout assumptions built into the simulation environment."},{"cited_title":"Analysis of a dual sourcing inventory model with normal unit demand and erlang mixture lead times","cited_arxiv_id":null,"evidence_quote":"Supplies the dual-sourcing model with normal demand and Erlang lead times used to frame supplier decisions."},{"cited_title":"Centralized planning models for multi-echelon inventory systems under uncertainty","cited_arxiv_id":null,"evidence_quote":"Sets out centralized planning for multi-echelon inventory systems, the basis of that model's coordination objective."}],"review_version":1}