Dynamic Retail Pricing via Q-Learning -- A Reinforcement Learning Framework for Enhanced Revenue Management

Ketan Kale; Mohit Apte; Pranav Datar; Pratiksha Deshmukh

REVIEW 4 major objections 6 minor 12 references

Dynamic Retail Pricing via Q-Learning -- A Reinforcement Learning Framework for Enhanced Revenue Management

T0 review · 4 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims a Q-learning agent learns retail prices that beat a traditional optimizer on a simulated demand model.

desk verdict The paper's own tables show scipy.optimize beating Q-learning on revenue for all 14 products, so the central claim is contradicted by the reported data. read the letter →

arxiv 2411.18261 v1 pith:JNZ4AHF2 submitted 2024-11-27 cs.LG

Mohit Apte , Ketan Kale , Pranav Datar , Pratiksha Deshmukh This is my paper · ORCID

classification cs.LG

keywords dynamicpricingQ-learningreinforcementlearningrevenuemanagementpriceelasticitydemandmodelingretailepsilon-greedy

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that a Q-learning agent can set retail prices dynamically and generate more revenue than a traditional operations-research optimizer applied to the same demand model. The authors build a simulated environment in which demand follows a linear function of price, with per-product base price, base demand, and price elasticity. The agent's state is the product type plus whether the day is a weekday or weekend, its actions are candidate prices, and its reward is unit margin times demand. They report 'optimal prices' learned by Q-learning for 14 electronics products and compare them with prices from a scipy-based optimizer, interpreting the comparisons as evidence that the RL approach adapts to market dynamics and price elasticity better than static optimization. The central claim is that this model-free learning approach surpasses traditional methods in revenue generation.

What carries the argument

The load-bearing machinery is the Q-learning update rule combined with a linear demand model. Demand is modeled as $\text{Demand} = \text{Base Demand} \times \left(1 + \text{Elasticity} \times \frac{\text{Price} - \text{Base Price}}{\text{Base Price}}\right)$, the reward is $(\text{Price} - \text{Cost}) \times \text{Demand}$, and the agent updates $Q(s,a) \leftarrow (1-\alpha) Q(s,a) + \alpha\,\left(\text{reward} + \gamma \max_{a'} Q(s',a')\right)$ under an epsilon-greedy policy. This machinery lets a model-free agent discover which prices are best for each product and day type without explicitly solving the optimization, and it is the same machinery that would let the policy update online if the demand function drifted.

What would settle it

Using the paper's own demand equation and Table I parameters, compute the revenue (price times demand) at the Q-learning prices from Table II and at the traditional optimizer's prices from Table III; if the traditional optimizer's prices produce equal or higher total revenue than the Q-learning prices, the paper's central claim of revenue superiority is refuted.

Watch

Extended reading notes

Core claim

On its own terms, the paper's discovery is that Q-learning, with no prior knowledge of the demand function, converges to a pricing policy that the authors say outperforms the prices produced by a traditional optimizer on the same simulated products. The learned policy chooses prices that differ from the traditional optimizer's prices, and for many products the Q-learning price is paired with a higher demand at a higher price, which the authors read as capturing market segments the static optimizer misses. The mechanism is straightforward: the agent receives a reward equal to margin times demand for each price it tries, and the Q-table learns to associate product-and-day states with the most profitable prices. The paper presents this as a general blueprint for data-driven dynamic pricing, where the demand model can be replaced by live market feedback.

Load-bearing premise

The linear demand equation with the per-product elasticity, base price, and base demand values is assumed to be the true market demand, but the paper gives no estimation procedure or validation for these parameters.

Editorial extensions

If this is right

If the claim holds, retailers can delegate price setting to an agent that re-optimizes continuously as market conditions change, without re-estimating a global demand model.
The state split by product and weekday/weekend shows that adding context features to the state space is a natural way to capture demand seasonality in pricing.
The same reward structure can be extended to include inventory costs, competitor prices, or customer segments, and the Q-learning algorithm would remain unchanged.
Because the agent is model-free, the approach transfers to markets where the true demand function is unknown, as long as a reward signal (margin) is observable.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The paper's tables report demand, not revenue; computing revenue from the listed prices and demands shows that the traditional optimizer yields higher revenue for every product in the tables, so the 'revenue superiority' claim is not actually demonstrated by the displayed numbers.
Because the demand parameters are presented without an estimation procedure or confidence intervals, the strongest defensible statement is that Q-learning can optimize the model it is given, not that it discovers true market optima.
A testable extension would be to run the same comparison on a nonlinear or stochastic demand function, where the linear optimizer's closed-form advantage disappears and the RL agent's exploration could matter more.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

You asked for a quick read of arXiv:2411.18261. Here's the short version: the paper's central claim — that Q-learning surpasses traditional methods in revenue — is contradicted by its own Tables II and III. For every one of the 14 products, the scipy.optimize price times demand exceeds the Q-learning result. The Samsung 49" 4K Q6F example they highlight is the worst: RL gives $820.3 × 101.5 ≈ $83k, traditional gives $509.5 × 260.1 ≈ $132k. That's a 60% gap, the other way.

To be fair, the paper has some virtues. It is plainly written, the demand model in Eq. (1) is explicit, and the authors actually print all results, so the discrepancy is visible to anyone who multiplies two numbers. That transparency is more than many submissions manage. The Q-learning update is standard and correctly stated. As a tutorial-style exposition of how one could set up tabular Q-learning for a linear demand curve, it is not misleading.

The problems go beyond the misread tables though. The reward function in Eq. (3) is profit, not revenue, but no cost values are reported anywhere, so even if we interpret the claim as 'higher profit', it is untestable from the data. The comparison is also circular: both the Q-learning environment and the scipy baseline optimize the same profit function built from the same fitted demand parameters. This is a convergence test on a known optimum, not a demonstration of RL beating OR. The demand parameters are said to come from the Datafiniti dataset, but no estimation procedure, validation, or error bars are given, so the whole experiment rests on unverified numbers. There are also no hyperparameters for Q-learning, no code, and no out-of-sample evaluation. Finally, the novelty is thin: the authors themselves cite Kim et al. [5] as already applying Q-learning to dynamic pricing in e-commerce, and the present work adds only a small 14-product simulation.

Who gets value from this? A reader completely new to RL might take away a basic template, but as a research paper it does not support its conclusions. I would desk-reject it rather than send it to reviewers. If the authors return with costs, the demand estimation procedure, convergence diagnostics, and a fair comparison on the same objective, it might be a reasonable workshop-level study, but the revenue claim should be dropped unless it actually holds up.

Referee Report

4 major / 6 minor

Summary. The paper proposes a tabular Q-learning approach for dynamic retail pricing, simulated on 14 consumer electronic products whose demand follows a linear function of price, elasticity, base price, and base demand (Eq. (1)). The Q-learning agent selects prices in a discrete action space with weekday/weekend states and a profit-based reward (Eq. (3)), and is compared with a scipy.optimize baseline that maximizes the same demand function. The abstract and conclusion claim that the RL model surpasses traditional methods in revenue generation and adapts to evolving market dynamics.

Significance. If the central claim were supported, the paper would provide a simple demonstration that tabular Q-learning can beat classical optimization on a deterministic pricing benchmark. However, the paper's own tables contradict the claim, and the experimental design does not test adaptation to changing market dynamics. The main strength is transparency: the demand equation and Tables I-III are explicit enough that the revenue comparison can be checked directly, which is exactly what reveals the contradiction. There is no reproducible code, no convergence analysis, and no statistical validation, so the paper offers little beyond that transparency.

major comments (4)

[Abstract and Section V] The central claim that the RL model 'surpasses traditional methods in terms of revenue generation' is contradicted by the paper's own Tables II and III. For every one of the 14 products, the scipy.optimize price times demand revenue exceeds the Q-learning price times demand revenue. For the product highlighted in Section V, Samsung 49" 4K Q6F, the RL result is 820.3 x 101.5 = $83,260 while the traditional result is 509.5 x 260.1 = $132,521, so the cited example shows lower revenue, not higher. Since the reward function in Eq. (3) is profit, (Price - Cost) x Demand, and no cost values are reported, the revenue claim cannot be rescued by interpreting the tables as profit. The numerical results therefore directly undermine the abstract and conclusion.
[Section III.A-III.B and Section IV] The comparison is circular in a way that invalidates the generalization claim. Both the Q-learning environment and the scipy baseline optimize the same deterministic demand function Eq. (1) built from the same per-product parameters in Table I. The experiment therefore tests whether tabular Q-learning converges to an optimum of a known static function; it does not test adaptation to changing market dynamics. The paper claims the parameters were estimated from the Datafiniti dataset but gives no estimation procedure, no validation, and no error bars, so every 'optimal' price in Tables II and III rests on an unverified demand model. This is a load-bearing gap for the stated conclusion that RL adapts to real-time market changes.
[Section III.C-III.F and Tables II-III] The Q-learning results appear not to have converged, and the missing hyperparameters make the experiments unreproducible. The paper does not report the learning rate alpha, discount factor gamma, epsilon schedule, number of episodes, action set granularity, or state transition dynamics. Several 'optimal' Q-learning prices are suspiciously close to base prices (e.g., VIZIO 70" 4K XHDR at 1300.2 versus base 1300.0) or far from the scipy optimum in ways inconsistent with a converged policy (e.g., Samsung 49" 4K Q6F at 820.3 versus 509.5). Without convergence curves or repeated-seed statistics, the tables cannot be interpreted as evidence for or against Q-learning's effectiveness.
[Section V] Section V misinterprets the results by treating higher optimized demand as evidence of advantage. Revenue management maximizes profit or revenue, not demand quantity. For instance, the paper praises the Sony 43" 4K UHD result because RL reaches 203.8 units at $610.5 versus the traditional 506.9 units at $382.0, but the traditional revenue is $193,636 versus the RL revenue of $124,420, so the RL outcome is worse on the stated objective. The evaluation criterion in the text is inconsistent with the metric used in the abstract and conclusion.

minor comments (6)

[Section III.E] Eq. (3) defines the reward as profit, but the abstract and conclusion repeatedly refer to 'revenue generation.' The paper should either report unit costs and evaluate profit, or consistently use revenue as the objective.
[Section III.D] The state space is described as product type and weekday/weekend, but the paper never specifies how many states exist, how the weekday/weekend distinction affects demand, or how the Q-table is structured. This makes the experiment impossible to reproduce.
[Section III.G] The dataset description says the Datafiniti data 'was instrumental in estimating price elasticities, base demand, and initial pricing,' but no estimation method, code, or validation is provided. A citation or appendix with the procedure is needed.
[References] Reference [9] has a missing opening quotation mark, and several references lack page ranges or have inconsistent formatting (e.g., [2], [5], [6]).
[Section III.A] There is a typo in 'a environment' that should read 'an environment.'
[Throughout] No code, data, or reproducibility statement is provided, which is unusual for an empirical machine learning paper and would be required for any verification of the claims.

Circularity Check

2 steps flagged · score 6.0 of 10

The 'optimal prices' are in-sample maximizers of the same fitted demand function used to train the RL agent, making the claimed price-elasticity insights and RL superiority reduce to the constructed simulation.

fitted input called prediction [Section III-G (Dataset Description); Eq. (1); Tables I-III]
"The dataset was instrumental in estimating price elasticities, base demand, and initial pricing for the products under consideration."

Eq. (1) defines demand using the base demand, base price, and elasticity values reported in Table I. Those same values are the inputs to the simulated retail environment in which the Q-learning agent acts and to the scipy objective built from the Eq. (3) reward. Tables II and III then present 'optimal' prices and demands for those same products. These outputs are in-sample maximizers of the fitted demand/profit function; no new market data or validation is introduced. The claimed 'insight into the complex interplay of price elasticity and consumer demand' is a restatement of the elasticity column of Table I, so the prediction is forced by the fitted input.
renaming known result [Section V, Samsung 49” 4K Q6F example]
"For instance, in the case of the Samsung 49” 4K Q6F, reinforcement learning achieves an optimized demand of 101.5 units at an optimal price of $820.3, compared to the 260.1 units at $509.5 obtained from traditional optimization."

Both price-demand pairs in this example are computed from the same Eq. (1) using the Table I elasticity (-4.4) for that product. The paper presents this as evidence that RL 'strategically adjust[s] prices to capture more market share while maximizing revenue' and adapts to 'market dynamics.' However, the demand response to price is entirely fixed by the fitted linear demand equation; the Q-learning agent simply picks an action in an environment whose reward is that equation. Interpreting the resulting demand difference as an RL-discovered insight is renaming the known fitted demand curve as an emergent result.

full rationale

The circularity is not a self-citation chain; it is the fitted-input-as-prediction pattern. The authors estimate base price, base demand, and elasticity for 14 products from the Datafiniti dataset (no procedure or error bars given), then build the RL environment and the scipy baseline from that exact demand model. Consequently, the 'optimal prices' in Tables II and III are solutions to the same constructed profit/reward function, and any 'insight into price elasticity and consumer demand' is just the assumed elasticity written out again. That is why the central quantitative content reduces to the fitted input by construction. Separately, and independent of circularity, the paper's numerical tables actually contradict the abstract's revenue-superiority claim: for the highlighted Q6F example, RL revenue is 820.3 × 101.5 ≈ $83,260, while traditional revenue is 509.5 × 260.1 ≈ $132,521, and the same pattern holds for all products if revenue is price × demand. This is a correctness/consistency problem rather than a circularity problem, so it is noted here but not counted as an additional circular step. Because at least the 'predictions' reduce to the fitted demand function by construction, the score is 6 (partial circularity), not higher, as the RL-vs-scipy comparison could in principle be a legitimate optimizer benchmark if reframed as such.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

All conclusions rest on unverified demand parameters and an unreported cost value. The Q-learning agent and the scipy baseline solve the same profit-maximization problem defined by these parameters, so there is no external benchmark or prediction.

free parameters (5)

per-product price elasticity = Table I, e.g., -0.5 to -8.4
The demand curve slope is set by these elasticity values; no fitting method or confidence intervals are provided.
per-product base demand = Table I, e.g., 27 to 154
Baseline quantity demanded at base price; extracted from dataset but method unstated.
per-product base price = Table I, e.g., 109.2 to 2411.6
Reference price used in the demand formula; source not described.
cost per unit = not reported (implicitly zero in revenue computations)
The reward function subtracts Cost times Demand, but no cost values are given anywhere; the traditional optimizer's prices match a zero-cost revenue maximization.
Q-learning hyperparameters = not reported
Learning rate, discount factor, epsilon, episode count, and action set are never specified, so the exact results are unreproducible.

assumptions (4)

domain assumption Demand is linear in price with constant elasticity at the base price (Eq. 1).
Used to define the simulated environment for both Q-learning and scipy; no empirical justification.
domain assumption Profit equals (price minus cost) times demand, with constant per-unit cost.
Eq. (3) defines the reward; cost values are never given.
domain assumption The Datafiniti product dataset is representative for estimating demand parameters.
Section III-G says the dataset was used to estimate elasticities and base demand, but no estimation method is shown.
ad hoc to paper Weekday/weekend states and the static demand function capture market dynamics.
The paper claims adaptation to market dynamics but never simulates any time-varying process; the results are on a static function.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Dynamic Retail Pricing via Q-Learning -- A Reinforcement Learning Framework for Enhanced Revenue Management." pith.science (2026). https://pith.science/paper/JNZ4AHF2

@misc{pith2026241118261,
  author       = {Pith},
  title        = {Pith review of: Dynamic Retail Pricing via Q-Learning -- A Reinforcement Learning Framework for Enhanced Revenue Management},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JNZ4AHF2}},
  note         = {Machine review of arXiv:2411.18261}
}

read the original abstract

This paper explores the application of a reinforcement learning (RL) framework using the Q-Learning algorithm to enhance dynamic pricing strategies in the retail sector. Unlike traditional pricing methods, which often rely on static demand models, our RL approach continuously adapts to evolving market dynamics, offering a more flexible and responsive pricing strategy. By creating a simulated retail environment, we demonstrate how RL effectively addresses real-time changes in consumer behavior and market conditions, leading to improved revenue outcomes. Our results illustrate that the RL model not only surpasses traditional methods in terms of revenue generation but also provides insights into the complex interplay of price elasticity and consumer demand. This research underlines the significant potential of applying artificial intelligence in economic decision-making, paving the way for more sophisticated, data-driven pricing models in various commercial domains.

Figures

Figures reproduced from arXiv: 2411.18261 by the authors.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

12 extracted references · 11 canonical work pages

[5]

Kim et al., ``Applying Q-Learning for dynamic pricing and inventory control,'' in IEEE Transactions on Systems, Man, and Cybernetics, vol

J. Kim et al., ``Applying Q-Learning for dynamic pricing and inventory control,'' in IEEE Transactions on Systems, Man, and Cybernetics, vol. 47, no. 8, pp. 2120-2130, 2017

work page 2017
[1]

R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction. MIT Press, 2018

2018
[2]

Smith et al., ``Pricing strategy optimization in the airline industry,'' in Journal of Revenue and Pricing Management, vol

B. Smith et al., ``Pricing strategy optimization in the airline industry,'' in Journal of Revenue and Pricing Management, vol. 11, no. 3, pp. 324-336, 1992

work page 1992
[3]

Jones and C

D. Jones and C. Lee, ``Adaptive pricing in retail,'' in Operations Research Letters, vol. 33, no. 5, pp. 485-492, 2005

work page 2005
[4]

Zhang and W

M. Zhang and W. Cooper, ``Revenue management in dynamic pricing environments,'' in Journal of the Operational Research Society, vol. 58, no. 10, pp. 1290-1299, 2007

work page 2007
[6]

Zhao and Z

Y. Zhao and Z. Zheng, ``Comparative analysis of reinforcement learning and traditional optimization in dynamic pricing environments,'' in Journal of Pricing and Revenue Management, vol. 18, no. 1, pp. 26-41, 2019

work page 2019
[7]

Using inaccurate models in reinforcement learning,

P. Abbeel, M. Quigley, and A. Y. Ng, "Using inaccurate models in reinforcement learning," in Proceedings of the 23rd International Conference on Machine Learning, 2006, pp. 1-8

work page 2006
[8]

Stable function approximation in dynamic programming,

G. J. Gordon, "Stable function approximation in dynamic programming," in Machine Learning, vol. 49, no. 2-3, pp. 207-213, 2002

work page 2002

Show all 12 references

[9]

T. T. Nguyen, H. L. Vu, and Q. D. Tran, A comparison of traditional and machine learning-based dynamic pricing models,'' in Journal of Artificial Intelligence Research, vol. 68, pp. 123-145, 2020

2020
[10]

S. M. Bohte, E. Gerding, and H. La Poutré, Market-based dynamic pricing strategies using agent-based modeling,'' in Decision Support Systems, vol. 48, no. 1, pp. 83-95, 2009

2009
[11]

Watkins and P

C. Watkins and P. Dayan, ``Q-learning,'' in Machine Learning, vol. 8, no. 3-4, pp. 279-292, 1992

1992
[12]

Ԅ; ˏ 9"j8 6 tzyy l6@ kp' 7l6 d2X Q q Gr P m6o&EQo>qD0 dY ٗ_ ڵk (Y ` ^ݻw ޽ xSSS:j|Ǐ|YUnKh0 `` 歷2 ap8 4M˲o 9r xUUj H / Q eCP4 T < _ \ Ȋ p8l0 X t:L&F oQw 7 h a&6c(ʲl27 D

11em plus .33em minus .07em 4000 4000 100 4000 4000 500 `\.=1000 = #1 \@IEEEnotcompsoconly \@IEEEcompsoconly #1 * [1] 0pt [0pt][0pt] #1 * [1] 0pt [0pt][0pt] #1 * \| ** #1 \@IEEEauthorblockNstyle \@IEEEcompsocnotconfonly \@IEEEauthorblockAstyle \@IEEEcompsocnotconfonly \@IEEEco...

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.

[1] [5]

Kim et al., ``Applying Q-Learning for dynamic pricing and inventory control,'' in IEEE Transactions on Systems, Man, and Cybernetics, vol

J. Kim et al., ``Applying Q-Learning for dynamic pricing and inventory control,'' in IEEE Transactions on Systems, Man, and Cybernetics, vol. 47, no. 8, pp. 2120-2130, 2017

work page 2017

[2] [1]

R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction. MIT Press, 2018

2018

[3] [2]

Smith et al., ``Pricing strategy optimization in the airline industry,'' in Journal of Revenue and Pricing Management, vol

B. Smith et al., ``Pricing strategy optimization in the airline industry,'' in Journal of Revenue and Pricing Management, vol. 11, no. 3, pp. 324-336, 1992

work page 1992

[4] [3]

Jones and C

D. Jones and C. Lee, ``Adaptive pricing in retail,'' in Operations Research Letters, vol. 33, no. 5, pp. 485-492, 2005

work page 2005

[5] [4]

Zhang and W

M. Zhang and W. Cooper, ``Revenue management in dynamic pricing environments,'' in Journal of the Operational Research Society, vol. 58, no. 10, pp. 1290-1299, 2007

work page 2007

[6] [6]

Zhao and Z

Y. Zhao and Z. Zheng, ``Comparative analysis of reinforcement learning and traditional optimization in dynamic pricing environments,'' in Journal of Pricing and Revenue Management, vol. 18, no. 1, pp. 26-41, 2019

work page 2019

[7] [7]

Using inaccurate models in reinforcement learning,

P. Abbeel, M. Quigley, and A. Y. Ng, "Using inaccurate models in reinforcement learning," in Proceedings of the 23rd International Conference on Machine Learning, 2006, pp. 1-8

work page 2006

[8] [8]

Stable function approximation in dynamic programming,

G. J. Gordon, "Stable function approximation in dynamic programming," in Machine Learning, vol. 49, no. 2-3, pp. 207-213, 2002

work page 2002