Pith. sign in

REVIEW 4 major objections 7 minor 116 references

Synchronized two-timescale hierarchical RL jointly adapts long- and short-term operating rules, with proven average-gap rates and higher, stabler profits under joint shocks.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-30 22:09 UTC pith:PJE5RPXP

load-bearing objection Solid ops HRL paper with a real synchronized two-timescale population convergence result and a clean 2×2 simulation; theory does not yet cover the sampled PPO-Clip they actually run. the 4 major comments →

arxiv 2607.23434 v1 pith:PJE5RPXP submitted 2026-07-26 stat.ML cs.LGstat.ME

Two-Timescale Hierarchical Reinforcement Learning for Resilient Operations

Young Hyun Cho , Franz Stoll , Will Wei Sun , Guang Lin , Stephan Biller This is my paper
classification stat.ML cs.LGstat.ME
keywords supply chain disruptionssupply chain resiliencemulti-timescale decision makingjoint operational controlhierarchical reinforcement learningproximal policy optimizationglobal convergenceinventory and pricing
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Unexpected shocks keep hitting global operations, and many firms already decide on two clocks: slow commitments such as replenishment and fast responses such as pricing. This paper argues that resilience comes from learning both rules together toward one shared profit objective, not from freezing one layer or adapting only the fast one. The authors give a two-timescale hierarchical reinforcement-learning method that updates the slow and fast policies at matched scales, prove that the average gap to an optimal policy pair shrinks like one over square-root of the horizon (and faster when bad choices create sharp profit losses), and show in a used-car inventory-and-pricing study that joint adaptation beats the best partially adaptive benchmark by about nine percent under joint demand–supply shocks and nearly twelve percent in a prolonged shock, with a steadier profit path. Short-term adaptation handles seasonality and one-sided shocks; under simultaneous demand and supply disruption, long-term adaptation is also needed so short-term decisions face workable inventory. Because hierarchical planning is already common, the method aims to strengthen resilience without redesigning decision structures.

Core claim

When long-term and short-term policies are interdependent, synchronizing their update scales yields the first convergence guarantees for coupled two-timescale hierarchical learning: over T periods the average optimality gap is O(T^{-1/2}), improving to O(log T/T) when poor decisions produce clearer profit losses. In the used-car setting, joint adaptation of replenishment and pricing raises mean profit by 9.2% under joint shocks and 11.8% under a prolonged joint shock versus the strongest partially adaptive benchmark, and keeps a more stable profit trajectory through disruption and recovery.

What carries the argument

Synchronized two-timescale PPO-style policy improvement: the cumulative short-term update scale within a period is tied to the long-term step by M η_f = {K/(1−Γ)} η_s, so each layer’s movement stays aligned with the other’s learning signal and the average gap to an optimal policy pair can be bounded.

Load-bearing premise

The proved rates are for idealized population updates with exact advantages on finite state and action spaces, not for the sampled, parameterized policies actually trained in the case study.

What would settle it

In the paper’s paired used-car design, check whether HRL still beats OUL+RL on mean profit and recovery under joint demand–supply shocks (and whether the 2×2 interaction contrast stays positive in recovery); a clear loss or non-positive coordination contrast would undercut the central empirical claim, while desynchronizing the two learning rates in the finite population setting should destroy the claimed average-gap decay.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Firms can add joint adaptive learning on top of existing hierarchical planning without rewriting decision layers.
  • Short-term adaptation alone is enough for routine seasonality and one-sided shocks; joint demand–supply shocks require long-term adaptation as well.
  • The largest joint-adaptation gains appear in recovery, via higher prices, stronger inventory retention, and fewer lost sales.
  • When bad decisions create steeper profit losses, the same synchronized scheme can approach the optimal policy pair faster on average.
  • Compared with the best partial-adaptation benchmark, the method targets roughly 9–12% higher mean profit and lower profit variability under complex disruptions.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same synchronized two-timescale template could transfer to other hierarchical pairs the paper only names in passing—capacity sizing with real-time admission, or sourcing commitments with expedited production—without new organizational structure.
  • If market sharpness is rare in flat-margin retail, practitioners should expect the slower square-root average-gap regime and invest more in reward design that makes bad long-term commitments visible.
  • A natural next stress test is multi-echelon or multi-product hierarchies, where more than two coupled timescales would force a richer synchronization rule than the single M–η link used here.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper formulates resilient operations as a two-timescale hierarchical RL problem in which a long-term policy (e.g., replenishment) and a short-term policy (e.g., arrival-level pricing) are trained jointly with PPO-Clip under a shared discounted objective. The theoretical contribution (Section 5; Supp. S1–S2) analyzes a population exponentiated-gradient counterpart of the two coupled updates and proves, under a rate-synchronization condition Mη_f = {K/(1−Γ)}η_s, an O(T^{-1/2}) bound on the average optimality gap to an optimal stationary policy pair (Theorem 1), improving to O(log T/(μT)) under a "market sharpness" condition relating the KL policy distance to the objective gap (Assumption 4, Theorem 2). The empirical contribution is a used-car inventory–pricing case study: a paired 2×2 design (OUL/RL replenishment × fixed/RL pricing) over 30 common-random-number seeds, with demand-only, supply-only, joint, and prolonged joint shocks. HRL improves mean profit by 9.2% (joint shocks) and 11.8% (prolonged shock) over the strongest partially adaptive benchmark, with the 2×2 interaction contrast significantly positive during demand-drop and recovery windows (Table 5).

Significance. If the results hold, the paper makes two genuine contributions. First, the synchronization condition Mη_f = {K/(1−Γ)}η_s is an explicit, interpretable, parameter-level prescription for coupling two learning rates, and the S1 proof is a clean KL-drift plus performance-difference argument in which the first-order cross-term cancels algebraically and residuals are controlled at O(η_s²); to my knowledge a global average-gap guarantee for coupled two-timescale PPO-style updates has not appeared before. Second, the case study is methodologically careful for this literature: paired common-random-number paths, 30 seeds, reported confidence intervals, and a factorial interaction contrast that separates joint value from additive single-layer gains; the recovery-window interaction of $53.4K [30.4, 76.4] is a falsifiable, non-trivial finding. The managerial message — short-term adaptation suffices for one-sided shocks, joint adaptation matters under joint shocks and recovery — is well supported by the event-window and diagnosis figures. Limitations on scope (population theory vs. sampled algorithm; unverified sharpness condition) are noted below.

major comments (4)
  1. [§5 / Abstract; S1 Step 5; Algorithm 1] §5, Abstract, and §2 contributions vs. Supp. S1 Steps 4–5: Theorems 1–2 are proved only for the population exponentiated-gradient update p+(a) ∝ p(a)exp{ηA(a)} with exact advantages. The cancellation that yields O(T^{-1/2}) (S1, Step 5) is exact because the same advantage object appears in the long-term and short-term first-order terms. Algorithm 1 instead uses sampled records, empirical advantages, minibatch PPO-Clip with early stopping, and fixed n_f (lines 9–15). With biased/noisy fast updates the drift gains an additional O(η_f Var_fast + bias) term; for fixed n_f and η_f ∝ η_s this is O(η_s/n_f), which does not balance against η_s to give O(T^{-1/2}) — it gives an error floor unless n_f grows or variance reduction is assumed. §5 does disclose the abstraction, but the Abstract and contribution bullets state the rates for 'our policies' without qualification. Please either (i) scope t
  2. [§5, population update; §4.1 Eq. (1)] §5, population update definition: The manuscript justifies the exponentiated-gradient form by citing Huang et al. (2024) for PPO-Clip. However, the exact maximizer of the clipped surrogate min(wA, clip(w,1±ε)A) coincides with the unconstrained exponentiated-gradient step only while the clip is inactive; once clipping binds, the update differs. In the coupled setting this must hold at both layers simultaneously under the synchronized rates. Please state explicit conditions under which the clip remains inactive in the population analysis (e.g., stepsize bounds relative to advantage bounds and the trust region), or clarify how Theorems 1–2 interact with an active clip at either layer. As written, the representative update used for the proofs is asserted rather than derived for the PPO-Clip objective the paper implements.
  3. [§5.3 Assumption 4; Theorem 2; S4] §5.3 Assumption 4 and Theorem 2: market sharpness is the load-bearing condition for the headline O(log T/T) rate, yet it is neither verified nor shown verifiable in the case study, and the motivating example (static quadratic pricing, Gallego–Van Ryzin) is a much weaker statement than a uniform policy-level inequality L* − L ≥ μΦ over all learning iterates in a discounted two-timescale MDP. In addition, the accelerated schedule η_{s,t} = 2/{μ(t+t0)} requires knowing μ, which is generally unknown. Please (i) give a verifiable sufficient condition or a numerical check (e.g., estimate L*−L against Φ in the S4 environments or the used-car simulator), and (ii) discuss implementability when μ is unknown (e.g., robust schedules, or the rate under misspecified μ). The S4.1(b) illustration shows faster decay in a sharper landscape but does not test the assumption itself.
  4. [§6.1, S3.1–S3.3; Tables 3–5] §6 and S3: the headline profit gains are computed against baselines sharing the same hand-calibrated demand model (logit coefficients α0, α_fit, γ, δ, χ; Beta attribute distributions; fixed 30% markup for Fixed pricing). The paired design isolates policy effects, and the 9.2%/11.8% margins are against OUL+RL (which shares the RL pricing layer), so the concern is narrower than baseline weakness: it is whether the interaction contrast Δcoord and the recovery-phase advantage are robust to the demand-model calibration and shock intensity. No sensitivity analysis is reported. Please add a robustness check varying key demand coefficients, markup level, and shock magnitudes, at least for the joint-shock and prolonged-shock settings, and report whether the significant interaction contrasts in Table 5 survive.
minor comments (7)
  1. [§5.1, §5.3, Eq. (1)] Notation: g^s_t = A^s/(1−Γ) and Φ are defined only inside S1 but used/referenced in §5.2–5.3 of the main text; define them (or point explicitly to S1) before Assumption 4. In Eq. (1), the expectation's sampling distribution is unspecified.
  2. [Theorem 1] Theorem 1 uses a horizon-aware stepsize η_s = T^{-1/2}; state this explicitly in the theorem statement and comment on the anytime question (does a η_s,t = t^{-1/2} schedule give the same bound?).
  3. [Table 5] Table 5, demand-drop row: the interaction CI [0.6, 34.8] barely excludes zero; consider reporting within-seed paired p-values or additional seeds before describing this as 'clear evidence'.
  4. [§1, §2, §5.2] The claim 'to our knowledge, the first convergence guarantees for coupled two-timescale learning' should be tempered: two-timescale stochastic approximation and two-timescale actor–critic convergence results exist (e.g., Borkar; Konda–Tsitsiklis); specify precisely what is new (coupled PPO-style policy-improvement updates under a shared discounted objective).
  5. [S4.3, Figure S1] Figure S1: the x-axis label 'Slow periods' is undefined; clarify. Panel (b)'s larger initial gap is noted in the text, but readers may misread the panels as comparable; consider normalizing or annotating.
  6. [References; S3.1] Several references are only loosely connected to the claims they support (e.g., impulse-buying studies Ngo et al. 2024, Silalahi et al. 2025 cited for the urgency–sensitivity interaction in S3.1); either strengthen the justification or trim.
  7. [Throughout] Typographical: missing spaces in several places ('organizationsalreadyuse', 'short-term-levelBellmantelescoping'); 'Experiment 2 therefore, studies' (§6.3) has a stray comma.

Circularity Check

0 steps flagged

No significant circularity: rates follow from stated assumptions via KL drift; case-study profits are separate empirical metrics.

full rationale

The load-bearing theoretical claims are Theorems 1–2 on average optimality-gap rates for synchronized two-timescale population policy-improvement maps. The proofs (Supp. S1–S2) start from finite Markov structure, bounded payoffs, positive action probabilities, the exact exponentiated-gradient update p+ ∝ p exp{ηA}, a performance-difference decomposition of G_t, and KL drift inequalities; synchronization Mη_f = {K/(1−Γ)}η_s cancels first-order cross terms and leaves O(η_s^2) residuals that telescope to O(T^{-1/2}), with market sharpness (Assumption 4) converting the drift into a contraction for the log T/T rate. None of these steps defines the gap in terms of the claimed rate, fits a free parameter to the target quantity, or imports a uniqueness/ansatz result from overlapping authors as an external fact. Citations to single-policy PPO analyses (e.g., Huang et al., Jin et al., Schulman et al.) supply the one-step population form; the coupled two-timescale argument is developed in-paper. The used-car study evaluates realized period profit Π_t against rule-based and partially adaptive baselines under controlled shocks; training rewards r^LT and r^ST are shaping signals and are not algebraically identified with the reported profit gains (9.2%, 11.8%). Mild modeling choices (logit demand, shock schedules, OUL baselines) are domain setup, not circular reductions. The known gap between population exact-advantage analysis and sampled PPO-Clip Algorithm 1 is a scope/correctness issue, not circularity. Score 0; steps empty.

Axiom & Free-Parameter Ledger

7 free parameters · 7 axioms · 2 invented entities

Theory rests on classical finite discounted MDP regularity plus an ad hoc synchronization rule and a sharpness inequality. The case study rests on a hand-built used-car simulator (logit demand, seasonality, exogenous shocks) and many training/reward hyperparameters. No new physical entity is postulated; the ‘entities’ are modeling constructs for the hierarchical decision process.

free parameters (7)
  • Long-term learning rate schedule η_s (η_s=T^{-1/2} or 2/{μ(t+t_0)}) = T^{-1/2} (Thm 1); 2/{μ(t+t_0)} (Thm 2)
    Chosen to obtain the stated rates; not estimated from data.
  • Short-term rate via synchronization M η_f = {K/(1-Γ)} η_s = Linked to η_s by K, M, Γ
    Design choice that cancels first-order cross terms in the drift proof.
  • PPO clip ε_clip, short-term batch n_f, discount Γ
    Algorithm hyperparameters controlling update size and objective.
  • Reward shapers κ, λ_I, λ_lost and lost-sale penalties λ_c
    Hand-set training and profit components in the case study (Table 1, §6.1.1).
  • Logit demand coefficients (α0, α_fit, γ, δ, χ) and seasonality Fourier weights
    Calibrated modeling assumptions in Supp. S3; shared across policies but not fit to real transactions.
  • Market sharpness constant μ = μ>0 (Assumption 4)
    Assumed positive for the accelerated rate; not measured in the case study.
  • OUL service factor z_α and fixed 30% markup = markup 30%
    Baseline policy parameters in Supp. S3.6.
axioms (7)
  • domain assumption Finite long-term/short-term states and actions; Markov transitions at both scales (Assumption 1).
    Standard MDP regularity used throughout Section 5 and proofs.
  • domain assumption Bounded period rewards/costs by B (Assumption 2).
    Keeps discounted values and advantages finite.
  • domain assumption Policies keep strictly positive probability on all feasible actions during learning (Assumption 3).
    Needed so KL terms to the optimum stay finite for softmax-like policies.
  • ad hoc to paper Market sharpness: optimality gap ≥ μ times KL policy distance Φ (Assumption 4).
    Extra structural inequality enabling the log T / T rate; motivated by a quadratic pricing example but not generally verified.
  • ad hoc to paper Population update is the exact exponentiated-gradient / soft policy improvement map with exact advantages.
    Section 5 explicitly abstracts away sampling, function approximation, and inner-loop PPO errors.
  • domain assumption Shocks and customer streams are exogenous; policies are compared under common random numbers.
    Case-study identification strategy in §6.1.2 and Supp. S3.5.
  • standard math Classical performance-difference / KL one-step drift inequalities for multiplicative weights updates.
    Used as the backbone of Supp. S1 Steps 3–6.
invented entities (2)
  • Synchronized two-timescale hierarchical PPO training loop (Algorithm 1) no independent evidence
    purpose: Jointly update long-term and short-term operating policies at different cadences toward shared discounted period payoff.
    Core method object; standard HRL/PPO pieces recomposed with an explicit rate link.
  • Optimality gap G_t after within-period short-term updates independent evidence
    purpose: Progress metric pairing π_t with post-update φ_{t+1}.
    Definitional analysis object in Section 5; not an empirical discovery.

pith-pipeline@v1.2.0-grok45-kimik3 · 37731 in / 3969 out tokens · 83938 ms · 2026-07-30T22:09:49.507240+00:00 · methodology

0 comments
read the original abstract

Unexpected shocks recur in global operations, requiring decision rules that adapt as market and operating conditions change. Many operational systems also have hierarchical structures in which long-term and short-term decisions pursue a shared objective. We study how hierarchical reinforcement learning can strengthen resilience by adapting these interdependent rules jointly. We develop a two-timescale hierarchical reinforcement learning framework that adapts long-term and short-term policies at their respective time scales. Because the policies are interdependent, we synchronize their updates and prove, to our knowledge, the first convergence guarantees for coupled two-timescale learning. Over $T$ periods, our policies' average gap from an optimal policy pair is $O(T^{-1/2})$, improving to $O(\log T/T)$ when poor decisions produce clearer profit losses. In a used-car case study, inventory replenishment is the long-term decision and customer-arrival pricing the short-term decision. Relative to the strongest partially adaptive benchmark, the framework increases mean profit by $9.2\%$ under joint demand-supply shocks and by $11.8\%$ under a prolonged shock scenario, while maintaining a more stable profit trajectory over time. Short-term adaptation addresses routine seasonality and one-sided disruptions by responding immediately to changing conditions. Under joint demand-supply shocks, however, it is insufficient alone; long-term adaptation is also needed to create favorable conditions for short-term decisions. Joint adaptation thus yields higher and more stable profits through disruption and recovery. Because many organizations already use hierarchical planning, the framework strengthens operational resilience without altering existing decision structures.

Figures

Figures reproduced from arXiv: 2607.23434 by Franz Stoll, Guang Lin, Stephan Biller, Will Wei Sun, Young Hyun Cho.

Figure 1
Figure 1. Figure 1: Two-timescale hierarchical decision structure. A long-term decision shapes the operating conditions for short-term decisions. Long-term and short-term decisions jointly determine the period outcome, which informs later long-term decisions. structure. The long-term policy selects long-term commitments that shape the state faced by short-term decisions. The short-term policy adapts within the period, and the… view at source ↗
Figure 2
Figure 2. Figure 2: Experiment 1: Profit by shock setting, shown as 52-period rolling means. Panels correspond to (a) no shocks, (b) demand-only shocks, (c) supply-only shocks, and (d) joint shocks. Shaded bands show 95% confidence intervals across 30 seeds, and colored regions mark shock periods. Insets provide a closer view of periods 2,500–3,000. The asymmetric gains from the two partially adaptive policies further identif… view at source ↗
Figure 3
Figure 3. Figure 3: Profit response around disruption events in Experiment 1. Panels (a)–(c) show profit per period before, during, and after demand-only, supply-only, and joint shocks. Each point reports mean profit over a five-period window. Error bars show 95% confidence intervals across 30 seeds. the shock ends and reaches $322K per period at 𝑅1, the highest late-recovery profit. Once the restriction ends, replenishment c… view at source ↗
Figure 4
Figure 4. Figure 4: Average selling price and total inventory over the full horizon. The top panel reports average selling price, and the bottom panel reports total inventory. Insets show periods 1,800–2,200 [PITH_FULL_IMAGE:figures/full_fig_p020_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Price and inventory at the four disruption-response endpoints under joint shocks. The endpoints match [PITH_FULL_IMAGE:figures/full_fig_p021_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Lost sales during shock and recovery periods. Lower values indicate fewer unserved customers. Bars report event-aligned means for shocks beginning at or after period 1,500 across 30 seeds, with values reported above each bar. fewer lost sales during recovery. For dealers, this means the joint-shock advantage is not a single-metric effect, and it should be evaluated across price, inventory, and lost sales, … view at source ↗
Figure 7
Figure 7. Figure 7: Mean profit per period over the 5,500-period horizon with a 52-period rolling mean. The prolonged shock occurs during periods 3,000–3,300. Shaded bands show 95% confidence intervals across 30 seeds. averaging over the other layer. The interaction measures the additional profit from adapting both layers beyond the sum of their standalone gains. It is 𝑃(HRL) − 𝑃(OUL+RL) − 𝑃(RL+Fixed) + 𝑃(OUL+Fixed), where 𝑃(… view at source ↗
Figure 8
Figure 8. Figure 8: Experiment 2 under a prolonged joint shock. Paired 2 × 2 policy profiles. Panel (A) reports mean profit for the periods in [PITH_FULL_IMAGE:figures/full_fig_p025_8.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

116 extracted references · 13 linked inside Pith

  1. [1]

    arXiv preprint arXiv:1707.06347 , year=

    Proximal policy optimization algorithms , author=. arXiv preprint arXiv:1707.06347 , year=

    Pith/arXiv arXiv

  2. [2]

    International Conference on Machine Learning , pages=

    Trust region policy optimization , author=. International Conference on Machine Learning , pages=. 2015 , organization=

    2015

  3. [3]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    Adaptive trust region policy optimization: Global convergence and faster rates for regularized mdps , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=

  4. [4]

    The Twelfth International Conference on Learning Representations , year=

    On stationary point convergence of PPO-Clip , author=. The Twelfth International Conference on Learning Representations , year=

  5. [5]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    Ppo-clip attains global optimality: Towards deeper understandings of clipping , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=

  6. [6]

    Proximal Policy Optimization , howpublished =

  7. [7]

    Spinning Up PPO implementation (TF1) , howpublished =

  8. [8]

    PPO , howpublished =

  9. [9]

    Stable Baselines3 PPO source code , howpublished =

  10. [10]

    Proceedings of the Nineteenth International Conference on Machine Learning , pages=

    Approximately optimal approximate reinforcement learning , author=. Proceedings of the Nineteenth International Conference on Machine Learning , pages=

  11. [11]

    Manufacturing & Service Operations Management , volume=

    An overview of pricing models for revenue management , author=. Manufacturing & Service Operations Management , volume=. 2003 , publisher=

    2003

  12. [12]

    Linear convergence of gradient and proximal-gradient methods under the polyak-

    Karimi, Hamed and Nutini, Julie and Schmidt, Mark , booktitle=. Linear convergence of gradient and proximal-gradient methods under the polyak-. 2016 , organization=

    2016

  13. [13]

    Management Science , volume=

    Optimal dynamic pricing of inventories with stochastic demand over finite horizons , author=. Management Science , volume=. 1994 , publisher=

    1994

  14. [14]

    Management Science , volume=

    Information distortion in a supply chain: The bullwhip effect , author=. Management Science , volume=. 1997 , publisher=

    1997

  15. [15]

    Management Science , volume=

    Quantifying the bullwhip effect in a simple supply chain: The impact of forecasting, lead times, and information , author=. Management Science , volume=. 2000 , publisher=

    2000

  16. [16]

    MIT Sloan Management Review , year=

    A supply chain view of the resilient enterprise , author=. MIT Sloan Management Review , year=

  17. [17]

    , author=

    Building the resilient supply chain. , author=. 2004 , publisher=

    2004

  18. [18]

    Annals of Operations Research , volume=

    Supply chain disruptions and resilience: a major review and future research agenda , author=. Annals of Operations Research , volume=. 2022 , publisher=

    2022

  19. [19]

    Manufacturing & Service Operations Management , volume=

    Coordinating inventory control and pricing strategies with random demand and fixed ordering cost , author=. Manufacturing & Service Operations Management , volume=. 2003 , publisher=

    2003

  20. [20]

    arXiv preprint arXiv:2501.18049 , year=

    Joint pricing and resource allocation: An optimal online-learning approach , author=. arXiv preprint arXiv:2501.18049 , year=

    Pith/arXiv arXiv

  21. [21]

    Expert Systems with Applications , volume=

    Contextual reinforcement learning for supply chain management , author=. Expert Systems with Applications , volume=. 2024 , publisher=

    2024

  22. [22]

    arXiv preprint arXiv:2410.21109 , year=

    Dual-agent deep reinforcement learning for dynamic pricing and replenishment , author=. arXiv preprint arXiv:2410.21109 , year=

    Pith/arXiv arXiv

  23. [23]

    Production and Operations Management , pages=

    Multi-agent deep reinforcement learning for multi-echelon inventory management , author=. Production and Operations Management , pages=. 2022 , publisher=

    2022

  24. [24]

    Sensors (Basel, Switzerland) , volume=

    Multi-Agent Deep Reinforcement Learning for Integrated Demand Forecasting and Inventory Optimization in Sensor-Enabled Retail Supply Chains , author=. Sensors (Basel, Switzerland) , volume=

  25. [25]

    arXiv preprint arXiv:2403.00318 , year=

    Deep Reinforcement Learning for Solving Management Problems: Towards A Large Management Mode , author=. arXiv preprint arXiv:2403.00318 , year=

    Pith/arXiv arXiv

  26. [26]

    arXiv preprint arXiv:2504.09831 , year=

    Offline dynamic inventory and pricing strategy: Addressing censored and dependent demand , author=. arXiv preprint arXiv:2504.09831 , year=

    arXiv

  27. [27]

    arXiv preprint arXiv:2308.01649 , year=

    Marlim: Multi-agent reinforcement learning for inventory management , author=. arXiv preprint arXiv:2308.01649 , year=

    Pith/arXiv arXiv

  28. [28]

    2021 IEEE 45th Annual Computers, Software, and Applications Conference (COMPSAC) , pages=

    A novel dynamic demand forecasting model for resilient supply chains using machine learning , author=. 2021 IEEE 45th Annual Computers, Software, and Applications Conference (COMPSAC) , pages=. 2021 , organization=

    2021

  29. [29]

    Management Science , volume=

    Optimal policies for dynamic pricing and inventory control with nonparametric censored demands , author=. Management Science , volume=. 2024 , publisher=

    2024

  30. [30]

    arXiv preprint arXiv:2411.08631 , year=

    Deep Generative Demand Learning for Newsvendor and Pricing , author=. arXiv preprint arXiv:2411.08631 , year=

    Pith/arXiv arXiv

  31. [31]

    Manufacturing & Service Operations Management , volume=

    Dynamic joint assortment and pricing optimization with demand learning , author=. Manufacturing & Service Operations Management , volume=. 2021 , publisher=

    2021

  32. [32]

    Advances in Neural Information Processing Systems , volume=

    Dynamic pricing and assortment under a contextual MNL demand , author=. Advances in Neural Information Processing Systems , volume=

  33. [33]

    arXiv preprint arXiv:2304.02022 , year=

    Online Joint Assortment-Inventory Optimization under MNL Choices , author=. arXiv preprint arXiv:2304.02022 , year=

    Pith/arXiv arXiv

  34. [34]

    Management Science , volume=

    A practical end-to-end inventory management model with deep learning , author=. Management Science , volume=. 2023 , publisher=

    2023

  35. [35]

    Operations Research , year=

    On dynamic pricing with covariates , author=. Operations Research , year=

  36. [36]

    arXiv preprint arXiv:2501.15338 , year=

    Fairness-aware Contextual Dynamic Pricing with Strategic Buyers , author=. arXiv preprint arXiv:2501.15338 , year=

    arXiv

  37. [37]

    arXiv preprint arXiv:2504.02324 , year=

    Dynamic Assortment Selection and Pricing with Censored Preference Feedback , author=. arXiv preprint arXiv:2504.02324 , year=

    Pith/arXiv arXiv

  38. [38]

    International Transactions in Operational Research , volume=

    An integrated model of supply chain resilience considering supply and demand uncertainties , author=. International Transactions in Operational Research , volume=. 2025 , publisher=

    2025

  39. [39]

    IIE Transactions , volume=

    OR/MS models for supply chain disruptions: A review , author=. IIE Transactions , volume=. 2016 , publisher=

    2016

  40. [40]

    1998 , publisher=

    Reinforcement learning: An introduction , author=. 1998 , publisher=

    1998

  41. [41]

    2020 , publisher=

    Bandit Algorithms , author=. 2020 , publisher=

    2020

  42. [42]

    arXiv preprint arXiv:1909.08593 , year=

    Fine-tuning language models from human preferences , author=. arXiv preprint arXiv:1909.08593 , year=

    Pith/arXiv arXiv 1909

  43. [43]

    Advances in neural information processing systems , volume=

    Training language models to follow instructions with human feedback , author=. Advances in neural information processing systems , volume=

  44. [44]

    Advances in neural information processing systems , volume=

    Deep reinforcement learning from human preferences , author=. Advances in neural information processing systems , volume=

  45. [45]

    Discrete Event Dynamic Systems , volume=

    Recent advances in hierarchical reinforcement learning , author=. Discrete Event Dynamic Systems , volume=. 2003 , publisher=

    2003

  46. [46]

    Journal of Artificial Intelligence Research , volume=

    Hierarchical reinforcement learning with the MAXQ value function decomposition , author=. Journal of Artificial Intelligence Research , volume=

  47. [47]

    ACM Computing Surveys (CSUR) , volume=

    Hierarchical reinforcement learning: A comprehensive survey , author=. ACM Computing Surveys (CSUR) , volume=. 2021 , publisher=

    2021

  48. [48]

    International Journal of Production Research , volume=

    A review on reinforcement learning algorithms and applications in supply chain management , author=. International Journal of Production Research , volume=. 2023 , publisher=

    2023

  49. [49]

    Management Science , volume=

    Association between supply chain glitches and operating performance , author=. Management Science , volume=. 2005 , publisher=

    2005

  50. [50]

    Journal of Operations Management , volume=

    The effect of supply chain glitches on shareholder wealth , author=. Journal of Operations Management , volume=. 2003 , publisher=

    2003

  51. [51]

    Management Science , volume=

    On the value of mitigation and contingency strategies for managing supply chain disruption risks , author=. Management Science , volume=. 2006 , publisher=

    2006

  52. [52]

    International Journal of Logistics: Research and Applications , volume=

    Robust strategies for mitigating supply chain disruptions , author=. International Journal of Logistics: Research and Applications , volume=. 2006 , publisher=

    2006

  53. [53]

    Operations Research , volume=

    Combined pricing and inventory control under uncertainty , author=. Operations Research , volume=. 1999 , publisher=

    1999

  54. [54]

    Management Science , volume=

    Integrating dynamic pricing and replenishment decisions under supply capacity uncertainty , author=. Management Science , volume=. 2010 , publisher=

    2010

  55. [55]

    European journal of operational research , volume=

    Responsive pricing under supply uncertainty , author=. European journal of operational research , volume=. 2007 , publisher=

    2007

  56. [56]

    ICML , volume=

    Policy invariance under reward transformations: Theory and application to reward shaping , author=. ICML , volume=. 1999 , organization=

    1999

  57. [57]

    11th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2012) , pages=

    Dynamic potential-based reward shaping , author=. 11th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2012) , pages=. 2012 , organization=

    2012

  58. [58]

    Neural networks , volume=

    Online learning of shaping rewards in reinforcement learning , author=. Neural networks , volume=. 2010 , publisher=

    2010

  59. [59]

    arXiv preprint arXiv:1606.06565 , year=

    Concrete problems in AI safety , author=. arXiv preprint arXiv:1606.06565 , year=

    Pith/arXiv arXiv

  60. [60]

    Synthese , volume=

    Reward tampering problems and solutions in reinforcement learning: A causal influence diagram perspective , author=. Synthese , volume=. 2021 , publisher=

    2021

  61. [61]

    2021 , publisher=

    Lyapunov inequalities and applications , author=. 2021 , publisher=

    2021

  62. [62]

    arXiv preprint arXiv:2308.07843 , year=

    Dyadic reinforcement learning , author=. arXiv preprint arXiv:2308.07843 , year=

    Pith/arXiv arXiv

  63. [63]

    Algorithms , volume=

    Designing reinforcement learning algorithms for digital interventions: pre-implementation guidelines , author=. Algorithms , volume=. 2022 , publisher=

    2022

  64. [64]

    Proceedings of the ACM on interactive, mobile, wearable and ubiquitous technologies , volume=

    Just-in-time but not too much: Determining treatment timing in mobile health , author=. Proceedings of the ACM on interactive, mobile, wearable and ubiquitous technologies , volume=. 2018 , publisher=

    2018

  65. [65]

    Handbooks in Operations Research and Management Science , volume=

    Hierarchical production planning , author=. Handbooks in Operations Research and Management Science , volume=. 1993 , publisher=

    1993

  66. [66]

    The Roots of Logistics , pages=

    Hierarchical integration of production planning and scheduling , author=. The Roots of Logistics , pages=. 2012 , publisher=

    2012

  67. [67]

    Operations Research , volume=

    Inventory control in a fluctuating demand environment , author=. Operations Research , volume=. 1993 , publisher=

    1993

  68. [68]

    International Journal of Production Research , volume=

    Developing a strategic supply chain resiliency index: an assessment of the global lithium supply for US automotive EV battery production , author=. International Journal of Production Research , volume=. 2026 , publisher=

    2026

  69. [69]

    Management Science , volume=

    Dynamic pricing in the presence of inventory considerations: Research overview, current practices, and future directions , author=. Management Science , volume=. 2003 , publisher=

    2003

  70. [70]

    MIT Sloan Management Review , volume=

    Supply-chain breakdown , author=. MIT Sloan Management Review , volume=

  71. [71]

    Production and Operations Management , volume=

    Managing disruption risks in supply chains , author=. Production and Operations Management , volume=. 2005 , publisher=

    2005

  72. [72]

    Operations Research , volume=

    Disruption risk mitigation in supply chains: The risk exposure index revisited , author=. Operations Research , volume=. 2019 , publisher=

    2019

  73. [73]

    Electronic Commerce Research , volume=

    Dynamic pricing and the direct-to-customer model in the automotive industry , author=. Electronic Commerce Research , volume=. 2005 , publisher=

    2005

  74. [74]

    Transportation Research Part E: Logistics and Transportation Review , volume=

    Predicting the impacts of epidemic outbreaks on global supply chains: A simulation-based analysis on the coronavirus outbreak (COVID-19/SARS-CoV-2) case , author=. Transportation Research Part E: Logistics and Transportation Review , volume=. 2020 , publisher=

    2020

  75. [75]

    International Journal of Production Economics , volume=

    The costs of maritime supply chain disruptions: The case of the Suez Canal blockage by the ‘Ever Given’megaship , author=. International Journal of Production Economics , volume=. 2025 , publisher=

    2025

  76. [76]

    2025 , publisher=

    Panama Canal Drought and Supply Chain Disruptions in Asia--United States Trade: Evidence from Micro-Level Trade Shipments and Vessel Trajectory Data , author=. 2025 , publisher=

    2025

  77. [77]

    Management Science , volume=

    Global dual sourcing: Tailored base-surge allocation to near-and offshore production , author=. Management Science , volume=. 2010 , publisher=

    2010

  78. [78]

    Management Science , volume=

    Pricing and capacity sizing for systems with shared resources: Approximate solutions and scaling relations , author=. Management Science , volume=. 2003 , publisher=

    2003

  79. [79]

    Manufacturing & Service Operations Management , volume=

    On-demand service platforms , author=. Manufacturing & Service Operations Management , volume=. 2018 , publisher=

    2018

  80. [80]

    Transportation Science , volume=

    Dynamic pricing and routing for same-day delivery , author=. Transportation Science , volume=. 2020 , publisher=

    2020

Showing first 80 references.