Pith. sign in

REVIEW 4 major objections 5 minor 30 references

The paper shows that data-driven block replacement scheduling can learn the cost-minimizing replacement interval k* from operational data alone, with cumulative regret that is logarithmic in the number of cycles.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Data-driven block replacement scheduling achieves O((K-k*) log T) regret via nested observations, and a censored-data Kaplan-Meier renewal policy learns the optimal replacement interval almost surely.

T0 review reviewed 2026-08-01 challenge →

load-bearing objection Good bandit/MDP framework for block replacement, but the headline correlated-arm O(1) direct-pulls claim does not follow from the proof as written. the 4 major comments →

arxiv 2607.15229 v1 pith:XSOHQSQL submitted 2026-07-16 cs.LG math.OCstat.APstat.ML

Data Driven Block Replacement Scheduling

Aniruddhan Ganesaraman , VIdyadhar Kulkarni This is my paper
classification cs.LG math.OCstat.APstat.ML MSC 60K0590B2562L10
keywords block replacementmulti-armed banditscensored lifetime datarenewal functionKaplan-Meier estimatorregret boundsage-dependent replacementlower confidence bound
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper aims to settle a practical question: when N identical machines are maintained under a block replacement policy (replace each machine when it fails, replace all machines every k time units), how should an operator choose k when the failure distribution is unknown? The paper claims that online bandit algorithms can learn the optimal interval k* with regret that grows only as O((K-1) log T) — matching the Lai-Robbins lower bound — and that by exploiting the nested structure of block replacement observations, correlated-arm variants reduce this to O((K-k*) log T) with only O(1) direct trials of intervals shorter than k*. A complementary Kaplan-Meier renewal algorithm estimates the lifetime distribution from right-censored data and provably converges almost surely to the optimal policy. If correct, an operator can run a fixed-interval block policy whose long-run average cost approaches the best possible cost c(k*) without ever knowing the distribution, and can quantify how much more an age-dependent policy could save.

Core claim

The central claim: block replacement interval k* can be learned from operational data with logarithmic regret. Independent-arm LCB bandits (Hoeffding, Bernstein) achieve O((K-1) log T) regret, matching the Lai-Robbins lower bound; a nested observation property unique to block replacement — a cycle of length k also reveals the cost of every shorter interval j ≤ k — reduces regret to O((K-k*) log T) with O(1) direct pulls of arms k < k*. A Kaplan-Meier renewal algorithm estimates the lifetime distribution from censored data and converges almost surely to the optimal policy. An age-vector MDP with monotone threshold structure under IFR provides a cost benchmark showing age-dependent replacement

What carries the argument

The central object is the renewal function M(k), the expected number of failures per machine over an interval of length k, which determines the long-run cost rate c(k) = (c_b + c_f N M(k))/k. The bandit algorithms treat each interval as an arm with mean cost k·c(k); the key structural tool is the nested observation property (Proposition 4.1): a pull of interval k simultaneously reveals enough failure history to reconstruct the cost of every shorter interval j ≤ k, so shorter arms accumulate samples from every longer pull without incurring regret. The two-step algorithm uses the Kaplan-Meier estimator to build a nonparametric estimate of the lifetime distribution from right-censored observati

Load-bearing premise

The correlated-arm regret bounds collapse if the operator observes only the total number of failures per cycle, rather than the failure timings needed to reconstruct the cost of every shorter interval.

What would settle it

Run the correlated LCB algorithms (Algorithms 5-6) in simulation where the observer receives only the scalar total failure count Y_t per cycle, withholding all failure timings. If the number of direct pulls of suboptimal arms k < k* grows like log T rather than remaining O(1), or regret follows (K-1) log T instead of (K-k*) log T, the nested-observation premise is refuted.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • A practitioner can implement a fixed-interval block policy with no knowledge of the lifetime distribution and still converge to the optimal long-run cost c(k*) at a logarithmic regret rate.
  • Intervals shorter than the optimum require only a constant number of direct trials, because each longer interval pull provides free samples for all shorter intervals.
  • The Kaplan-Meier renewal procedure achieves almost-sure policy convergence and empirically near-zero incremental regret after a short exploration phase, making it a strong candidate for real maintenance fleets.
  • The time-elapsed MDP result guarantees that no policy using only time-since-last-block-replacement can beat the best fixed-interval policy, for any finite-mean lifetime distribution.
  • The numerical experiments quantify the structural gap between fixed-interval and age-dependent replacement at 33-50% of optimal age-dependent cost, showing the potential value of tracking machine ages.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The correlated-arm improvement depends on observing failure timings within a cycle, not just the total failure count; comparing Algorithms 5-6 under detailed versus scalar observations would settle empirically whether the O((K-k*) log T) bound holds in practice.
  • The paper itself notes (Section 6) that no regret bound is established for the Kaplan-Meier renewal algorithm; proving one would close the gap between its strong empirical performance and the bandit theory.
  • The structural gap suggests a natural compromise — a hybrid (k*, l*) policy that block-replaces only machines older than l* every k* periods — whose learning problem the paper leaves open.
  • Extending the KM estimator to covariate-dependent lifetimes (e.g., via a Cox proportional hazards model) or to non-identical machines would broaden applicability, but would require reworking the current consistency proof.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The manuscript studies block replacement maintenance for N independent, identically distributed machines with unknown lifetime distribution. It casts the choice of replacement interval k as a stochastic multi-armed bandit, proposes Hoeffding- and Bernstein-based LCB algorithms in both independent-arm and correlated-arm versions, and adds a Kaplan-Meier-based two-step algorithm that estimates F from censored observations and then uses the renewal equation to select k. It also formulates two average-cost MDPs: a time-elapsed MDP (claimed to justify block replacement within its policy class) and an age-vector MDP (claimed to have a monotone threshold optimal policy under IFR), the latter serving as a gold-standard benchmark. The headline theoretical claims are: Theorems 4.1-4.2 give O((K-1) log T) regret for independent LCB algorithms; Theorems 4.3-4.4 give O((K-k*) log T) regret for correlated LCB algorithms with only O(1) direct pulls of suboptimal arms k<k*; and Theorems 4.5-4.6 give almost-sure consistency of the KM-based policy. Numerical experiments compare all algorithms on Binomial and Poisson lifetime models.

Significance. The paper contains useful components: the independent-arm LCB analysis follows standard bandit theory and is credible; the KM renewal algorithm with censored observations is a sensible two-step procedure with plausible consistency; the MDP formulations provide a clear benchmark framework; and the authors ship code and detailed appendices. However, the truly distinctive contribution is the correlated-arm improvement, and that part is not supported by the manuscript's own proof. The claimed O(1) direct-pull count and the resulting O((K-k*) log T) regret separation are the central advertised results, appearing in the abstract, the introduction, and the conclusion. Since the proof of Theorems 4.3-4.4 contains a miscalibrated exploration bonus, an incorrectly specified good event, and a final step in which a Theta(log T) quantity is concluded to be O(1), the paper's main novelty is currently unsubstantiated. If the O(1) claim can be either proven under a clearly stated observation model or removed from the claims, the remaining paper is still of interest, but the current version overclaims.

major comments (4)
  1. [Appendix B, proof of Theorem 4.3] The good event G_t is one-sided in the wrong direction. The proof defines G_t = {bc_k <= c(k)+Delta/4} cap {bc_k* >= c(k*)-Delta/4}, but to obtain the displayed lower bound on LCB_k one needs bc_k >= c(k)-Delta/4, and to obtain the upper bound on LCB_k* one needs bc_k* <= c(k*)+Delta/4. The stated one-sided inequalities imply the opposite signs from what is used. The event should be two-sided: |bc_k-c(k)|<=Delta/4 and |bc_k*-c(k*)|<=Delta/4.
  2. [Appendix B, n_0 calculation] Even after correcting the good event, the bonus calculation is off by a factor: with n_0 = ceil(16 b^2 Delta_k^{-2} log(KT^2)), Hoeffding gives b sqrt(2 log(KT^2)/n_0) = Delta_k/(2 sqrt(2)), not Delta_k/4. Thus the claimed separation LCB_k(t) > LCB_{k*}(t) on G_t is not established. Enlarging n_0 by a constant factor repairs this, but it does not repair the next issue.
  3. [Appendix B, E[n_k(T)] bound] The proof concludes E[n_k(T)] <= 2 n_0 + O(1) = O(1), but n_0 = Theta(log(KT^2)) = Theta(log T), so the bound is O(log T), not O(1). Consequently the proof yields the same logarithmic direct-pull count as the independent-arm algorithms. Since the O((K-k*) log T) regret bound in Theorem 4.3 depends on arms k<k* having O(1) direct pulls, the advertised improvement over independent LCB is not supported. Theorem 4.4 and Appendix C inherit the same problem, as n_0 and n_{0,k*} are both Theta(log T).
  4. [Section 4.1 and Algorithms 5-6] The bandit observation model is not stated consistently. Section 4.1 says the operator observes the total number of failures Y_t per cycle, one scalar. Proposition 4.1 and Algorithms 5-6 require the full failure timing within the cycle, so that the cost C(j) for every j <= k_t can be reconstructed. If only the aggregate Y_t is observable, Proposition 4.1(a) is false. If full per-failure timing is intended, it must be stated explicitly as part of the model; the pseudocode in Appendix A2.5/A2.6 ('Observe cumulative failures: Z_j <- sum_{i=1}^j Y_i') currently leaves this undefined.
minor comments (5)
  1. [Algorithm 5/6 pseudocode] The line 'Observe cumulative failures: Z_j <- sum_{i=1}^j Y_i' uses Y_i without defining the per-period failure counts; this is the same ambiguity as Major Comment 4 and should be resolved in the pseudocode.
  2. [Remark 4.5] The remark states that the Lai-Robbins lower bound requires every consistent algorithm to pull each suboptimal arm Omega(log T) times. The bound in Eq. (5) is on regret, not directly on per-arm pull counts for arbitrary gaps; the statement should be qualified.
  3. [Section 5] The numerical experiments report results from a single run with a common random seed and no confidence intervals or multiple-seed summaries. Given that the theoretical separation is asymptotic, this limits the strength of the empirical comparison.
  4. [Table 2 and text near it] The phrase 'Note that tau* = 0, meaning any age triggers block replacement' is confusing because tau*(b) is a function of the other machine's age; the sentence should be clarified.
  5. [General] There are several typographical issues: 'Proof.Proof.' appears in Theorems 3.1 and 3.3, and the phrase 'Table 1' is used inconsistently across the complexity discussion. These are minor and do not affect the results.

Circularity Check

0 steps flagged

No significant circularity: the regret and consistency results are derived from stated renewal-theory and concentration-inequality assumptions; the main caveats are proof/modeling gaps, not circular reductions.

full rationale

The paper's central results are not obtained by fitting the target quantity or by a self-citation chain. The independent-arm regret bounds (Theorems 4.1-4.2) are standard LCB arguments relying on external concentration inequalities and the Lai-Robbins lower bound. The correlated-arm bounds (Theorems 4.3-4.4) are attempted proofs from Hoeffding/Bernstein bounds plus the nested-observation property of Proposition 4.1, which is an algebraic identity about renewal counts: C(l) is a deterministic function of the lifetimes observed by time k>=l. The Kaplan-Meier results (Theorems 4.5-4.6) are derived from uniform consistency of the KM estimator under independent censoring and the renewal equation; the estimated cost convergence is proved by induction, not assumed. The time-elapsed MDP result (Theorem 3.2) is a reformulation of the definition of c(k): any stationary deterministic policy has cycle cost c(min S), so the optimum is c(k*). This is a definitional equivalence, not a circular prediction; it does not smuggle in k*. The only self-citation is Ge-Kulkarni [2024], which is cited as related prior work and is not load-bearing for any proof here. There are legitimate correctness concerns, but they are not circularity: Appendix B concludes E[n_k(T)] = O(1) from n0 = Theta(log T) and miscomputes the Hoeffding bonus as Delta/4 instead of Delta/(2 sqrt 2); the observation model in Section 4.1 states only total failures Y_t are observed while Algorithms 5-6 require per-period failure counts; and Algorithm 7's pseudocode says 'sample l~F' although F is unknown, which appears to be a simulation shorthand rather than a learned input. None of these reduce a claimed result to its own assumptions by construction.

Axiom & Free-Parameter Ledger

2 free parameters · 10 axioms · 0 invented entities

The central claims are built on renewal theory and concentration inequalities; no fitted parameters enter the asymptotic bounds. The main additional premises are the full-history observation model for correlated arms, the boundary forcing h(K+1)=1 in the age-vector MDP, and the independent-censoring structure for KM, any of which, if false, would change the results.

free parameters (2)
  • KM exploration rate epsilon = not reported
    Algorithm 7 uses a decaying exploration schedule epsilon_t = epsilon/sqrt(t); the value used in the experiments is not stated, so replication is incomplete.
  • KM refit frequency R = not reported
    Algorithm 7 recomputes the KM estimator and renewal function every R cycles; R is not stated in the numerical section.
axioms (10)
  • domain assumption Lifetimes are positive integer-valued, iid across machines and over renewals, with finite mean
    Model specification in Section 3.1, assumptions (i)-(ii).
  • standard math Renewal reward theorem gives c(k) = (c_b + c_f N M(k))/k
    Used throughout Section 3; see Ross [1996].
  • standard math Lai-Robbins lower bound applies to the per-cycle regret
    Equation (5), Section 3.2.
  • domain assumption c_f > c_b/N
    Given in Section 3.1; motivates block replacement.
  • domain assumption f(k) > 0 for all k in [K]
    Used in Theorem 3.3 to ensure all machines can fail, making the age-vector MDP unichain.
  • domain assumption The optimal interval k* is unique
    Stated in Section 3.1; used in regret proofs and Theorem 4.6.
  • domain assumption F has increasing failure rate for Theorem 3.4
    Required for the monotone threshold policy; Section 3.4.
  • ad hoc to paper Full failure history of each machine is observable within a cycle
    Correlated-arm algorithms (Algorithms 5-6, Proposition 4.1) reconstruct costs for all j <= k from a pull of k; Section 4.1 only says total failure count Y_t is observed, so this is an unstated strengthening of the observation model.
  • ad hoc to paper Boundary h(K+1) = 1: machines older than K fail with certainty
    Age-vector MDP state-space cap in Section 3.4; modifies the tail of F and is not justified physically.
  • domain assumption Residual lifetimes at block replacement are independent of their censoring times and have distribution F
    Needed for KM consistency in Theorem 4.5 and Appendix D.

reviewed 2026-08-01 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Data Driven Block Replacement Scheduling." pith.science (2026). https://pith.science/paper/XSOHQSQL

@misc{pith2026260715229,
  author       = {Pith},
  title        = {Pith review of: Data Driven Block Replacement Scheduling},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XSOHQSQL}},
  note         = {Machine review of arXiv:2607.15229}
}
Share X Bluesky LinkedIn Reddit HN
abstract

We develop data-driven algorithms for maintaining $N$ independent identical machines under a \textit{block replacement policy}, in which each machine is replaced upon failure and all machines are jointly replaced at regular intervals of length $k$. The goal is to learn the cost-minimizing interval $k^*$ from operational data when the lifetime distribution is unknown. At each decision epoch, the operator selects $k \in \{1, 2, \ldots, K\}$, observes the resulting failure history (a mixture of complete and right-censored lifetimes) and incurs a per-unit-time cost governed by the renewal function. We formulate this as a stochastic multi-armed bandit and propose Hoeffding- and Bernstein-based lower-confidence-bound algorithms achieving $O(K \log T)$ regret, matching the Lai--Robbins lower bound. Exploiting a nested observation property unique to block replacement, correlated variants attain $O((K-k^*)\log T)$ regret and require only $O(1)$ direct pulls of suboptimal arms $k < k^*$. A complementary Kaplan--Meier renewal algorithm estimates the lifetime distribution nonparametrically from censored data, achieving almost-sure policy consistency and empirically near-zero incremental regret at long horizons. We additionally analyze two average-cost MDPs: a time-elapsed formulation establishing that block replacement is optimal within its policy class for any lifetime distribution, and an age-vector formulation proving a monotone threshold structure under increasing failure rate distributions and providing a gold-standard cost benchmark. Numerical experiments confirm the theoretical ordering and reveal structural cost gaps between optimal block and age-dependent replacement.

Figures

Figures reproduced from arXiv: 2607.15229 by Aniruddhan Ganesaraman, Vidyadhar Kulkarni.

Figure 1
Figure 1. Figure 1: True long-run average cost c(k) vs k under each lifetime distribution. The optimal interval k ∗ is marked with a red diamond. For each distribution the true optimal interval k ∗ and cost c(k ∗ ) are computed via the renewal equation (1) using a PMF truncated at Krenew = 45 to ensure accuracy under the Poisson tail. We have tested all five algorithms using simulations. The simulations need to run over long … view at source ↗
Figure 2
Figure 2. Figure 2: Cumulative regret Reg(T) for Algorithms 3–7 over T = 10, 000 decision epochs (cb = 1, cf = 2.6, N = 2, K = 12) [PITH_FULL_IMAGE:figures/full_fig_p021_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Cumulative regret Reg g(T) for Algorithms 3–7 over T = 10, 000 decision epochs (cb = 1, cf = 2.6, N = 2, K = 12). differs across arms. The cumulative regret plots reveal a clear performance ordering: among the bandit algorithms, the correlated Bernstein LCB (Algorithm 6) achieves the lowest regret, consistent with its O((K − k ∗ ) log T) guarantee and tighter variance-aware confidence bounds; the Kaplan–Me… view at source ↗
Figure 4
Figure 4. Figure 4: Histogram of replacement intervals k selected by the Correlated Bernstein LCB (Algorithm 6) and the Kaplan–Meier Renewal (Algorithm 7) algorithms over the first T = 50 decision epochs, under the Binomial(10, 0.5) (left) and Poisson(4) (right) lifetime distributions. The dashed vertical line marks the optimal interval k ∗ . time-elapsed MDP (Section 3.3), computed analytically in O(K2 ) operations via Algor… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

30 extracted references · 1 canonical work pages

  1. [1]

    European Journal of Operational Research 299:898--909

    Andersen JF, Andersen AR, Kulahci M, Nielsen BF (2022) A numerical study of Markov decision process algorithms for multi-component replacement problems. European Journal of Operational Research 299:898--909

    2022

  2. [2]

    Theoretical Computer Science 410(19):1876--1902

    Audibert J-Y, Munos R, Szepesv \'a ri C (2009) Exploration--exploitation tradeoff using variance estimates in multi-armed bandits. Theoretical Computer Science 410(19):1876--1902

    2009

  3. [3]

    Machine Learning 47:235--256

    Auer P, Cesa-Bianchi N, Fischer P (2002) Finite-time analysis of the multiarmed bandit problem. Machine Learning 47:235--256

    2002

  4. [4]

    Operations Research 8(1):90--100

    Barlow RE, Hunter L (1960) Optimum preventive maintenance policies. Operations Research 8(1):90--100

    1960

  5. [5]

    Journal of Nonparametric Statistics 4:317--326

    Baxter LA, Li L (1995) Nonparametric confidence intervals for the renewal function with censored data. Journal of Nonparametric Statistics 4:317--326

    1995

  6. [6]

    Stochastic Processes and their Applications 4:89--106

    Berg M (1976) Optimal replacement policies for two-unit machines with increasing running costs. Stochastic Processes and their Applications 4:89--106

    1976

  7. [7]

    The Annals of Statistics 2(3):437--453

    Breslow N, Crowley J (1974) A large sample study of the life table and product limit estimates under random censorship. The Annals of Statistics 2(3):437--453

    1974

  8. [8]

    Foundations and Trends in Machine Learning 5:1--122

    Bubeck S, Cesa-Bianchi N (2012) Regret analysis of stochastic and nonstochastic multi-armed bandit problems. Foundations and Trends in Machine Learning 5:1--122

    2012

  9. [9]

    Proceedings of the 28th Conference on Uncertainty in Artificial Intelligence 142--151

    Caron S, Kveton B, Lelarge M, Bhagat S (2012) Leveraging side observations in stochastic bandits. Proceedings of the 28th Conference on Uncertainty in Artificial Intelligence 142--151

    2012

  10. [10]

    European Journal of Operational Research 51(1):1--23

    Cho DI, Parlar M (1991) A survey of maintenance models for multi-unit systems. European Journal of Operational Research 51(1):1--23

    1991

  11. [11]

    Journal of Risk and Reliability 231:534--545

    Corman F, Kraijema S, Godjevac M, Lodewijks G (2017) Optimizing preventive maintenance policy: a data-driven application for a light rail braking system. Journal of Risk and Reliability 231:534--545

    2017

  12. [12]

    Mathematical Methods of Operations Research 45:411--435

    Dekker R, Wildeman RE, van der Duyn Schouten F (1997) A review of multi-component maintenance models with economic dependence. Mathematical Methods of Operations Research 45:411--435

    1997

  13. [13]

    European Journal of Operational Research 290:530--545

    Deprez L, Antonio K, Boute R (2021) Pricing service maintenance contracts using predictive analytics. European Journal of Operational Research 290:530--545

    2021

  14. [14]

    SSRN Electronic Journal doi:10.2139/ssrn.4959529

    Ge P, Kulkarni VG (2024) Data driven machine maintenance: a renewal reward bandit framework. SSRN Electronic Journal doi:10.2139/ssrn.4959529

    work page doi:10.2139/ssrn.4959529 2024

  15. [15]

    The Annals of Statistics 11(1):49--58

    Gill RD (1983) Large sample behaviour of the product-limit estimator on the whole line. The Annals of Statistics 11(1):49--58

    1983

  16. [16]

    The Annals of Statistics 21:1431--1451

    Gr \"u bel R, Pitts SM (1993) Nonparametric estimation in renewal theory I : the empirical renewal function. The Annals of Statistics 21:1431--1451

    1993

  17. [17]

    Journal of the American Statistical Association 53(282):457--481

    Kaplan EL, Meier P (1958) Nonparametric estimation from incomplete observations. Journal of the American Statistical Association 53(282):457--481

    1958

  18. [18]

    (Chapman and Hall/CRC, Boca Raton)

    Kulkarni VG (2009) Modeling and Analysis of Stochastic Systems, 2nd ed. (Chapman and Hall/CRC, Boca Raton)

    2009

  19. [19]

    Operations Research Letters 38:273--279

    Kurt M, Kharoufeh JP (2010) Monotone optimal replacement policies for a Markovian deteriorating system in a controllable environment. Operations Research Letters 38:273--279

    2010

  20. [20]

    Proceedings of the Eighteenth International Conference on Artificial Intelligence and Statistics 535--543

    Kveton B, Wen Z, Ashkan A, Szepesv \'a ri C (2015) Tight regret bounds for stochastic combinatorial semi-bandits. Proceedings of the Eighteenth International Conference on Artificial Intelligence and Statistics 535--543

    2015

  21. [21]

    Advances in Applied Mathematics 6(1):4--22

    Lai TL, Robbins H (1985) Asymptotically efficient adaptive allocation rules. Advances in Applied Mathematics 6(1):4--22

    1985

  22. [22]

    Lattimore T, Szepesv \'a ri C (2020) Bandit Algorithms (Cambridge University Press, Cambridge)

    2020

  23. [23]

    Advances in Neural Information Processing Systems 24

    Mannor S, Shamir O (2011) From bandits to experts: on the value of side-observations. Advances in Neural Information Processing Systems 24

    2011

  24. [24]

    Puterman ML (1994) Markov Decision Processes: Discrete Stochastic Dynamic Programming (John Wiley & Sons, New York)

    1994

  25. [25]

    (John Wiley & Sons, New York)

    Ross SM (1996) Stochastic Processes, 2nd ed. (John Wiley & Sons, New York)

    1996

  26. [26]

    Foundations and Trends in Machine Learning 11:1--96

    Russo DJ, Van Roy B, Kazerouni A, Osband I, Wen Z (2018) A tutorial on Thompson sampling. Foundations and Trends in Machine Learning 11:1--96

    2018

  27. [27]

    Journal of Applied Probability 29(4):932--946

    Shaked M, Zhu H (1992) Some results on block replacement policies and renewal theory. Journal of Applied Probability 29(4):932--946

    1992

  28. [28]

    European Journal of Operational Research 96:232--247

    Sheu S-H, Jhang J-P (1996) A generalized group maintenance policy. European Journal of Operational Research 96:232--247

    1996

  29. [29]

    Biometrika 25:285--294

    Thompson WR (1933) On the likelihood that one unknown probability exceeds another in view of the evidence of two samples. Biometrika 25:285--294

    1933

  30. [30]

    The Annals of Statistics 10:772--785

    Vardi Y (1982) Nonparametric estimation in renewal processes. The Annals of Statistics 10:772--785

    1982

This paper was first reviewed by deepseek-v4-flash on August 1, 2026.