DARLING: Detection Augmented Reinforcement Learning with Non-Stationary Guarantees

Argyrios Gerogiannis; Venugopal V. Veeravalli; Yu-Han Huang

arxiv: 2604.16684 · v2 · submitted 2026-04-17 · 💻 cs.LG · stat.ML

DARLING: Detection Augmented Reinforcement Learning with Non-Stationary Guarantees

Argyrios Gerogiannis , Yu-Han Huang , Venugopal V. Veeravalli This is my paper

Pith reviewed 2026-05-13 07:36 UTC · model grok-4.3

classification 💻 cs.LG stat.ML

keywords reinforcement learningnon-stationary MDPspiecewise stationarydynamic regretchange detectiontabular MDPslinear MDPsminimax bounds

0 comments

The pith

DARLING augments RL algorithms with change detection to match minimax-optimal dynamic regret in non-stationary tabular MDPs.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces DARLING as a modular wrapper that adds a change-point detection component to existing reinforcement learning methods for piecewise-stationary environments where rewards and transitions shift at unknown times. It first derives the first minimax lower bounds on dynamic regret for both tabular and linear finite-horizon episodic MDPs. Under change-point separability and reachability conditions, DARLING improves prior regret guarantees in the tabular case and exactly meets the lower bound; in the linear case it meets the bound when reachability parameters are known. A reader would care because the result supplies performance guarantees for agents that must operate in realistic settings where the world changes without warning and without supplying the agent any schedule of those changes.

Core claim

DARLING is a detection-augmented wrapper for piecewise-stationary reinforcement learning that requires no prior knowledge of change times or magnitudes. In tabular MDPs that satisfy change-point separability and reachability, it improves the best previously known dynamic-regret bounds and matches the minimax lower bound established in the paper. In linear MDPs the same wrapper matches the lower bound once reachability parameters are supplied, while the analysis identifies structural obstacles that prevent the same tight result without those parameters.

What carries the argument

DARLING, a modular wrapper that combines a change-point detection module with any base PS-RL algorithm to identify and adapt to unknown shifts in rewards and dynamics.

If this is right

Dynamic regret for tabular piecewise-stationary RL becomes minimax-optimal under the stated conditions.
The method operates without any supplied schedule or magnitude information about environment changes.
The same wrapper attains the lower bound in linear MDPs once reachability parameters are known.
Empirical tests across multiple non-stationary benchmarks show consistent improvement over prior methods.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The detection wrapper could be tested in continuous-state or function-approximation settings where the current analysis does not yet apply.
Relaxing reachability would enlarge the class of non-stationary problems for which near-optimal guarantees are possible.
The lower bounds indicate that any future non-stationary RL method must pay at least this cost when changes are completely unknown.

Load-bearing premise

Change-point separability and reachability conditions must hold so the detection module can reliably locate shifts without knowing their timing or size in advance.

What would settle it

Construct a tabular MDP that satisfies the separability and reachability conditions, run DARLING, and check whether its dynamic regret stays within a constant factor of the derived minimax lower bound; exceeding that factor on such an instance would falsify the optimality claim.

Figures

Figures reproduced from arXiv: 2604.16684 by Argyrios Gerogiannis, Venugopal V. Veeravalli, Yu-Han Huang.

**Figure 1.** Figure 1: Cumulative reward results for the experiments (higher=better). Top row: Tabular MDPs, Bottom row: Linear MDPs. giannis et al., 2025b) to stress-test prior-free adaptation: segment lengths are i.i.d. geometric with parameter T −ξ for ξ ∈ {0.4, 0.6, 0.8}, yielding an average of up to 659 changes over the horizon. This is substantially more challenging than the settings used in Mao et al. (2022) (5 changes) … view at source ↗

**Figure 2.** Figure 2: States of the MDP Mi’s Let r k h,i and P k h,i denote the reward function and the transition kernel of MDP Mi at step h over the k th stationary segment, i.e., r t h = r k h,i and P t h = P k h,i for Mi at t ∈ {νk−1, . . . , νk − 1}. We set the rewards of the MDP Mi’s to be deterministic and binary. To be specific, The reward function is defined as follows: for all k ∈ [NT + 1], h ∈ [H], i ∈ {0, 1} NT +1 ,… view at source ↗

**Figure 3.** Figure 3: The transition kernel of MDP Mi with ik = 0 Fix i ∈ {0, 1} NT +1 and k ∈ [NT + 1], and let j be the index obtained by flipping the k th bit, i.e., jk ̸= ik and jl = il for all l ̸= k; the map i 7→ j is bijective. Without loss of generality, we assume that ik = 0 and jk = 1. Then Ri,k(π) = X (h,ℓ,a)̸=(1+D,1,ag) εk(H − H¯ − D) Ei,π[nk(leafℓ, a, h)] = εk(H − H¯ − D) (νk − νk−1 − Ei,π[nk(leaf1, ag, 1 + D)]) ≥ … view at source ↗

**Figure 4.** Figure 4: The transition kernel of MDP Mi with ik = 1 Let ˜i := (i1, . . . , ik, 0, . . . , 0) be the index vector obtained by making the bits of i after the k th bit become 0. Similarly, let ˜j := (j1, . . . , jk, 0, . . . , 0) be the index vector obtained by making the bits of j after the k th bit become 0. Then, due to the fact that P k h,i and P k h,˜i are identical up to the k th interval, and that P k h,j and … view at source ↗

**Figure 5.** Figure 5: The process of transition probability assignment for the MDPs with NT = 2. The lines represent the value of the transition probability P k h,i (sg|leafℓ, a), and the intervals are the stationary segments. The colored triple underneath each interval denotes the triple (leafℓ˜ i , a˜i, h˜i) in (8). In step (a), we use the fact that the transition kernel of M˜i and that of M˜j differ only at step h˜ i during … view at source ↗

read the original abstract

We study model-free reinforcement learning (RL) in non-stationary finite-horizon episodic Markov decision processes (MDPs) without prior knowledge of the non-stationarity. We focus on the piecewise stationary (PS) setting, where both rewards and transition dynamics can change at unknown times. We first revisit existing state-of-the-art approaches and identify theoretical and practical limitations that change the current landscape of performance guarantees. To characterize the difficulty of the problem, we establish the first minimax lower bounds for PS-RL in tabular and linear MDPs. We then introduce Detection Augmented Reinforcement Learning (DARLING), a modular wrapper for PS-RL that applies to both tabular and linear MDPs, without knowledge of the changes. In tabular MDPs, under change-point separability and reachability conditions, DARLING improves the best known dynamic regret bounds and matches our minimax lower bound. In linear MDPs, DARLING matches the minimax lower bound when the relevant reachability parameters are known, and our analysis clarifies the structural obstacles that distinguish this setting from the tabular case. Finally, through extensive experimentation across diverse non-stationary benchmarks, we show that DARLING consistently surpasses the state-of-the-art methods.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

DARLING gives the first minimax lower bounds for piecewise stationary RL and matches them in the tabular case with a modular detection wrapper, but only when change points are separable and reachable.

read the letter

The main thing to know is that this paper derives the first minimax lower bounds for piecewise stationary RL in both tabular and linear MDPs, then shows DARLING, a detection-augmented wrapper, matches those bounds in the tabular setting under change-point separability and reachability conditions. It improves on prior dynamic regret rates when those hold and does so without prior knowledge of change times or sizes. The modular design is a practical plus because it can wrap existing algorithms rather than requiring a full redesign. Experiments on diverse non-stationary benchmarks indicate consistent gains over current methods. The lower bounds appear derived independently, which avoids the usual circularity issues. The soft spots are the assumptions themselves. If change points are too close or the shifts too small relative to mixing time, detection fails and the regret reverts to slower rates from earlier work. Reachability is needed for reliable visitation and false-positive control, so violations break the sample-complexity arguments. In the linear case the paper itself flags structural obstacles that prevent a clean match without knowing reachability parameters in advance. Full proofs and experiment details would need checking to confirm no gaps in the linear analysis or data handling. This is for RL theorists working on non-stationary settings who want tight bounds rather than heuristic improvements. It has enough new content and grounding to deserve a serious referee, even if revisions are needed around the conditions.

Referee Report

3 major / 2 minor

Summary. The paper studies model-free RL in piecewise-stationary non-stationary finite-horizon episodic MDPs without prior knowledge of change points. It derives the first minimax lower bounds on dynamic regret for both tabular and linear MDPs, then proposes DARLING as a modular detection-augmented wrapper that improves upon prior dynamic-regret bounds. In tabular MDPs, under explicit change-point separability and reachability conditions, DARLING matches the derived minimax lower bound; in linear MDPs it matches the lower bound when reachability parameters are known. Experiments across non-stationary benchmarks show consistent outperformance of existing methods.

Significance. If the lower-bound constructions and matching upper bounds hold, the work provides the first tight characterization of the difficulty of piecewise-stationary RL and supplies a practical, modular algorithm that attains the optimal rate under stated conditions. The clarification of structural obstacles separating the tabular and linear cases, together with the experimental validation, would constitute a substantive advance for non-stationary RL theory and practice.

major comments (3)

[Abstract / §4] Abstract and §4 (presumably the tabular analysis): the headline claim that DARLING matches the minimax lower bound is explicitly conditional on change-point separability and reachability. The manuscript must state the precise theorem (including the detection-error term in the regret decomposition) under which these conditions suffice to control false-positive/negative rates, and must clarify what happens to the dynamic regret when separability is violated (e.g., change points closer than the mixing time).
[Linear-MDP analysis] Linear-MDP section: the matching result requires that the relevant reachability parameters are known a priori. The paper should either (a) provide a data-driven estimator for these parameters whose error does not degrade the leading term, or (b) explicitly quantify the additional regret incurred when they must be estimated, as this appears to be the central structural obstacle distinguishing the linear from the tabular case.
[Lower-bound section] Lower-bound constructions: because the abstract asserts these are the first minimax lower bounds, the proof sketches or appendices must be checked for whether the hard instances satisfy the same piecewise-stationary episodic structure used in the upper-bound analysis; any mismatch in horizon, episode length, or change-point placement would prevent the claimed tightness.

minor comments (2)

[Notation / §3] Notation for the detection module (thresholds, window sizes) should be introduced once and used consistently; currently the abstract refers to “the detection module” without a forward reference to its definition.
[Experiments] The experimental section would benefit from an explicit statement of the change-point generation process and the precise metrics used to declare “consistent superiority,” including confidence intervals or statistical tests.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the detailed and constructive comments. We address each major comment point by point below, indicating the revisions we will incorporate to improve clarity and completeness.

read point-by-point responses

Referee: [Abstract / §4] Abstract and §4 (presumably the tabular analysis): the headline claim that DARLING matches the minimax lower bound is explicitly conditional on change-point separability and reachability. The manuscript must state the precise theorem (including the detection-error term in the regret decomposition) under which these conditions suffice to control false-positive/negative rates, and must clarify what happens to the dynamic regret when separability is violated (e.g., change points closer than the mixing time).

Authors: We agree that greater precision is needed. In the revision we will restate the main tabular theorem in full (including the explicit detection-error term arising from the change-point detector in the regret decomposition) and specify the exact conditions on separability and reachability that guarantee the false-positive/negative rates remain controlled. We will also add a short discussion clarifying the consequence of violating separability: when change points are closer than the mixing time the dynamic regret acquires an additional linear term in the number of such violations and no longer matches the minimax lower bound. revision: yes
Referee: [Linear-MDP analysis] Linear-MDP section: the matching result requires that the relevant reachability parameters are known a priori. The paper should either (a) provide a data-driven estimator for these parameters whose error does not degrade the leading term, or (b) explicitly quantify the additional regret incurred when they must be estimated, as this appears to be the central structural obstacle distinguishing the linear from the tabular case.

Authors: We acknowledge this limitation. The revised linear-MDP section will explicitly quantify the extra regret incurred when the reachability parameters must be estimated from data; the estimation error contributes only a lower-order additive term that does not change the leading minimax rate. While constructing a fully data-driven estimator that preserves the exact leading term remains technically challenging (and is highlighted as the key structural difference from the tabular case), the added quantification directly addresses the referee’s request. revision: partial
Referee: [Lower-bound section] Lower-bound constructions: because the abstract asserts these are the first minimax lower bounds, the proof sketches or appendices must be checked for whether the hard instances satisfy the same piecewise-stationary episodic structure used in the upper-bound analysis; any mismatch in horizon, episode length, or change-point placement would prevent the claimed tightness.

Authors: The hard instances were constructed to obey exactly the same piecewise-stationary episodic MDP structure (identical finite horizon, episode length, and admissible change-point placements) used in the upper-bound analysis. We will insert a brief clarifying remark in the lower-bound section and ensure the appendix proofs explicitly verify this alignment, thereby confirming that the upper and lower bounds are tight under the stated conditions. revision: yes

Circularity Check

0 steps flagged

No significant circularity: lower bounds derived first and independently; algorithm matches them under external conditions rather than by construction.

full rationale

The paper establishes minimax lower bounds for PS-RL before introducing DARLING, then shows the algorithm achieves matching upper bounds in tabular MDPs only under explicitly stated change-point separability and reachability conditions. These conditions are external assumptions required for the detection module's guarantees and are not derived from or fitted to the algorithm's outputs. No self-citations are load-bearing for the core claims, no parameters are fitted then renamed as predictions, and the derivation does not reduce any result to its own inputs by definition. The analysis remains self-contained against the stated benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

The central claims rest on change-point separability, reachability conditions, and the piecewise stationary assumption; these are domain assumptions rather than derived quantities. No free parameters or invented entities are explicitly introduced in the abstract.

axioms (2)

domain assumption The environment is piecewise stationary with changes at unknown times.
Stated in the abstract as the setting under study.
domain assumption Change-point separability and reachability conditions hold in tabular MDPs.
Required for the matching regret bound in the abstract.

pith-pipeline@v0.9.0 · 5523 in / 1372 out tokens · 34815 ms · 2026-05-13T07:36:10.070305+00:00 · methodology

Review history (2 revisions) →

discussion (0)

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

IndisputableMonolith/Cost/FunctionalEquation.lean washburn_uniqueness_aczel unclear

?

unclear
Relation between the paper passage and the cited Recognition theorem.

DARLING combines effective mean-shift detection ideas... two tests tailored to identify changes in the reward functions and transition dynamics... GDT test statistic GTSk... Property 4.8: ℓD, mD = O(log T + log(1/(δD δF)))
IndisputableMonolith/Foundation/RealityFromDistinction.lean reality_from_one_distinction unclear

?

unclear
Relation between the paper passage and the cited Recognition theorem.

Theorem 4.11... R(π,T)=Õ((NT+1)RL(T/(NT+1)))... matches minimax lower bound under Assumptions 4.7,4.10

What do these tags mean?

matches: The paper's claim is directly supported by a theorem in the formal canon.
supports: The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends: The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses: The paper appears to rely on the theorem as machinery.
contradicts: The paper's claim conflicts with a theorem or certificate in the canon.
unclear: Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

11 extracted references · 11 canonical work pages

[1]

cc/paper_files/paper/2014/file/ 903ce9225fca3e988c2af215d4e544d3-Paper

URL https://proceedings.neurips. cc/paper_files/paper/2014/file/ 903ce9225fca3e988c2af215d4e544d3-Paper. pdf. Besson, L., Kaufmann, E., Maillard, O.-A., and Seznec, J. Efficient Change-Point Detection for Tackling Piecewise- Stationary Bandits.Journal of Machine Learning Re- search, 23(77):1–40, 2022. Cai, H., Ren, K., Zhang, W., Malialis, K., Wang, J., Y...

work page arXiv 2014
[2]

Domingues, O

URL https://proceedings.mlr.press/ v151/deng22b.html. Domingues, O. D., M ´enard, P., Kaufmann, E., and Valko, M. Episodic reinforcement learning in finite 9 Detection Augmented Reinforcement Learning with Non-Stationary Guarantees mdps: Minimax lower bounds revisited. In Feld- man, V ., Ligett, K., and Sabato, S. (eds.),Proceed- ings of the 32nd Internat...

work page arXiv 2021
[3]

Hong, K., Li, Y ., and Tewari, A

URL https://proceedings.mlr.press/ v202/he23d.html. Hong, K., Li, Y ., and Tewari, A. An Optimization- based Algorithm for Non-stationary Kernel Bandits without Prior Knowledge. In Ruiz, F., Dy, J., and van de Meent, J.-W. (eds.),Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, volume 206 ofProceedings of Machine...

work page
[4]

Hu, P., Chen, Y ., and Huang, L

URL https://proceedings.mlr.press/ v206/hong23b.html. Hu, P., Chen, Y ., and Huang, L. Nearly minimax optimal re- inforcement learning with linear function approximation. In Chaudhuri, K., Jegelka, S., Song, L., Szepesvari, C., Niu, G., and Sabato, S. (eds.),Proceedings of the 39th International Conference on Machine Learning, volume 162 ofProceedings of ...

work page doi:10.1109/isit63088.2025.11195515 2022
[5]

Kocsis, L

URL https://proceedings.mlr.press/ v125/jin20a.html. Kocsis, L. and Szepesv ´ari, C. Discounted ucb. In2nd PASCAL Challenges Workshop, volume 2, pp. 51–134, 2006. Liu, F., Lee, J., and Shroff, N. A Change-Detection Based Framework for Piecewise-Stationary Multi-Armed Ban- dit Problem.Proceedings of the AAAI Conference on Artificial Intelligence, 32(1), Ap...

work page arXiv 2006
[6]

Ortner, R., Gajane, P., and Auer, P

URL https://proceedings.mlr.press/ v139/menard21b.html. Ortner, R., Gajane, P., and Auer, P. Variational regret bounds for reinforcement learning. InUncertainty in Artificial Intelligence, pp. 81–90, 2019. Ortner, R., Gajane, P., and Auer, P. Variational regret bounds for reinforcement learning. In Adams, R. P. and Gogate, V . (eds.),Proceedings of The 35...

work page 2019
[7]

Russac, Y ., Vernade, C., and Capp´e, O

PMLR, 25–28 Feb 2024. Russac, Y ., Vernade, C., and Capp´e, O. Weighted Linear Bandits for Non-Stationary Environments. In Wallach, H., Larochelle, H., Beygelzimer, A., d'Alch´e-Buc, F., Fox, E., and Garnett, R. (eds.),Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc.,

work page 2024
[8]

arXiv preprint arXiv:2003.10113 , year=

URL https://proceedings.neurips. cc/paper_files/paper/2019/file/ 263fc48aae39f219b4c71d9d4bb4aed2-Paper. pdf. Russac, Y ., Capp´e, O., and Garivier, A. Algorithms for Non-Stationary Generalized Linear Bandits, 2020. URL https://arxiv.org/abs/2003.10113. Russac, Y ., Faury, L., Capp´e, O., and Garivier, A. Self- Concordant Analysis of Generalized Linear Ba...

work page arXiv 2019
[9]

Zhao, P., Zhang, L., Jiang, Y ., and Zhou, Z.-H

URL https://proceedings.mlr.press/ v134/wei21b.html. Zhao, P., Zhang, L., Jiang, Y ., and Zhou, Z.-H. A Simple Approach for Non-stationary Linear Bandits. In Chiappa, S. and Calandra, R. (eds.),Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, volume 108 ofProceedings of Machine Learning Research, pp. 746–...

work page
[10]

Zhou, D., Gu, Q., and Szepesvari, C

URL https://proceedings.mlr.press/ v108/zhao20a.html. Zhou, D., Gu, Q., and Szepesvari, C. Nearly minimax opti- mal reinforcement learning for linear mixture markov decision processes. In Belkin, M. and Kpotufe, S. (eds.),Proceedings of Thirty Fourth Conference on Learn- ing Theory, volume 134 ofProceedings of Machine Learning Research, pp. 4532–4576. PML...

work page
[11]

,1,1+𝐷)1/2for others(leaf#,3,3+𝐷)(leaf$,5,2+𝐷)(leaf%,5,4+𝐷) One-hot vectors: 𝐢=[1,0,0] (leaf

URL https://proceedings.mlr.press/ v134/zhou21a.html. 11 Detection Augmented Reinforcement Learning with Non-Stationary Guarantees Zhou, H., Chen, J., Varshney, L. R., and Jagmohan, A. Nonstationary reinforcement learning with linear func- tion approximation.Transactions on Machine Learn- ing Research, 2022. ISSN 2835-8856. URL https: //openreview.net/for...

work page doi:10.1109/ciss50987.2021.9400292 2022

[1] [1]

cc/paper_files/paper/2014/file/ 903ce9225fca3e988c2af215d4e544d3-Paper

URL https://proceedings.neurips. cc/paper_files/paper/2014/file/ 903ce9225fca3e988c2af215d4e544d3-Paper. pdf. Besson, L., Kaufmann, E., Maillard, O.-A., and Seznec, J. Efficient Change-Point Detection for Tackling Piecewise- Stationary Bandits.Journal of Machine Learning Re- search, 23(77):1–40, 2022. Cai, H., Ren, K., Zhang, W., Malialis, K., Wang, J., Y...

work page arXiv 2014

[2] [2]

Domingues, O

URL https://proceedings.mlr.press/ v151/deng22b.html. Domingues, O. D., M ´enard, P., Kaufmann, E., and Valko, M. Episodic reinforcement learning in finite 9 Detection Augmented Reinforcement Learning with Non-Stationary Guarantees mdps: Minimax lower bounds revisited. In Feld- man, V ., Ligett, K., and Sabato, S. (eds.),Proceed- ings of the 32nd Internat...

work page arXiv 2021

[3] [3]

Hong, K., Li, Y ., and Tewari, A

URL https://proceedings.mlr.press/ v202/he23d.html. Hong, K., Li, Y ., and Tewari, A. An Optimization- based Algorithm for Non-stationary Kernel Bandits without Prior Knowledge. In Ruiz, F., Dy, J., and van de Meent, J.-W. (eds.),Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, volume 206 ofProceedings of Machine...

work page

[4] [4]

Hu, P., Chen, Y ., and Huang, L

URL https://proceedings.mlr.press/ v206/hong23b.html. Hu, P., Chen, Y ., and Huang, L. Nearly minimax optimal re- inforcement learning with linear function approximation. In Chaudhuri, K., Jegelka, S., Song, L., Szepesvari, C., Niu, G., and Sabato, S. (eds.),Proceedings of the 39th International Conference on Machine Learning, volume 162 ofProceedings of ...

work page doi:10.1109/isit63088.2025.11195515 2022

[5] [5]

Kocsis, L

URL https://proceedings.mlr.press/ v125/jin20a.html. Kocsis, L. and Szepesv ´ari, C. Discounted ucb. In2nd PASCAL Challenges Workshop, volume 2, pp. 51–134, 2006. Liu, F., Lee, J., and Shroff, N. A Change-Detection Based Framework for Piecewise-Stationary Multi-Armed Ban- dit Problem.Proceedings of the AAAI Conference on Artificial Intelligence, 32(1), Ap...

work page arXiv 2006

[6] [6]

Ortner, R., Gajane, P., and Auer, P

URL https://proceedings.mlr.press/ v139/menard21b.html. Ortner, R., Gajane, P., and Auer, P. Variational regret bounds for reinforcement learning. InUncertainty in Artificial Intelligence, pp. 81–90, 2019. Ortner, R., Gajane, P., and Auer, P. Variational regret bounds for reinforcement learning. In Adams, R. P. and Gogate, V . (eds.),Proceedings of The 35...

work page 2019

[7] [7]

Russac, Y ., Vernade, C., and Capp´e, O

PMLR, 25–28 Feb 2024. Russac, Y ., Vernade, C., and Capp´e, O. Weighted Linear Bandits for Non-Stationary Environments. In Wallach, H., Larochelle, H., Beygelzimer, A., d'Alch´e-Buc, F., Fox, E., and Garnett, R. (eds.),Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc.,

work page 2024

[8] [8]

arXiv preprint arXiv:2003.10113 , year=

URL https://proceedings.neurips. cc/paper_files/paper/2019/file/ 263fc48aae39f219b4c71d9d4bb4aed2-Paper. pdf. Russac, Y ., Capp´e, O., and Garivier, A. Algorithms for Non-Stationary Generalized Linear Bandits, 2020. URL https://arxiv.org/abs/2003.10113. Russac, Y ., Faury, L., Capp´e, O., and Garivier, A. Self- Concordant Analysis of Generalized Linear Ba...

work page arXiv 2019

[9] [9]

Zhao, P., Zhang, L., Jiang, Y ., and Zhou, Z.-H

URL https://proceedings.mlr.press/ v134/wei21b.html. Zhao, P., Zhang, L., Jiang, Y ., and Zhou, Z.-H. A Simple Approach for Non-stationary Linear Bandits. In Chiappa, S. and Calandra, R. (eds.),Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, volume 108 ofProceedings of Machine Learning Research, pp. 746–...

work page

[10] [10]

Zhou, D., Gu, Q., and Szepesvari, C

URL https://proceedings.mlr.press/ v108/zhao20a.html. Zhou, D., Gu, Q., and Szepesvari, C. Nearly minimax opti- mal reinforcement learning for linear mixture markov decision processes. In Belkin, M. and Kpotufe, S. (eds.),Proceedings of Thirty Fourth Conference on Learn- ing Theory, volume 134 ofProceedings of Machine Learning Research, pp. 4532–4576. PML...

work page

[11] [11]

,1,1+𝐷)1/2for others(leaf#,3,3+𝐷)(leaf$,5,2+𝐷)(leaf%,5,4+𝐷) One-hot vectors: 𝐢=[1,0,0] (leaf

URL https://proceedings.mlr.press/ v134/zhou21a.html. 11 Detection Augmented Reinforcement Learning with Non-Stationary Guarantees Zhou, H., Chen, J., Varshney, L. R., and Jagmohan, A. Nonstationary reinforcement learning with linear func- tion approximation.Transactions on Machine Learn- ing Research, 2022. ISSN 2835-8856. URL https: //openreview.net/for...

work page doi:10.1109/ciss50987.2021.9400292 2022