Pith. sign in

REVIEW 2 major objections 4 minor 36 references

Follow-the-Perturbed-Leader for Decoupled Bandits: Best-of-Both-Worlds and Practicality

T0 review · 2 major / 4 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that a Follow-the-Perturbed-Leader rule with Pareto-distributed perturbations achieves best-of-both-worlds regret in the decoupled multi-armed bandit — O(sqrt(KT)) against adversarial losses and time-independent O(K/Delta_m

desk verdict First FTPL policy for decoupled bandits with best-of-both-worlds guarantees and no resampling/convex optimization; the stochastic-regime proof has a small but real gap for K=2 that is likely patchable. read the letter →

arxiv 2510.12152 v2 pith:RN6ZNQFL submitted 2025-10-14 stat.ML cs.LG

Chaiwon Kim , Jongyeong Lee , Min-hwan Oh This is my paper
classification stat.MLcs.LG
keywords decoupledmulti-armedbanditfollow-the-perturbed-leaderParetoperturbationsbest-of-both-worldsregretadversarialbanditsstochasticbounds
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

In the decoupled multi-armed bandit — where at each round the learner may choose one arm to explore and a different arm to exploit — the paper proposes a Follow-the-Perturbed-Leader policy with Pareto-distributed random perturbations and shows it enjoys the best of both worlds: minimax-optimal O(sqrt(KT)) regret in the adversarial regime and a time-independent O(K/Delta_min) regret in the stochastically constrained adversarial regime, which includes the stochastic case. The policy is deliberately practical: it never solves a convex optimization problem, as earlier best-of-both-worlds policies did, and it never resamples to estimate arm-selection probabilities, as standard FTPL bandit policies do. Instead, a closed-form surrogate exploration rule approximates the true selection probabilities tightly enough to drive the proof. If correct, this turns best-of-both-worlds guarantees from a theoretical artifact into a fast, parameter-light algorithm, with experiments showing an order-of-magnitude speedup while maintaining or improving regret.

What carries the argument

The central object is the surrogate exploration vector q_t whose normalized version p_t drives exploration. Its entries are (min{(1+η_t hat L_{t,i})^{-1}, σ_{t,i}^{-1/α}})^{(α+1)/2}, where σ_{t,i} is the rank of arm i's estimated accumulated loss. This closed-form rule stands in for w_t^{1/2+1/(2α)} — a power of the true (closed-form-lacking) FTPL selection probability — and is computable in O(K log K) per step. Lemma 10 is the workhorse: it bounds the true selection probability w_{t,i} above and below by (1+η_t hat L_{t,i})^{-α} (up to constants on D_t), allowing the regret proof to replace w_t with q_t. The proof then splits regret into stability and penalty terms, bounds stability via sum

What would settle it

Set K=2 and choose losses so that at some round the sum in Eq. (10) equals exactly 1/2 with a suboptimal arm's estimated cumulative loss equal to zero while the optimal arm's estimated loss is positive; if such a history exists, the implication 'D_t ⇒ hat L_{t,i*}=0' used in Lemma 10 fails, and the stochastic proof as written does not cover it.

Watch

Extended reading notes

Core claim

The paper's central claim is that an FTPL rule with i.i.d. Pareto perturbations of shape α > 1, using exploitation arm i_t = argmin_i (hat L_{t,i} − r_{t,i}/η_t) and an exploration rule p_t derived from q_t = (min{(1+η_t hat L_{t,i})^{-1}, σ_{t,i}^{-1/α}})^{(α+1)/2} normalized over arms, attains O(sqrt(KT)) expected regret against an adaptive adversary (Theorem 1) and O(K/Delta_min) regret in the stochastically constrained adversarial regime for α ∈ (1,3] (Theorem 2), with α=3 sharpening the second term to O(sqrt(K/Delta_min) Σ_{i≠i*} 1/Delta_i + K/Delta_min) (Corollary 3). The proof couples the exploration distribution to the true FTPL selection probabilities through a tight upper bound — w

Load-bearing premise

The proof assumes that the event D_t — defined by a sum of terms being at most 1/2 — guarantees that the estimated cumulative loss of the true optimal arm is zero; for K=2, the equality case can make this implication false, and Lemmas 9 and 10 depend on it.

Editorial extensions

If this is right

  • The decoupled bandit admits a best-of-both-worlds policy that is fully implementable in O(K log K) per round, removing the main computational obstacle to using such guarantees in practice.
  • In the adversarial regime, regret matches the minimax lower bound up to constants, so no further improvement is possible; in the stochastic regime, regret is independent of T, unlike the log T bound of standard MABs.
  • The surrogate technique for approximating FTPL selection probabilities may transfer to other bandit settings where resampling is too costly.
  • Naively pairing a pure-exploration rule with a standard exploitation rule is empirically suboptimal; the coupling between exploitation and exploration probabilities is what carries the regret bound.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the K=2 boundary issue in the analysis is patched (e.g., a strict threshold in D_t or a separate two-arm argument), the algorithmic result is untouched and the theorem likely holds as stated; the issue is confined to the proof's event, not to the policy.
  • The same closed-form surrogate trick could replace geometric resampling in FTPL algorithms for other online decision problems (semi-bandits, combinatorial action sets), where resampling costs grow even faster.
  • Adaptive or arm-dependent learning rates, which the paper mentions but does not pursue, could plausibly tighten the stochastic regret to match the sharpest known bound for this setting without the additive K/Delta_min term.
  • Given the empirical 20x speedup, FTPL with Pareto perturbations looks like the practical default for decoupled problems when K is large, provided the gap-dependent stochastic bound is acceptable.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper proposes Algorithm 1, a Follow-the-Perturbed-Leader policy with Pareto perturbations for the decoupled multi-armed bandit, together with a closed-form exploration distribution q_t. The central theoretical claims are Theorem 1 (O(√(KT)) regret in the adversarial regime) and Theorem 2/Corollary 3 (time-independent O(K/Δ_min) and improved gap-dependent regret in the stochastically constrained adversarial regime), achieved without the convex optimization step of Decoupled-Tsallis-INF and without the resampling step usually required by FTPL. The paper also provides numerical experiments showing improved runtime and regret relative to Decoupled-Tsallis-INF and pure-exploration baselines.

Significance. If the stochastic-regime proof is repaired, the result is a genuinely useful contribution: it gives the first BOBW FTPL policy for decoupled bandits that avoids both convex optimization and resampling, with a simple per-round procedure and detailed proofs. The paper is also careful to benchmark runtime and to demonstrate the suboptimality of naively mixing pure exploration with standard exploitation. The main proof uses standard FTPL–FTRL equivalence and self-bounding, with no evidence of circularity. However, two technical gaps in the current manuscript—one in the definition of the event D_t and one in a divergent infinite sum—mean that Theorems 2 and 3 are not fully established as written.

major comments (2)
  1. [Appendix C, Eq. (10); Section 3.4] The statement 'When D_t occurs, it implies that \hat L_{t,i*}=0' is false for K=2 at the equality boundary. Since D_t is defined by a non-strict inequality summing only over i≠i*, a suboptimal arm with zero shifted loss contributes exactly 1/2 and D_t holds even when \hat L_{t,i*}>0. Concretely, take K=2, α=3, shifted losses (0,δ) for (suboptimal, optimal), δ>0; then D_t holds but q_{t,i*} = (1/(1+η_tδ))^2 < 1 and w_{t,i*} can be made arbitrarily small, contradicting Lemma 10's w_{t,i*}≥1/(2e). Lemma 9 likewise uses q_{t,i*}=1 on D_t. Thus the proof of Theorem 2 and Corollary 3 does not cover K=2 as written. A strict threshold (<1/2) or a separate K=2 argument is needed.
  2. [Appendix C, Eq. (37) / Lemma 9] The bound \sum_{t=1}^\infty (ζ/η_t)(ζ/η_t+e) ≤ O(c^2 K^{2/α-1}) is not correct. With η_t = c K^{1/α-1/2}/√t, we have ζ/η_t = Θ(√t), so the summand is Θ(√t + t) and the infinite sum diverges. This bound is used to control the large-estimator contribution in Lemma 9 and hence the optimal-arm contribution to the stability term on D_t in Theorem 2. Even if this term is intended only on D_t, D_t can hold for all large t in the stochastic regime, so the divergence is not avoided. The proof of Theorem 2 is therefore incomplete unless (37) is replaced by a correctly convergent estimate or the argument is restructured.
minor comments (4)
  1. [Appendix C, Eq. (26)] The citation '(by (11))' appears to be a typo: the displayed inequality uses Lemma 11 (the lower bound on ∑Δ_i w_{t,i} on D_t^c), not an equation numbered (11). Please correct the cross-reference.
  2. [Lemma 9 and Appendix C] The expression '1/(1-e^{-1}) (1-e^{-1}) ζ/η_t (ζ/η_t + e)' contains a cancelling (1-e^{-1}) factor that makes the bound unnecessarily large and obscures its origin. Please state the exact form of the cited lemma from Honda et al. (2023) / Lee et al. (2024) and verify the constants.
  3. [Section 3.4] The proof sketch says 'where the last step follows from the definition of q_{t,i}, which implies ∑_i q_{t,i} ≤ ∑_i i^{-1/2-1/(2α)}'. This is correct only after relabeling arms by increasing shifted loss; please make the sorting/rank-based convention explicit here, as it is in Lemma 5.
  4. [Abstract and Section 5] The phrase 'constant regret in the stochastic regime' is standard for time-independent regret, but the bound O(K/Δ_min) is problem-dependent. Consider clarifying this to avoid confusion with a universal constant.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: the regret bounds follow from external FTPL/FTRL equivalences, Bregman-divergence decompositions, and published perturbation tail bounds, not from fitted values or self-referential definitions.

full rationale

The paper's claimed predictions are upper bounds on regret (Theorems 1, 2, and Corollary 3) derived through a standard regret decomposition (Lemma 4, adapted from Zhan et al. 2025), a stability-term bound (Lemma 5), and a penalty-term bound (Lemma 6). The exploration distribution p_t in Eq. (7) is chosen directly from the shifted-loss estimates; it is not fitted to the regret targets, and the learning-rate constant c is either set to 2 following prior work or chosen analytically, not calibrated to the theorem outputs. The stochastic argument relates q_t to w_t on the event D_t via Lemma 10, which is a lower bound on FTPL selection probabilities obtained from Pareto-perturbation CDFs; this is a derivation step, not an identity with the final regret expression. The paper does rely on several results from overlapping prior work (Lee et al. 2024; Honda et al. 2023; Lee et al. 2025), but those are published, citable proofs whose assumptions do not include the decoupled regret bound, and the core FTPL-FTRL equivalence is attributed to Abernethy et al. (2015) and Suggala & Netrapalli (2020). No fitted parameter is renamed as a prediction, and no load-bearing premise is justified solely by a same-author citation. The flagged K=2 issue with the implication "When D_t occurs, it implies \hat L_{t,i^*}=0" is a correctness gap in the stochastic proof, not a circular reduction: the implication can fail at the equality boundary, but this does not make the theorem's bound equal to an input of the algorithm.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new physical or conceptual entities. Its free parameters are hyperparameters and analysis constants (alpha, c, zeta, y, kappa), none fitted to target regret values. The axioms are standard convex-analysis facts plus the already-defined decoupled bandit model.

free parameters (5)
  • alpha (Pareto shape) = 3 (experiments); alpha in (1,3] for Theorem 2
    Chosen by hand. The theory holds for a range; the stochastic-regime proof uses alpha<=3 to bound (w_i)^(1-1/alpha) against q_i.
  • c (learning-rate constant) = 2 (experiments); c* ~= 0.128 in Corollary 3 constant optimization
    Learning rate eta_t = c K^(1/alpha - 1/2)/sqrt(t). c is not fitted to data; it only changes constants in the regret bounds, but it is a hand-chosen hyperparameter.
  • zeta (Lemma 9 threshold parameter) = 10^-1 in the constant computation of Appendix C
    Introduced to split the analysis of the optimal-arm stability contribution. It does not affect the asymptotic order but appears in constants.
  • y (D_t threshold base) = 2
    The event D_t in Eq. (10) uses y=2. A different y would change constants, and Remark 8 notes y can be optimized. This is an analysis choice, not a data-fit.
  • kappa (self-bounding coefficient) = 1/2
    In the self-bounding step the proof uses Reg = 2*Reg - Reg and sets kappa=1/2 in Remark 8. This is an analysis constant.
assumptions (4)
  • standard math FTPL can be represented as FTRL with convex potential Phi*(x) (Abernethy et al. 2015; Suggala & Netrapalli 2020).
    Used in the proof of Lemma 4 (Appendix A.1) to derive the regret decomposition via Bregman divergences.
  • standard math Bregman divergences are nonnegative and satisfy the four-point identity DPhi(x,y)+DPhi(z,x)-DPhi(z,y) = <grad Phi(x)-grad Phi(y), x-z>.
    Used in Lemma 4's proof to convert FTPL dynamics into a stability-plus-penalty decomposition.
  • domain assumption In the stochastically constrained adversarial regime, the pseudo-regret is Reg(T)=E[sum_t sum_{i != i*} Delta_i w_{t,i}] (Eq. (2)).
    This is the defining model assumption that lets the proof use gap-weighted exploitation probabilities.
  • standard math The self-bounding technique of Zimmert & Seldin (2021) can be applied to convert a bound in terms of w_i^(1-1/alpha) into a bound in terms of Delta_i^(1-alpha) t^(-alpha/2).
    Used in Theorem 2's proof and Corollary 3; it is an established proof technique from the cited literature.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Follow-the-Perturbed-Leader for Decoupled Bandits: Best-of-Both-Worlds and Practicality." pith.science (2026). https://pith.science/paper/RN6ZNQFL

@misc{pith2026251012152,
  author       = {Pith},
  title        = {Pith review of: Follow-the-Perturbed-Leader for Decoupled Bandits: Best-of-Both-Worlds and Practicality},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RN6ZNQFL}},
  note         = {Machine review of arXiv:2510.12152}
}
abstract

We study the decoupled multi-armed bandit problem, where the learner separately selects one arm for exploration and one, possibly different, arm for exploitation at each round. In this setting, the loss of the explored arm is observed but not incurred, whereas the loss of the exploited arm is incurred without being observed. We propose an efficient Follow-the-Perturbed-Leader (FTPL) policy that achieves Best-of-Both-Worlds (BOBW) guarantee with constant regret in the stochastic regime and optimal $O(\sqrt{KT})$ regret in the adversarial regime. A key feature of our method is that it completely avoids both the convex optimization required by prior BOBW policies and the resampling procedures typically used in FTPL bandit policies. This allows FTPL to fully realize its computational efficiency advantages, leading to substantial reductions in computational cost. We empirically confirm that our policy not only improves the runtime but also demonstrates superior regret performance in both regimes.

Figures

Figures reproduced from arXiv: 2510.12152 by the authors.

Figure 1
Figure 1. Adversarial regret with ∆ = 0.125 2 2 2 4 2 6 2 8 Number of Arms (K) 0.00 0.25 0.50 0.75 1.00 Average Runtime per Step (ms) FTRL FTPL (Ours) [PITH_FULL_IMAGE:figures/full_fig_p009_1.png] view at source ↗
Figure 3
Figure 3. Stochastic regret with EB-TC, a pure exploration policy attaining constant cumulative regret. See Section 4 for details. 0 2000 4000 6000 8000 10000 Time Step 0 1000 2000 3000 4000 Regret FTRL (Mixed) FTPL (Mixed) FTRL FTPL (Ours) [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figure 5
Figure 5. Adversarial regret with ∆ = 0.0625 2 2 2 4 2 6 Number of Arms (K) 250 500 750 1000 1250 Runtime Ratio per Step FTRL / FTPL [PITH_FULL_IMAGE:figures/full_fig_p029_5.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

36 extracted references · 4 linked inside Pith

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    Fighting bandits with a new kind of smoothness

    Jacob Abernethy, Chansoo Lee, and Ambuj Tewari. Fighting bandits with a new kind of smoothness. In Advances in Neural Information Processing Systems, volume 28, 2015

    2015

  3. [3]

    The nonstochastic multiarmed bandit problem

    Peter Auer, Nicolo Cesa-Bianchi, Yoav Freund, and Robert E Schapire. The nonstochastic multiarmed bandit problem. SIAM Journal on Computing, 32 0 (1): 0 48--77, 2002

    2002

  4. [4]

    Decoupling exploration and exploitation in multi-armed bandits

    Orly Avner, Shie Mannor, and Ohad Shamir. Decoupling exploration and exploitation in multi-armed bandits. In International Conference on Machine Learning, pp.\ 1107--1114, 2012

    2012

  5. [5]

    Bandit algorithms for e-commerce recommender systems

    Bj \"o rn Brod \'e n, Mikael Hammar, Bengt J Nilsson, and Dimitris Paraschakis. Bandit algorithms for e-commerce recommender systems. In The ACM Conference on Recommender Systems, pp.\ 349--349, 2017

    2017

  6. [6]

    Five miracles of mirror descent, 2019

    S \'e bastien Bubeck. Five miracles of mirror descent, 2019. URL https://hdpa2019.sciencesconf.org/data/pages/bubeck_hdpa.pdf. Lecture note of HDPA-2019

    2019

  7. [7]

    Regret analysis of stochastic and nonstochastic multi-armed bandit problems

    S \'e bastien Bubeck and Nicolo Cesa-Bianchi. Regret analysis of stochastic and nonstochastic multi-armed bandit problems. Foundations and Trends in Machine Learning , 5 0 (1): 0 1--122, 2012

    2012

  8. [8]

    The best of both worlds: Stochastic and adversarial bandits

    S \'e bastien Bubeck and Aleksandrs Slivkins. The best of both worlds: Stochastic and adversarial bandits. In Annual Conference on Learning Theory, volume 23, pp.\ 42.1--42.23. PMLR, 2012

    2012

Show all 36 references
  1. [9]

    A survey of online experiment design with the stochastic multi-armed bandit

    Giuseppe Burtini, Jason Loeppky, and Ramon Lawrence. A survey of online experiment design with the stochastic multi-armed bandit. arXiv preprint arXiv:1510.00757, 2015

    2015 arXiv

  2. [10]

    Optimization of epsilon-greedy exploration

    Ethan Che, Hakan Ceylan, James McInerney, and Nathan Kallus. Optimization of epsilon-greedy exploration. arXiv preprint arXiv:2506.03324, 2025

    2025 arXiv

  3. [11]

    Geometric resampling in nearly linear time for Follow-the-Perturbed-Leader with Best-of-Both-Worlds guarantee in bandit problems

    Botao Chen, Jongyeong Lee, and Junya Honda. Geometric resampling in nearly linear time for Follow-the-Perturbed-Leader with Best-of-Both-Worlds guarantee in bandit problems. In International Conference on Machine Learning, 2025

    2025

  4. [12]

    Action elimination and stopping conditions for the multi-armed bandit and reinforcement learning problems

    Eyal Even-Dar, Shie Mannor, and Yishay Mansour. Action elimination and stopping conditions for the multi-armed bandit and reinforcement learning problems. The Journal of Machine Learning Research, 7 0 (39): 0 1079--1105, 2006

    2006

  5. [13]

    On explore-then-commit strategies

    Aur \'e lien Garivier, Tor Lattimore, and Emilie Kaufmann. On explore-then-commit strategies. Advances in Neural Information Processing Systems, 29, 2016

    2016

  6. [14]

    Follow-the-Perturbed-Leader achieves Best-of-Both-Worlds for bandit problems

    Junya Honda, Shinji Ito, and Taira Tsuchiya. Follow-the-Perturbed-Leader achieves Best-of-Both-Worlds for bandit problems. In International Conference on Algorithmic Learning Theory, volume 201, pp.\ 726--754. PMLR, 2023

    2023

  7. [15]

    Adaptive learning rate for Follow-the-Regularized-Leader : Competitive analysis and Best-of-Both-Worlds

    Shinji Ito, Taira Tsuchiya, and Junya Honda. Adaptive learning rate for Follow-the-Regularized-Leader : Competitive analysis and Best-of-Both-Worlds . In Annual Conference on Learning Theory, volume 247, pp.\ 2522--2563. PMLR, 2024

    2024

  8. [16]

    Improved Best-of-Both-Worlds guarantees for multi-armed bandits: FTRL with general regularizers and multiple optimal arms

    Tiancheng Jin, Junyan Liu, and Haipeng Luo. Improved Best-of-Both-Worlds guarantees for multi-armed bandits: FTRL with general regularizers and multiple optimal arms. In Advances in Neural Information Processing Systems, volume 36, pp.\ 30918--30978, 2023

    2023

  9. [17]

    An -best-arm identification algorithm for fixed-confidence and beyond

    Marc Jourdan, R \'e my Degenne, and Emilie Kaufmann. An -best-arm identification algorithm for fixed-confidence and beyond. In Advances in Neural Information Processing Systems, volume 36, pp.\ 16578--16649, 2023

    2023

  10. [18]

    On the optimality of perturbations in stochastic and adversarial multi-armed bandit problems

    Baekjin Kim and Ambuj Tewari. On the optimality of perturbations in stochastic and adversarial multi-armed bandit problems. In Advances in Neural Information Processing Systems, volume 242, pp.\ 2695--2704, 2019

    2019

  11. [19]

    Follow-the-Perturbed-Leader with F r\'echet-type tail distributions: Optimality in adversarial bandits and best-of-both-worlds

    Jongyeong Lee, Junya Honda, Shinji Ito, and Min-hwan Oh. Follow-the-Perturbed-Leader with F r\'echet-type tail distributions: Optimality in adversarial bandits and best-of-both-worlds. In Conference on Learning Theory, volume 247, pp.\ 3375--3430. PMLR, 2024

    2024

  12. [20]

    Revisiting Follow-the-Perturbed-Leader with unbounded perturbations in bandit problems

    Jongyeong Lee, Junya Honda, Shinji Ito, and Min hwan Oh. Revisiting Follow-the-Perturbed-Leader with unbounded perturbations in bandit problems. In Advances in Neural Information Processing Systems, 2025

    2025

  13. [21]

    Schwartz, and Jacob Abernethy

    Kanishka Misra, Eric M. Schwartz, and Jacob Abernethy. Dynamic online pricing with incomplete information using multiarmed bandit experiments. Marketing Science, 38 0 (2): 0 226--252, 2019

    2019

  14. [22]

    Low-rank bandit methods for high-dimensional dynamic pricing

    Jonas W Mueller, Vasilis Syrgkanis, and Matt Taddy. Low-rank bandit methods for high-dimensional dynamic pricing. In Advances in Neural Information Processing Systems, volume 32, 2019

    2019

  15. [23]

    Importance weighting without importance weights: An efficient algorithm for combinatorial semi-bandits

    Gergely Neu and G \'a bor Bart \'o k. Importance weighting without importance weights: An efficient algorithm for combinatorial semi-bandits. Journal of Machine Learning Research, 17 0 (1): 0 5355--5375, 2016

    2016

  16. [24]

    Conic optimization via operator splitting and homogeneous self-dual embedding

    Brendan O'Donoghue, Eric Chu, Neal Parikh, and Stephen Boyd. Conic optimization via operator splitting and homogeneous self-dual embedding. Journal of Optimization Theory and Applications, 169 0 (3): 0 1042--1068, 2016

    2016

  17. [25]

    NIST handbook of mathematical functions hardback and CD-ROM

    Frank WJ Olver. NIST handbook of mathematical functions hardback and CD-ROM. Cambridge university press, 2010

    2010

  18. [26]

    Tsallis-inf for decoupled exploration and exploitation in multi-armed bandits

    Chlo \'e Rouyer and Yevgeny Seldin. Tsallis-inf for decoupled exploration and exploitation in multi-armed bandits. In Conference on Learning Theory, volume 125, pp.\ 3227--3249. PMLR, 2020

    2020

  19. [27]

    Follow the perturbed leader: Optimism and fast parallel algorithms for smooth minimax games

    Arun Suggala and Praneeth Netrapalli. Follow the perturbed leader: Optimism and fast parallel algorithms for smooth minimax games. Advances in Neural Information Processing Systems, 33: 0 22316--22326, 2020

    2020

  20. [28]

    Stability-penalty-adaptive follow-the-regularized-leader: Sparsity, game-dependency, and best-of-both-worlds

    Taira Tsuchiya, Shinji Ito, and Junya Honda. Stability-penalty-adaptive follow-the-regularized-leader: Sparsity, game-dependency, and best-of-both-worlds. Advances in Neural Information Processing Systems, 36: 0 47406--47437, 2023

    2023

  21. [29]

    More adaptive algorithms for adversarial bandits

    Chen-Yu Wei and Haipeng Luo. More adaptive algorithms for adversarial bandits. In Conference on Learning Theory, volume 75, pp.\ 1--29. PMLR, 2018

    2018

  22. [30]

    Follow-the-perturbed-leader approaches best-of-both-worlds for the m-set semi-bandit problems

    Jingxin Zhan, Yuchen Xin, Chenjie Sun, and Zhihua Zhang. Follow-the-perturbed-leader approaches best-of-both-worlds for the m-set semi-bandit problems. arXiv preprint arXiv:2504.07307, 2025

    2025 arXiv

  23. [31]

    Sim-to-real transfer in deep reinforcement learning for robotics: a survey

    Wenshuai Zhao, Jorge Pe \ n a Queralta, and Tomi Westerlund. Sim-to-real transfer in deep reinforcement learning for robotics: a survey. In 2020 IEEE symposium series on computational intelligence (SSCI), pp.\ 737--744. IEEE, 2020

    2020

  24. [32]

    Large-scale bandit approaches for recommender systems

    Qian Zhou, XiaoFang Zhang, Jin Xu, and Bin Liang. Large-scale bandit approaches for recommender systems. In International Conference on Neural Information Processing, pp.\ 811--821. Springer, 2017

    2017

  25. [33]

    Tsallis- INF : A n optimal algorithm for stochastic and adversarial bandits

    Julian Zimmert and Yevgeny Seldin. Tsallis- INF : A n optimal algorithm for stochastic and adversarial bandits. The Journal of Machine Learning Research, 22 0 (1): 0 1310--1358, 2021

    2021

  26. [34]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  27. [35]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  28. [36]

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

    2021 arXiv

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.