Best Arm Identification with Possibly Biased Offline Data

Le Yang; Vincent Y. F. Tan; Wang Chi Cheung

REVIEW 3 major objections 5 minor 2 cited by

Best Arm Identification with Possibly Biased Offline Data

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read With a valid bound on the offline-online bias, the LUCB-H algorithm matches pure online sample complexity when historical data misleads and improves on it when historical data helps.

desk verdict Worth sending to review, but Theorem 4.1 needs a real time-uniform concentration fix before the main upper bound can stand. read the letter →

arxiv 2505.23165 v1 pith:DHQFF3KL submitted 2025-05-29 cs.LG cs.AIcs.ITmath.IT

Le Yang , Vincent Y. F. Tan , Wang Chi Cheung This is my paper

classification cs.LGcs.AIcs.ITmath.IT MSC 62L05

keywords bestarmidentificationofflinedatadistributionshiftbiasboundfixedconfidencesamplecomplexitymulti-armedbanditsLUCBalgorithm

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether historical (offline) reward data can safely speed up best-arm identification when that data may come from a different distribution than the current online rewards. It first shows an impossibility result: without an upper bound on the distributional shift, no adaptive algorithm can both exploit helpful offline data and ignore misleading offline data across all instances. It then proposes LUCB-H, which takes such a bias bound $V(i)$ as input and, at each round, keeps the tighter of an online-only confidence interval and an offline-augmented confidence interval. The paper claims that LUCB-H's expected stopping time matches standard LUCB when the offline data is misleading and strictly improves on it when the per-arm discrepancy satisfies $\eta(i) < \Delta_i/4$. If true, this gives a practical way to warm-start fixed-confidence best-arm identification without risking worse-than-online performance, provided a valid bias bound is available.

What carries the argument

The load-bearing object is the mixed confidence interval built from two estimators: the online sample mean $\hat Y_t(i)$ with radius $\sqrt{2\log(kt/\delta)/N_t(i)}$, and the pooled estimator $\frac{N_t(i)\hat Y_t(i)+TS(i)\hat X(i)}{N_t(i)+TS(i)}$ with radius $\sqrt{\frac{2\log(kt/\delta)}{N_t(i)+TS(i)}} + \frac{TS(i)}{N_t(i)+TS(i)}\cdot V(i)$. Taking the minimum for upper bounds and the maximum for lower bounds gives the tightest valid interval at every round; the bias bound $V(i)$ converts the unknown shift $|\mu_{\mathrm{off}}(i)-\mu_{\mathrm{on}}(i)|$ into a worst-case correction, and the discrepancy measure $\eta(i)=V(i)+\mu_{\mathrm{off}}(i)-\mu_{\mathrm{on}}(i)$ decides when the offline-augmented bound is active. When $\eta(i) \le \Delta_i/4$ the offline data saves samples; when $\eta(i)>\Delta_i/4$ the offline bound is looser than the online one, so it is effectively discarded.

What would settle it

Run LUCB-H on a fixed two-arm instance with known Gaussian means and a valid bias bound, and compare the empirical failure probability at the stopping time $\tau_\delta$ against $\delta$. If the confidence intervals are not time-uniform as claimed, repeated runs at a horizon of order $\sum \Delta_i^{-2}\log(1/\delta)$ will show the error probability exceeding $\delta$; the missing union bound in Lemma C.1 would be exposed by checking the maximum deviation of $\hat Y_t(i)$ across all $t$ rather than at a single $t$.

Watch

Extended reading notes

Core claim

LUCB-H identifies the best arm with probability at least $1-\delta$ while adaptively deciding per arm whether the historical data is worth using. The algorithm maintains two confidence systems: standard online LUCB bounds and offline-augmented bounds whose radius includes a bias-correction term $\frac{TS(i)}{N_t(i)+TS(i)}\cdot V(i)$. At round $t$ it forms $\mathrm{UCB}_{\mathrm{mix}}(i)=\min\{\mathrm{UCB}_t(i),\mathrm{UCB}^S_t(i)\}$ and $\mathrm{LCB}_{\mathrm{mix}}(i)=\max\{\mathrm{LCB}_t(i),\mathrm{LCB}^S_t(i)\}$. Theorem 4.1 states that with a valid bias bound $V(i) \ge |\mu_{\mathrm{off}}(i)-\mu_{\mathrm{on}}(i)|$, the expected stopping time is $O\left(\sum_{\Delta_i>0}\left(\Delta_i^{-2}\log(1/\delta) - TS(i)\cdot\max\left\{1-\frac{4\eta(i)}{\Delta_i},0\right\}\right)\right)$, where $\eta(i)=V(i)+\mu_{\mathrm{off}}(i)-\mu_{\mathrm{on}}(i)$. Thus the sample complexity is never worse in order than LUCB without offline data, and it is strictly smaller exactly when $\eta(i)<\Delta_i/4$; the paper also proves an instance-dependent lower bound with a saving term $TS(i)\cdot\max\left\{\frac{\mu_{\mathrm{off}}(1)-\mu_{\mathrm{off}}(i)}{\Delta_i},0\right\}^2$ and shows the two saving terms match in some special cases.

Load-bearing premise

The main performance theorem assumes that the algorithm's uncertainty intervals remain correct simultaneously at every time step; the proof shows correctness only at one time step with high probability, and the missing step that bridges 'one step' to 'all steps' is load-bearing.

Editorial extensions

If this is right

When the offline distribution equals the online one, the saving term reaches $TS(i)$ and LUCB-H recovers the batch Track-and-Stop sample complexity.
When offline means are far enough that $\eta(i) \ge \Delta_i/4$ for every suboptimal arm, LUCB-H reverts to exactly the standard LUCB bound, so a practitioner never pays for carrying bad history.
For instances with equal bias bounds across arms, the gap between the upper bound's saving term and the lower bound's saving term is provably nonnegative, so in that regime LUCB-H sits within the lower-bound constant.
The impossibility result means any practical system that wants to exploit offline data without a bias bound must first estimate a valid upper bound on the shift; the bound is an input, not an output, of the algorithm.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Beyond the paper, the min/max mixing rule suggests a general recipe for fixed-confidence BAI: run two confidence systems, one online-only and one history-augmented, and take the tighter bound; this could be ported to successive-elimination or Track-and-Stop variants.
A natural testable extension is to replace the known-variance Gaussian assumption with bounded or sub-Gaussian rewards; the saving-term logic should survive once variance proxies replace the unit-variance radii.
Appendix E's finding that underestimating $V(i)$ for the best arm degrades performance implies a practical design principle: budget the bias bound conservatively for the empirically best-looking arms, because those are the arms where the offline data can do the most harm.
The lower bound's saving term suggests an arm-by-arm inclusion criterion for offline data: use arm $i$'s history only when $\mu_{\mathrm{off}}(1)-\mu_{\mathrm{off}}(i)$ is positive, a quantity that could be estimated directly from offline moments in applications.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: this paper deserves review, and the main idea is sound, but the proof of Theorem 4.1 has a real gap. It should not be accepted as-is; it needs a time-uniform concentration argument first.

The genuinely new part is the BAI problem with potentially biased offline data and a known bias bound. The impossibility result — no adaptive policy can exploit offline data without an upper bound on bias — is a clean transfer of the Cheung–Lyu regret tradeoff to BAI, and it is stated and argued correctly. LUCB-H is a natural combination of LUCB confidence intervals with bias-corrected historical estimates; the saving term in the upper bound and the matching lower bound give a believable instance-dependent picture. The experiments, while not extensive, are consistent with the theory and show the algorithm discards misleading offline data. The lower-bound proof is a standard change-of-measure argument and appears correct. Citation practice looks fair; the debt to Cheung–Lyu is explicit and the BAI analysis is distinct.

The soft spot is Lemma C.1, and it is load-bearing. The lemma asserts that certain confidence-bound events hold simultaneously for all rounds with probability at least 1 − δ/k. The proof applies the Chernoff bound at one round and unions only over arms. If you try to union over all t with the stated radius sqrt(2 log(kt/δ)/N_t), the failure probabilities sum to infinity, so the simultaneous clean event is not established. Since Theorem 4.1 relies on that clean event to bound the number of pulls per arm, the main upper bound does not follow as written. This looks fixable — use a time-uniform confidence radius such as sqrt(2 log(k t^2/δ)/N_t) or a peeling argument — but the resulting bound may pick up extra log-type factors. The theorem statement could need amendment. I did not find other flaws of comparable size; the impossibility result and lower bound stand independently.

For whom: anyone working on fixed-confidence BAI or offline-to-online transfer. The problem is well motivated, the algorithm is simple, and the theory is close to complete. My recommendation for peer review: send it out, but make the referee focus on Lemma C.1 and require either a correct time-uniform proof or an honest restatement of the sample complexity.

Referee Report

3 major / 5 minor

Summary. The paper studies fixed-confidence best-arm identification when the learner is given offline (historical) samples whose distributions may differ from the online reward distributions. It first claims an impossibility result (Proposition 3.1) showing that no delta-PAC policy can both exploit helpful offline data and ignore misleading offline data unless it knows a bound on the offline-online mean shift. It then proposes LUCB-H, a modification of LUCB that maintains two confidence intervals per arm, one from online data and one from a bias-corrected mixture of offline and online data, and takes the tighter of the two. Theorem 4.1 claims an expected stopping-time bound matching standard LUCB plus a saving term when the bias is small, and Theorem 5.1 gives an instance-dependent lower bound. Numerical experiments illustrate the claimed behavior.

Significance. The problem is timely and the algorithmic idea is natural: LUCB-H is a clean way to make offline data useful without risking unbounded degradation when the offline data are misleading. If the proof gaps were repaired, the paper would make a useful contribution to the fixed-confidence BAI literature and would extend the bias-bound framework of Cheung and Lyu (2024) from regret minimization to BAI. The paper does not ship code or machine-checked proofs, so the contribution rests on the mathematical arguments; at present the two central arguments are incomplete.

major comments (3)

[Appendix C, Lemma C.1] The lemma asserts that the confidence-bound events hold simultaneously for all rounds t=1,2,... with probability at least 1-delta/k, and this time-uniform event is used in the proof of Theorem 4.1 to conclude that no arm is pulled after its counter exceeds tau_i. The proof, however, only establishes (at best) the event for a fixed t: the Chernoff bound is applied to the event over n=1,...,t for that t, and then a union bound is taken over the k arms. No union over t is performed. This is not a minor omission: with the stated radius sqrt(2 log(kt/delta)/n), a union over t would have failure probability sum_{t>=1} delta/(kt)=infinity, so the claimed simultaneous event is not proven. The derivation of E[tau_delta]=O(sum Delta_i^{-2} log(1/delta) - Savu(i)) therefore does not follow. A standard repair is to use a time-uniform radius such as sqrt(2 log(k t^2/delta)/N_t(i)), possibly with a peeling argument, and then re-derive the sample-complexity bound; the O(log(1/delta)) rate may survive, but the theorem statement must be re-verified.
[Appendix B, Proposition 3.1] The proof uses the inequality d(1-delta,delta/2) > delta^{-epsilon'} d(1-delta,delta), claimed to follow from monotonicity of the binary relative entropy. This is false for any fixed epsilon'>0 as delta tends to 0: d(1-delta,delta/2) is asymptotic to log(1/delta) and d(1-delta,delta) is asymptotic to log(1/delta), so the ratio tends to 1, not to infinity. Consequently the displayed bound E_sigma(Q,2) >= 2 delta^{-2 beta - epsilon'} log(1/(2.4 delta)) does not follow from the preceding lines. In addition, the subtraction term TS(2) KL(Qoff(2),Poff(2)) / KL(Qon(2),Pon(2)) is not bounded by delta^{-2 beta - epsilon'} log(1/delta) as the proof asserts: with the constructed Qoff(2), KL(Qoff(2),Poff(2)) is of order delta^{-2 beta - epsilon}, KL(Qon(2),Pon(2)) is of order delta^{2 beta}, and TS(2) can be of order delta^{-2 beta}, making the quotient of order delta^{-6 beta - epsilon} up to constants, which swamps the leading term. Thus Proposition 3.1, the paper's impossibility result, is not established by the submitted proof.
[Theorem 4.1, Eq. (4.5)] The statement E[tau_delta]=O(sum_{Delta_i>0}(Delta_i^{-2} log(1/delta) - Savu(i))) is not meaningful when the saving term exceeds the log term: for example, if eta(i)=0 and TS(i) is very large, the right-hand side is negative, but a stopping time cannot be negative. The bound should be written with max{...,0} inside the sum, or with an explicit cap on Savu(i), as is presumably intended in the proof's derivation of a lower bound on N_t(i).

minor comments (5)

[Algorithm 1, line 15] The notation 'Nt+1(i) = Nt(i) + /BD {i in {ht, lt}}' uses an undefined symbol; it should be written with the indicator function 1{...}.
[Section 6] The sentence 'We set the confidence levels at delta = 0.1, 0.12, ..., 0.110' appears to contain a typo; the range and step are unclear from the text and should be corrected.
[Lemma C.1 proof] Even for a fixed t, the proof derives two one-sided inequalities each with probability at least 1-delta/k, so their intersection has probability at least 1-2delta/k; the claimed failure probability 1-delta/k needs an additional argument or an absolute-value concentration bound.
[Appendix D] In the definition of Savl(i), the square is written outside the max term without parentheses: TS(i) max{...}^2; adding parentheses would remove ambiguity about the scope of the square.
[General notation] The paper uses 'delta-PAC' without defining the acronym at first use, and the symbol 'P^2' in Assumption 5.1 is not explained; clarifying these would help the reader.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular reduction found; Theorem 4.1 and 5.1 are derived from the algorithm's own confidence bounds and a KL change-of-measure, with only a contextual self-citation to Cheung–Lyu (2024).

full rationale

Walking the claimed derivation chain, Theorem 4.1's saving term Savu(i) = TS(i)·max{1 − 4η(i)/Δi, 0} is not a fitted input or a relabeled quantity; it is solved out in Appendix C from the algorithm's own stopping inequality min{2√(2 log(kt/δ))/(Nt(i)+TS(i)) + (TS(i)/(Nt(i)+TS(i)))·η(i), 2√(2 log(kt/δ))/Nt(i)} ≤ Δi/2, with η(i) = V(i) + μoff(i) − μon(i) the deterministic bias of the pooled estimator. The 'helpful when η(i) < Δi/4' threshold is a byproduct of this same algebra, not an assumption chosen to make the bound come out. Theorem 5.1's lower-bound saving Savl(i) = TS(i)·max{(μoff(1) − μoff(i))/Δi, 0}^2 is derived independently by a change-of-measure argument (Appendix D, Lemma A.1), and the matching discussion in Remark 5.1 is algebra on the two formulas, explicitly leaving the general gap unproved ('we leave the proof of this to future work'). No parameter is estimated from data and then 'predicted': V(i) is a declared input, both proofs carry their own concentration and transport arguments, and the experiments simulate the same algorithm rather than fitting the theory. The one self-citation, Cheung and Lyu (2024) (coauthored by W. C. Cheung), motivates the bias-bound framework in related work and is not a black box in the proofs of Proposition 3.1, Theorem 4.1, or Theorem 5.1. What the paper actually suffers from are correctness gaps, which are not circularity: Lemma C.1 claims the confidence events hold for all rounds t with probability ≥ 1 − δ/k, but the proof unions only over arms for a fixed t (the per-round failure δ/(kt) would sum to infinity over t), so the time-uniform clean event is unproven and the stated O(Σ Δ^{-2} log(1/δ)) complexity does not follow as written; Theorem 4.1 also omits a positive-part on the whole bracket so the bound can be formally negative; and the general gap(i) ≥ 0 is conceded to be unproved. Appendix E's honest admission that an underestimated V can cause underperformance is a limitation of the method, not a circular step. These are soundness issues, not reductions to inputs.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central derivation rests on the Gaussian known-variance assumption and on the availability of a valid bias bound V. The proof also implicitly assumes a time-uniform concentration result that is not established. No new physical or mathematical entities are introduced; LUCB-H is an algorithm, not an entity.

free parameters (1)

Bias bound V(i) = External input; must satisfy V(i) >= |mu_off(i)-mu_on(i)|
LUCB-H takes V(i) as an input; the saving term and the decision to pool offline data depend on it. The paper does not estimate V from data, and Appendix E shows that an invalid (too small) V breaks the robustness guarantee.

assumptions (4)

domain assumption Online and offline rewards are Gaussian with known variance; experiments use unit variance.
Section 2 states P_on and P_off are normal distributions with known variance. This justifies the sub-Gaussian confidence intervals and the KL computations in the lower bound.
domain assumption A finite valid bias bound V(i) >= |mu_off(i) - mu_on(i)| is known to the decision maker before the online phase.
Introduced in Section 3 after Proposition 3.1. LUCB-H's correctness and saving term depend on this bound; Appendix E shows underestimation can make LUCB-H underperform the online-only baseline.
standard math Assumption 5.1: the distribution family allows local perturbations of the KL divergence in either mean direction.
Used in Appendix D to construct alternative bandit models for the lower bound. This is a standard change-of-measure assumption in the BAI literature.
ad hoc to paper Lemma C.1: the confidence intervals contain the true online mean uniformly over all rounds t with probability at least 1-delta/k.
The provided proof only controls failure at a fixed t with probability delta/(kt), and the union over all t diverges. Theorem 4.1's proof relies on this unproven uniform event.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Best Arm Identification with Possibly Biased Offline Data." pith.science (2026). https://pith.science/paper/DHQFF3KL

@misc{pith2026250523165,
  author       = {Pith},
  title        = {Pith review of: Best Arm Identification with Possibly Biased Offline Data},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DHQFF3KL}},
  note         = {Machine review of arXiv:2505.23165}
}

read the original abstract

We study the best arm identification (BAI) problem with potentially biased offline data in the fixed confidence setting, which commonly arises in real-world scenarios such as clinical trials. We prove an impossibility result for adaptive algorithms without prior knowledge of the bias bound between online and offline distributions. To address this, we propose the LUCB-H algorithm, which introduces adaptive confidence bounds by incorporating an auxiliary bias correction to balance offline and online data within the LUCB framework. Theoretical analysis shows that LUCB-H matches the sample complexity of standard LUCB when offline data is misleading and significantly outperforms it when offline data is helpful. We also derive an instance-dependent lower bound that matches the upper bound of LUCB-H in certain scenarios. Numerical experiments further demonstrate the robustness and adaptability of LUCB-H in effectively incorporating offline data.

Figures

Figures reproduced from arXiv: 2505.23165 by the authors.

**Figure 6.** Evolution of E[τδ] with V in group 2 Arm 1 Arm 2 Arm 3 Arm 4 Arm 5 Arms 0 100 200 300 400 500 600 700 Samples Pure LUCB LUCB-H (misleading) LUCB-H (beneficial) LUCB-H (partial) [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗

**Figure 8.** Samples for different arms when δ = 0.01 in group 2 (a) Case 1 (Misleading Bias): In this case, all arms are adversely affected. Thus, the offline mean for the best arm decreases while the means for the suboptimal arms increase. The offline means in Group 1 are (0.4, 0.6, 0.6, 0.6, 0.6), resulting in the offline phase identifying a best arm that is not arm 1. The offline means in Group 2 are (0.4, 0.8, 0.7, 0.6, 0.… view at source ↗

Figures from the paper (1 more)

**Figure 10.** Figure 10: Evolution of E[τδ] with V in group 2. In particular, on the bandit model v ′ the best arm is no longer arm 1. Introducing the event E = { ˆI ∗ = 1} ∈ Fτδ , any δ-PAC algorithm satisfies Pτδ (v, E) ≥ 1 − δ and Pτδ (v ′ , E) ≤ δ. By Lemma A.1, we have X k i=1 Ev[Nτδ (i)…

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Decentralized Relaxed Smooth Optimization with Gradient Descent Methods
math.OC 2025-08 unverdicted novelty 6.0 of 10

A decentralized gradient descent method with adaptive clipping is claimed to reach best-known convergence rates for convex and nonconvex problems under (L0,L1)-smoothness without knowing the constants.
Using causal abstractions to accelerate decision-making in complex bandit problems
cs.LG 2025-09 conditional novelty 5.0 of 10

AT-UCB uses a cheap abstracted causal model to filter out suboptimal actions before running UCB on the expensive base model, with a regret bound that improves when the abstraction is accurate.

Reference graph

Works this paper leans on

12 extracted references · 4 canonical work pages · cited by 2 Pith papers

[1]

Optimal best-arm iden- tiﬁcation in bandits with access to ofﬂine data

Shubhada Agrawal, Sandeep Juneja, Karthikeyan Shan- mugam, and Arun Sai Suggala. Optimal best-arm iden- tiﬁcation in bandits with access to ofﬂine data. arXiv preprint arXiv:2306.09048,

arXiv
[2]

0.1 0.125 0.15 0.175 0.2 0.225 0 2000 4000 6000 8000 10000 pure LUCB LUCB-H (misleading) LUCB-H (beneficial) Figure 10: Evolution of E[τδ ] with V in group

work page 2000
[9]

Warm-starting con- textual bandits: Robustly combining supervised and ban- dit feedback

Chicheng Zhang, Alekh Agarwal, Hal Daumé III, John Langford, and Sahand N Negahban. Warm-starting con- textual bandits: Robustly combining supervised and ban- dit feedback. arXiv preprint arXiv:1901.00301 ,

arXiv 1901
[10]

Let IP = ( P oﬀ , P on) and IQ = ( Qoﬀ , Qon) be two problem instances deﬁned over a common ﬁnite arm set A

Let A = ((πt), τ, ˆI ∗) be a δ-P AC algorithm. Let IP = ( P oﬀ , P on) and IQ = ( Qoﬀ , Qon) be two problem instances deﬁned over a common ﬁnite arm set A. Both instances share the same ofﬂine sample sizes {TS(i)}i∈A and conﬁdence level δ, but may differ in their ofﬂine and online reward distributions. Let τδ(Q, i) be the (random) number of online samples...

work page 2016
[11]

, k}, from Assumption 5.1 there exists an alternative model v′ = (v1,

For all i ∈ {1, 2, . . . , k}, from Assumption 5.1 there exists an alternative model v′ = (v1, . . . , vi−1, v′ i, vi+1, . . . , vk) in which the only arm modiﬁed is arm i, i ⁄= 1 and v′ i is such that: KL(vi, v1) < KL(vi, v′ i) < KL(vi, v1) + a, µ on′ (i) > µ on′ (1) and µoﬀ ′ (i) = { µoﬀ (i), if µoﬀ (i) > µ oﬀ (1) µoﬀ (1), if µoﬀ (i) < µ oﬀ (1). 0.1 0.1...

work page 2000
[2016]

Ofﬂine learning for combinatorial multi-armed bandits

Xutong Liu, Xiangxiang Dai, Jinhang Zuo, Siwei Wang, Carlee-Joe Wong, John Lui, and Wei Chen. Ofﬂine learning for combinatorial multi-armed bandits. arXiv preprint arXiv:2501.19300,

arXiv
[2019]

Optimal exploitation of clus- tering and history information in multi-armed bandit

Djallel Bouneffouf, Srinivasan Parthasarathy, Horst Samu - lowitz, and Martin Wistub. Optimal exploitation of clus- tering and history information in multi-armed bandit. arXiv preprint arXiv:1906.03979 ,

arXiv 1906
[2020]

Data-pooling reinforcement learning for personalized healthcare inter - vention

Xinyun Chen, Pengyi Shi, and Shanwen Pu. Data-pooling reinforcement learning for personalized healthcare inter - vention. arXiv preprint arXiv:2211.08998 ,

arXiv

Show all 12 references

[2021]

Combining of- ﬂine causal inference and online bandit learning for data driven decision

Li Y e, Yishi Lin, Hong Xie, and John Lui. Combining of- ﬂine causal inference and online bandit learning for data driven decision. arXiv preprint arXiv:2001.05699 ,

2001 arXiv
[2022]

Leveraging (biased) in- formation: Multi-armed bandits with ofﬂine data

Wang Chi Cheung and Lixing Lyu. Leveraging (biased) in- formation: Multi-armed bandits with ofﬂine data. arXiv preprint arXiv:2405.02594,
[2023]

Best-arm identiﬁca- tion algorithms for multi-armed bandits in the ﬁxed con- ﬁdence setting

Kevin Jamieson and Robert Nowak. Best-arm identiﬁca- tion algorithms for multi-armed bandits in the ﬁxed con- ﬁdence setting. In 2014 48th annual conference on infor- mation sciences and systems (CISS) , pages 1–6. IEEE,

2014
[2025]

Domain adaptation: Learning bounds and al- gorithms

Yishay Mansour, Mehryar Mohri, and Afshin Ros- tamizadeh. Domain adaptation: Learning bounds and al- gorithms. arXiv preprint arXiv:0902.3430 ,

arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Optimal best-arm iden- tiﬁcation in bandits with access to ofﬂine data

Shubhada Agrawal, Sandeep Juneja, Karthikeyan Shan- mugam, and Arun Sai Suggala. Optimal best-arm iden- tiﬁcation in bandits with access to ofﬂine data. arXiv preprint arXiv:2306.09048,

arXiv

[2] [2]

0.1 0.125 0.15 0.175 0.2 0.225 0 2000 4000 6000 8000 10000 pure LUCB LUCB-H (misleading) LUCB-H (beneficial) Figure 10: Evolution of E[τδ ] with V in group

work page 2000

[3] [9]

Warm-starting con- textual bandits: Robustly combining supervised and ban- dit feedback

Chicheng Zhang, Alekh Agarwal, Hal Daumé III, John Langford, and Sahand N Negahban. Warm-starting con- textual bandits: Robustly combining supervised and ban- dit feedback. arXiv preprint arXiv:1901.00301 ,

arXiv 1901

[4] [10]

Let IP = ( P oﬀ , P on) and IQ = ( Qoﬀ , Qon) be two problem instances deﬁned over a common ﬁnite arm set A

Let A = ((πt), τ, ˆI ∗) be a δ-P AC algorithm. Let IP = ( P oﬀ , P on) and IQ = ( Qoﬀ , Qon) be two problem instances deﬁned over a common ﬁnite arm set A. Both instances share the same ofﬂine sample sizes {TS(i)}i∈A and conﬁdence level δ, but may differ in their ofﬂine and online reward distributions. Let τδ(Q, i) be the (random) number of online samples...

work page 2016

[5] [11]

, k}, from Assumption 5.1 there exists an alternative model v′ = (v1,

For all i ∈ {1, 2, . . . , k}, from Assumption 5.1 there exists an alternative model v′ = (v1, . . . , vi−1, v′ i, vi+1, . . . , vk) in which the only arm modiﬁed is arm i, i ⁄= 1 and v′ i is such that: KL(vi, v1) < KL(vi, v′ i) < KL(vi, v1) + a, µ on′ (i) > µ on′ (1) and µoﬀ ′ (i) = { µoﬀ (i), if µoﬀ (i) > µ oﬀ (1) µoﬀ (1), if µoﬀ (i) < µ oﬀ (1). 0.1 0.1...

work page 2000

[6] [2016]

Ofﬂine learning for combinatorial multi-armed bandits

Xutong Liu, Xiangxiang Dai, Jinhang Zuo, Siwei Wang, Carlee-Joe Wong, John Lui, and Wei Chen. Ofﬂine learning for combinatorial multi-armed bandits. arXiv preprint arXiv:2501.19300,

arXiv

[7] [2019]

Optimal exploitation of clus- tering and history information in multi-armed bandit

Djallel Bouneffouf, Srinivasan Parthasarathy, Horst Samu - lowitz, and Martin Wistub. Optimal exploitation of clus- tering and history information in multi-armed bandit. arXiv preprint arXiv:1906.03979 ,

arXiv 1906

[8] [2020]

Data-pooling reinforcement learning for personalized healthcare inter - vention

Xinyun Chen, Pengyi Shi, and Shanwen Pu. Data-pooling reinforcement learning for personalized healthcare inter - vention. arXiv preprint arXiv:2211.08998 ,

arXiv