Pith. sign in

REVIEW 2 major objections 6 minor 60 references

Online Clustering of Dueling Bandits

T0 review · 2 major / 6 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper introduces the first clustering-of-dueling-bandits algorithms, proving that when users give only pairwise preference feedback, sharing data inside learned clusters makes expected regret scale with the number of clusters rather…

desk verdict First CB + dueling combination with plausible but incompletely verified regret bounds; worth refereeing, but the adaptive-context eigenvalue step and the outsourced CONDB lemmas are the main risk points. read the letter →

arxiv 2502.02079 v1 pith:IBPCQZG7 submitted 2025-02-04 cs.LG cs.AI

Zhiyong Wang , Jiahang Sun , Mingze Kong , Jize Xie , Qinghua Hu , John C.S. Lui , Zhongxiang Dai This is my paper · ORCID
classification cs.LGcs.AI MSC 68T0568W27
keywords clusteringofbanditsduelingpreferencefeedbackcontextualregretboundsneuralBradley-Terry-Lucemodelusercollaboration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to prove that collaborative learning is possible when the only feedback available is a binary preference between two options. It introduces the first clustering-of-dueling-bandits algorithms, COLDB (linear reward assumptions) and CONDB (neural-network rewards), which group users online into estimated clusters and pool each cluster's preference data to choose which pair of items to show next. The central result is that expected regret is bounded by $O(\frac{1}{\kappa_\mu} d \sqrt{mT})$ for COLDB and a corresponding effective-dimension bound for CONDB, where $m$ is the number of unknown user clusters rather than the total number of users $u$. Because $m$ can be far smaller than $u$, the bound formalizes why collaboration helps: a larger average cluster size gives a provably tighter regret guarantee. The work matters because real recommendation and prompt-optimization systems often solicit relative preferences instead of absolute scores, and until now no clustering-of-bandits method existed for that feedback type.

What carries the argument

The load-bearing machinery is a dynamic graph over the user set whose connected components serve as the current estimate of the clustering, updated by a threshold-based edge-deletion rule: after each round the algorithm re-estimates the served user's preference parameters and removes the edge to any user whose estimate differs by more than $f(T_{i,t}) + f(T_{\ell,t})$, where the thresholds are the widths of per-user confidence regions. Confidence comes from a self-normalized concentration bound applied to the maximum-likelihood estimate of the preference vector, giving $\|\hat\theta_{i,t} - \theta_{j(i)}\|_2 \leq \frac{\sqrt{\lambda\kappa_\mu} + \sqrt{2\log(u/\delta) + d\log(1 + 4T_{i,t}\kappa_\mu/(d\lambda))}}{\kappa_\mu \sqrt{\lambda_{\min}(V_{i,t-1})}}$. The item-regularity condition (Assumption 2.4), which makes arm features i.i.d. with full-rank covariance, is what converts the information matrix's minimum eigenvalue into a quantity growing linearly in the number of observations, fixing the time $T_0$ by which every user's estimate is within $\gamma/4$ of its cluster's true parameter and the graph provably matches the ground-truth partition. After that point the algorithm effectively runs a cluster-shared dueling bandit per cluster: the first arm is greedy, the second arm is an optimistic UCB along the difference direction $\phi(x) - \phi(x_{t,1})$, and the regret telescopes through the elliptical-potential sum of the context differences.

What would settle it

Fix $u=200$ users with $m \in \{2,5,10,20\}$ ground-truth clusters on the same context distribution and compare cumulative regret slopes after each algorithm's clustering phase: the theorem predicts curves that separate in proportion to $\sqrt{m}$ (a factor $\sqrt{10}$ between $m=2$ and $m=20$), so statistically indistinguishable slopes would refute the claimed $m$-dependence. A second check targets Assumption 2.4: draw contexts i.i.d. from a distribution whose covariance has minimal eigenvalue $\lambda_x$ near zero while keeping cluster separation $\gamma$ fixed; the full bound of Theorem 4.1 predicts a long initial regime of order $u(d/(\kappa_\mu^2\tilde\lambda_x\gamma^2) + 1/\tilde\lambda_x^2)\log T$ before the $\sqrt{mT}$ term dominates, and the proof's auxiliary conditions ($\lambda\kappa_\mu$ small, $u/\delta$ large) become testable there, so a measured delay wildly different from that prediction would show the mechanism is not what carries the bound.

Watch

Extended reading notes

Core claim

The paper's claim is that preference-only feedback does not block cross-user collaboration: both COLDB and CONDB learn the unknown user clustering online and, once clustering is correct, achieve regret that grows with the number of ground-truth user clusters $m$ instead of the number of users $u$. The regret of COLDB satisfies $R(T) = O(\frac{1}{\kappa_\mu} d \sqrt{mT})$ (Theorem 4.1), and CONDB satisfies $R(T) = O\big((\frac{\sqrt{e_d}}{\kappa_\mu} + B\sqrt{\lambda/\kappa_\mu})\sqrt{e_d m T}\big)$ (Theorem 4.2), with $e_d$ the effective dimension of the neural feature map. Both bounds decompose into a term that is logarithmic in $T$ and accounts for the rounds needed to cluster all users correctly, plus a dominant $\sqrt{T}$ term whose coefficient scales with $\sqrt{m}$. The paper reads this as a theoretical guarantee that more collaboration (fewer clusters, more users per cluster) strictly improves the bound, matching known single-user dueling-bandit rates when $m=1$; the extra $1/\kappa_\mu$ factor, where $\kappa_\mu$ is the minimum slope of the preference curve, exactly captures the added difficulty of binary relative feedback versus numeric rewards.

Load-bearing premise

The whole argument rests on Assumption 2.4: every round's arm features are drawn independently from one fixed distribution whose covariance is full-rank, so that accumulating observations always shrinks uncertainty at a guaranteed rate; if the features are chosen adversarially or the covariance is nearly degenerate, the promised clustering time and the regret bound no longer follow.

Editorial extensions

If this is right

  • With a single cluster ($m=1$), the COLDB bound reduces to $O(d\sqrt{T}/\kappa_\mu)$, matching the established single-user linear dueling bandit rate, so the clustered algorithm is never asymptotically worse than running independent dueling bandits per user.
  • Preference feedback is provably harder than numeric feedback in this setting: the extra $1/\kappa_\mu$ factor (with $\kappa_\mu < 1/4$ for the logistic link) makes the clustered-preference bounds larger than classical clustered-linear-bandit bounds, exactly quantifying the cost of binary relative feedback.
  • For systems with millions of users but relatively few behavior types, the regret shrinks as $m$ decreases, so the algorithm's guarantee improves precisely when the user population is more homogeneous.
  • Because the clustering is learned online with no prior knowledge of $m$ or the partition, the algorithms can be deployed cold-start: they spend $O\big(u(d/(\kappa_\mu^2 \tilde\lambda_x \gamma^2) + 1/\tilde\lambda_x^2)\log T\big)$ rounds identifying the partition and then harvest the collaboration benefit.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The graph-clustering skeleton is modular: the edge-deletion rule only needs per-user confidence widths, so the same scheme should transfer to variance-aware or Thompson-sampling dueling bandits and inherit any improvement in their per-user rates, plausibly retaining the $\sqrt{m}$-type dependence.
  • A testable relaxation of Assumption 2.4 would allow contexts to be non-i.i.d. but diverse, each round providing an arm difference with a bounded-below projection onto the preference subspace; if the guarantees survive that relaxation, the algorithms extend to real recommendation logs where item features repeat.
  • Applied to prompt optimization, the framework suggests a concrete deployment: group LLM users or prompt-tuning tasks by inferred reward function, present pairwise response comparisons within a cluster, and share winners across the group, with error scaling in the number of distinct user types rather than the number of prompt queries.
  • The simplified bounds drop the $T_0$ clustering term, but when clusters are close (small $\gamma$) or item covariance is weak (small $\tilde\lambda_x$), that term, not $\sqrt{mT}$, will dominate in practice; this regime is not advertised by the headline bounds.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. The paper proposes two graph-based online clustering algorithms for contextual dueling bandits: COLDB, which models user reward functions linearly, and CONDB, which models them with neural networks. At each round the served user is assigned to its connected component in a dynamically updated graph, a shared reward model is fitted on that component's preference data, the first arm is chosen greedily and the second by a UCB-type bonus on the pair difference, and edges are deleted when per-user parameter estimates differ by more than a threshold. The main theoretical claims are an expected regret bound of O((1/\kappa_\mu)d\sqrt{mT}) for COLDB (Theorem 4.1) and O((\sqrt{ed}/\kappa_\mu + B\sqrt{\lambda/\kappa_\mu})\sqrt{ed\,mT}) for CONDB (Theorem 4.2), after an initial O(u(\cdots)\log T) clustering phase, so that regret depends on the number m of ground-truth clusters rather than the number u of users. Experiments on synthetic data and MovieLens compare the two algorithms with independently run single-user dueling bandit baselines.

Significance. If the proofs are completed, this would be a useful first extension of online clustering of bandits to preference feedback, and the claimed dependence on m rather than u would give a precise sense in which user collaboration provably helps. The algorithmic framework is clearly described and the experiments support the qualitative benefit of collaboration. However, two load-bearing parts of the analysis are currently imported from prior work rather than proved in the submitted manuscript, and one of these imports concerns the adaptive, history-dependent arm-pair selection that is central to the algorithm. The central regret claim is therefore not yet established by the submitted proof, although the issues may be repairable with additional arguments or assumptions.

major comments (2)
  1. [Appendix B, Lemma B.2, Eq. (36)] The proof that all users are correctly clustered after T0 depends on the bound \lambda_{\min}(V_{i,t}) \ge 2\tilde{\lambda}_x T_{i,t}. This is obtained by invoking Lemma J.1 of Wang et al. (2024a) and Lemma 7 of Li & Zhang (2018), but the matrix V_{i,t} is built from the vectors z_s = \phi(x_{s,1}) - \phi(x_{s,2}), where x_{s,1} is the greedy maximizer and x_{s,2} is the UCB maximizer at round s. Neither external lemma is stated, and the paper does not show that it applies to this adaptively selected, history-dependent sequence rather than to independent draws from the context distribution \rho. Since T0, the clustering guarantee, and the post-T0 regret decomposition all rely on Eq. (36), the claimed O(d\sqrt{mT}/\kappa_\mu) collaboration benefit is not established by the submitted proof. The same issue appears in the neural case in Lemma C.5, Eq. (70). Please either prove the eigenvalue lower bound for the actual selection rule or state and verify an additional assumption under which it holds.
  2. [Appendix C, Lemmas C.4 and C.6] The two central confidence bounds for CONDB, namely the per-user bound \sqrt{m_{\rm NN}}\|\theta_{f,i} - \hat\theta_{i,t}\|_2 and the cluster-level bound \sqrt{m_{\rm NN}}\|\theta_{f,i_t} - \theta_t\|_{V_{t-1}}, are asserted to follow by 'modifying the proof of Lemma 6 from Verma et al. (2024)', but no modification is shown. These lemmas are load-bearing: Lemma C.4 drives the edge-deletion rule in Lemma C.5, and Lemma C.6 drives the immediate regret decomposition after T0. The constants, the definition of \beta_T with ed + 2\log(u/\delta), the pair-selection rule, and the clustering context all differ from the reference setting. A citation with 'can be proved similarly' is not sufficient for a main theorem of this paper; the adapted proof must be supplied or the reduction made rigorous.
minor comments (6)
  1. [Theorem 4.2] The theorem statement lists Assumptions 2.1, 2.4, 2.5, and 2.7, but the proof of Lemma C.5 explicitly uses Assumption 2.3 (uniform user arrival) and Lemma 8 of Li & Zhang (2018). Assumption 2.3 must be included in the theorem statement.
  2. [Equations (13)-(14) and (16)-(17)] The equality O(u(\cdots)\log T + \cdots) = O(\cdots) collapses the initial clustering phase into the leading term. This is only valid asymptotically in T for fixed u and m; please state this asymptotic interpretation explicitly.
  3. [Algorithm 1, line 1; Lemma B.4] The algorithm's input defines \beta_t with a term d\log(1 + tL^2\kappa_\mu/(d\lambda)), while the proof and Theorem 4.1 use \beta_T with 4T in place of tL^2, and the linear setting does not define L. These definitions should be aligned.
  4. [Assumption 2.5] The condition 'x_j = x_{j+d/2}' appears to be a typo for x_j = x_{j+d/2} (where j+d/2 is a coordinate index). Please clarify the notation.
  5. [Lemmas B.5 and C.9] Both lemmas choose \lambda such that 4\kappa_\mu/\lambda \le 1. Since \lambda is listed as an input parameter, the paper should state a concrete admissible choice, e.g., \lambda = 4\kappa_\mu, in the algorithm and theorem statements rather than leaving it implicit.
  6. [Lemma B.2] The proof uses the conditions \lambda\kappa_\mu \le 2\log(u/\delta)+\cdots and u/\delta \ge 512/(\lambda\kappa_\mu\tilde\lambda_x\gamma^2) and calls them 'typically held'. For a rigorous theorem these should be stated as explicit conditions or replaced by a concrete choice of \lambda and \delta that guarantees them.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the regret bounds are derived analytically from stated assumptions, and the self-citations to prior work by the same group are technical lemmas with independent content, not the target result.

full rationale

The paper's central claims are upper bounds on the expected regret of COLDB and CONDB. The proofs are conventional: Lemma B.1 and Lemma C.4 give self-normalized concentration bounds for the per-user MLE and neural estimates; Lemmas B.2 and C.5 convert these into a sufficient clustering time T0 using the item-regularity assumption (Assumption 2.4) and the cited lower bound on the minimum eigenvalue of the accumulated information matrix; the post-clustering regret is then bounded by the elliptical-potential lemma and Cauchy-Schwarz over the m ground-truth clusters. The quantity m appears because the post-T0 analysis groups users by cluster, and the u-dependence is confined to the logarithmic T0 term; this is a derived structural feature, not a quantity fitted to empirical regret. The cited results from Wang et al. (2024a) and Li & Zhang (2018) are parameter-free technical lemmas about the information matrix of adaptively chosen i.i.d. contexts; they do not state or assume the target regret bound and are therefore independent support. The neural variant inherits the effective dimension bound from Verma et al. (2024), which is again external to the target claim. The algorithm's deletion threshold is calibrated to the same confidence bound used in the proof, which is standard confidence-bound design rather than a reduction of the prediction to the input. The only substantive concern is whether Lemma J.1 applies to the adaptively selected arm-pair differences, but that is a possible proof gap or correctness risk, not circularity under the standards of this review, because no equation is shown to be equivalent to its own input by construction.

Assumptions & free parameters 3 free parameters · 8 assumptions · 0 invented entities

The central regret claims rest on a stack of standard bandit assumptions (BTL model, cluster separation, uniform arrivals, stochastic contexts) plus two borrowed results from prior neural bandit papers. No new entities are postulated. The main practical unknowns are the constants λ̃_x and B that the algorithms need to set their deletion thresholds.

free parameters (3)
  • λ (ridge regularization)
    Chosen by hand in both algorithms; the theory requires λ to satisfy 4κµ/λ ≤ 1 and λκµ ≤ O(log...), but no principled selection rule is given.
  • λ̃_x (minimum eigenvalue rate)
    Introduced in Assumption 2.4 and used in the deletion threshold f(T_i,t) in Algorithms 1 and 2; it is a problem-dependent constant that is not estimated or specified in practice.
  • B (NTK norm bound)
    Appears in Lemma 2.6 / C.1 and is used in CONDB's line 9 and deletion threshold. The paper does not show how to compute or estimate B from data.
assumptions (8)
  • domain assumption Assumption 2.1: BTL preference model with logistic link, Lipschitz constant Lµ and minimum gradient κµ.
    Standard dueling bandits assumption; used throughout the concentration and regret proofs.
  • domain assumption Assumption 2.2: Pairwise preference vectors of different clusters are separated by at least γ.
    Needed to guarantee that edge deletion separates clusters in time T0; if γ is small, T0 grows.
  • domain assumption Assumption 2.3: Users arrive uniformly at random each round.
    Used in Lemma B.2 to guarantee every user is served enough times within T0.
  • domain assumption Assumption 2.4: Contexts are drawn i.i.d. from a full-rank distribution with sub-Gaussian tails.
    Load-bearing for the lower bound on λ_min(V_i,t); without it the clustering time T0 is not bounded.
  • domain assumption Assumption 2.5: Bounded rewards, NTK matrix lower bounded by λ0, and normalized doubled contexts.
    Standard neural bandit assumptions; needed for the NTK linearization lemma.
  • domain assumption Assumption 2.7: Reward functions of different clusters are separated by γ′ at every arm.
    Used in Lemma C.2 to translate function separation into parameter separation for CONDB.
  • standard math Lemma C.1 (NTK linearization from Zhang et al. 2021): each cluster's reward function is linear in the initial gradient features with norm bound B.
    Cited theorem; not proven in this paper. It is the bridge between neural networks and linear analysis.
  • domain assumption Lemma 6 of Verma et al. 2024 (neural dueling bandit concentration), modified without proof in Lemmas C.4 and C.6.
    The paper states the result follows by modification but does not show the modified proof; this is a derivation gap.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Online Clustering of Dueling Bandits." pith.science (2026). https://pith.science/paper/IBPCQZG7

@misc{pith2026250202079,
  author       = {Pith},
  title        = {Pith review of: Online Clustering of Dueling Bandits},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IBPCQZG7}},
  note         = {Machine review of arXiv:2502.02079}
}
read the original abstract

The contextual multi-armed bandit (MAB) is a widely used framework for problems requiring sequential decision-making under uncertainty, such as recommendation systems. In applications involving a large number of users, the performance of contextual MAB can be significantly improved by facilitating collaboration among multiple users. This has been achieved by the clustering of bandits (CB) methods, which adaptively group the users into different clusters and achieve collaboration by allowing the users in the same cluster to share data. However, classical CB algorithms typically rely on numerical reward feedback, which may not be practical in certain real-world applications. For instance, in recommendation systems, it is more realistic and reliable to solicit preference feedback between pairs of recommended items rather than absolute rewards. To address this limitation, we introduce the first "clustering of dueling bandit algorithms" to enable collaborative decision-making based on preference feedback. We propose two novel algorithms: (1) Clustering of Linear Dueling Bandits (COLDB) which models the user reward functions as linear functions of the context vectors, and (2) Clustering of Neural Dueling Bandits (CONDB) which uses a neural network to model complex, non-linear user reward functions. Both algorithms are supported by rigorous theoretical analyses, demonstrating that user collaboration leads to improved regret bounds. Extensive empirical evaluations on synthetic and real-world datasets further validate the effectiveness of our methods, establishing their potential in real-world applications involving multiple users with preference-based feedback.

Figures

Figures reproduced from arXiv: 2502.02079 by the authors.

Figure 1
Figure 1. Experimental results for our COLDB algorithm with a linear reward function. 5. Experimental Results We use both synthetic and real-world experiments to evalu￾ate the performance of our COLDB and CONDB algorithms. For both algorithms, we compare them with their corre￾sponding single-user variant as the baseline. Specifically, for COLDB, we compare it with the baseline of LDB IND, which refers to Linear Dueling Bandit… view at source ↗
Figure 2
Figure 2. Experimental results for our CONDB algorithm with a non-linear (square) reward function. COLDB algorithm described above. The major difference is that instead of using linear reward functions, here we adopt a non-linear reward function, i.e., a square function: fi(x) = (θ ⊤ i x) 2 . The results in this setting are plotted in [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

60 extracted references · 49 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    Improved algorithms for linear stochastic bandits

    Abbasi-Yadkori, Y., P \'a l, D., and Szepesv \'a ri, C. Improved algorithms for linear stochastic bandits. Advances in neural information processing systems, 24, 2011

    2011

  3. [3]

    Reducing dueling bandits to cardinal bandits

    Ailon, N., Karnin, Z., and Joachims, T. Reducing dueling bandits to cardinal bandits. In Proc. ICML, pp.\ 856--864, 2014

    work page 2014

  4. [4]

    and He, J

    Ban, Y. and He, J. Convolutional neural bandit: Provable algorithm for visual-aware advertising. arXiv :2107.07438, 2021 a

    arXiv 2021

  5. [5]

    and He, J

    Ban, Y. and He, J. Local clustering in contextual multi-armed bandits. In Proceedings of the Web Conference 2021, pp.\ 2335--2346, 2021 b

    work page 2021

  6. [6]

    Ee-net: Exploitation-exploration neural networks in contextual bandits

    Ban, Y., Yan, Y., Banerjee, A., and He, J. Ee-net: Exploitation-exploration neural networks in contextual bandits. In Proc. ICLR, 2022

    work page 2022

  7. [7]

    Meta clustering of neural bandits

    Ban, Y., Qi, Y., Wei, T., Liu, L., and He, J. Meta clustering of neural bandits. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp.\ 95--106, 2024

    work page 2024

  8. [8]

    Stochastic contextual dueling bandits under linear stochastic transitivity models

    Bengs, V., Saha, A., and H \"u llermeier, E. Stochastic contextual dueling bandits under linear stochastic transitivity models. In Proc. ICML, pp.\ 1764--1786, 2022

    work page 2022

Show all 60 references
  1. [9]

    Contextual bandits with linear payoff functions

    Chu, W., Li, L., Reyzin, L., and Schapire, R. Contextual bandits with linear payoff functions. In Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, pp.\ 208--214. JMLR Workshop and Conference Proceedings, 2011

    2011

  2. [10]

    Dai, Z., Shu, Y., Low, B. K. H., and Jaillet, P. Sample-then-optimize batch neural Thompson sampling. In Proc. NeurIPS , 2022

    2022

  3. [11]

    X., Low, B

    Dai, Z., Shu, Y., Verma, A., Fan, F. X., Low, B. K. H., and Jaillet, P. Federated neural bandits. In Proc. ICLR, 2023

    2023

  4. [12]

    Variance-aware regret bounds for stochastic contextual dueling bandits

    Di, Q., Jin, T., Wu, Y., Zhao, H., Farnoud, F., and Gu, Q. Variance-aware regret bounds for stochastic contextual dueling bandits. arXiv:2310.00968, 2023

    2023 arXiv

  5. [13]

    A relative exponential weighing algorithm for adversarial utility-based dueling bandits

    Gajane, P., Urvoy, T., and Cl \'e rot, F. A relative exponential weighing algorithm for adversarial utility-based dueling bandits. In Proc. ICML, pp.\ 218--227, 2015

    2015

  6. [14]

    Online clustering of bandits

    Gentile, C., Li, S., and Zappella, G. Online clustering of bandits. In International Conference on Machine Learning, pp.\ 757--765. PMLR, 2014

    2014

  7. [15]

    On context-dependent clustering of bandits

    Gentile, C., Li, S., Kar, P., Karatzoglou, A., Zappella, G., and Etrue, E. On context-dependent clustering of bandits. In International Conference on machine learning, pp.\ 1253--1262. PMLR, 2017

    2017

  8. [16]

    Batched neural bandits

    Gu, Q., Karbasi, A., Khosravi, K., Mirrokni, V., and Zhou, D. Batched neural bandits. arXiv :2102.13028, 2021

    2021 arXiv

  9. [17]

    Harper, F. M. and Konstan, J. A. The movielens datasets: History and context. Acm transactions on interactive intelligent systems (tiis), 5 0 (4): 0 1--19, 2015

    2015

  10. [18]

    Hunter, D. R. Mm algorithms for generalized bradley-terry models. Annals of Statistics, pp.\ 384--406, 2004

    2004

  11. [19]

    Combinatorial neural bandits

    Hwang, T., Chai, K., and Oh, M.-h. Combinatorial neural bandits. In International Conference on Machine Learning, pp.\ 14203--14236. PMLR, 2023

    2023

  12. [20]

    Neural tangent kernel: Convergence and generalization in neural networks

    Jacot, A., Gabriel, F., and Hongler, C. Neural tangent kernel: Convergence and generalization in neural networks. In Proc. NeurIPS , 2018

    2018

  13. [21]

    Learning neural contextual bandits through perturbed rewards

    Jia, Y., Zhang, W., Zhou, D., Gu, Q., and Wang, H. Learning neural contextual bandits through perturbed rewards. In Proc. ICLR, 2021

    2021

  14. [22]

    and Krause, A

    Kassraie, P. and Krause, A. Neural contextual bandits without regret. In Proc. AISTATS, pp.\ 240--278, 2022

    2022

  15. [23]

    Graph neural network bandits

    Kassraie, P., Krause, A., and Bogunovic, I. Graph neural network bandits. In Proc. NeurIPS , 2022

    2022

  16. [24]

    Regret lower bound and optimal algorithm in dueling bandit problem

    Komiyama, J., Honda, J., Kashima, H., and Nakagawa, H. Regret lower bound and optimal algorithm in dueling bandit problem. In Proc. COLT, pp.\ 1141--1154, 2015

    2015

  17. [25]

    and Szepesv \'a ri, C

    Lattimore, T. and Szepesv \'a ri, C. Bandit algorithms. Cambridge University Press, 2020

    2020

  18. [26]

    Li, L., Chu, W., Langford, J., and Schapire, R. E. A contextual-bandit approach to personalized news article recommendation. In Proceedings of the 19th international conference on World wide web, pp.\ 661--670, 2010

    2010

  19. [27]

    Provably optimal algorithms for generalized linear contextual bandits

    Li, L., Lu, Y., and Zhou, D. Provably optimal algorithms for generalized linear contextual bandits. In Proc. ICML, pp.\ 2071--2080, 2017

    2017

  20. [28]

    and Zhang, S

    Li, S. and Zhang, S. Online clustering of contextual cascading bandits. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018

    2018

  21. [29]

    Collaborative filtering bandits

    Li, S., Karatzoglou, A., and Gentile, C. Collaborative filtering bandits. In Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval, pp.\ 539--548, 2016

    2016

  22. [30]

    Improved algorithm on online clustering of bandits

    Li, S., Chen, W., Li, S., and Leung, K.-S. Improved algorithm on online clustering of bandits. In Proceedings of the 28th International Joint Conference on Artificial Intelligence, IJCAI'19, pp.\ 2923–2929. AAAI Press, 2019. ISBN 9780999241141

    2019

  23. [31]

    Feel-good thompson sampling for contextual dueling bandits

    Li, X., Zhao, H., and Gu, Q. Feel-good thompson sampling for contextual dueling bandits. arXiv:2404.06013, 2024

    2024 arXiv

  24. [32]

    Lin, X., Dai, Z., Verma, A., Ng, S.-K., Jaillet, P., and Low, B. K. H. Prompt optimization with human feedback. arXiv preprint arXiv:2405.17346, 2024 a

    2024 arXiv

  25. [33]

    Lin, X., Wu, Z., Dai, Z., Hu, W., Shu, Y., Ng, S.-K., Jaillet, P., and Low, B. K. H. Use your INSTINCT : Instruction optimization using neural bandits coupled with transformers. In Proc. ICML, 2024 b

    2024

  26. [34]

    Lisicki, M., Afkanpour, A., and Taylor, G. W. An empirical study of neural kernel bandits. In NeurIPS Workshop on Bayesian Deep Learning, 2021

    2021

  27. [35]

    Federated online clustering of bandits

    Liu, X., Zhao, H., Yu, T., Li, S., and Lui, J. Federated online clustering of bandits. In The 38th Conference on Uncertainty in Artificial Intelligence, 2022

    2022

  28. [36]

    Luce, R. D. Individual choice behavior: A theoretical analysis. Courier Corporation, 2005

    2005

  29. [37]

    Online limited memory neural-linear bandits with likelihood matching

    Nabati, O., Zahavy, T., and Mannor, S. Online limited memory neural-linear bandits with likelihood matching. In Proc. ICML , 2021

    2021

  30. [38]

    T., and Venkatesh, S

    Nguyen-Tang, T., Gupta, S., Nguyen, A. T., and Venkatesh, S. Offline neural contextual bandits: Pessimism, optimization and generalization. In Proc. ICLR, 2022

    2022

  31. [39]

    Graph neural bandits

    Qi, Y., Ban, Y., and He, J. Graph neural bandits. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp.\ 1920--1931, 2023

    1920

  32. [40]

    Meta-learning with neural bandit scheduler

    Qi, Y., Ban, Y., Wei, T., Zou, J., Yao, H., and He, J. Meta-learning with neural bandit scheduler. Advances in Neural Information Processing Systems, 36, 2024

    2024

  33. [41]

    Optimal algorithms for stochastic contextual preference bandits

    Saha, A. Optimal algorithms for stochastic contextual preference bandits. In Proc. NeurIPS, pp.\ 30050--30062, 2021

    2021

  34. [42]

    and Ghoshal, S

    Saha, A. and Ghoshal, S. Exploiting correlation to achieve faster learning rates in low-rank preference bandits. In Proc. AISTATS, pp.\ 456--482, 2022

    2022

  35. [43]

    and Gopalan, A

    Saha, A. and Gopalan, A. Battle of bandits. In Proc. UAI, pp.\ 805--814, 2018

    2018

  36. [44]

    and Gopalan, A

    Saha, A. and Gopalan, A. Active ranking with subset-wise preferences. In Proc. AISTATS, pp.\ 3312--3321, 2019 a

    2019

  37. [45]

    and Gopalan, A

    Saha, A. and Gopalan, A. Pac battling bandits in the plackett-luce model. In Proc. ALT, pp.\ 700--737, 2019 b

    2019

  38. [46]

    and Krishnamurthy, A

    Saha, A. and Krishnamurthy, A. Efficient and optimal algorithms for contextual dueling bandits under realizability. In Proc. ALT, pp.\ 968--994, 2022

    2022

  39. [47]

    Provably and practically efficient neural contextual bandits

    Salgia, S., Vakili, S., and Zhao, Q. Provably and practically efficient neural contextual bandits. arXiv :2206.00099, 2022

    2022 arXiv

  40. [48]

    Verma, A., Dai, Z., Lin, X., Jaillet, P., and Low, B. K. H. Neural dueling bandits. arXiv preprint arXiv:2407.17112, 2024

    2024 arXiv

  41. [49]

    Online clustering of bandits with misspecified user models

    Wang, Z., Xie, J., Liu, X., Li, S., and Lui, J. Online clustering of bandits with misspecified user models. Advances in Neural Information Processing Systems, 36, 2024 a

    2024

  42. [50]

    Online corrupted user detection and regret minimization

    Wang, Z., Xie, J., Yu, T., Li, S., and Lui, J. Online corrupted user detection and regret minimization. Advances in Neural Information Processing Systems, 36, 2024 b

    2024

  43. [51]

    Neural contextual bandits with deep representation and shallow exploration

    Xu, P., Wen, Z., Zhao, H., and Gu, Q. Neural contextual bandits with deep representation and shallow exploration. arXiv :2012.01780, 2020

    2012 arXiv

  44. [52]

    and Joachims, T

    Yue, Y. and Joachims, T. Interactively optimizing information retrieval systems as a dueling bandits problem. In Proc. ICML, pp.\ 1201--1208, 2009

    2009

  45. [53]

    and Joachims, T

    Yue, Y. and Joachims, T. Beat the mean bandit. In Proc. ICML, pp.\ 241--248, 2011

    2011

  46. [54]

    The k-armed dueling bandits problem

    Yue, Y., Broder, J., Kleinberg, R., and Joachims, T. The k-armed dueling bandits problem. Journal of Computer and System Sciences, pp.\ 1538--1556, 2012

    2012

  47. [55]

    Neural Thompson sampling

    Zhang, W., Zhou, D., Li, L., and Gu, Q. Neural Thompson sampling. In Proc. ICLR, 2021

    2021

  48. [56]

    Neural contextual bandits with UCB -based exploration

    Zhou, D., Li, L., and Gu, Q. Neural contextual bandits with UCB -based exploration. In Proc. ICML, pp.\ 11492--11502, 2020

    2020

  49. [57]

    Principled reinforcement learning with human feedback from pairwise or k-wise comparisons

    Zhu, B., Jordan, M., and Jiao, J. Principled reinforcement learning with human feedback from pairwise or k-wise comparisons. In Proc. ICML, pp.\ 43037--43067, 2023

    2023

  50. [58]

    Pure exploration in kernel and neural bandits

    Zhu, Y., Zhou, D., Jiang, R., Gu, Q., Willett, R., and Nowak, R. Pure exploration in kernel and neural bandits. In Proc. NeurIPS , volume 34, pp.\ 11618--11630, 2021

    2021

  51. [59]

    Relative upper confidence bound for the k-armed dueling bandit problem

    Zoghi, M., Whiteson, S., Munos, R., and Rijke, M. Relative upper confidence bound for the k-armed dueling bandit problem. In Proc. ICML, pp.\ 10--18, 2014 a

    2014

  52. [60]

    A., De Rijke, M., and Munos, R

    Zoghi, M., Whiteson, S. A., De Rijke, M., and Munos, R. Relative confidence sampling for efficient on-line ranker evaluation. In Proc. WSDM, pp.\ 73--82, 2014 b

    2014

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.