Pith. sign in

REVIEW 4 major objections 5 minor 61 references

Bayesian Optimization from Human Feedback: Near-Optimal Regret Bounds

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that Bayesian optimization with only pairwise preference feedback can achieve the same regret rate as conventional scalar-feedback Bayesian optimization, and proves it for a multi-round elimination algorithm.

desk verdict The paper has a real algorithmic idea and a real gap: the confidence bound in Theorem 4.7 leans on an unproved Loewner inequality that fails for small lambda, which collapses the regret bound as written. read the letter →

arxiv 2505.23673 v1 pith:IHXPJR27 submitted 2025-05-29 cs.LG

Aya Kayal , Sattar Vakili , Laura Toni , Da-shan Shiu , Alberto Bernacchia This is my paper
classification cs.LG
keywords BayesianoptimizationpreferencefeedbackBradley-Terry-Lucemodelregretboundskernelmethodsmulti-roundalgorithmsmaximuminformationgainsamplecomplexity
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that in Bayesian optimization with binary preference queries, a carefully structured algorithm can match the regret rate of conventional scalar-feedback Bayesian optimization: $\tilde{O}(\sqrt{\Gamma(T)T})$. If true, this means pairwise "which is better?" queries are as informative, in a worst-case sense, as direct numeric evaluations of the hidden utility function. The proposed algorithm, MR-LPF, works in rounds, queries the most uncertain pair inside a shrinking set of plausible optima, and eliminates actions whose preference confidence bounds fall below $\tfrac12$. The paper further derives sample complexities for linear, squared-exponential, and Mat\'ern kernels that match, up to logarithms, the known lower bounds for scalar-feedback Bayesian optimization.

What carries the argument

The carrying mechanism is MR-LPF itself: rounds of increasing size ($N_1 = \lceil\sqrt{T}\rceil$, $N_r = \lceil\sqrt{N_{r-1}T}\rceil$), pair queries chosen within the active set $\mathcal{M}_r$ by maximizing kernel uncertainty $\sigma_{n-1,r}(x,x')$, and end-of-round elimination that keeps an action $x$ only if $\mu(h(x,x')) + \beta^{(r)}\sigma(x,x') \ge 1/2$ for every rival $x'$. Three ingredients do the analytical work: the dueling kernel $k((x,x'),(u,u')) = k(x,u)+k(x',u')-k(x,u')-k(x',u)$, which preserves the RKHS norm and makes ridge-type regression applicable to utility differences; the maximum information gain $\Gamma_\lambda(T)$, whose sublinear growth for a kernel family guarantees sublinear regret; and the Loewner inequality $G_t(\theta_\star,\theta_t) \succeq \kappa^{-1}V_t$ taken from the prior MaxMinLCB analysis, which converts a mean-value-theorem derivative integral into the ridge covariance $V_t = S_t + \kappa\lambda I$ and yields the confidence bound $|\mu(h_t(z))-\mu(h(z))| \le \beta\sigma_t(z)$ with $\beta$ free of $\sqrt{\Gamma(T)}$.

What would settle it

Run MR-LPF on a small finite grid with a Mat\'ern kernel and small regularization $\lambda$, and at each round compute the smallest eigenvalue of $G_t(\theta_\star,\theta_t) - \kappa^{-1}V_t$; a single negative eigenvalue in a trajectory where the true parameter is known refutes Eq. (41) and with it the confidence bound of Theorem 4.7, independently of any regret experiment.

Watch

Extended reading notes

Core claim

The central claim is that the Multi-Round Learning from Preference-based Feedback (MR-LPF) algorithm achieves cumulative regret $R(T) \le 2CR\,\beta^{(R)}(\delta)\sqrt{\Gamma_{4\lambda}(T)}\,(T^{1/2}+1)$ with probability at least $1-\delta$, which simplifies to $\tilde{O}(\sqrt{\Gamma_\lambda(T)T\log(|\mathcal{X}|/\delta)})$. This is the same rate as order-optimal scalar-feedback BO and improves over the two existing bounds $\tilde{O}(\Gamma(T)\kappa^2\sqrt{T})$ and $\tilde{O}((\Gamma(T)T)^{3/4})$, removing both the $\kappa^2$ dependence and the extra $\sqrt{\Gamma(T)}$ factor. From the regret bound, the paper obtains simple-regret sample complexities that match, up to logarithmic factors, the known lower bounds for scalar-feedback BO with Mat\'ern kernels, with the caveat that the comparison is informal because the preference-noise model is Gumbel-like rather than Gaussian. The improvement comes from the round structure: observation points are selected by pure variance, independent of past feedback values, which yields tighter confidence intervals; and elimination keeps only nearly tied actions, so the sigmoid curvature bound $\kappa_r$ collapses to a fixed value of $6$ after the first round.

Load-bearing premise

The whole bound hinges on the inequality $G_t(\theta_\star,\theta_t) \succeq \kappa^{-1}V_t$, which requires the integrated sigmoid slope between the true and fitted preference functions to stay above $1/\kappa$; a regularized logistic fit can leave the bounded utility interval where the sigmoid is flat, so this slope bound is not automatic and the proof collapses if it fails.

Editorial extensions

If this is right

  • For linear kernels, MR-LPF finds an $\epsilon$-good action with $T = \tilde{O}(d\log(1/\delta)/\epsilon^2)$ preference queries; for squared-exponential kernels, $T = \tilde{O}(\log(1/\delta)/\epsilon^2)$; for Mat\'ern kernels of smoothness $\nu$, $T = \tilde{O}(\log(1/\delta)/\epsilon^{2+d/\nu})$, each matching the corresponding scalar-feedback rate up to logs.
  • Because $\kappa$ disappears from the dominant regret term, a preference-learning algorithm no longer needs to know or control the maximum sigmoid curvature of the utility range to get near-optimal guarantees.
  • The regret bound is sublinear for every kernel whose maximum information gain grows sublinearly; for Mat\'ern and neural tangent kernels, where $\Gamma(T)$ can grow faster than $\sqrt{T}$, the prior bounds could become vacuous while this one remains sublinear.
  • Outputting any action that survives the final round yields simple regret at most $2\beta^{(R)}C\sqrt{R\Gamma_{4\lambda}(T)/T}$, so the final recommendation, not only the cumulative query loss, reaches $\epsilon$ accuracy at the stated sample counts.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct but untested consequence of the paper's own logic is that the query count needed to reach fixed $\epsilon$ should trace the $\epsilon^{-(2+d/\nu)}$ curve kernel by kernel; measuring this across SE, Mat\'ern with several $\nu$, and neural tangent kernels would provide a scaling check the paper does not report.
  • The independence assumption on which the tight confidence bound rests is specific to a learner that ignores past outcomes within a round; an adversary who conditions action choices on observed preferences would break that independence, so the near-optimal rate likely does not extend to adversarial or non-stationary preference feedback.
  • If Gumbel-noise lower bounds were derived for scalar-feedback BO, the paper's informal tightness argument would become a formal equivalence, converting 'same sample complexity as scalar feedback' from a suggestive match into a proven one; this is the missing step the authors explicitly acknowledge.
  • The $\kappa_r = 6$ argument predicts that after one round of pure uncertainty exploration the surviving candidates are nearly tied in preference probability; this structural prediction could be checked directly by inspecting the empirical distribution of $\mu(h(x,x'))$ over surviving pairs in the paper's experiments.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This paper studies Bayesian optimization from preference feedback (BOHF) under the Bradley-Terry-Luce (BTL) model. It introduces MR-LPF, a multi-round elimination algorithm that queries the pair of currently plausible actions with maximum kernel variance and, at the end of each round, eliminates actions whose upper confidence bound against every surviving action is below 1/2. The main theorem (Theorem 4.1) claims a regret bound of order O~(sqrt(Gamma(T) T log|X|)), improving on the O~(Gamma(T) kappa^2 sqrt(T)) and O~((Gamma(T) T)^{3/4}) bounds of Pasztor et al. (2024) and Xu et al. (2024), respectively. The proof strategy exploits the fact that, within a round, query locations are chosen independently of the labels, yielding tighter logistic-regression confidence intervals (Theorem 4.7). Corollaries give sample-complexity statements, and experiments compare MR-LPF against MaxMinLCB on synthetic kernels and the Yelp dataset.

Significance. If Theorem 4.1 were fully proved, the paper would settle an important open question: preference feedback would impose no asymptotic penalty over scalar feedback in kernelized BO, at least for the BTL model. The multi-round structure is a clean and potentially transferable idea, and the within-round separation of query design from labels is a genuine methodological contribution. The paper also responsibly flags in Remark 4.6 that its lower-bound comparison to conventional BO is informal. However, as I detail below, the confidence-interval proof contains a substantial unproved Loewner bound, the finite-action requirement is not stated in the setup, and the regret definition is inconsistent with the proof. These issues prevent the main claim from being accepted as written, despite the value of the algorithmic idea.

major comments (4)
  1. [Section 2.1, Eq. (1); Appendix B, Eq. (18)] Equation (1) defines regret as R(T) = sum_t [P(x* ≻ x_t) + P(x* ≻ x'_t) - 1/2]. As printed, this quantity can be negative when both selected actions are much worse than the optimum, and it is not bounded by 1/2 per step. The proof, however, repeatedly uses per-step bounds of the form P(x* ≻ x) - 1/2 (see Eqs. (18), (30), (31)), which correspond to the average sub-optimality gap (P(x* ≻ x_t) + P(x* ≻ x'_t))/2 - 1/2. The theorem is therefore stated for a different quantity than the one analyzed. This is fixable by changing the definition (and the abstract/introduction accordingly), but the current mismatch makes the main theorem ill-posed.
  2. [Section 2.1 and Theorem 4.1] The setup explicitly allows a continuous action set X, but Theorem 4.1 and Lemma B.1 rely on a union bound over the |X| action pairs, with beta(r) in Eq. (14) containing log(2R|X|/delta). For continuous X, log|X| is undefined and the stated regret bound has no meaning. The theorem should either explicitly assume that X is finite or provide a separate discretization argument. As written, the advertised applicability to continuous domains is not supported by the analysis.
  3. [Appendix C, Eq. (41)] The proof uses the Loewner bound G_t(theta*, theta_t) ⪰ kappa^{-1} V_t, citing Pasztor et al. (2024, Lemma 12). The coefficient alpha(z_i; theta*, theta_t) in the definition of G_t is the average of mu' along the segment between theta*^T phi(z_i) and theta_t^T phi(z_i), and (41) requires this average to be at least kappa^{-1}. The kappa in Eq. (2) is defined from the true utility range h(X × X), but theta_t is the regularized logistic minimizer of Eq. (7), and the paper gives no bound on theta_t^T phi(z_i). For a one-point dataset with y_1=0, the first-order condition is mu(theta_t) + lambda theta_t = 0 (up to the feature norm); as lambda -> 0, theta_t -> -infinity and the averaged derivative alpha tends to 0, so for any fixed kappa the inequality alpha >= kappa^{-1} fails for sufficiently small lambda. Such a dataset is admissible in MR-LPF because within a round the query locations are chosen from the variance alone, independently of the labels. Since both the stochastic and bias bounds in Eqs. (42) and (43) rely on (41), Theorem 4.7 is not proved as stated, and the regret bound of Theorem 4.1 does not follow from the given argument. A repair is needed, for example a norm bound on theta_t, a lower bound on lambda, or a different confidence-interval argument.
  4. [Remark 4.6 and Abstract] The abstract and Section 1 claim that the order-optimal sample complexities of conventional BO are recovered, but Remark 4.6 acknowledges that the comparison with Scarlett et al. (2017) is not strictly valid: the lower bound there assumes Gaussian noise while BTL preference feedback corresponds to Gumbel/logistic noise, and the reduction changes constants and even the value of the target function. Since the paper provides only an informal justification, the word 'recovered' in the abstract overstates what is proved. I recommend either providing a formal lower bound or tempering the optimality claim in the abstract and introduction.
minor comments (5)
  1. [Section 2.1] There is a typo in 'Bradeley-Terry-Luce'; it should be 'Bradley-Terry-Luce'.
  2. [Section 1.1.1] The text contains 'Supvariations of UCB'; this should likely be 'Subvariations' or a related intended term.
  3. [Appendix B, Lemma B.1] The proof of Lemma B.1 is only sketched ('follows from Theorem 4.7, a union bound ...'). Please spell out the union bound over rounds and pairs and the precise role of the kappa_r = 6 bound derived in Eqs. (27)-(28).
  4. [Theorem 4.1 and Eq. (13)] The notation Gamma_{(4 lambda)}(T) in the theorem is introduced without explanation; since the definition in Eq. (13) is Gamma_lambda(T), the reader should be told that the scaling 4 lambda arises from kappa_r >= 4 for r >= 2.
  5. [Section 5] The experiments compare MR-LPF only against MaxMinLCB, not against POP-BO of Xu et al. (2024), even though the related-work table discusses both. Adding the second baseline would make the experimental comparison align with the paper's claimed improvements.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation: the regret bound is derived from kernel confidence intervals and variance summation; self-cited information-gain scalings are external and not load-bearing.

full rationale

The paper's central claim (Theorem 4.1) is not obtained by fitting a parameter to the data it then 'predicts.' The regret bound follows from round-wise variance summation [Eqs. (20)-(33)], a confidence interval (Theorem 4.7), and an elimination argument (Lemma B.2). The confidence interval in Theorem 4.7 is proved from the first-order condition of the regularized logistic loss, sub-Gaussian concentration, and the Loewner bound G_t(theta*,theta_t) >= kappa^{-1} V_t cited from Pasztor et al. (2024, Lemma 12); this is an external citation to non-overlapping authors, not a self-citation, and no equation in the paper defines kappa or V_t in terms of the final bound. The only self-citations are to Vakili et al. (2021a,b) for conventional BO confidence bounds and for Gamma(T) scalings of SE/Matérn/linear kernels; these are independent, parameter-free published results used only to simplify the stated rates, and the paper's regret proof does not rest on them. Remark 4.6's comparison with Scarlett et al. (2017) is explicitly labeled informal and not used as a proof step. The potential failure of inequality (41) for regularized logistic fits that leave the utility range is a correctness/technical-support concern, not a circularity: it would invalidate Theorem 4.7, but it is not an instance of the derivation assuming what it proves. Hence no significant circularity; score 2 reflects only the presence of minor, non-load-bearing self-citations.

Assumptions & free parameters 1 free parameters · 7 assumptions · 0 invented entities

The central claim rests on standard RKHS assumptions, the BTL sigmoid feedback model, a finite-action-set requirement that is implicit, a key Loewner bound imported from Pasztor et al., and standard concentration tools. No new physical or mathematical entities are introduced; MR-LPF is an algorithm, and the dueling kernel was defined in prior work.

free parameters (1)
  • lambda (regularization parameter) = 0.05 in experiments; arbitrary in theory
    Enters the regret and confidence bounds through Gamma_lambda(T) and beta; the theorem holds for any lambda>0, but the choice changes constants and, in practice, the tightness of the logistic-regression Hessian bound.
assumptions (7)
  • domain assumption The unknown utility f lies in the RKHS of a known kernel k with ||f||_{H_k} <= B and k<=1 (Assumption 2.1).
    Standard for GP bandits; all confidence and regret statements are conditional on this.
  • domain assumption Preferences follow the BTL model with sigmoid link: P(y=1|x,x') = mu(f(x)-f(x')), mu(s) = (1+e^{-s})^{-1}.
    Defines the feedback model in Section 2.1; the analysis is specific to this link.
  • ad hoc to paper The action set X is finite so that union bounds over |X| pairs apply.
    beta^(r) in Eq (14) contains log(2R|X|/delta), but Section 2.1 allows continuous X; no discretization argument is provided.
  • domain assumption Pasztor et al. (2024), Lemma 12: G_t(theta1,theta2) >= kappa^{-1} V_t for all theta1, theta2.
    Inherited external lemma used in Eq (41) of Appendix C; not proved and not obviously valid for all regularized logistic predictors.
  • standard math Mercer theorem and representer theorem give the feature representation and the parametric form of h_t (Appendix A).
    Foundation of the RKHS model; standard functional analysis result.
  • standard math Standard concentration inequalities (sub-Gaussian tail bounds) and union bounds are used in Theorem 4.7 and Lemma B.1.
    Standard tools; the paper cites Vershynin 2018.
  • domain assumption Known bounds on Gamma_lambda(T) for linear, SE, and Matérn kernels (Vakili et al., 2021b).
    Used in Corollary 4.5 to translate the regret bound into sample complexity.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Bayesian Optimization from Human Feedback: Near-Optimal Regret Bounds." pith.science (2026). https://pith.science/paper/IHXPJR27

@misc{pith2026250523673,
  author       = {Pith},
  title        = {Pith review of: Bayesian Optimization from Human Feedback: Near-Optimal Regret Bounds},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IHXPJR27}},
  note         = {Machine review of arXiv:2505.23673}
}
abstract

Bayesian optimization (BO) with preference-based feedback has recently garnered significant attention due to its emerging applications. We refer to this problem as Bayesian Optimization from Human Feedback (BOHF), which differs from conventional BO by learning the best actions from a reduced feedback model, where only the preference between two actions is revealed to the learner at each time step. The objective is to identify the best action using a limited number of preference queries, typically obtained through costly human feedback. Existing work, which adopts the Bradley-Terry-Luce (BTL) feedback model, provides regret bounds for the performance of several algorithms. In this work, within the same framework we develop tighter performance guarantees. Specifically, we derive regret bounds of $\tilde{\mathcal{O}}(\sqrt{\Gamma(T)T})$, where $\Gamma(T)$ represents the maximum information gain$\unicode{x2014}$a kernel-specific complexity term$\unicode{x2014}$and $T$ is the number of queries. Our results significantly improve upon existing bounds. Notably, for common kernels, we show that the order-optimal sample complexities of conventional BO$\unicode{x2014}$achieved with richer feedback models$\unicode{x2014}$are recovered. In other words, the same number of preferential samples as scalar-valued samples is sufficient to find a nearly optimal solution.

Figures

Figures reproduced from arXiv: 2505.23673 by the authors.

Figure 1
Figure 1. Average Regret against T with RKHS test functions (top row) and Ackley test function (bottom row). The shaded area represents the standard error. each round, (x(n,r) , x′ (n,r) ), are collected solely based on the variance, which is independent of the observation values by definition. In contrast, both the MaxMinLCB algorithm in Pasztor et al. ´ (2024) and the POP-BO algorithm in Xu et al. (2024) select observation … view at source ↗
Figure 2
Figure 2. Average regret against T for the experiment with Yelp Open Dataset. The shaded area represents the standard error. 5. Experiments We run numerical experiments to evaluate the performance of MR-LPF and compare it to MaxMinLCB (Pasztor et al. ´ , 2024, Algorithm 1) on various test functions, including both synthetic and real-world cases. Our implementation is publicly available.3 We first select the test function f as… view at source ↗
Figure 3
Figure 3. Plots of the utility function f(x), the preference function h(x, x′ ) = f(x)−f(x ′ ), and the probability of preference µ(h(x, x′ )) for synthetic experiments. The rows correspond to: (1st row) SE kernel (RKHS), (2nd row) Matern kernel with ´ ν = 2.5 (RKHS), (3rd row) Matern kernel with ´ ν = 1.5 (RKHS), and (4th row) Ackley function. 19 [PITH_FULL_IMAGE:figures/full_fig_p019_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

61 extracted references · 48 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    Online learning for linearly parametrized control problems

    Abbasi-Yadkori, Y. Online learning for linearly parametrized control problems. 2013

    work page 2013

  3. [3]

    Reducing dueling bandits to cardinal bandits

    Ailon, N., Karnin, Z., and Joachims, T. Reducing dueling bandits to cardinal bandits. In International Conference on Machine Learning, pp.\ 856--864. PMLR, 2014

    work page 2014

  4. [4]

    S., Hu, W., Li, Z., Salakhutdinov, R

    Arora, S., Du, S. S., Hu, W., Li, Z., Salakhutdinov, R. R., and Wang, R. On exact computation with an infinitely wide neural net. Advances in Neural Information Processing Systems, 32, 2019

    2019

  5. [5]

    R., Daulton, S., Letham, B., Wilson, A

    Balandat, M., Karrer, B., Jiang, D. R., Daulton, S., Letham, B., Wilson, A. G., and Bakshy, E. BoTorch: A Framework for Efficient Monte-Carlo Bayesian Optimization . In Advances in Neural Information Processing Systems 33, 2020

    work page 2020

  6. [6]

    Preference-based online learning with dueling bandits: A survey

    Bengs, V., Busa-Fekete, R., El Mesaoudi-Paul, A., and H \"u llermeier, E. Preference-based online learning with dueling bandits: A survey. Journal of Machine Learning Research, 22 0 (7): 0 1--108, 2021

    work page 2021

  7. [7]

    Stochastic contextual dueling bandits under linear stochastic transitivity models

    Bengs, V., Saha, A., and H \"u llermeier, E. Stochastic contextual dueling bandits under linear stochastic transitivity models. In International Conference on Machine Learning, pp.\ 1764--1786. PMLR, 2022

    work page 2022

  8. [8]

    Mat \'e rn gaussian processes on riemannian manifolds

    Borovitskiy, V., Terenin, A., Mostowsky, P., et al. Mat \'e rn gaussian processes on riemannian manifolds. Advances in Neural Information Processing Systems, 33: 0 12426--12437, 2020

    work page 2020

Show all 61 references
  1. [9]

    Bradley, R. A. and Terry, M. E. Rank analysis of incomplete block designs: I. the method of paired comparisons. Biometrika, 39 0 (3/4): 0 324--345, 1952

    1952

  2. [10]

    M., and De Freitas, N

    Brochu, E., Cora, V. M., and De Freitas, N. A tutorial on bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning. arXiv preprint arXiv:1012.2599, 2010

    2010 arXiv

  3. [11]

    Instructzero: Efficient instruction optimization for black-box large language models

    Chen, L., Chen, J., Goldstein, T., Huang, H., and Zhou, T. Instructzero: Efficient instruction optimization for black-box large language models. In International Conference on Machine Learning, pp.\ 6503--6518. PMLR, 2024

    2024

  4. [12]

    Human-in-the-loop: Provably efficient preference-based reinforcement learning with general function approximation

    Chen, X., Zhong, H., Yang, Z., Wang, Z., and Wang, L. Human-in-the-loop: Provably efficient preference-based reinforcement learning with general function approximation. In International Conference on Machine Learning, pp.\ 3773--3793. PMLR, 2022

    2022

  5. [13]

    Chowdhury, S. R. and Gopalan, A. On kernelized multi-armed bandits. In International Conference on Machine Learning, pp.\ 844--853. PMLR, 2017

    2017

  6. [14]

    and Steinwart, I

    Christmann, A. and Steinwart, I. Support Vector Machines. Springer New York, NY, 2008

    2008

  7. [15]

    Das, N., Chakraborty, S., Pacchiano, A., and Chowdhury, S. R. Active preference optimization for sample efficient rlhf. In ICML 2024 Workshop on Theoretical Foundations of Foundation Models, 2024

    2024

  8. [16]

    E., Slivkins, A., and Zoghi, M

    Dud \' k, M., Hofmann, K., Schapire, R. E., Slivkins, A., and Zoghi, M. Contextual dueling bandits. In Conference on Learning Theory, pp.\ 563--587. PMLR, 2015

    2015

  9. [17]

    Falahatgar, M., Orlitsky, A., Pichapati, V., and Suresh, A. T. Maximum selection and ranking under noisy comparisons. In International Conference on Machine Learning, pp.\ 1088--1096. PMLR, 2017

    2017

  10. [18]

    Improved optimistic algorithms for logistic bandits

    Faury, L., Abeille, M., Calauz \`e nes, C., and Fercoq, O. Improved optimistic algorithms for logistic bandits. In International Conference on Machine Learning, pp.\ 3052--3060. PMLR, 2020

    2020

  11. [19]

    Frazier, P. I. A tutorial on bayesian optimization. arXiv preprint arXiv:1807.02811, 2018

    2018 arXiv

  12. [20]

    R., Pleiss, G., Bindel, D., Weinberger, K

    Gardner, J. R., Pleiss, G., Bindel, D., Weinberger, K. Q., and Wilson, A. G. Gpytorch: Blackbox matrix-matrix gaussian process inference with gpu acceleration. In Advances in Neural Information Processing Systems, 2018

    2018

  13. [21]

    Gonz \'a lez, J., Dai, Z., Damianou, A., and Lawrence, N. D. Preferential bayesian optimization. In International Conference on Machine Learning, pp.\ 1282--1291. PMLR, 2017

    2017

  14. [22]

    L., and Thomaz, A

    Griffith, S., Subramanian, K., Scholz, J., Isbell, C. L., and Thomaz, A. L. Policy shaping: Integrating human feedback with reinforcement learning. Advances in Neural Information Processing Systems, 26, 2013

    2013

  15. [23]

    and Yang, X.-S

    Jamil, M. and Yang, X.-S. A literature survey of benchmark functions for global optimisation problems. International Journal of Mathematical Modelling and Numerical Optimisation, 4 0 (2): 0 150--194, 2013

    2013

  16. [24]

    R., Schonlau, M., and Welch, W

    Jones, D. R., Schonlau, M., and Welch, W. J. Efficient global optimization of expensive black-box functions. Journal of Global optimization, 13: 0 455--492, 1998

    1998

  17. [25]

    Feel-good thompson sampling for contextual dueling bandits

    Li, X., Zhao, H., and Gu, Q. Feel-good thompson sampling for contextual dueling bandits. In Forty-first International Conference on Machine Learning, 2024

    2024

  18. [26]

    and Scarlett, J

    Li, Z. and Scarlett, J. Gaussian process bandit optimization with few batches. In International Conference on Artificial Intelligence and Statistics, pp.\ 92--107. PMLR, 2022

    2022

  19. [27]

    Lin, X., Wu, Z., Dai, Z., Hu, W., Shu, Y., Ng, S.-K., Jaillet, P., and Low, B. K. H. Use your instinct: Instruction optimization using neural bandits coupled with transformers. In NeurIPS 2023 Workshop on Instruction Tuning and Instruction Following, 2023

    2023

  20. [28]

    Lin, X., Dai, Z., Verma, A., Ng, S.-K., Jaillet, P., and Low, B. K. H. Prompt optimization with human feedback. In ICML 2024 Workshop on Models of Human Feedback for AI Alignment, 2024

    2024

  21. [29]

    Sample efficient reinforcement learning from human feedback via active exploration

    Mehta, V., Das, V., Neopane, O., Dai, Y., Bogunovic, I., Schneider, J., and Neiswanger, W. Sample efficient reinforcement learning from human feedback via active exploration. arXiv preprint arXiv:2312.00267, 2023 a

    2023 arXiv

  22. [30]

    Kernelized offline contextual dueling bandits

    Mehta, V., Neopane, O., Das, V., Lin, S., Schneider, J., and Neiswanger, W. Kernelized offline contextual dueling bandits. In ICML Workshop The Many Facets of Preference-Based Learning, 2023 b

    2023

  23. [31]

    Functions of positive and negative type, and their connection with the theory of integral equations

    Mercer, J. Functions of positive and negative type, and their connection with the theory of integral equations. Philosophical Transactions of the Royal Society of London. Series A, Containing Papers of a Mathematical or Physical Character, 209: 0 415--446, 1909. ISSN 02643952

    1909

  24. [32]

    Projective preferential bayesian optimization

    Mikkola, P., Todorovi \'c , M., J \"a rvi, J., Rinke, P., and Kaski, S. Projective preferential bayesian optimization. In International Conference on Machine Learning, pp.\ 6884--6892. PMLR, 2020

    2020

  25. [33]

    Dueling posterior sampling for preference-based reinforcement learning

    Novoseller, E., Wei, Y., Sui, Y., Yue, Y., and Burdick, J. Dueling posterior sampling for preference-based reinforcement learning. In Conference on Uncertainty in Artificial Intelligence, pp.\ 1029--1038. PMLR, 2020

    2020

  26. [34]

    Training language models to follow instructions with human feedback

    Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35: 0 27730--27744, 2022

    2022

  27. [35]

    Bandits with preference feedback: A stackelberg game perspective

    P \'a sztor, B., Kassraie, P., and Krause, A. Bandits with preference feedback: A stackelberg game perspective. In Advances in Neural Information Processing Systems 38, 2024

    2024

  28. [36]

    Scikit-learn: Machine learning in P ython

    Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., and Duchesnay, E. Scikit-learn: Machine learning in P ython. Journal of ...

    2011

  29. [37]

    Optimal algorithms for stochastic contextual preference bandits

    Saha, A. Optimal algorithms for stochastic contextual preference bandits. Advances in Neural Information Processing Systems, 34: 0 30050--30062, 2021

    2021

  30. [38]

    and Krishnamurthy, A

    Saha, A. and Krishnamurthy, A. Efficient and optimal algorithms for contextual dueling bandits under realizability. In International Conference on Algorithmic Learning Theory, pp.\ 968--994. PMLR, 2022

    2022

  31. [39]

    Dueling rl: Reinforcement learning with trajectory preferences

    Saha, A., Pacchiano, A., and Lee, J. Dueling rl: Reinforcement learning with trajectory preferences. In International Conference on Artificial Intelligence and Statistics, pp.\ 6263--6289. PMLR, 2023

    2023

  32. [40]

    A domain-shrinking based bayesian optimization algorithm with order-optimal regret performance

    Salgia, S., Vakili, S., and Zhao, Q. A domain-shrinking based bayesian optimization algorithm with order-optimal regret performance. Advances in Neural Information Processing Systems, 34: 0 28836--28847, 2021

    2021

  33. [41]

    Lower bounds on regret for noisy gaussian process bandit optimization

    Scarlett, J., Bogunovic, I., and Cevher, V. Lower bounds on regret for noisy gaussian process bandit optimization. In Conference on Learning Theory, pp.\ 1723--1742. PMLR, 2017

    2017

  34. [42]

    Sch \"o lkopf, B., Herbrich, R., and Smola, A. J. A generalized representer theorem. In International conference on computational learning theory, pp.\ 416--426. Springer, 2001

    2001

  35. [43]

    P., and De Freitas, N

    Shahriari, B., Swersky, K., Wang, Z., Adams, R. P., and De Freitas, N. Taking the human out of the loop: A review of bayesian optimization. Proceedings of the IEEE, 104 0 (1): 0 148--175, 2015

    2015

  36. [44]

    M., and Seeger, M

    Srinivas, N., Krause, A., Kakade, S. M., and Seeger, M. Gaussian process optimization in the bandit setting: No regret and experimental design. In International Conference on Machine Learning, 2010

    2010

  37. [45]

    Towards practical preferential bayesian optimization with skew gaussian processes

    Takeno, S., Nomura, M., and Karasuyama, M. Towards practical preferential bayesian optimization with skew gaussian processes. In International Conference on Machine Learning, pp.\ 33516--33533. PMLR, 2023

    2023

  38. [46]

    Thompson, W. R. On the likelihood that one unknown probability exceeds another in view of the evidence of two samples. Biometrika, 25 0 (3-4): 0 285--294, 1933

    1933

  39. [47]

    Optimal order simple regret for gaussian process bandits

    Vakili, S., Bouziani, N., Jalali, S., Bernacchia, A., and Shiu, D.-s. Optimal order simple regret for gaussian process bandits. Advances in Neural Information Processing Systems, 34: 0 21202--21215, 2021 a

    2021

  40. [48]

    On information gain and regret bounds in gaussian process bandits

    Vakili, S., Khezeli, K., and Picheny, V. On information gain and regret bounds in gaussian process bandits. In International Conference on Artificial Intelligence and Statistics, pp.\ 82--90. PMLR, 2021 b

    2021

  41. [49]

    Finite-time analysis of kernelised contextual bandits

    Valko, M., Korda, N., Munos, R., Flaounas, I., and Cristianini, N. Finite-time analysis of kernelised contextual bandits. In Uncertainty in Artificial Intelligence, 2013

    2013

  42. [50]

    Verma, A., Dai, Z., Lin, X., Jaillet, P., and Low, B. K. H. Neural dueling bandits: Preference-based optimization with human feedback. In The Thirteenth International Conference on Learning Representations, 2025

    2025

  43. [51]

    High-dimensional probability: An introduction with applications in data science, volume 47

    Vershynin, R. High-dimensional probability: An introduction with applications in data science, volume 47. Cambridge university press, 2018

    2018

  44. [52]

    Whitehouse, J., Ramdas, A., and Wu, S. Z. On the sublinear regret of gp-ucb. Advances in Neural Information Processing Systems, 36, 2024

    2024

  45. [53]

    and Sun, W

    Wu, R. and Sun, W. Making RL with preference-based feedback efficient via randomization. In The Twelfth International Conference on Learning Representations, 2024

    2024

  46. [54]

    Principled preferential bayesian optimization

    Xu, W., Wang, W., Jiang, Y., Svetozarevic, B., and Jones, C. Principled preferential bayesian optimization. In International Conference on Machine Learning, pp.\ 55305--55336. PMLR, 2024

    2024

  47. [55]

    Zeroth order non-convex optimization with dueling-choice bandits

    Xu, Y., Joshi, A., Singh, A., and Dubrawski, A. Zeroth order non-convex optimization with dueling-choice bandits. In Conference on Uncertainty in Artificial Intelligence, pp.\ 899--908. PMLR, 2020 a

    2020

  48. [56]

    Preference-based reinforcement learning with finite-time guarantees

    Xu, Y., Wang, R., Yang, L., Singh, A., and Dubrawski, A. Preference-based reinforcement learning with finite-time guarantees. Advances in Neural Information Processing Systems, 33: 0 18784--18794, 2020 b

    2020

  49. [57]

    and Joachims, T

    Yue, Y. and Joachims, T. Interactively optimizing information retrieval systems as a dueling bandits problem. In Proceedings of the 26th Annual International Conference on Machine Learning, pp.\ 1201--1208, 2009

    2009

  50. [58]

    The k-armed dueling bandits problem

    Yue, Y., Broder, J., Kleinberg, R., and Joachims, T. The k-armed dueling bandits problem. Journal of Computer and System Sciences, 78 0 (5): 0 1538--1556, 2012

    2012

  51. [59]

    D., and Sun, W

    Zhan, W., Uehara, M., Kallus, N., Lee, J. D., and Sun, W. Provable offline preference-based reinforcement learning. In The Twelfth International Conference on Learning Representations, 2024

    2024

  52. [60]

    Relative upper confidence bound for the k-armed dueling bandit problem

    Zoghi, M., Whiteson, S., Munos, R., and Rijke, M. Relative upper confidence bound for the k-armed dueling bandit problem. In International Conference on Machine Learning, pp.\ 10--18. PMLR, 2014

    2014

  53. [61]

    Mergerucb: A method for large-scale online ranker evaluation

    Zoghi, M., Whiteson, S., and de Rijke, M. Mergerucb: A method for large-scale online ranker evaluation. In Proceedings of the Eighth ACM International Conference on Web Search and Data Mining, pp.\ 17--26, 2015

    2015

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.