pith. sign in

arxiv: 2405.14657 · v2 · pith:RGZD2C57new · submitted 2024-05-23 · 💻 cs.LG · stat.ML

Anchor-Based Heteroscedastic Noise for Preferential Bayesian Optimization

Marshal Arijona Sinaga , Julien Martinelli , Samuel Kaski This is my paper

Pith reviewed 2026-05-24 01:07 UTC · model grok-4.3

classification 💻 cs.LG stat.ML
keywords preferential Bayesian optimizationheteroscedastic noiseanchor exampleskernel density estimationrisk-averse acquisition functionshuman preferencesGaussian processes
0
0 comments X

The pith

User-supplied anchors enable modeling of input-dependent comparison noise in preferential Bayesian optimization.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper develops a heteroscedastic noise model for preferential Bayesian optimization, where users provide a few reliable anchor examples before optimization begins. These anchors are processed with a kernel density estimator to produce a map of where comparisons are likely to be noisy or reliable. This map is integrated into Gaussian process surrogates to derive acquisition functions that account for both expected utility and the risk of uncertain comparisons. Theoretical results establish that a risk-adjusted version of the expected utility of the best option maintains near-optimality, and the estimator is consistent under ideal conditions. Experiments demonstrate better risk-adjusted performance on synthetic and real human preference data.

Core claim

By incorporating an input-dependent uncertainty map derived from user-provided anchors via kernel density estimation into preferential Gaussian process models, the method allows derivation of risk-averse acquisition functions for preferential Bayesian optimization. A risk-adjusted variant of EUBO preserves the one-step Bayes-optimality guarantee up to an additive constant, while the KDE estimator achieves standard consistency and concentration rates under an idealized i.i.d. anchor model.

What carries the argument

The anchor-based kernel density estimator that converts a small set of reliable examples into an input-dependent map of user uncertainty for comparison noise.

If this is right

  • Risk-averse acquisition functions balance high utility with ease of reliable comparison.
  • The approach maintains one-step Bayes-optimality for the adjusted EUBO up to an additive constant.
  • KDE-based uncertainty estimates are consistent and concentrate at standard rates under i.i.d. anchors.
  • Experiments show improved performance on synthetic problems and human-preference datasets.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Anchor placement could be chosen adaptively to maximize the informativeness of the noise map.
  • The method may generalize to other sequential decision problems with heterogeneous query difficulty.
  • Users might benefit from feedback on how their anchors influence the optimization path.
  • Combining this with other noise models could handle both input-dependent and output-dependent heteroscedasticity.

Load-bearing premise

Users can supply a small set of reliable anchor examples whose locations predict comparison noise levels at other points.

What would settle it

A dataset of human comparisons where the noise level does not correlate with proximity to the provided anchors, resulting in no performance gain from the heteroscedastic model.

Figures

Figures reproduced from arXiv: 2405.14657 by Julien Martinelli, Marshal Arijona Sinaga, Samuel Kaski.

Figure 1
Figure 1. Figure 1: Heteroscedastic Preferential Bayesian Optimization. Top left: latent user utility with [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Results for three synthetic problems: Sine1D [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Results for the Hartmann4D test function using different approximated inference techniques. [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
read the original abstract

Preferential Bayesian optimization (PBO) learns latent utilities from pairwise comparisons, but most existing methods assume homoscedastic comparison noise. This is inadequate in human-in-the-loop settings, where a user may compare some designs reliably and others only hesitantly. We propose a heteroscedastic noise model for PBO: before optimization, the user provides a small set of reliable examples, called anchors, and a kernel density estimator (KDE) turns these anchors into an input-dependent map of user uncertainty. We incorporate this map into preferential GP surrogates and derive risk-averse acquisition functions that trade off utility and ease of comparison. We further show that a risk-adjusted variant of the popular expected utility of the best option (EUBO) preserves the one-step Bayes-optimality guarantee up to an additive constant, and that under an idealized i.i.d. anchor model the KDE estimator enjoys standard consistency and concentration rates. Experiments on synthetic problems and human-preference datasets show improved risk-adjusted performance and clarify how anchor placement affects the method.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript proposes an anchor-based heteroscedastic noise model for preferential Bayesian optimization (PBO). Users supply a small set of reliable anchor examples before optimization; a kernel density estimator (KDE) converts these into an input-dependent map of comparison uncertainty. This map is incorporated into preferential Gaussian process surrogates to derive risk-averse acquisition functions. A risk-adjusted variant of expected utility of the best option (EUBO) is shown to preserve one-step Bayes optimality up to an additive constant. Under an idealized i.i.d. anchor model, the KDE is claimed to enjoy standard consistency and concentration rates. Experiments on synthetic problems and human-preference datasets report improved risk-adjusted performance.

Significance. If the central claims hold, the work provides a practical mechanism to handle varying comparison reliability in human-in-the-loop PBO, which is a common but under-modeled issue. The preservation of a near-optimality guarantee for the risk-adjusted EUBO and the explicit use of user-supplied anchors are concrete contributions. The approach could improve reliability in preference-based optimization tasks, though its impact depends on the realism of the anchor model.

major comments (2)
  1. [theoretical analysis section on KDE] Theoretical analysis (consistency claim): The KDE consistency and concentration rates are stated to follow standard results under an idealized i.i.d. anchor model. However, the method description specifies that anchors are a small fixed set deliberately supplied by the user and chosen for reliability rather than sampled i.i.d. from a noise-generating distribution. Standard KDE rates require random sampling and do not automatically extend to deliberate selection, which can introduce location bias and density distortion. This assumption is load-bearing for the justification of the input-dependent uncertainty map used by the risk-averse acquisitions.
  2. [section deriving risk-adjusted EUBO] Acquisition function derivation (risk-adjusted EUBO): The one-step Bayes-optimality guarantee for the risk-adjusted EUBO is established only up to an additive constant. The manuscript does not provide a bound on the size of this constant or analyze its dependence on the heteroscedasticity map or anchor placement. Without such quantification, it is unclear whether the guarantee remains meaningful for the risk-averse setting that is the paper's central motivation.
minor comments (2)
  1. [abstract and introduction] The abstract and introduction could more explicitly distinguish the idealized i.i.d. assumption used for the consistency proof from the deliberate anchor selection used in the algorithm and experiments.
  2. [model and acquisition sections] Notation for the heteroscedasticity map (e.g., how the KDE output enters the GP likelihood) should be introduced earlier and used consistently in the acquisition-function sections.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback. We address each major comment below and indicate planned revisions where appropriate.

read point-by-point responses

  1. Referee: Theoretical analysis (consistency claim): The KDE consistency and concentration rates are stated to follow standard results under an idealized i.i.d. anchor model. However, the method description specifies that anchors are a small fixed set deliberately supplied by the user and chosen for reliability rather than sampled i.i.d. from a noise-generating distribution. Standard KDE rates require random sampling and do not automatically extend to deliberate selection, which can introduce location bias and density distortion. This assumption is load-bearing for the justification of the input-dependent uncertainty map used by the risk-averse acquisitions.

    Authors: The manuscript already qualifies the consistency and concentration results as holding specifically 'under an idealized i.i.d. anchor model.' We agree that deliberate, fixed user selection of anchors does not satisfy the random sampling assumption required for standard KDE theory, and that this distinction merits explicit discussion. We will revise the theoretical analysis section to clarify the idealized nature of the assumption, note that the rates provide justification only in that setting, and discuss how non-random anchor placement may affect the uncertainty map in practice while emphasizing that the method's utility is ultimately validated empirically. revision: partial

  2. Referee: Acquisition function derivation (risk-adjusted EUBO): The one-step Bayes-optimality guarantee for the risk-adjusted EUBO is established only up to an additive constant. The manuscript does not provide a bound on the size of this constant or analyze its dependence on the heteroscedasticity map or anchor placement. Without such quantification, it is unclear whether the guarantee remains meaningful for the risk-averse setting that is the paper's central motivation.

    Authors: Because the additive constant is independent of the queried pair, the argmax of the risk-adjusted EUBO is identical to that of the standard EUBO; thus the one-step optimality property is preserved exactly in terms of the selected query. We nevertheless agree that a bound on the constant and its dependence on the heteroscedasticity map would strengthen the result. The manuscript does not derive such a bound. We will revise the acquisition-function section to explicitly note this limitation and identify bounding the constant as an item for future work. revision: partial

Circularity Check

0 steps flagged

No significant circularity; claims rest on external rates and explicit derivations

full rationale

The paper states that a risk-adjusted EUBO preserves one-step Bayes-optimality up to an additive constant and that KDE enjoys standard consistency rates under an explicitly idealized i.i.d. anchor model. These statements appeal to external statistical results rather than reducing any prediction or guarantee to a fitted parameter or self-citation by construction. No equations or steps in the provided text equate an output to its input via definition, renaming, or load-bearing self-reference. The derivation chain remains self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

The central claims rest on the existence of reliable user-provided anchors whose density can be used to predict comparison noise, plus standard KDE consistency results under i.i.d. assumptions. No explicit free parameters or invented entities are named in the abstract.

axioms (2)
  • domain assumption User-provided anchors are reliable examples whose locations predict input-dependent comparison noise via KDE
    This premise is required to construct the heteroscedastic noise map before optimization begins.
  • standard math Standard KDE consistency and concentration rates hold under an idealized i.i.d. anchor model
    Invoked to support the theoretical properties of the uncertainty estimator.

pith-pipeline@v0.9.0 · 5713 in / 1541 out tokens · 28484 ms · 2026-05-24T01:07:38.153869+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. A tutorial on learning from preferences and choices with Gaussian Processes

    cs.LG 2024-03 unverdicted novelty 3.0

    Tutorial on a GP-based framework for preference and choice learning that unifies random utility models, limits of discernment, and multi-utility scenarios via customized likelihoods for object and label preferences.

Reference graph

Works this paper leans on

42 extracted references · 42 canonical work pages · cited by 1 Pith paper · 1 internal anchor

  1. [1]

    Howey, Krikamol Muandet, Michael A

    Masaki Adachi, Brady Planden, David A. Howey, Krikamol Muandet, Michael A. Osborne, and Siu Lun Chau. Looping in the human: Collaborative and explainable bayesian optimization, 2023

    work page 2023

  2. [2]

    qeubo: A decision-theoretic acquisition function for preferential bayesian optimization

    Raul Astudillo, Zhiyuan Jerry Lin, Eytan Bakshy, and Peter Frazier. qeubo: A decision-theoretic acquisition function for preferential bayesian optimization. In Francisco Ruiz, Jennifer Dy, and Jan-Willem van de Meent, editors, Proceedings of The 26th International Conference on Artificial Intelligence and Statistics , volume 206 of Proceedings of Machine ...

    work page 2023

  3. [3]

    The skew-normal and related families, volume 3

    Adelchi Azzalini. The skew-normal and related families, volume 3. Cambridge University Press, 2013

    work page 2013

  4. [4]

    Ideals, central tendency, and frequency of instantiation as determinants of graded structure in categories

    Lawrence W Barsalou. Ideals, central tendency, and frequency of instantiation as determinants of graded structure in categories. Journal of experimental psychology: learning, memory, and cognition, 11(4):629, 1985

    work page 1985

  5. [5]

    Preferential bayesian optimisation with skew gaussian processes

    Alessio Benavoli, Dario Azzimonti, and Dario Piga. Preferential bayesian optimisation with skew gaussian processes. In Proceedings of the Genetic and Evolutionary Computation Con- ference Companion, GECCO ’21, page 1842–1850, New York, NY , USA, 2021. Association for Computing Machinery. ISBN 9781450383516. doi: 10.1145/3449726.3463128. URL https://doi.or...

    work page doi:10.1145/3449726.3463128 2021

  6. [6]

    On a modification of chebyshev’s inequality and of the error formula of laplace

    Sergei Bernstein. On a modification of chebyshev’s inequality and of the error formula of laplace. Ann. Sci. Inst. Sav. Ukraine, Sect. Math, 1(4):38–49, 1924

    work page 1924

  7. [7]

    A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning

    Eric Brochu, Vlad M Cora, and Nando De Freitas. A tutorial on bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning. arXiv preprint arXiv:1012.2599, 2010

    work page internal anchor Pith review Pith/arXiv arXiv 2010

  8. [8]

    The Adaptive Web - Methods and Strategies of Web Personalization

    Peter Brusilovsky, Alfred Kobsa, and Wolfgang Nejdl. The Adaptive Web - Methods and Strategies of Web Personalization . 01 2007. ISBN 978-3-540-72078-2. doi: 10.1007/ 978-3-540-72079-9

    work page 2007

  9. [9]

    Preference learning with gaussian processes

    Wei Chu and Zoubin Ghahramani. Preference learning with gaussian processes. In Proceedings of the 22nd international conference on Machine learning, pages 137–144, 2005

    work page 2005

  10. [10]

    Hebo: Pushing the limits of sample-efficient hyperparameter optimisation

    Alexander Cowen-Rivers, Wenlong Lyu, Rasul Tutunov, Zhi Wang, Antoine Grosnit, Ryan-Rhys Griffiths, Alexandre Maravel, Jianye Hao, Jun Wang, Jan Peters, and Haitham Bou Ammar. Hebo: Pushing the limits of sample-efficient hyperparameter optimisation. Journal of Artificial Intelligence Research, 74, 07 2022

    work page 2022

  11. [11]

    Afonso Eduardo and Michael U. Gutmann. Bayesian optimization with informative covariance, 2023

    work page 2023

  12. [12]

    Expertise in context: Human and machine

    Paul J Feltovic, Kenneth M Ford, and Robert R Hoffman. Expertise in context: Human and machine. AI Magazine, 22(3):62, 2001

    work page 2001

  13. [13]

    Rates of strong uniform consistency for multivariate kernel density estimators

    Evarist Giné and Armelle Guillou. Rates of strong uniform consistency for multivariate kernel density estimators. In Annales de l’Institut Henri Poincare (B) Probability and Statistics , volume 38, pages 907–921. Elsevier, 2002

    work page 2002

  14. [14]

    Bandwidth selection in kernel density estimation: Oracle inequalities and adaptive minimax optimality

    Alexander Goldenshluger and Oleg Lepski. Bandwidth selection in kernel density estimation: Oracle inequalities and adaptive minimax optimality. The Annals of Statistics, 39(3):1608 – 1632, 2011. doi: 10.1214/11-AOS883. URL https://doi.org/10.1214/11-AOS883

    work page doi:10.1214/11-aos883 2011

  15. [15]

    Risk management: approaches for fixed income markets, volume 73

    Bennett W Golub and Leo M Tilman. Risk management: approaches for fixed income markets, volume 73. John Wiley & Sons, 2000

    work page 2000

  16. [16]

    Preferential bayesian optimization

    Javier González, Zhenwen Dai, Andreas Damianou, and Neil D Lawrence. Preferential bayesian optimization. In International Conference on Machine Learning, pages 1282–1291. PMLR, 2017. 10

    work page 2017

  17. [17]

    Achieving robustness to aleatoric uncertainty with heteroscedastic bayesian optimisation

    Ryan-Rhys Griffiths, Alexander A Aldrick, Miguel Garcia-Ortegon, Vidhi Lalchand, et al. Achieving robustness to aleatoric uncertainty with heteroscedastic bayesian optimisation. Ma- chine Learning: Science and Technology, 3(1):015004, 2021

    work page 2021

  18. [18]

    The elements of statistical learning: data mining, inference, and prediction, volume 2

    Trevor Hastie, Robert Tibshirani, Jerome H Friedman, and Jerome H Friedman. The elements of statistical learning: data mining, inference, and prediction, volume 2. Springer, 2009

    work page 2009

  19. [19]

    The psychology of expertise: Cognitive research and empirical AI

    Robert R Hoffman. The psychology of expertise: Cognitive research and empirical AI. Psychol- ogy Press, 2014

    work page 2014

  20. [20]

    Augmenting bayesian optimization with preference-based expert feedback

    Daolang Huang, Louis Filstroff, Petrus Mikkola, Runkai Zheng, Milica Todorovic, and Samuel Kaski. Augmenting bayesian optimization with preference-based expert feedback. InICML 2023 Workshop The Many Facets of Preference-Based Learning, 2023. URL https://openreview. net/forum?id=OJXfwj2hiP

    work page 2023

  21. [21]

    Stochastic bayesian optimization with unknown continuous context distribution via kernel density estimation, 2023

    Xiaobin Huang, Lei Song, Ke Xue, and Chao Qian. Stochastic bayesian optimization with unknown continuous context distribution via kernel density estimation, 2023

    work page 2023

  22. [22]

    Risk management and financial institutions,+ Web Site, volume 733

    John Hull. Risk management and financial institutions,+ Web Site, volume 733. John Wiley & Sons, 2012

    work page 2012

  23. [23]

    A general framework for user-guided bayesian optimization, 2023

    Carl Hvarfner, Frank Hutter, and Luigi Nardi. A general framework for user-guided bayesian optimization, 2023

    work page 2023

  24. [24]

    Prospect theory: An analysis of decision under risk

    Daniel Kahneman and Amos Tversky. Prospect theory: An analysis of decision under risk. In Handbook of the fundamentals of financial decision making: Part I, pages 99–127. World Scientific, 2013

    work page 2013

  25. [25]

    Sequential gallery for interactive visual design optimization

    Yuki Koyama, Issei Sato, and Masataka Goto. Sequential gallery for interactive visual design optimization. ACM Transactions on Graphics (TOG), 39(4):88–1, 2020

    work page 2020

  26. [26]

    Adaptivity to local smoothness and dimension in kernel regression

    Samory Kpotufe and Vikas Garg. Adaptivity to local smoothness and dimension in kernel regression. In C.J. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K.Q. Weinberger, editors, Advances in Neural Information Processing Systems, volume 26. Curran Associates, Inc., 2013. URL https://proceedings.neurips.cc/paper_files/paper/2013/file/ 28fc2782ea7ef51c11...

    work page 2013

  27. [27]

    A tutorial on energy- based learning

    Yann LeCun, Sumit Chopra, Raia Hadsell, M Ranzato, and Fujie Huang. A tutorial on energy- based learning. Predicting structured data, 1(0), 2006

    work page 2006

  28. [28]

    Risk-averse heteroscedastic bayesian optimization

    Anastasia Makarova, Ilnura Usmanova, Ilija Bogunovic, and Andreas Krause. Risk-averse heteroscedastic bayesian optimization. Advances in Neural Information Processing Systems, 34: 17235–17245, 2021

    work page 2021

  29. [29]

    Psychological essentialism

    Douglas L Medin and Andrew Ortony. Psychological essentialism. Similarity and analogical reasoning, 179:195, 1989

    work page 1989

  30. [30]

    Updating quasi-newton matrices with limited storage

    Jorge Nocedal. Updating quasi-newton matrices with limited storage. Mathematics of computa- tion, 35(151):773–782, 1980

    work page 1980

  31. [31]

    Financial econometrics: from basics to advanced modeling techniques

    Svetlozar T Rachev, Stefan Mittnik, Frank J Fabozzi, Sergio M Focardi, et al. Financial econometrics: from basics to advanced modeling techniques. John Wiley & Sons, 2007

    work page 2007

  32. [32]

    Carl Edward Rasmussen and Christopher K. I. Williams. Gaussian processes for machine learning. Adaptive computation and machine learning. MIT Press, 2006. ISBN 026218253X

    work page 2006

  33. [33]

    Fast computation of kernel estimators

    Vikas C Raykar, Ramani Duraiswami, and Linda H Zhao. Fast computation of kernel estimators. Journal of Computational and Graphical Statistics, 19(1):205–220, 2010

    work page 2010

  34. [34]

    Distinguishing types of superficial similarities: Different effects on the access and use of earlier problems

    Brian H Ross. Distinguishing types of superficial similarities: Different effects on the access and use of earlier problems. Journal of Experimental Psychology: Learning, memory, and cognition, 15(3):456, 1989

    work page 1989

  35. [35]

    Introduction to Nonparametric Statistics

    Bodhisattva Sen. Introduction to Nonparametric Statistics. 2020. 11

    work page 2020

  36. [36]

    Preferential batch bayesian optimization

    Eero Siivola, Akash Kumar Dhaka, Michael Riis Andersen, Javier González, Pablo García Moreno, and Aki Vehtari. Preferential batch bayesian optimization. In 2021 IEEE 31st International Workshop on Machine Learning for Signal Processing (MLSP), pages 1–6, 2021. doi: 10.1109/MLSP52302.2021.9596494

    work page doi:10.1109/mlsp52302.2021.9596494 2021

  37. [37]

    Judgment under uncertainty: Heuristics and biases

    Paul Slovic and Amos Tversky. Judgment under uncertainty: Heuristics and biases. Cambridge University Press, 1982

    work page 1982

  38. [38]

    Bayesian optimization with conformal prediction sets

    Samuel Stanton, Wesley Maddox, and Andrew Gordon Wilson. Bayesian optimization with conformal prediction sets. In Francisco Ruiz, Jennifer Dy, and Jan-Willem van de Meent, editors, Proceedings of The 26th International Conference on Artificial Intelligence and Statistics , volume 206 of Proceedings of Machine Learning Research, pages 959–986. PMLR, 25–27 Apr

    work page

  39. [39]

    URL https://proceedings.mlr.press/v206/stanton23a.html

    work page

  40. [40]

    Cross-validatory choice and assessment of statistical predictions

    Mervyn Stone. Cross-validatory choice and assessment of statistical predictions. Journal of the royal statistical society: Series B (Methodological), 36(2):111–133, 1974

    work page 1974

  41. [41]

    Human-in-the-loop assisted de novo molecular design

    Iiris Sundin, Alexey V oronov, Haoping Xiao, Kostas Papadopoulos, Esben Jannik Bjerrum, Markus Heinonen, Atanas Patronov, Samuel Kaski, and Ola Engkvist. Human-in-the-loop assisted de novo molecular design. Journal of Cheminformatics, 14(1), 2022

    work page 2022

  42. [42]

    Towards practical preferential bayesian optimization with skew gaussian processes

    Shion Takeno, Masahiro Nomura, and Masayuki Karasuyama. Towards practical preferential bayesian optimization with skew gaussian processes. In International Conference on Machine Learning, pages 33516–33533. PMLR, 2023. 12 Appendix — Heteroscedastic Preferential Bayesian Optimization with Informative Noise Distributions Outline. The appendix is organized a...

    work page 2023