Pith. sign in

REVIEW 2 major objections 4 minor 116 references

Multi-Metric Adaptive Experimental Design Under a Fixed Budget with Validation

T0 review · 2 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read SHRVar selects the treatment most likely to pass a multi-metric A/B test, with error probability that decays exponentially in the exploration budget.

desk verdict Proposition 1 is algebraically wrong: the sqrt(Tv/2) belongs on the effect-size term, not on the quantile term, so SHRVar optimizes the wrong objective. read the letter →

arxiv 2506.03062 v2 pith:PJV3JXNW submitted 2025-06-03 cs.LG stat.ML

Qining Zhang , Tanner Fiez , Yi Liu , Wenyang Liu This is my paper
classification cs.LGstat.ML MSC 62L05
keywords multi-armedbanditsbest-armidentificationsequentialhalvingA/Btestingadaptiveexperimentaldesignheterogeneousvariancemultiplemetricsz-values
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to show that a two-phase setup, an adaptive exploration phase that picks a treatment and then a classical A/B test that verifies it, can be made statistically efficient even with many treatments and multiple success metrics such as revenue, safety, and engagement. Its central claim is that the right exploration objective is not the treatment with the largest average reward but the treatment most likely to pass every metric's validation test, which reduces to maximizing the smallest z-value across metrics. The proposed algorithm, SHRVar, allocates samples by relative variance to equalize estimation error and eliminates half the candidates each round using estimated z-values. Theorem 1 bounds the probability of recommending the wrong treatment by an exponentially decaying term governed by a new instance-dependent complexity $H_3$. If correct, the framework lets practitioners keep the inferential benefits of an A/B test while using adaptivity to screen a large candidate set.

What carries the argument

The central object is the z-value $z_{a,i}=(\mu_{a,i}-\mu_{0,i})/\sqrt{\sigma^2_{a,i}+\sigma^2_{0,i}}+\xi_{a,i}$, a per-metric signal-to-noise ratio plus a validation constant; the best treatment maximizes the minimum z-value across metrics, and the minimizing metric is called the bottleneck metric. The algorithm's sampling rule uses the relative variances $\rho^2_{a,i}=\sigma^2_{a,i}/(\sigma^2_{a,i}+\sigma^2_{0,i})$ and $\lambda^2_{a,i}=\sigma^2_{0,i}/(\sigma^2_{a,i}+\sigma^2_{0,i})$ to allocate pulls so that the variance of the empirical z-value estimates is approximately equalized across active treatments. The $H_3$ complexity measure aggregates z-value gaps over elimination stages, weighted by the total relative variance of control and treatments, and it appears in the exponential error bound.

What would settle it

Run SHRVar on an instance with non-normal but sub-Gaussian rewards where the treatment maximizing the normal z-value differs from the treatment maximizing the empirically measured validation pass rate; if SHRVar's recommended treatment does not pass validation at the highest measured rate, the normal-theory characterization behind Proposition 1 is the part that fails.

Watch

Extended reading notes

Core claim

The paper proves that, under normal-theory validation, the treatment maximizing the minimum validation-pass probability is exactly the treatment maximizing the minimum z-value $z_{a,i}=(\mu_{a,i}-\mu_{0,i})/\sqrt{\sigma^2_{a,i}+\sigma^2_{0,i}}+\xi_{a,i}$, where $\xi_{a,i}$ is a known constant set by the confidence level or Bayesian threshold and the validation horizon. It then gives SHRVar, a sequential-halving algorithm whose sampling rule equalizes the variance of the z-value estimates using relative variances $\rho^2_{a,i}=\sigma^2_{a,i}/(\sigma^2_{a,i}+\sigma^2_{0,i})$, and whose elimination rule keeps the half of active treatments with the largest estimated minimum z-value. The main theorem states that SHRVar identifies the best treatment with error at most $6M\log_2 A\,\exp(-T/(2H_3\log_2 A))$, where $H_3$ is an instance-dependent complexity measure generalizing the $H_2$ measures of sequential halving and its heterogeneous-variance variant. The paper also reports experiments showing the algorithm outperforms uniform and variance-only sampling baselines, and that its known-variance performance is preserved when variances must be estimated.

Load-bearing premise

The mapping from treatment quality to the normal-CDF z-value is proved only when rewards are normally distributed, while the formal model only assumes sub-Gaussian rewards; if real rewards are not normal, the treatment chosen may not be the one most likely to pass the actual validation test.

Editorial extensions

If this is right

  • A fixed-budget exploration phase can be aimed at downstream A/B-test success rather than at raw reward, so practitioners can screen hundreds of variants while still reporting valid average treatment effects from the validation phase.
  • SHRVar reduces to sequential halving when there is one metric with equal variances, and to its heterogeneous-variance variant when variances differ, so the exponential guarantee comes without sacrificing performance in simpler models.
  • The z-value view makes the validation design, including confidence level, Bayesian prior, and horizon, part of the exploration objective, meaning the choice of validation test changes which treatment is worth finding.
  • The error bound grows only logarithmically in the number of treatments and metrics, so the method is designed to scale to hundreds of candidates and several guardrail metrics.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same maximize-the-minimum-z-value principle could be adapted to non-normal validation tests, such as permutation or bootstrap tests, but the exponential bound would then need a different large-deviation argument than the normal CDF.
  • One testable extension is to treat the validation constant $\xi_{a,i}$ as a tunable knob: by raising or lowering the Bayesian prior variance or confidence threshold, a platform could deliberately bias exploration toward conservative or aggressive deployments.
  • The relative-variance allocation could be combined with sequential variance estimation, as in the paper's SHRVar-Ada variant, in non-stationary settings where the equalization property would need to be re-derived.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. This paper proposes a fixed-budget multi-metric best-arm identification framework with a downstream validation A/B test. It defines the best treatment as the one maximizing the minimum per-metric validation success probability, introduces the SHRVar algorithm with a relative-variance-based sampling rule and elimination based on estimated z-values, and claims an exponential error-probability bound governed by a new complexity measure H3 that generalizes the complexity measures of SH and SHVar. Numerical experiments compare SHRVar with several baselines and support the algorithm's practical performance.

Significance. If the central characterization and the bound were correct, the paper would make a valuable contribution: it connects best-arm identification with a validation phase, handles heterogeneous variances across metrics and treatments, and provides a proof with an explicit exponential rate. The relative-variance sampling idea is a natural and potentially useful extension of SHVar. The paper also includes a complete proof and extensive numerical experiments. However, because Proposition 1 is algebraically incorrect, the algorithm's stated target is not the actual validation objective, and the main theoretical claim is therefore not established as stated. The issues are significant but appear repairable.

major comments (2)
  1. [Section 4, Eq. (2) and Appendix C] Proposition 1 contains a scaling error in the definition of the z-value. For non-Bayesian validation, the validation success probability is P(E_{v,i}) = Phi( sqrt(T_v/2) * (mu_{a,i}-mu_{0,i}) / sqrt(sigma^2_{a,i}+sigma^2_{0,i}) + Phi^{-1}(delta_i) ), as the proof in Appendix C itself derives. The paper instead writes Eq. (2) with the factor sqrt(T_v/2) multiplying Phi^{-1}(delta_i) and not multiplying the signal-to-noise term. The two expressions are not related by a common positive transformation when the validation constants vary across metrics, so the argmax of the minimum can differ. For example, with T_v=4, delta_1=0.5, delta_2=0.9772, and control variances chosen so that sigma^2_a+sigma^2_0=1, take Treatment 1 with normalized effects (1,0) and Treatment 2 with normalized effects (1.1,-0.5). Treatment 1 has true minimum success probability Phi(1.414)=0.921 and Eq. (2) minimum z-value 1.0; Treatment 2 has true minimum success probability Phi(1.293)=0.902 and Eq. (2) minimum z-value 1.1. Thus Eq. (2) selects Treatment 2 while Eq. (1) selects Treatment 1. This means Theorem 1 is a guarantee about a target different from the stated validation objective.
  2. [Section 3 vs. Appendix C] The formal model in Section 3 assumes sub-Gaussian rewards, but the proof of Proposition 1 in Appendix C states 'Since the reward follows a normal distribution' and derives the validation probability as an exact normal CDF. Under the stated sub-Gaussian model, the validation success probability is not exactly the normal CDF, and the equivalence in Proposition 1 is not exact. The paper should either strengthen the model to Gaussian rewards (which would be consistent with the known-variance z-test) or restate the characterization as an approximation and adjust the surrounding claims accordingly.
minor comments (4)
  1. [Algorithm 1, line 8] The final recommendation line should read 'recommend the arm in A_{rlog2 As+1}' with the subscript typeset correctly.
  2. [Section 7.1] The text says 'We report the error probability Ppˆa“a˚q that the algorithm successfully recommends a˚'; this is a success probability, not an error probability, and the wording should be corrected.
  3. [Section 7.2] The experiments use A=27 treatments, which is not a power of two and so falls outside the theoretical setting of Theorem 1; if 2^7 is intended, the text should be corrected.
  4. [Section 3] The model says each reward is sub-Gaussian with a variance proxy sigma^2_{a,i}, but the algorithm later treats sigma^2_{a,i} as the exact variance in the z-test and in the sampling rule; this distinction should be clarified.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the H3 exponent is derived from the proof, and self-citations are contextual rather than load-bearing.

full rationale

The paper's central claim is not circular. The best-treatment characterization in Proposition 1 is presented as a derivation from the validation event in Eq. (1) via a normal-CDF calculation; whether that calculation is correct is a separate question. The SHRVar error bound in Theorem 1 is obtained from the proof's union-bound/Markov argument in Appendix D: H3 is defined from the same z-gaps, relative variances, and kappa terms that appear in the stage-wise probability estimates, so the exponential rate is a derived instance-dependent statement rather than an assumed input. The paper's comparisons to SH and SHVar are algebraic reductions of H3 to the complexity measures of Karnin et al. (2013) and Lalitha et al. (2023); they do not import those results as the conclusion. Self-citations occur in related work (Zhang & Ying 2023; Kanarios et al. 2024) and as background for the variance-equalization principle (Weltz et al. 2023, which includes co-author Fiez), but the sampling rule is derived by an explicit min-max variance calculation in Section 5 and Theorem 1 is proved in the appendix, so no self-citation is load-bearing. The main caveat is a correctness concern, not a circularity: the proof of Proposition 1 says 'Since the reward follows a normal distribution' although Section 3 only assumes sub-Gaussian rewards, and the appended derivation appears to place sqrt(Tv/2) on the validation constant in the z-value. If that scaling is wrong, SHRVar may be optimizing a proxy objective, but this would be an algebraic/modeling error rather than a reduction of the paper's predictions to its inputs. For these reasons the circularity score is at the low end, 2.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The main theorem rests on the stated sub-Gaussian model, independence across metrics, known variances, and an implicit normality assumption for the validation-phase probability calculations that is used in Proposition 1 but not stated in the model. No parameters are fit to data.

assumptions (4)
  • domain assumption Reward distributions are sub-Gaussian with known variances for all arms and metrics
    Stated in Section 3; the exploration-phase concentration bounds rely on it.
  • domain assumption Reward metrics are independent within each arm
    Stated in Section 3; used in Lemma 1 to factor probabilities over metrics.
  • domain assumption Validation rejection probabilities follow the normal CDF
    Used in the proof of Proposition 1, where the text says 'Since the reward follows a normal distribution,' but this is not stated in the Section 3 model, which only assumes sub-Gaussianity.
  • domain assumption Validation horizon and confidence parameters are known and fixed
    The z-value objective in Equation 2 depends on known delta_i, q_i, and tau_i, which are treated as given inputs.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Multi-Metric Adaptive Experimental Design Under a Fixed Budget with Validation." pith.science (2026). https://pith.science/paper/PJV3JXNW

@misc{pith2026250603062,
  author       = {Pith},
  title        = {Pith review of: Multi-Metric Adaptive Experimental Design Under a Fixed Budget with Validation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PJV3JXNW}},
  note         = {Machine review of arXiv:2506.03062}
}
read the original abstract

A/B tests in online experiments face statistical power challenges when testing multiple candidates simultaneously, while adaptive experimental designs (AED) alone fall short in inferring experiment statistics such as the average treatment effect, especially with many metrics (e.g., revenue, safety) and heterogeneous variances. This paper proposes a fixed-budget multi-metric AED framework with a two-phase structure: an adaptive exploration phase to identify the best treatment, and a validation phase with an A/B test to verify the treatment's quality and infer statistics. We propose SHRVar, which generalizes sequential halving (SH) with a novel relative-variance-based sampling and an elimination strategy built on reward z values. It achieves a provable error probability that decreases exponentially, where the exponent H3 generalizes the complexity measure for SH and SHVar with homogeneous and heterogeneous variances, respectively. Numerical experiments demonstrate its performance and robustness.

Figures

Figures reproduced from arXiv: 2506.03062 by the authors.

Figure 1
Figure 1. E-Commerce Webpage Development: the framework consists of two phases: exploration and validation. [PITH_FULL_IMAGE:figures/full_fig_p007_1.png] view at source ↗
Figure 2
Figure 2. (a) Exploration Accuracy: rate of identifying the best treatment in exploration as time horizon [PITH_FULL_IMAGE:figures/full_fig_p015_2.png] view at source ↗
Figure 3
Figure 3. (a) Exploration Accuracy: rate of identifying the best treatment in exploration as time horizon [PITH_FULL_IMAGE:figures/full_fig_p016_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: (a) Exploration Accuracy: rate of identifying the best treatment in exploration as time horizon [PITH_FULL_IMAGE:figures/full_fig_p017_4.png]
Figure 5
Figure 5. Figure 5: (a) Standard instance: the best treatment is better than control in all metrics. (b) Null instance: no [PITH_FULL_IMAGE:figures/full_fig_p018_5.png]
Figure 6
Figure 6. Figure 6: A toy example to compute δsp1q where treatment 2 has the largest LCB, i.e., arg maxa LCBδ s paq “ 2. Empirical z values are marked with solid lines and confidence intervals are marked with color bars. In (a), δ is too small so that UCBδ s p1q ą LCBδ s p2q. In (c), δ is…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

116 extracted references · 65 canonical work pages

  1. [1]

    Almost optimal exploration in multi-armed bandits

    Zohar Karnin, Tomer Koren, and Oren Somekh. Almost optimal exploration in multi-armed bandits. In Proceedings of the 30th International Conference on Machine Learning, volume 28 of Proceedings of Machine Learning Research, pages 1238--1246, Atlanta, Georgia, USA, 17--19 Jun 2013. PMLR

    2013

  2. [2]

    Fixed-budget best-arm identification with heterogeneous reward variances

    Anusha Lalitha Lalitha, Kousha Kalantari, Yifei Ma, Anoop Deoras, and Branislav Kveton. Fixed-budget best-arm identification with heterogeneous reward variances. In Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence, volume 216 of Proceedings of Machine Learning Research, pages 1164--1173. PMLR, 31 Jul--04 Aug 2023

    2023

  3. [3]

    Trustworthy online controlled experiments: A practical guide to a/b testing

    Ron Kohavi, Diane Tang, and Ya Xu. Trustworthy online controlled experiments: A practical guide to a/b testing. Cambridge University Press, 2020

    2020

  4. [4]

    Guido W. Imbens. Nonparametric estimation of average treatment effects under exogeneity: A review. The Review of Economics and Statistics, 86 0 (1): 0 4--29, 02 2004. ISSN 0034-6535. doi:10.1162/003465304323023651

    doi:10.1162/003465304323023651 2004

  5. [5]

    Semiparametric efficient inference in adaptive experiments

    Thomas Cook, Alan Mishler, and Aaditya Ramdas. Semiparametric efficient inference in adaptive experiments. In NeurIPS 2023 Workshop on Adaptive Experimental Design and Active Learning in the Real World, 2023

    2023

  6. [6]

    Asymptotically optimal adaptive a/b tests for average treatment effect

    Vikas Deep, Achal Bassamboo, and Sandeep Juneja. Asymptotically optimal adaptive a/b tests for average treatment effect. SSRN, 2023

    2023

  7. [7]

    Best-arm identification algorithms for multi-armed bandits in the fixed confidence setting

    Kevin Jamieson and Robert Nowak. Best-arm identification algorithms for multi-armed bandits in the fixed confidence setting. In 2014 48th Annual Conference on Information Sciences and Systems (CISS), pages 1--6, 2014. doi:10.1109/CISS.2014.6814096

    work page arXiv 2014

  8. [8]

    On the Complexity of Best Arm Identification in Multi-Armed Bandit Models

    Emilie Kaufmann, Olivier Capp \'e , and Aur \'e lien Garivier. On the Complexity of Best Arm Identification in Multi-Armed Bandit Models . Journal of Machine Learning Research , 17: 0 1--42, January 2016

    2016

Show all 116 references
  1. [9]

    A/B testing: The most powerful way to turn clicks into customers

    Dan Siroker and Pete Koomen. A/B testing: The most powerful way to turn clicks into customers. John Wiley & Sons, 2015

    2015

  2. [10]

    On the complexity of a/b testing

    Emilie Kaufmann, Olivier Cappé, and Aurélien Garivier. On the complexity of a/b testing. In Proceedings of The 27th Conference on Learning Theory, volume 35 of Proceedings of Machine Learning Research, pages 461--481, Barcelona, Spain, 13--15 Jun 2014. PMLR

    2014

  3. [11]

    Pure exploration in finitely-armed and continuous-armed bandits

    Sébastien Bubeck, Rémi Munos, and Gilles Stoltz. Pure exploration in finitely-armed and continuous-armed bandits. Theoretical Computer Science, 412 0 (19): 0 1832--1852, 2011. ISSN 0304-3975. doi:https://doi.org/10.1016/j.tcs.2010.12.059. Algorithmic Learning Theory (ALT 2009)

    2011 doi

  4. [12]

    Optimal best arm identification with fixed confidence

    Aurélien Garivier and Emilie Kaufmann. Optimal best arm identification with fixed confidence. In 29th Annual Conference on Learning Theory, volume 49 of Proceedings of Machine Learning Research, pages 998--1027, Columbia University, New York, New York, USA, 23--26 Jun 2016. PMLR

    2016

  5. [13]

    Feasible bidding strategies through pure-exploration bandits

    Julian Katz-Samuels and Abraham Bagherjeiran. Feasible bidding strategies through pure-exploration bandits. 2019

    2019

  6. [14]

    Feasible arm identification

    Julian Katz-Samuels and Clay Scott. Feasible arm identification. In Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 2535--2543. PMLR, 10--15 Jul 2018

    2018

  7. [15]

    Top feasible arm identification

    Julian Katz-Samuels and Clayton Scott. Top feasible arm identification. In Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics, volume 89 of Proceedings of Machine Learning Research, pages 1593--1601. PMLR, 16--18 Apr 2019

    2019

  8. [16]

    Pareto front identification from stochastic bandit feedback

    Peter Auer, Chao-Kai Chiang, Ronald Ortner, and Madalina Drugan. Pareto front identification from stochastic bandit feedback. In Proceedings of the 19th International Conference on Artificial Intelligence and Statistics, volume 51 of Proceedings of Machine Learning Research, p...

    2016

  9. [17]

    A/b/n testing with control in the presence of subpopulations

    Yoan Russac, Christina Katsimerou, Dennis Bohle, Olivier Capp\' e , Aur\' e lien Garivier, and Wouter M Koolen. A/b/n testing with control in the presence of subpopulations. In Advances in Neural Information Processing Systems, volume 34, pages 25100--25110. Curran Associates,...

  10. [18]

    Finite-time analysis of the multiarmed bandit problem

    Peter Auer, Nicolo Cesa-Bianchi, and Paul Fischer. Finite-time analysis of the multiarmed bandit problem. Machine learning, 47: 0 235--256, 2002

    2002

  11. [19]

    Ucb revisited: Improved regret bounds for the stochastic multi-armed bandit problem

    Peter Auer and Ronald Ortner. Ucb revisited: Improved regret bounds for the stochastic multi-armed bandit problem. Periodica Mathematica Hungarica, 61 0 (1-2): 0 55 -- 65, 2010. doi:10.1007/s10998-010-3055-6

    2010 doi

  12. [20]

    Best Arm Identification in Multi-Armed Bandits

    Jean-Yves Audibert and S \'e bastien Bubeck. Best Arm Identification in Multi-Armed Bandits . In COLT - 23th Conference on Learning Theory - 2010 , page 13 p., Haifa, Israel, June 2010

    2010

  13. [21]

    Fast and regret optimal best arm identification: Fundamental limits and low-complexity algorithms

    Qining Zhang and Lei Ying. Fast and regret optimal best arm identification: Fundamental limits and low-complexity algorithms. In Advances in Neural Information Processing Systems, volume 36, pages 16729--16769. Curran Associates, Inc., 2023

    2023

  14. [22]

    Simple bayesian algorithms for best arm identification

    Daniel Russo. Simple bayesian algorithms for best arm identification. In 29th Annual Conference on Learning Theory, volume 49 of Proceedings of Machine Learning Research, pages 1417--1418, Columbia University, New York, New York, USA, 23--26 Jun 2016. PMLR

    2016

  15. [23]

    Best of both worlds: Stochastic & adversarial best-arm identification

    Yasin Abbasi-Yadkori, Peter Bartlett, Victor Gabillon, Alan Malek, and Michal Valko. Best of both worlds: Stochastic & adversarial best-arm identification . In Proceedings of the 31st Conference On Learning Theory, volume 75 of Proceedings of Machine Learning Research, pages 9...

    2018

  16. [24]

    Top two algorithms revisited

    Marc Jourdan, R\' e my Degenne, Dorian Baudry, Rianne de Heide, and Emilie Kaufmann. Top two algorithms revisited. In Advances in Neural Information Processing Systems, volume 35, pages 26791--26803. Curran Associates, Inc., 2022

    2022

  17. [25]

    Efficient selection of multiple bandit arms: theory and practice

    Shivaram Kalyanakrishnan and Peter Stone. Efficient selection of multiple bandit arms: theory and practice. In Proceedings of the 27th International Conference on International Conference on Machine Learning, ICML'10, page 511–518, Madison, WI, USA, 2010. Omnipress. ISBN 9781605589077

    2010

  18. [26]

    Pac subset selection in stochastic multi-armed bandits

    Shivaram Kalyanakrishnan, Ambuj Tewari, Peter Auer, and Peter Stone. Pac subset selection in stochastic multi-armed bandits. In Proceedings of the 29th International Coference on International Conference on Machine Learning, ICML'12, page 227–234, Madison, WI, USA, 2012. Omnip...

    2012

  19. [27]

    Nearly Instance Optimal Sample Complexity Bounds for Top-k Arm Selection

    Lijie Chen, Jian Li, and Mingda Qiao. Nearly Instance Optimal Sample Complexity Bounds for Top-k Arm Selection . In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, volume 54 of Proceedings of Machine Learning Research, pages 101--110...

  20. [28]

    Approximate top- m arm identification with heterogeneous reward variances

    Ruida Zhou and Chao Tian. Approximate top- m arm identification with heterogeneous reward variances. In Proceedings of The 25th International Conference on Artificial Intelligence and Statistics, volume 151 of Proceedings of Machine Learning Research, pages 7483--7504. PMLR, 2...

    2022

  21. [29]

    Finding all epsilon-good arms in stochastic bandits

    Blake Mason, Lalit Jain, Ardhendu Tripathy, and Robert Nowak. Finding all epsilon-good arms in stochastic bandits. In Advances in Neural Information Processing Systems, volume 33, pages 20707--20718. Curran Associates, Inc., 2020

    2020

  22. [30]

    The true sample complexity of identifying good arms

    Julian Katz-Samuels and Kevin Jamieson. The true sample complexity of identifying good arms. In Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, volume 108 of Proceedings of Machine Learning Research, pages 1781--1791. PMLR, 2...

    2020

  23. [31]

    Revisiting simple regret: Fast rates for returning a good arm

    Yao Zhao, Connor Stephens, Csaba Szepesvari, and Kwang-Sung Jun. Revisiting simple regret: Fast rates for returning a good arm. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 42110--42158. ...

    2023

  24. [32]

    Active learning for level set estimation

    Alkis Gotovos, Nathalie Casati, Gregory Hitz, and Andreas Krause. Active learning for level set estimation. In Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence, IJCAI '13, page 1344–1350. AAAI Press, 2013. ISBN 9781577356332

    2013

  25. [33]

    Good arm identification via bandit feedback

    Hideaki Kano, Junya Honda, Kentaro Sakamaki, Kentaro Matsuura, Atsuyoshi Nakamura, and Masashi Sugiyama. Good arm identification via bandit feedback. Mach. Learn., 108 0 (5): 0 721–745, may 2019. ISSN 0885-6125. doi:10.1007/s10994-019-05784-4

    2019 doi

  26. [34]

    Pure exploration with multiple correct answers

    R\' e my Degenne and Wouter Koolen. Pure exploration with multiple correct answers. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019

    2019

  27. [35]

    An anytime algorithm for good arm identification

    Marc Jourdan and Cl \'e mence R \'e da. An anytime algorithm for good arm identification. arXiv preprint arXiv:2310.10359, 2023

    2023

  28. [36]

    Differential good arm identification

    Yun-Da Tsai, Tzu-Hsien Tsai, and Shou-De Lin. Differential good arm identification. arXiv preprint arXiv:2303.07154, 2023

    2023 arXiv

  29. [37]

    lil’hdoc: An algorithm for good arm identification under small threshold gap

    Tzu-Hsien Tsai, Yun-Da Tsai, and Shou-De Lin. lil’hdoc: An algorithm for good arm identification under small threshold gap. In Advances in Knowledge Discovery and Data Mining: 28th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2024, Taipei, Taiwan, May ...

    2024 doi

  30. [38]

    Sequential test for the lowest mean: From thompson to murphy sampling

    Emilie Kaufmann, Wouter M Koolen, and Aur\' e lien Garivier. Sequential test for the lowest mean: From thompson to murphy sampling. In Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018

    2018

  31. [39]

    An optimal algorithm for the thresholding bandit problem

    Andrea Locatelli, Maurilio Gutzeit, and Alexandra Carpentier. An optimal algorithm for the thresholding bandit problem. In Proceedings of The 33rd International Conference on Machine Learning, volume 48 of Proceedings of Machine Learning Research, pages 1690--1698, New York, N...

    2016

  32. [40]

    Thresholding bandits with augmented ucb

    Subhojyoti Mukherjee, Naveen Kolar Purushothama, Nandan Sudarsanam, and Balaraman Ravindran. Thresholding bandits with augmented ucb. In Proceedings of the 26th International Joint Conference on Artificial Intelligence, IJCAI'17, page 2515–2521. AAAI Press, 2017. ISBN 9780999241103

    2017

  33. [41]

    Online sign identification: Minimization of the number of errors in thresholding bandits

    Reda Ouhamma, Odalric-Ambrym Maillard, and Vianney Perchet. Online sign identification: Minimization of the number of errors in thresholding bandits. In Advances in Neural Information Processing Systems, volume 34, pages 18577--18589. Curran Associates, Inc., 2021

    2021

  34. [42]

    Nichol, Christopher R Genovese, Jeff Schneider, Christopher J

    Brent Bryan, Robert C. Nichol, Christopher R Genovese, Jeff Schneider, Christopher J. Miller, and Larry Wasserman. Active learning for identifying function threshold boundaries. In Advances in Neural Information Processing Systems, volume 18. MIT Press, 2005

    2005

  35. [43]

    Thresholding bandit with optimal aggregate regret

    Chao Tao, Sa\' u l Blanco, Jian Peng, and Yuan Zhou. Thresholding bandit with optimal aggregate regret. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019

    2019

  36. [44]

    Multiscale gaussian process level set estimation

    Shubhanshu Shekhar and Tara Javidi. Multiscale gaussian process level set estimation. In Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics, volume 89 of Proceedings of Machine Learning Research, pages 3283--3291. PMLR, 16--18 Apr 2019

    2019

  37. [45]

    Active level set estimation for continuous search space with theoretical guarantee

    Giang Ngo, Dang Nguyen, Dat Phan-Trong, and Sunil Gupta. Active level set estimation for continuous search space with theoretical guarantee. In Proceedings of the 15th Asian Conference on Machine Learning, volume 222 of Proceedings of Machine Learning Research, pages 943--958....

    2024

  38. [46]

    Kochenderfer

    Andrea Zanette, Junzi Zhang, and Mykel J. Kochenderfer. Robust super-level set estimation using gaussian processes. In Machine Learning and Knowledge Discovery in Databases, pages 276--291, Cham, 2019. Springer International Publishing. ISBN 978-3-030-10928-8

    2019

  39. [47]

    Level set estimation with search space warping

    Manisha Senadeera, Santu Rana, Sunil Gupta, and Svetha Venkatesh. Level set estimation with search space warping. In Advances in Knowledge Discovery and Data Mining, pages 827--839, Cham, 2020. Springer International Publishing. ISBN 978-3-030-47436-2

    2020

  40. [48]

    Truncated variance reduction: A unified approach to bayesian optimization and level-set estimation

    Ilija Bogunovic, Jonathan Scarlett, Andreas Krause, and Volkan Cevher. Truncated variance reduction: A unified approach to bayesian optimization and level-set estimation. In Advances in Neural Information Processing Systems, volume 29. Curran Associates, Inc., 2016

    2016

  41. [49]

    Active learning for level set estimation under cost-dependent input uncertainty

    Yu Inatsu, Masayuki Karasuyama, Keiichi Inoue, and Ichiro Takeuchi. Active learning for level set estimation under cost-dependent input uncertainty. arXiv preprint arXiv:1909.06064, 2019

    1909 arXiv

  42. [50]

    Bayesian experimental design for finding reliable level set under input uncertainty

    Shogo Iwazaki, Yu Inatsu, and Ichiro Takeuchi. Bayesian experimental design for finding reliable level set under input uncertainty. IEEE Access, 8: 0 203982--203993, 2020. doi:10.1109/ACCESS.2020.3036863

    2020

  43. [51]

    Nearly optimal algorithms for level set estimation

    Blake Mason, Lalit Jain, Subhojyoti Mukherjee, Romain Camilleri, Kevin Jamieson, and Robert Nowak. Nearly optimal algorithms for level set estimation. In Proceedings of The 25th International Conference on Artificial Intelligence and Statistics, volume 151 of Proceedings of Ma...

    2022

  44. [52]

    High dimensional level set estimation with bayesian neural network

    Huong Ha, Sunil Gupta, Santu Rana, and Svetha Venkatesh. High dimensional level set estimation with bayesian neural network. Proceedings of the AAAI Conference on Artificial Intelligence, 35 0 (13): 0 12095--12103, May 2021. doi:10.1609/aaai.v35i13.17436

    2021 doi

  45. [53]

    The sample complexity of level set approximation

    Fran c ois Bachoc, Tommaso Cesari, and S \'e bastien Gerchinovitz. The sample complexity of level set approximation. In Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, volume 130 of Proceedings of Machine Learning Research, pages 424...

    2021

  46. [54]

    A new perspective on pool-based active classification and false-discovery control

    Lalit Jain and Kevin G Jamieson. A new perspective on pool-based active classification and false-discovery control. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019

    2019

  47. [55]

    Drugan and Ann Nowe

    Madalina M. Drugan and Ann Nowe. Designing multi-objective multi-armed bandits algorithms: A study. In The 2013 International Joint Conference on Neural Networks (IJCNN), pages 1--8, 2013. doi:10.1109/IJCNN.2013.6707036

    2013

  48. [56]

    Yahyaa, Madalina M

    Saba Q. Yahyaa, Madalina M. Drugan, and Bernard Manderick. Annealing-pareto multi-objective multi-armed bandit algorithm. In 2014 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL), pages 1--8, 2014 a . doi:10.1109/ADPRL.2014.7010619

    2014

  49. [57]

    Knowledge gradient for multi-objective multi-armed bandit algorithms

    Saba Yahyaa, Madalina Drugan, and Bernard Manderick. Knowledge gradient for multi-objective multi-armed bandit algorithms. In ICAART 2014 - Proceedings of the 6th International Conference on Agents and Artificial Intelligence, March 2014 b . ISBN 978-989-8565-39-6

    2014

  50. [58]

    Sequential learning of the P areto front for multi-objective bandits

    Elise Crepon, Aur\' e lien Garivier, and Wouter Koolen. Sequential learning of the P areto front for multi-objective bandits. In Proceedings of The 27th International Conference on Artificial Intelligence and Statistics, volume 238 of Proceedings of Machine Learning Research, ...

    2024

  51. [59]

    Multi-objective -armed bandits

    Kristof Van Moffaert, Kevin Van Vaerenbergh, Peter Vrancx, and Ann Nowe. Multi-objective -armed bandits. In 2014 International Joint Conference on Neural Networks (IJCNN), pages 2331--2338, 2014. doi:10.1109/IJCNN.2014.6889753

    2014

  52. [60]

    Drugan and Ann Nowe

    Madalina M. Drugan and Ann Nowe. Scalarization based pareto optimal set of arms identification algorithms. In 2014 International Joint Conference on Neural Networks (IJCNN), pages 2690--2697, 2014. doi:10.1109/IJCNN.2014.6889484

    2014

  53. [61]

    Exploration versus exploitation trade-off in infinite horizon pareto multi-armed bandits algorithms

    Madalina Drugan and Bernard Manderick. Exploration versus exploitation trade-off in infinite horizon pareto multi-armed bandits algorithms. In Proceedings of the International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,, pages 66--77. SciTePress, 2015....

    2015 doi

  54. [62]

    Roijers, Luisa M

    Diederik M. Roijers, Luisa M. Zintgraf, and Ann Now \'e . Interactive thompson sampling for multi-objective multi-armed bandits. In Algorithmic Decision Theory, pages 18--34, Cham, 2017. Springer International Publishing. ISBN 978-3-319-67504-6

    2017

  55. [63]

    Multi-objective recommendation via multivariate policy learning

    Olivier Jeunen, Jatin Mandav, Ivan Potapov, Nakul Agarwal, Sourabh Vaid, Wenzhe Shi, and Aleksei Ustimenko. Multi-objective recommendation via multivariate policy learning. arXiv preprint arXiv:2405.02141, 2024

    2024 arXiv

  56. [64]

    Vector optimization with stochastic bandit feedback

    Cagin Ararat and Cem Tekin. Vector optimization with stochastic bandit feedback. In Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, volume 206 of Proceedings of Machine Learning Research, pages 2165--2190. PMLR, 25--27 Apr 2023

    2023

  57. [65]

    Zintgraf, Diederik M

    Luisa M. Zintgraf, Diederik M. Roijers, Sjoerd Linders, Catholijn M. Jonker, and Ann Now\' e . Ordered preference elicitation strategies for supporting multi-objective decision making. In Proceedings of the 17th International Conference on Autonomous Agents and MultiAgent Syst...

    2018

  58. [66]

    Stochastic bandits with linear constraints

    Aldo Pacchiano, Mohammad Ghavamzadeh, Peter Bartlett, and Heinrich Jiang. Stochastic bandits with linear constraints. In Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, volume 130 of Proceedings of Machine Learning Research, pages 28...

    2021

  59. [67]

    An efficient pessimistic-optimistic algorithm for stochastic linear bandits with general constraints

    Xin Liu, Bin Li, Pengyi Shi, and Lei Ying. An efficient pessimistic-optimistic algorithm for stochastic linear bandits with general constraints. In Advances in Neural Information Processing Systems, volume 34, pages 24075--24086. Curran Associates, Inc., 2021

    2021

  60. [68]

    Bandits with knapsacks

    Ashwinkumar Badanidiyuru, Robert Kleinberg, and Aleksandrs Slivkins. Bandits with knapsacks. J. ACM, 65 0 (3), mar 2018. ISSN 0004-5411. doi:10.1145/3164539

    2018 doi

  61. [69]

    Multi-armed bandit with budget constraint and variable costs

    Wenkui Ding, Tao Qin, Xudong Zhang, and TieYan Liu. Multi-armed bandit with budget constraint and variable costs. Proceedings of the AAAI Conference on Artificial Intelligence, 27 0 (1): 0 232--238, Jun. 2013. doi:10.1609/aaai.v27i1.8637

    2013 doi

  62. [70]

    Cost aware best arm identification

    Kellen Kanarios, Qining Zhang, and Lei Ying. Cost aware best arm identification. Reinforcement Learning Journal, 4: 0 1533--1545, 2024

    2024

  63. [71]

    Pure exploration of multi-armed bandit under matroid constraints

    Lijie Chen, Anupam Gupta, and Jian Li. Pure exploration of multi-armed bandit under matroid constraints. In 29th Annual Conference on Learning Theory, volume 49 of Proceedings of Machine Learning Research, pages 647--669, Columbia University, New York, New York, USA, 23--26 Ju...

    2016

  64. [72]

    Wagenmaker, and Kevin Jamieson

    Zhenlin Wang, Andrew J. Wagenmaker, and Kevin Jamieson. Best arm identification with safety constraints. In Proceedings of The 25th International Conference on Artificial Intelligence and Statistics, volume 151 of Proceedings of Machine Learning Research, pages 9114--9146. PML...

    2022

  65. [73]

    Active learning with safety constraints

    Romain Camilleri, Andrew Wagenmaker, Jamie H Morgenstern, Lalit Jain, and Kevin G Jamieson. Active learning with safety constraints. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems, volume 35, pag...

    2022

  66. [74]

    Price of safety in linear best arm identification

    Xuedong Shang, Igor Colin, Merwan Barlier, and Hamza Cherkaoui. Price of safety in linear best arm identification. arXiv preprint arXiv:2309.08709, 2023

    2023 arXiv

  67. [75]

    Pure exploration in bandits with linear constraints

    Emil Carlsson, Debabrota Basu, Fredrik Johansson, and Devdatt Dubhashi. Pure exploration in bandits with linear constraints. In Proceedings of The 27th International Conference on Artificial Intelligence and Statistics, volume 238 of Proceedings of Machine Learning Research, p...

    2024

  68. [76]

    Yunlong Hou, Vincent Y. F. Tan, and Zixin Zhong. Almost optimal variance-constrained best arm identification. IEEE Transactions on Information Theory, 69 0 (4): 0 2603--2634, 2023. doi:10.1109/TIT.2022.3222231

    2023

  69. [77]

    Pac bandits with risk constraints

    Yahel David, Bal \'a zs Sz \"o r \'e nyi, Mohammad Ghavamzadeh, Shie Mannor, and Nahum Shimkin. Pac bandits with risk constraints. In ISAIM, 2018

    2018

  70. [78]

    Best arm identification with fairness constraints on subpopulations

    Yuhang Wu, Zeyu Zheng, and Tingyu Zhu. Best arm identification with fairness constraints on subpopulations. In 2023 Winter Simulation Conference (WSC), pages 540--551, 2023. doi:10.1109/WSC60868.2023.10408550

    2023

  71. [79]

    Best arm identification with resource constraints

    Zitian Li and Wang Chi Cheung. Best arm identification with resource constraints. In Proceedings of The 27th International Conference on Artificial Intelligence and Statistics, volume 238 of Proceedings of Machine Learning Research, pages 253--261. PMLR, 02--04 May 2024

    2024

  72. [80]

    Safe exploration for optimization with gaussian processes

    Yanan Sui, Alkis Gotovos, Joel Burdick, and Andreas Krause. Safe exploration for optimization with gaussian processes. In Proceedings of the 32nd International Conference on Machine Learning, volume 37 of Proceedings of Machine Learning Research, pages 997--1005, Lille, France...

    2015

  73. [81]

    Stagewise safe B ayesian optimization with G aussian processes

    Yanan Sui, Vincent Zhuang, Joel Burdick, and Yisong Yue. Stagewise safe B ayesian optimization with G aussian processes. In Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 4781--4789. PMLR, 10--...

    2018

  74. [82]

    Schoellig, and Andreas Krause

    Felix Berkenkamp, Angela P. Schoellig, and Andreas Krause. Safe controller optimization for quadrotors with gaussian processes. In 2016 IEEE International Conference on Robotics and Automation (ICRA), pages 491--496, 2016. doi:10.1109/ICRA.2016.7487170

    2016

  75. [83]

    Bayesian optimization with safety constraints: safe and automatic parameter tuning in robotics

    Felix Berkenkamp, Andreas Krause, and Angela P Schoellig. Bayesian optimization with safety constraints: safe and automatic parameter tuning in robotics. Machine learning, 112 0 (10): 0 3713—3747, 2023. ISSN 0885-6125. doi:10.1007/s10994-021-06019-1

    2023 doi

  76. [84]

    Safe exploration for interactive machine learning

    Matteo Turchetta, Felix Berkenkamp, and Andreas Krause. Safe exploration for interactive machine learning. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019

    2019

  77. [85]

    Information-theoretic safe exploration with gaussian processes

    Alessandro Bottero, Carlos Luis, Julia Vinogradska, Felix Berkenkamp, and Jan R Peters. Information-theoretic safe exploration with gaussian processes. In Advances in Neural Information Processing Systems, volume 35, pages 30707--30719. Curran Associates, Inc., 2022

    2022

  78. [86]

    Adaptive and safe B ayesian optimization in high dimensions via one-dimensional subspaces

    Johannes Kirschner, Mojmir Mutny, Nicole Hiller, Rasmus Ischebeck, and Andreas Krause. Adaptive and safe B ayesian optimization in high dimensions via one-dimensional subspaces. In Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings o...

    2019

  79. [87]

    Duivenvoorden, Felix Berkenkamp, Nicolas Carion, Andreas Krause, and Angela P

    Rikky R.P.R. Duivenvoorden, Felix Berkenkamp, Nicolas Carion, Andreas Krause, and Angela P. Schoellig. Constrained bayesian optimization with particle swarms for safe adaptive controller tuning. IFAC-PapersOnLine, 50 0 (1): 0 11800--11807, 2017. ISSN 2405-8963. doi:https://doi...

    2017 doi

  80. [88]

    Gelbart, Jasper Snoek, and Ryan P

    Michael A. Gelbart, Jasper Snoek, and Ryan P. Adams. Bayesian optimization with unknown constraints. In Proceedings of the Thirtieth Conference on Uncertainty in Artificial Intelligence, UAI'14, page 250–259, Arlington, Virginia, USA, 2014. AUAI Press. ISBN 9780974903910

    2014

  81. [89]

    Gelbart, Ryan P

    Jos \'e Miguel Hern\' a ndez-Lobato, Michael A. Gelbart, Ryan P. Adams, Matthew W. Hoffman, and Zoubin Ghahramani. A general framework for constrained bayesian optimization using information-based search. Journal of Machine Learning Research, 17 0 (160): 0 1--53, 2016

    2016

  82. [90]

    Gardner, Matt J

    Jacob R. Gardner, Matt J. Kusner, Zhixiang Xu, Kilian Q. Weinberger, and John P. Cunningham. Bayesian optimization with inequality constraints. In Proceedings of the 31st International Conference on International Conference on Machine Learning - Volume 32, ICML'14, page II–937...

    2014

  83. [91]

    An asymptotically optimal strategy for constrained multi-armed bandit problems

    Hyeong Soo Chang. An asymptotically optimal strategy for constrained multi-armed bandit problems. Mathematical Methods of Operations Research, 91 0 (3): 0 545--557, 2020

    2020

  84. [92]

    Constrained pure exploration multi-armed bandits with a fixed budget

    Fathima Zarin Faizal and Jayakrishnan Nair. Constrained pure exploration multi-armed bandits with a fixed budget. arXiv preprint arXiv:2211.14768, 2022

    2022 arXiv

  85. [93]

    Pure exploration for constrained best mixed arm identification with a fixed budget

    Dengwang Tang, Rahul Jain, Ashutosh Nayyar, and Pierluigi Nuzzo. Pure exploration for constrained best mixed arm identification with a fixed budget. arXiv preprint arXiv:2405.15090, 2024

    2024 arXiv

  86. [94]

    Interactively learning preference constraints in linear bandits

    David Lindner, Sebastian Tschiatschek, Katja Hofmann, and Andreas Krause. Interactively learning preference constraints in linear bandits. In Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pages 135...

    2022

  87. [95]

    Nonconvex robust optimization for problems with constraints

    Dimitris Bertsimas, Omid Nohadani, and Kwong Meng Teo. Nonconvex robust optimization for problems with constraints. INFORMS Journal on Computing, 22 0 (1): 0 44--58, 2010. doi:10.1287/ijoc.1090.0319

    2010

  88. [96]

    Chen, Brendan Lucier, Yaron Singer, and Vasilis Syrgkanis

    Robert S. Chen, Brendan Lucier, Yaron Singer, and Vasilis Syrgkanis. Robust optimization for non-convex objectives. In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017 b

    2017

  89. [97]

    On Minmax Theorems for Multiplayer Games, pages 217--234

    Yang Cai and Constantinos Daskalakis. On Minmax Theorems for Multiplayer Games, pages 217--234. SIAM, 2011. doi:10.1137/1.9781611973082.20

    2011 doi

  90. [98]

    Solving a class of non-convex min-max games using iterative first order methods

    Maher Nouiehed, Maziar Sanjabi, Tianjian Huang, Jason D Lee, and Meisam Razaviyayn. Solving a class of non-convex min-max games using iterative first order methods. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019

    2019

  91. [99]

    Max-min grouped bandits

    Zhenlin Wang and Jonathan Scarlett. Max-min grouped bandits. Proceedings of the AAAI Conference on Artificial Intelligence, 36 0 (8): 0 8603--8611, Jun. 2022. doi:10.1609/aaai.v36i8.20838

    2022 doi

  92. [100]

    Aurélien Garivier, Emilie Kaufmann, and Wouter M. Koolen. Maximin action identification: A new bandit framework for games. In 29th Annual Conference on Learning Theory, volume 49 of Proceedings of Machine Learning Research, pages 1028--1050, Columbia University, New York, New ...

    2016

  93. [101]

    Learning maximin strategies in simulation-based games with infinite strategy spaces

    Alberto Marchesi, Francesco Trov \`o , Nicola Gatti, et al. Learning maximin strategies in simulation-based games with infinite strategy spaces. In Proceedings of the Smooth Games Optimization and Machine Learning Workshop, pages 1--5, 2019

    2019

  94. [102]

    Bayesian fixed-budget best-arm identification

    Alexia Atsidakou, Sumeet Katariya, Sujay Sanghavi, and Branislav Kveton. Bayesian fixed-budget best-arm identification. arXiv preprint arXiv:2211.08572, 2022

    2022 arXiv

  95. [103]

    Multi-fidelity best-arm identification

    Riccardo Poiani, Alberto Maria Metelli, and Marcello Restelli. Multi-fidelity best-arm identification. In Advances in Neural Information Processing Systems, volume 35, pages 17857--17870. Curran Associates, Inc., 2022

    2022

  96. [104]

    Optimal multi-fidelity best-arm identification

    Riccardo Poiani, R \'e my Degenne, Emilie Kaufmann, Alberto Maria Metelli, and Marcello Restelli. Optimal multi-fidelity best-arm identification. arXiv preprint arXiv:2406.03033, 2024

    2024 arXiv

  97. [105]

    Experimental designs for heteroskedastic variance

    Justin Weltz, Tanner Fiez, Alexander Volfovsky, Eric Laber, Blake Mason, houssam nassif, and Lalit Jain. Experimental designs for heteroskedastic variance. In Advances in Neural Information Processing Systems, volume 36, pages 65967--66005. Curran Associates, Inc., 2023

    2023

  98. [106]

    Active learning in heteroscedastic noise

    András Antos, Varun Grover, and Csaba Szepesvári. Active learning in heteroscedastic noise. Theoretical Computer Science, 411 0 (29): 0 2712--2728, 2010. ISSN 0304-3975. Algorithmic Learning Theory (ALT 2008)

    2010

  99. [107]

    Active heteroscedastic regression

    Kamalika Chaudhuri, Prateek Jain, and Nagarajan Natarajan. Active heteroscedastic regression. In Doina Precup and Yee Whye Teh, editors, Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pages 694--702....

    2017

  100. [108]

    Experimental design

    Roger E Kirk. Experimental design. Sage handbook of quantitative methods in psychology, pages 23--45, 2009

    2009

  101. [109]

    Experimental design and analysis, 2012

    Howard J Seltman. Experimental design and analysis, 2012

    2012

  102. [110]

    Bayesian inference in statistical analysis

    George EP Box and George C Tiao. Bayesian inference in statistical analysis. John Wiley & Sons, 2011

    2011

  103. [111]

    The probable error of a mean

    Student. The probable error of a mean. Biometrika, pages 1--25, 1908

    1908

  104. [112]

    Bayesian a/b testing for business decisions

    Shafi Kamalbasha and Manuel Eugster. Bayesian a/b testing for business decisions. In Data Science -- Analytics and Applications, pages 50--57, Wiesbaden, 2021. Springer Fachmedien Wiesbaden. ISBN 978-3-658-32182-6

    2021

  105. [113]

    On the two different aspects of the representative method: the method of stratified sampling and the method of purposive selection

    Jerzy Neyman. On the two different aspects of the representative method: the method of stratified sampling and the method of purposive selection. In Breakthroughs in statistics: Methodology and distribution, pages 123--150. Springer, 1992

    1992

  106. [114]

    Double q-learning

    Hado Hasselt. Double q-learning. In Advances in Neural Information Processing Systems, volume 23. Curran Associates, Inc., 2010

    2010

  107. [115]

    A large deviations perspective on ordinal optimization

    Peter Glynn and Sandeep Juneja. A large deviations perspective on ordinal optimization. In Proceedings of the 2004 Winter Simulation Conference, 2004., volume 1. IEEE, 2004

    2004

  108. [116]

    Optimal treatment allocation for efficient policy evaluation in sequential decision making

    Ting Li, Chengchun Shi, Jianing Wang, Fan Zhou, et al. Optimal treatment allocation for efficient policy evaluation in sequential decision making. Advances in Neural Information Processing Systems, 36: 0 48890--48905, 2023

    2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.