Stochastic Multi-Objective Multi-Armed Bandits: Regret Definition and Algorithm

Mansoor Davoodi; Setareh Maghsudi

REVIEW 2 major objections 6 minor 21 references

Stochastic Multi-Objective Multi-Armed Bandits: Regret Definition and Algorithm

T0 review · 2 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper proposes a two-part regret metric and a set-cover algorithm that reaches every Pareto-optimal arm with sublinear regret.

desk verdict New two-part regret metric is a real contribution, but the polynomial-time greedy variant's cumulative adjustment-regret bound is not proven. read the letter →

arxiv 2506.13125 v1 pith:BM55MR6E submitted 2025-06-16 cs.LG cs.DS

Mansoor Davoodi , Setareh Maghsudi This is my paper · ORCID

classification cs.LGcs.DS MSC 68W2790C2962L05

keywords stochasticmulti-armedbanditsmulti-objectiveoptimizationParetoregretcoverageefficientPareto-optimalarmssetcoverminimization

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the standard Pareto-regret metric for multi-objective multi-armed bandits is too weak: it measures only the distance to the Pareto front in one direction, so an algorithm that optimizes a single objective can look good while ignoring the others. It proposes a two-part regret definition that holds across all objectives at once: coverage-regret, which requires every Pareto-optimal arm to be within regret $R$ of some arm the algorithm selects, and cumulative adjustment-regret, which bounds the total adjustments needed for selected arms to weakly dominate the optimal set. It also introduces the class of efficient Pareto-optimal (EPO) arms, the subset of Pareto-optimal arms that lie on the convex hull of the front. The paper's algorithm explores each arm $T'$ times, builds a minimum set cover of arms whose inflated empirical means dominate all arms, and then pulls only that cover; for both regret parts it claims $O(T^{2/3}(n\log T)^{1/3})$ regret, with an extra $\log n$ factor when the set cover is computed by a polynomial-time approximation.

What carries the argument

The load-bearing object is the minimum set cover $B$ of arms whose inflated empirical reward vectors $b+2r$ weakly dominate every arm, where $r=\sqrt{2\log T/T'}$ is the confidence radius after $T'$ pulls. Each arm carries a domination list $\mathrm{Dom}(a)=\{a': a+2r \succeq a'\}$, and the algorithm pulls only the cover $B$. The proofs rest on two covering facts: every Pareto-optimal arm is within $4r$ of some cover arm in every objective, and the minimality of $B$ forces every non-optimal cover arm to be within $4r$ of a Pareto-optimal arm that dominates it. Efficient Pareto-optimal arms are defined separately as PO arms that no convex combination of other PO arms weakly dominates, i.e., the arms on the convex hull of the Pareto front, and they are identified by a linear program once $A^*$ is known.

What would settle it

Run Algorithm 1 with the greedy set cover on small instances and record, for every arm $b\in B$, the largest coordinate gap to a Pareto-optimal arm that dominates it; if any gap exceeds $4r=\sqrt{8\log T/T'}$, the contradiction argument in Theorem 3 fails. Alternatively, enumerate the exact and greedy covers on the same instance and check whether the empirical cumulative adjustment-regret scales with $|B_{\text{greedy}}|$ rather than with $|A^*|$.

Watch

Extended reading notes

Core claim

The central claim is that a multi-objective bandit algorithm should be evaluated against every Pareto-optimal arm simultaneously, not merely by its minimum distance to the Pareto front. Formally, coverage-regret is the minimum over the selected arms of the worst-objective gap to each Pareto-optimal arm, while cumulative adjustment-regret is the total minimal $\epsilon$ such that each pulled arm plus $\epsilon$ in every coordinate weakly dominates some Pareto-optimal arm. On this footing, Algorithm 1 solves the problem: it pulls every arm $T'=(T/n)^{2/3}(2\log T)^{1/3}$ times, estimates each mean within radius $r=\sqrt{2\log T/T'}$, deletes arms dominated by an improved competitor, and computes a minimum set cover $B$ such that every arm is weakly dominated by $b+2r$ for some $b\in B$; it then pulls only the arms in $B$. Theorems 1 and 3 give coverage-regret and cumulative adjustment-regret of $O(T^{2/3}(n\log T)^{1/3})$; Theorem 4 extends the same guarantees when $B$ is pruned to the efficient Pareto-optimal arms; and Theorem 5 gives the same rate for a single-arm-pulling version compared with the average of a diverse Pareto-optimal set.

Load-bearing premise

The proof of the cumulative adjustment-regret bound (Theorem 3) assumes the set-cover step returns the exact minimum set cover $B$; the polynomial-time greedy variant only approximates the cover, so the stated $O(\log n\,T^{2/3}(n\log T)^{1/3})$ regret bound is not supported by the proof as written.

Editorial extensions

If this is right

If the regret bounds hold, any algorithm evaluated under the new metric must balance all objectives, so single-objective shortcuts that pass the old Pareto-regret test would no longer pass.
Both coverage-regret and cumulative adjustment-regret are sublinear, so Algorithm 1 can run for long horizons and still provably approach the Pareto front.
The polynomial-time variant with the $O(\log n)$-approximate set cover makes the sublinear guarantee available without exponential computation, at a factor $\log n$ in regret.
Theorem 5 gives the same $O(T^{2/3}(n\log T)^{1/3})$ rate when only one cover arm is pulled per round against the average of a diverse Pareto-optimal set, extending the result to a single-arm online setting.
As $T\to\infty$, the confidence radius shrinks to zero and the returned cover set converges to the full Pareto-optimal set, or to the efficient Pareto-optimal set after pruning, so the algorithm is asymptotically consistent.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The minimum set cover in Algorithm 1 is essentially an $\epsilon$-net of the Pareto front with radius $2r$; viewing it that way connects the regret rate to classical covering numbers of the front.
The EPO definition identifies the convex hull of the Pareto front, which suggests that pure-exploration variants could identify EPO arms with linear-programming separability tests rather than exhaustive searches.
The $T^{2/3}$ rate is characteristic of explore-then-commit designs; an adaptive UCB-style exploration could plausibly push the regret to $\tilde{O}(\sqrt{T})$ while keeping the same metric, but that is not shown here.
The two-part regret metric could be adapted to constrained or adversarial MO-MAB settings, since it only requires per-objective gaps and set-cover structure.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

The paper's main contribution is a two-part regret definition for MO-MAB: coverage-regret (each optimal arm is approximated across all objectives) and cumulative adjustment-regret (penalizing redundant or dominated pulls). That is a genuine improvement over the Drugan-Nowe Pareto regret, which as they correctly note can be gamed by single-objective algorithms. The EPO set is a bandit-flavored repackaging of supported Pareto solutions, but the formalization is clean and the LP characterization is useful.

The exact set-cover version of the analysis is mostly sound. Theorem 1's concentration argument is standard; Theorem 3's per-arm adjustment bound uses a transitivity-plus-minimality argument that checks out when B is exactly the minimum cover. The coverage-regret proof correctly exploits the fact that exploration pulls all arms.

The soft spots are real, though concentrated. The polynomial-time variant using an O(log n)-approximate greedy set cover does not inherit Theorem 3's bound. That proof relies on every arm in B being irredundant: if a selected arm had a large gap to all Pareto arms, it would be covered by another selected arm and could be removed. Greedy covers are not minimal, so an arm selected early can remain even once redundant, and the per-arm epsilon(b) <= 4r guarantee is simply not established. Since the final paragraph only bounds |B|, not the per-arm gaps, the claimed O(log n T^(2/3)(n log T)^(1/3)) cumulative adjustment regret is unsupported. The stress-test note is right: such an arm would contribute linearly in T. This is not a cosmetic gap; the polynomial-time claim is the headline engineering result.

There are smaller issues. Theorem 4's argument that non-efficient arms can be replaced by randomized mixtures of other arms is hand-wavy, and Algorithm 1 does not actually perform the mixing. Theorem 5's 'average best arm' bound is asserted more than proven. The experiments recalibrate r below the theoretical value, so they don't validate the guarantees.

The exact-cover theory and the metric itself deserve serious refereeing. I'd send it to review, but the authors should be told the greedy bound needs either a new proof or a weakened claim, and the EPO mixing needs to be made actual or dropped.

Referee Report

2 major / 6 minor

Summary. The paper proposes a new regret measure for multi-objective multi-armed bandits (MO-MAB), consisting of a coverage-regret component and a cumulative adjustment-regret component, together with the notion of Efficient Pareto-Optimal (EPO) arms. It presents a two-phase algorithm (Algorithm 1): explore each arm for T' rounds, compute a confidence radius r, discard arms clearly dominated by others, build a set cover B of arms whose inflated empirical reward vectors dominate all remaining arms, and then pull the arms in B for the remaining rounds. The paper claims sublinear regret O(T^{2/3}(n log T)^{1/3}) when the exact minimum set cover is used and O(log n · T^{2/3}(n log T)^{1/3}) for a polynomial-time greedy set-cover variant, as well as an extension of the guarantees to EPO arms.

Significance. If the central claims hold, the paper makes a useful contribution by proposing a regret notion that penalizes poor performance in every objective rather than only the scalarized distance to the Pareto front, and by connecting multi-objective bandit exploration to set cover. The exact-cover analysis is coherent and rests on a standard concentration argument: the coverage-regret bound follows immediately from the clean-event analysis, and the cumulative adjustment-regret bound for exact minimum set covers is justified by the minimality-based redundancy argument. The paper is also honest in its experiments, comparing exact and greedy cover sizes on synthetic data. However, the polynomial-time variant's regret guarantee and the EPO extension are not adequately proven as written, so the paper needs revision before the main claims can be accepted.

major comments (2)

[Section 5, Theorem 3 and the concluding paragraph of its proof] The claimed cumulative adjustment-regret bound for the polynomial-time greedy variant is unsupported. The proof of the per-arm bound epsilon(b) <= 4r for b in B relies on B being the exact minimum set cover: if a Pareto-optimal arm a* dominates b with gap greater than 4r, then Dom(b) is contained in Dom(a*), making b redundant, which contradicts minimality. An O(log n)-approximate greedy set cover need not be inclusion-wise minimal; it can retain an arm b even after Dom(b) is covered by other selected arms, and for such b the contradiction argument does not apply. The final sentence of the proof only states |B| <= log n · |A*|, which bounds the number of terms in Term 2, not their magnitudes. An arm with epsilon(b) much larger than 4r would contribute (T - T') * epsilon(b) to Term 2, which is linear in T and would violate the claimed O(log n · T^{2/3}(n log T)^{1/3}) bound. A local repair would be to add a post-processing step that prunes B to an inclusion-wise minimal cover, which restores the redundancy argument (a redundant arm can be removed) while preserving |B| <= O(log n) · |A*|; alternatively, a different argument for the greedy variant must be supplied.
[Section 5, Theorem 4] The extension of the regret guarantees to EPO arms is not rigorously established. The proof argues that a removed non-efficient arm a can be replaced by an artificial arm b_alpha = sum_i alpha_i a_i, because the rewards of b_alpha can be 'approximated for sufficiently large T by selecting a_i with probability alpha_i'. However, Algorithm 1 as written does not randomize in the exploitation phase; it pulls the remaining arms in B. The coverage-regret definition (Eq. (2)) requires, for each EPO arm a*, the existence of an actual arm a_t in the selected set A_t whose cumulative reward vector is within R of a*. A convex combination of arms is not an element of A_t, and in general no single arm in the convex combination dominates a* even when the combination does (for example, a* = (0.4, 0.4) is dominated by 0.5(1,0) + 0.5(0,1), but by neither (1,0) nor (0,1)). Thus, removing non-efficient arms can destroy coverage. The proof must either modify the algorithm to randomize and prove the regret bound for the resulting randomized process, or prove that some remaining arm in B covers each EPO arm within the claimed radius.

minor comments (6)

[Section 1] The fourth contribution bullet duplicates the third bullet verbatim; one of them should be removed.
[Keywords] The keyword 'sochastic' is a typo; it should read 'stochastic'.
[Section 4.1] The definition of the EPO set uses sequences S and an asymptotic threshold T0, but the notation is not made precise: the length of the sequences, the quantifier order, and the meaning of 'for sufficiently large T' for a fixed sequence should be clarified, and the two displayed definitions of set-wise domination should be aligned.
[Section 5, Theorem 2] The proof of convergence is informal; it should be made rigorous by using the minimum positive gap between the mean rewards of PO and non-PO arms and showing that for r below half that gap, B must equal A*.
[Section 4 / Section 5] The regret bounds implicitly assume bounded rewards (e.g., rewards in [0,1]) since the proofs bound per-round regret by constants and Term 1 of Theorem 3 bounds the adjustment by 1 per pulled arm; this assumption should be stated explicitly in the model.
[Section 5, Theorem 5 and the diversity paragraph] The term 'DPO' is used before its definition as 'diverse set of Pareto Optimal arms', and the proof of Theorem 5 is terse: the claim that uniform random selection from B yields expected per-round difference at most O(2r) requires a more detailed derivation.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the regret bounds are derived from concentration inequalities and an externally defined Pareto front, not from fitted inputs or self-citation.

full rationale

The paper's central derivation is self-contained against externally defined objects. The Pareto-optimal set A* is defined in Section 4.1 from the true reward means mu(a), and the EPO set is defined by convex-position dominance, independent of the algorithm. The exploration length T' = (T/n)^{2/3}(2 log T)^{1/3} and confidence radius r = sqrt(2 log T / T') are chosen from the horizon and Hoeffding-style concentration, not fitted to any target regret value. The coverage-regret proof (Theorem 1) bounds (T-T')4r under the clean event and then substitutes T', yielding O(T^{2/3}(n log T)^{1/3}); this is a standard algebraic reduction, not a renaming of the conclusion. The cumulative adjustment-regret proof (Theorem 3) decomposes the regret into Term 1 and Term 2, and bounds each selected arm's adjustment by 4r using the minimality of the exact minimum set cover plus the clean event; the size factor |B| <= |A*| comes from Lemma 1, which uses the fact that improved Pareto-optimal arms cover all arms. No fitted quantity is renamed as a prediction, and no load-bearing premise is justified by a self-citation: the bibliography contains no works by the present authors. The notable weakness in the paper is that the polynomial-time greedy set-cover variant's cumulative adjustment-regret guarantee is not established as written, because Theorem 3's per-arm 4r argument relies on exact minimality of B and the final |B| <= O(log n)|A*| bound only controls the number of terms, not their magnitudes. That is a correctness or proof-gap concern about an external approximation algorithm, not circularity: the greedy approximation ratio is cited from Vazirani's textbook and does not smuggle the paper's own conclusion into its assumptions. Therefore, the circularity score is 0.

Assumptions & free parameters 2 free parameters · 4 assumptions · 1 invented entities

The central regret bounds rest on standard concentration and set-cover results; no parameter is fitted to data. The only hand-chosen quantities are the exploration length T' and confidence radius r, which are horizon-dependent design choices. The EPO set is a new formal definition equivalent to supported Pareto solutions.

free parameters (2)

exploration length T' = (T/n)^(2/3) (2 log T)^(1/3)
Chosen by the authors to balance exploration and exploitation in the regret bound. It is a horizon-dependent design choice, not fitted to data, and is listed for completeness.
confidence radius r = sqrt(2 log T / T')
Derived from Hoeffding's inequality, not fitted. In the experiments r is hand-calibrated to about 0.02 instead of the theoretical value, but this only affects the reported experiments, not the stated theorems.

assumptions (4)

domain assumption Reward vectors are bounded in [0,1]^D.
Hoeffding's inequality and the union bound in Theorem 1 require bounded rewards. The paper does not explicitly state this assumption; experiments use Bernoulli rewards.
standard math Hoeffding's inequality and union bounds over arms, objectives, and rounds.
Used in Theorem 1 to bound the probability of the unclean event by O(1/T).
standard math Minimum set cover is NP-hard and admits an O(log n)-approximation.
Invoked in the Complexity section of Section 5 and used for the polynomial-time variant.
standard math The expected number of Pareto maxima in a random set of n vectors is polylogarithmic in n.
Used for the expected-case complexity claim in Section 5, citing Bentley, Kung, Schkolnick, and Thompson.

invented entities (1)

Efficient Pareto-Optimal (EPO) set EA*
purpose: Identify Pareto-optimal arms that are not weakly dominated by convex combinations of other Pareto-optimal arms, so they are the arms an online decision-maker may need to keep in a cover.
Formal definition in Eq. (1). It is equivalent to supported (convex-hull) Pareto points in multi-objective optimization, so it is not an empirical entity and has no external falsifiable handle.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Stochastic Multi-Objective Multi-Armed Bandits: Regret Definition and Algorithm." pith.science (2026). https://pith.science/paper/BM55MR6E

@misc{pith2026250613125,
  author       = {Pith},
  title        = {Pith review of: Stochastic Multi-Objective Multi-Armed Bandits: Regret Definition and Algorithm},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BM55MR6E}},
  note         = {Machine review of arXiv:2506.13125}
}

read the original abstract

Multi-armed bandit (MAB) problems are widely applied to online optimization tasks that require balancing exploration and exploitation. In practical scenarios, these tasks often involve multiple conflicting objectives, giving rise to multi-objective multi-armed bandits (MO-MAB). Existing MO-MAB approaches predominantly rely on the Pareto regret metric introduced in \cite{drugan2013designing}. However, this metric has notable limitations, particularly in accounting for all Pareto-optimal arms simultaneously. To address these challenges, we propose a novel and comprehensive regret metric that ensures balanced performance across conflicting objectives. Additionally, we introduce the concept of \textit{Efficient Pareto-Optimal} arms, which are specifically designed for online optimization. Based on our new metric, we develop a two-phase MO-MAB algorithm that achieves sublinear regret for both Pareto-optimal and efficient Pareto-optimal arms.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

21 extracted references · 19 canonical work pages

[1]

In: The 2013 International Joint Conference on Neural Networks (IJCNN), pp

Drugan, M.M., Nowe, A.: Designing multi-objective multi-armed bandits algo- rithms: A study. In: The 2013 International Joint Conference on Neural Networks (IJCNN), pp. 1–8 (2013). IEEE

work page 2013
[2]

Foundations and Trends®in Machine Learning12(1-2), 1–286 (2019)

Slivkins, A.,et al.: Introduction to multi-armed bandits. Foundations and Trends®in Machine Learning12(1-2), 1–286 (2019)

work page 2019
[3]

IEEE Transactions on Intelligent Transportation Systems23(12), 25536–25545 (2021)

Wei, W., Yang, R., Gu, H., Zhao, W., Chen, C., Wan, S.: Multi-objective opti- mization for resource allocation in vehicular cloud computing networks. IEEE Transactions on Intelligent Transportation Systems23(12), 25536–25545 (2021)

work page 2021
[4]

In: International Conference on Machine Learning, pp

Xu, M., Klabjan, D.: Pareto regret analyses in multi-objective multi-armed ban- dit. In: International Conference on Machine Learning, pp. 38499–38517 (2023). PMLR

work page 2023
[5]

In: 2014 IEEE Symposium on Adaptive Dynamic 19 Programming and Reinforcement Learning (ADPRL), pp

Drugan, M.M., Now´ e, A., Manderick, B.: Pareto upper confidence bounds algo- rithms: an empirical study. In: 2014 IEEE Symposium on Adaptive Dynamic 19 Programming and Reinforcement Learning (ADPRL), pp. 1–8 (2014). IEEE

work page 2014
[6]

Advances in neural information processing systems26(2013)

Mahdavi, M., Yang, T., Jin, R.: Stochastic convex optimization with multiple objectives. Advances in neural information processing systems26(2013)

work page 2013
[7]

In: 2014 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL), pp

Yahyaa, S.Q., Drugan, M.M., Manderick, B.: Annealing-pareto multi-objective multi-armed bandit algorithm. In: 2014 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL), pp. 1–8 (2014). IEEE

work page 2014
[8]

In: ESANN (2015)

Yahyaa, S.Q., Manderick, B.: Thompson sampling for multi-objective multi- armed bandits problem. In: ESANN (2015)

work page 2015

Show all 21 references

[9]

In: International Conference on Machine Learning, pp

Busa-Fekete, R., Sz¨ or´ enyi, B., Weng, P., Mannor, S.: Multi-objective bandits: Optimizing the generalized gini index. In: International Conference on Machine Learning, pp. 625–634 (2017). PMLR

2017
[10]

arXiv preprint arXiv:1803.04039 (2018)

¨Oner, D., Karakurt, A., Eryılmaz, A., Tekin, C.: Combinatorial multi-objective multi-armed bandit problem. arXiv preprint arXiv:1803.04039 (2018)

2018 arXiv
[11]

arXiv preprint arXiv:1905.12879 (2019)

Lu, S., Wang, G., Hu, Y., Zhang, L.: Multi-objective generalized linear bandits. arXiv preprint arXiv:1905.12879 (2019)

2019 arXiv
[12]

In: International Conference on Artificial Intelligence and Statistics, pp

Turgay, E., Oner, D., Tekin, C.: Multi-objective contextual bandit problem with similarity information. In: International Conference on Artificial Intelligence and Statistics, pp. 1673–1681 (2018). PMLR

2018
[13]

Machine Learning110(6), 1233–1266 (2021)

H¨ uy¨ uk, A., Tekin, C.: Multi-objective multi-armed bandit with lexicographically ordered and satisficing objectives. Machine Learning110(6), 1233–1266 (2021)

2021
[14]

Xue, B., Lin, X., Zhang, X., Zhang, Q.: Multiobjective stochastic linear bandits under lexicographic ordering
[15]

In: Proceedings of the AAAI Conference on Artificial Intelligence, vol

Cheng, J., Xue, B., Yi, J., Zhang, Q.: Hierarchize pareto dominance in multi- objective stochastic linear bandits. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, pp. 11489–11497 (2024)

2024
[16]

In: International Conference on Artificial Intelligence and Statistics, pp

Ararat, C., Tekin, C.: Vector optimization with stochastic bandit feedback. In: International Conference on Artificial Intelligence and Statistics, pp. 2165–2190 (2023). PMLR

2023
[17]

In: International Conference on Artificial Intelligence and Statistics, pp

Garivier, A., Koolen, W.M.,et al.: Sequential learning of the pareto front for multi-objective bandits. In: International Conference on Artificial Intelligence and Statistics, pp. 3583–3591 (2024). PMLR

2024
[18]

In: Interna- tional Conference on Artificial Intelligence and Statistics, pp

Karag¨ ozl¨ u, E.M., Yıldırım, Y.C., Ararat, C., Tekin, C.: Learning the pareto set under incomplete preferences: Pure exploration in vector bandits. In: Interna- tional Conference on Artificial Intelligence and Statistics, pp. 3070–3078 (2024). PMLR 20

2024
[19]

Springer, New York (2007)

Coello, C.A.C.: Evolutionary Algorithms for Solving Multi-Objective Problems. Springer, New York (2007)

2007
[20]

Springer, Berlin (2003)

Vazirani, V.V.: Approximation Algorithms. Springer, Berlin (2003). https://doi. org/10.1007/978-3-662-04565-7

2003 doi
[21]

Journal of the ACM (JACM)25(4), 536–543 (1978) 21

Bentley, J.L., Kung, H.-T., Schkolnick, M., Thompson, C.D.: On the average number of maxima in a set of vectors and applications. Journal of the ACM (JACM)25(4), 536–543 (1978) 21

1978

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

In: The 2013 International Joint Conference on Neural Networks (IJCNN), pp

Drugan, M.M., Nowe, A.: Designing multi-objective multi-armed bandits algo- rithms: A study. In: The 2013 International Joint Conference on Neural Networks (IJCNN), pp. 1–8 (2013). IEEE

work page 2013

[2] [2]

Foundations and Trends®in Machine Learning12(1-2), 1–286 (2019)

Slivkins, A.,et al.: Introduction to multi-armed bandits. Foundations and Trends®in Machine Learning12(1-2), 1–286 (2019)

work page 2019

[3] [3]

IEEE Transactions on Intelligent Transportation Systems23(12), 25536–25545 (2021)

Wei, W., Yang, R., Gu, H., Zhao, W., Chen, C., Wan, S.: Multi-objective opti- mization for resource allocation in vehicular cloud computing networks. IEEE Transactions on Intelligent Transportation Systems23(12), 25536–25545 (2021)

work page 2021

[4] [4]

In: International Conference on Machine Learning, pp

Xu, M., Klabjan, D.: Pareto regret analyses in multi-objective multi-armed ban- dit. In: International Conference on Machine Learning, pp. 38499–38517 (2023). PMLR

work page 2023

[5] [5]

In: 2014 IEEE Symposium on Adaptive Dynamic 19 Programming and Reinforcement Learning (ADPRL), pp

Drugan, M.M., Now´ e, A., Manderick, B.: Pareto upper confidence bounds algo- rithms: an empirical study. In: 2014 IEEE Symposium on Adaptive Dynamic 19 Programming and Reinforcement Learning (ADPRL), pp. 1–8 (2014). IEEE

work page 2014

[6] [6]

Advances in neural information processing systems26(2013)

Mahdavi, M., Yang, T., Jin, R.: Stochastic convex optimization with multiple objectives. Advances in neural information processing systems26(2013)

work page 2013

[7] [7]

In: 2014 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL), pp

Yahyaa, S.Q., Drugan, M.M., Manderick, B.: Annealing-pareto multi-objective multi-armed bandit algorithm. In: 2014 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL), pp. 1–8 (2014). IEEE

work page 2014

[8] [8]

In: ESANN (2015)

Yahyaa, S.Q., Manderick, B.: Thompson sampling for multi-objective multi- armed bandits problem. In: ESANN (2015)

work page 2015