Pith. sign in

REVIEW 4 major objections 4 minor 36 references

Optimizing Online Advertising with Multi-Armed Bandits: Mitigating the Cold Start Problem under Auction Dynamics

T0 review · 4 major / 4 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A UCB-style ranking algorithm for pay-per-click ad auctions achieves logarithmic regret, letting platforms explore cold-start ads without sacrificing long-term revenue.

desk verdict A plausible UCB extension to multi-slot pay-per-click auctions under the position-based model, but the K=L assumption means it is a ranking result, not an auction-selection result. read the letter →

arxiv 2502.01867 v1 pith:4PLTEXY6 submitted 2025-02-03 cs.LG

Anastasiia Soboleva , Andrey Pudovikov , Roman Snetkov , Alina Babenko , Egor Samosvat , Yuriy Dorn This is my paper · ORCID
classification cs.LG MSC 68T0591B2662L05
keywords coldstartmulti-armedbanditposition-basedmodelpay-per-clickauctionUCBregretboundonlineadvertisingfirst-price
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that the cold-start problem in online advertising—new ads lacking click data being ranked too low to ever get clicks—can be treated as a multi-armed bandit problem under auction dynamics, and that a UCB-style algorithm still enjoys strong theoretical guarantees. The authors propose AuctionUCB-PBM, which ranks ads by price times an upper confidence bound on click-through rate, in a multi-slot first-price pay-per-click auction with a position-based click model. The central theoretical result is a logarithmic regret bound, meaning the average regret per auction round shrinks to zero as the number of rounds grows. On synthetic data and on logged data from a real platform, the algorithm's per-round regret decreases, and a tail-exploration deployment scheme is proposed to keep short-term revenue from collapsing during exploration. A sympathetic reader would take the paper's message to be that controlled exploration of cold ads is not just practically useful but provably safe in this auction setting.

What carries the argument

The carrying object is the UCB eCPI index $$U_k(t) = \frac{S_k(t)}{N_k(t)} + \sqrt{\frac{\delta \ln t}{N_k(t)}},$$ where $S_k(t)$ counts clicks for ad $k$ and $N_k(t) = \sum_{l=1}^{L}\sum_{s=1}^{t-1} \gamma_l \mathbf{1}\{a_l(s)=k\}$ is the cumulative effective visibility of ad $k$, weighted by the position-dependent observation probabilities $\gamma_l$. The algorithm sorts ads by $P_k U_k(t)$ and shows the list. This index inherits the standard UCB concentration argument: with $\delta=1.5$, the probability that a true CTR lies outside its confidence interval is small enough that the bad-event regret is a constant, while the counting lemma of Combes et al. bounds how many rounds an under-observed ad can be responsible for a suboptimal ranking, producing the logarithmic term.

What would settle it

Run AuctionUCB-PBM with $K>L$—say 100 candidate ads and 10 slots—keeping prices and the same PBM. The index and proof rely on every ad being displayed every round; once the algorithm must choose a subset, the effective-impression statistic $N_k(t)$ is no longer a fixed exposure count, and a simulation where per-round regret fails to decay toward zero would show the paper's guarantee does not extend to general auction cold start.

Watch

Extended reading notes

Core claim

The paper's central claim is Theorem 1: for a $K$-slot first-price pay-per-click auction under the position-based model, with the number of ads equal to the number of slots and prices known, the AuctionUCB-PBM algorithm with $\delta = 1.5$ satisfies $$\mathbb{E}[R(T)] \leq \frac{\$pi^{2}$}{3} K \Delta_{\max} + \frac{64 K C(\gamma) P_{\max}}{\Delta_{\min}} \ln T,$$ where $\Delta_{\min}$ and $\Delta_{\max}$ are the smallest and largest regret gaps between the optimal ranking and any other ranking, $P_{\max}$ is the maximum click price, and $C(\gamma)$ is a constant depending only on the position visibility coefficients. In words, regret grows only logarithmically in the horizon $T$. The algorithm builds a UCB index for each ad from an effective-impression-weighted click estimator, ranks ads by decreasing $P_k U_k(t)$, and thereby balances exploration of cold ads with exploitation of known-good ads. The proof decomposes regret into a bad-event term, bounded by a constant through Chernoff–Hoeffding concentration, and a good-event term, bounded by $O(\ln T)$ via a counting lemma that limits how often any ad can be under-observed. The paper also claims experimental support: per-round regret decreases on synthetic data and on real logged auctions, and a top-$m$ deployment rule keeps short-term revenue stable while bandit exploration runs on the tail positions.

Load-bearing premise

The entire guarantee assumes the number of ads $K$ is exactly the number of slots $L$, so the learner only reranks a fixed set of ads and never has to decide which ads appear at all; real auctions have far more candidates than slots, and the proof does not cover selecting a subset of arms.

Editorial extensions

If this is right

  • Because regret is $O(\ln T)$, the average regret per round $\mathbb{E}[R(T)]/T$ tends to zero, so a platform running AuctionUCB-PBM indefinitely pays a vanishing per-auction penalty for exploring cold ads.
  • The algorithm needs only clicks, prices, and position visibility constants; it has no neural CTR model, so it can be deployed where labeled click data for new items is scarce.
  • The top-$m$ deployment rule lets a platform fix the most visible slots to baseline-ranked confident ads and run bandit exploration only in less visible tail slots, controlling short-term revenue loss by choosing the visibility share $\beta$.
  • The finite-time bound gives a concrete calibration rule: set $\delta=1.5$ and initialize each ad with one random ordering so that $N_k(t)>0$, and the logarithmic guarantee follows.
  • On logged real-auction data, per-round regret and CTR estimation error decrease with the number of opportunities, supporting deployment of the same algorithm on a live pay-per-click platform.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper fixes $K=L$, so the algorithm never faces the harder problem of selecting a subset of ads when candidates vastly outnumber slots. Extending AuctionUCB-PBM to $K>L$ would require a combinatorial arm-selection step, and it is an open question whether the same $O(\ln T)$ rate survives that change.
  • The reliance on known position-bias coefficients $\gamma_l$ means the regret guarantee silently assumes the click model is correctly specified; a deployment on a real platform would need to test whether estimation error in $\gamma_l$ dominates the exploration benefit.
  • Because the regret bound is in terms of the worst-case gap $\Delta_{\min}$, the algorithm may be slow to identify the optimal ranking when two ads have nearly equal eCPI; a gap-free or anytime variant would be a natural follow-up.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes AuctionUCB-PBM, a UCB-style algorithm for ranking ads in a multi-slot first-price pay-per-click auction under the position-based model (PBM). The authors claim a logarithmic upper bound on budget regret (Theorem 1), evaluate the algorithm on synthetic data and on logged real-platform data through an offline simulation, and propose a practical mechanism for mixing bandit exploration with a baseline ranking to control short-term revenue loss. The paper explicitly restricts attention to the case where the number of ads K equals the number of slots L, so the learner chooses a permutation of all ads rather than a subset.

Significance. If the theoretical and empirical claims are correct, the paper offers a simple UCB-style ranking method for a PBM-based PPC auction with a regret guarantee, and a practical deployment heuristic for balancing exploration with short-term revenue. The strengths are the clear problem formulation, the explicit algorithm, and the use of both synthetic and logged real-world data. However, the central contribution is incremental relative to prior multiple-play/PBM bandit work, the proof is presented as a sketch with several typos and an unstated external lemma, and the K=L restriction means the motivating auction-selection problem is not actually analyzed or tested.

major comments (4)
  1. [§3.2 and §5.2] The paper explicitly assumes K=L ("we consider the task of ranking, where the number of items equals the number of slots"), so every ad is displayed in every round and the learner only chooses a ranking, never a subset of candidate ads. Theorem 1 and Algorithm 1 depend on this: all arms receive effective impressions every round, so every UCB bonus decays, and the optimal action is the fixed sorted list a*=(1,...,K). In a real PPC auction the candidate pool is larger than the number of slots, and Algorithm 1 cannot be executed because it sorts all K ads into K slots. The real-data experiment in Section 5.2 sidesteps this by selecting only high-opportunity ads, restoring K=L. Consequently, the claimed applicability to the motivating auction setting is not supported. The authors should either extend the algorithm and regret analysis to K>L with subset selection, or reframe the paper's claims and title to the full-ranking regime and state the K>L case as an explicit limitation.
  2. [Theorem 1 and Appendix A] Theorem 1 states "Using L2R-BudgetUCB" although the proposed algorithm is AuctionUCB-PBM, and Appendix A begins with "In our regression analysis" where "regret analysis" is clearly intended. These are not merely cosmetic typos: they indicate that the proof may have been assembled from other papers without careful adaptation. The appendix also invokes an unstated external result, "[Combes et al.] Lemma 2," without stating its conditions or proof. A referee cannot verify the central regret bound from the material provided.
  3. [Appendix A, first term of the regret decomposition] The proof bounds the first term by (pi^2/3) K Delta_1, where Delta_1 = sum_k gamma_k P_k, but Theorem 1 states the bound with Delta_max = max_a Delta_a. Since Delta_max is generally smaller than Delta_1, the stated theorem does not follow from the proof as written. The authors must either prove the first term is bounded by (pi^2/3) K Delta_max or state the theorem with Delta_1 (or another justified constant).
  4. [Appendix A, events G_t and H_t] The definitions of the events are ambiguous and appear to be corrupted. G_t = {|S_t| >= l} and H_t = {|S_t| < l} use l both as a position index and as a cardinality threshold, while S_t is defined with the subscript N_{a_l(t)} that does not match the later use of N_k(t). This makes the chain F_t subset (G_t union H_t) and the subsequent splitting into G_{k,t}, H_{k,t} impossible to verify. The notation should be rewritten with distinct symbols and consistent indices.
minor comments (4)
  1. [Algorithm 1, lines 7-8] The loop increments t before updating N_k(t) and S_k(t), but the update formulas use N_k(t) = N_k(t-1) + gamma_k and S_k(t) = S_k(t-1) + r_k, which suggests the increment should occur after the update or the index should be shifted consistently.
  2. [Section 5.2] The real-data evaluation is an offline simulation that assumes ideal users and uses logged auctions, not a live experiment. This is a reasonable first step, but the wording in the abstract and conclusion that the method "confirm[s] the applicability" on the real platform is stronger than the evidence supports.
  3. [Section 5.1, Figure 1] The caption says "Upper is average Regret/t" and "lower is instant regret," but the figure labels should be clarified so the reader knows which panel corresponds to which metric.
  4. [Section 4.2] The paragraph explaining the warm-start phase says the regret from the initial random ordering "can be easily bounded by a constant," but this assertion is not justified in the proof. Either give a short argument or cite the relevant part of the appendix.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the regret bound is derived from standard concentration inequalities and external combinatorial-bandit lemmas, and the experiments do not fit the claimed prediction.

full rationale

The paper's central claim, Theorem 1, is a regret upper bound for AuctionUCB-PBM under a position-based model with pay-per-click first-price auctions. The proof in Appendix A decomposes regret into a concentration-failure term, bounded via Chernoff-Hoeffding with delta = 1.5, and a second term bounded by adapting external results from Lagrée et al. (2016), Combes et al. (2015), and Kveton et al. (2015). These are independent, non-overlapping prior proofs and are used as technical lemmas rather than as a substitute for the argument. The constant delta = 1.5 is chosen analytically so that the series t^{1-2delta} converges; it is not fitted to experimental outcomes. The regret bound's constants depend on problem parameters (gaps, prices, visibility coefficients) rather than on measured regret values, and the experiments compare algorithm regret against an optimal ranking computed from true CTRs without using the bound as an input. The main limitation, namely that the model assumes K = L so the learner ranks all ads rather than selecting a subset, is a scope and applicability concern, not a circularity: the theorem explicitly states this setting and the proof does not presuppose its conclusion. No load-bearing self-citation, fitted-input-renamed-as-prediction, or author-imported uniqueness result was found. The derivation is self-contained apart from standard external lemmas, so the circularity score is 0.

Assumptions & free parameters 3 free parameters · 8 assumptions · 0 invented entities

The central claim rests on standard MAB and PBM modeling assumptions plus external regret-analysis lemmas. No entity is invented and no parameter is fitted to the target result; delta is a proof-convenience constant, while beta and alpha are user-chosen deployment dials in Section 6.

free parameters (3)
  • delta (UCB exploration parameter) = 1.5
    Appears in the upper-confidence formula U_k(t)=hat_theta_k(t)+sqrt(delta ln t / N_k(t)) and in Theorem 1; chosen so the constant-sum term converges via zeta(2). It is a proof-convenience constant, not fitted to observed data.
  • beta (top-slot exploit share) = e.g., 0.2, 0.3, 0.4, 0.8
    In Section 6, beta sets how much total visibility is reserved for baseline-confident ads in the top m slots; the paper gives examples but no rule for choosing beta in production.
  • alpha (confidence-interval threshold for top-m participation) = not specified
    In Section 6, alpha decides whether the bandit CTR estimate can be used for an ad competing for the top m slots; the paper defines the rule but gives no value or estimation procedure.
assumptions (8)
  • domain assumption Click probability under PBM is examination probability gamma_l times ad relevance theta_k.
    Adopted in Sections 3.1 and 3.2 from Craswell et al.; if the user behavior model is wrong, the reward model and the regret bound do not apply.
  • domain assumption Visibility coefficients gamma_l are known, strictly decreasing, and fixed for all rounds.
    Stated in Section 3.2.1; the proof uses them as constants.
  • domain assumption Each ad has a known, fixed per-click price P_k.
    Section 3.2.1; prices are inputs to the auction and to the UCB index P_k U_k(t).
  • domain assumption True eCPIs are strictly ordered: P_1 theta_1 > ... > P_K theta_K.
    Section 3.2.2; this identifies the optimal ranking and defines the regret gaps Delta_a; ties are excluded from Delta_min.
  • domain assumption Click events are independent Bernoulli draws across rounds and arms.
    Section 3.2.1; the concentration inequality and regret analysis require independence.
  • standard math Chernoff-Hoeffding concentration applies to the ratio S_k(t)/N_k(t) with random effective-impression counts N_k(t).
    Used in Proposition 1; the conditional-on-N argument makes it plausible, but the paper does not formalize the stopping-time details.
  • standard math Combes et al. Lemma 2 bounds sums of under-explored arm counts.
    Invoked in Appendix A as a black box to bound the G_t and H_t terms; no proof is reproduced in this paper.
  • ad hoc to paper Regret from the initial random warm-up round can be bounded by a constant and ignored.
    Section 4.2 asserts this without derivation; it is plausible because the warm-up is a single round, but it is a stated rather than proved part of Theorem 1.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Optimizing Online Advertising with Multi-Armed Bandits: Mitigating the Cold Start Problem under Auction Dynamics." pith.science (2026). https://pith.science/paper/4PLTEXY6

@misc{pith2026250201867,
  author       = {Pith},
  title        = {Pith review of: Optimizing Online Advertising with Multi-Armed Bandits: Mitigating the Cold Start Problem under Auction Dynamics},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4PLTEXY6}},
  note         = {Machine review of arXiv:2502.01867}
}
read the original abstract

Online advertising platforms often face a common challenge: the cold start problem. Insufficient behavioral data (clicks) makes accurate click-through rate (CTR) forecasting of new ads challenging. CTR for "old" items can also be significantly underestimated due to their early performance influencing their long-term behavior on the platform. The cold start problem has far-reaching implications for businesses, including missed long-term revenue opportunities. To mitigate this issue, we developed a UCB-like algorithm under multi-armed bandit (MAB) setting for positional-based model (PBM), specifically tailored to auction pay-per-click systems. Our proposed algorithm successfully combines theory and practice: we obtain theoretical upper estimates of budget regret, and conduct a series of experiments on synthetic and real-world data that confirm the applicability of the method on the real platform. In addition to increasing the platform's long-term profitability, we also propose a mechanism for maintaining short-term profits through controlled exploration and exploitation of items.

Figures

Figures reproduced from arXiv: 2502.01867 by the authors.

Figure 1
Figure 1. Evaluation of AuctionUCB-PBM on synthetic data, generated as 1(c) combination - fixed [PITH_FULL_IMAGE:figures/full_fig_p008_1.png] view at source ↗
Figure 2
Figure 2. Evaluation of AuctionUCB-PBM on synthetic data, generated as 1(c) combination - fixed [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 3
Figure 3. Evaluating AuctionUCB-PBM on real data. Upper graphic describes the average on time cumulative regret [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Evaluating AuctionUCB-PBM on real data. The upper picture describes the distribution of absolute error [PITH_FULL_IMAGE:figures/full_fig_p011_4.png]
Figure 5
Figure 5. Figure 5: Evaluating AuctionUCB-PBM on synthetic data. Comparing [PITH_FULL_IMAGE:figures/full_fig_p016_5.png]
Figure 6
Figure 6. Figure 6: Evaluating AuctionUCB-PBM on synthetic data. Comparing [PITH_FULL_IMAGE:figures/full_fig_p017_6.png]
Figure 7
Figure 7. Figure 7: Evaluating AuctionUCB-PBM on synthetic data. Comparing [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

36 extracted references · 35 canonical work pages

  1. [1]

    Wide & deep learning for recommender systems

    [Cheng et al.(2016)Cheng, Koc, Harmsen, Shaked, Chandra, Aradhye, Anderson, Corrado, Chai, Ispir, et al.] Heng- Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, et al. Wide & deep learning for recommender systems. In Proceedings of the 1st workshop on deep learning f...

    work page 2016

  2. [5]

    Cold start to improve market thickness on online advertising platforms: Data-driven algorithms and field experiments

    [Ye et al.(2023)Ye, Zhang, Zhang, Zhang, Chen, and Xu] Zikun Ye, Dennis J Zhang, Heng Zhang, Renyu Zhang, Xin Chen, and Zhiwei Xu. Cold start to improve market thickness on online advertising platforms: Data-driven algorithms and field experiments. Management Science, 69(7):3838–3860,

    work page 2023

  3. [10]

    Finite-time analysis of the multiarmed bandit problem

    [Auer et al.(2002)Auer, Cesa-Bianchi, and Fischer] Peter Auer, Nicolo Cesa-Bianchi, and Paul Fischer. Finite-time analysis of the multiarmed bandit problem. Machine learning, 47:235–256,

    work page 2002

  4. [13]

    Bandits and recom- mender systems

    [Mary et al.(2015)Mary, Gaudel, and Preux] Jérémie Mary, Romaric Gaudel, and Philippe Preux. Bandits and recom- mender systems. In Machine Learning, Optimization, and Big Data: First International Workshop, MOD 2015, Taormina, Sicily, Italy, July 21-23, 2015, Revised Selected Papers 1, pages 325–336. Springer,

    work page 2015

  5. [15]

    Social learning in multi agent multi armed bandits

    [Sankararaman et al.(2019)Sankararaman, Ganesh, and Shakkottai] Abishek Sankararaman, Ayalvadi Ganesh, and Sanjay Shakkottai. Social learning in multi agent multi armed bandits. Proceedings of the ACM on Measurement and Analysis of Computing Systems, 3(3):1–35,

    work page 2019

  6. [16]

    An experimental comparison of click position-bias models

    [Craswell et al.(2008)Craswell, Zoeter, Taylor, and Ramsey] Nick Craswell, Onno Zoeter, Michael Taylor, and Bill Ramsey. An experimental comparison of click position-bias models. In Proceedings of the 2008 international conference on web search and data mining, pages 87–94,

    work page 2008

  7. [18]

    Multiple-play bandits in the position-based model

    [Lagrée et al.(2016)Lagrée, Vernade, and Cappe] Paul Lagrée, Claire Vernade, and Olivier Cappe. Multiple-play bandits in the position-based model. Advances in Neural Information Processing Systems, 29,

    work page 2016

  8. [19]

    Regression-based latent factor models

    [Agarwal and Chen(2009)] Deepak Agarwal and Bee-Chung Chen. Regression-based latent factor models. In Pro- ceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 19–28,

    work page 2009

Show all 36 references
  1. [21]

    Content-based recommender systems: State of the art and trends

    [Lops et al.(2011)Lops, De Gemmis, and Semeraro] Pasquale Lops, Marco De Gemmis, and Giovanni Semeraro. Content-based recommender systems: State of the art and trends. Recommender systems handbook , pages 73–105,

    2011

  2. [22]

    Col- laborative filtering and deep learning based hybrid recommendation for cold start problem

    [Wei et al.(2016)Wei, He, Chen, Zhou, and Tang] Jian Wei, Jianhua He, Kai Chen, Yi Zhou, and Zuoyin Tang. Col- laborative filtering and deep learning based hybrid recommendation for cold start problem. In 2016 IEEE 14th Intl Conf on Dependable, Autonomic and Secure Computing, ...

    2016

  3. [23]

    A meta-learning perspective on cold-start recommendations for items

    [Vartak et al.(2017)Vartak, Thiagarajan, Miranda, Bratman, and Larochelle] Manasi Vartak, Arvind Thiagarajan, Con- rado Miranda, Jeshua Bratman, and Hugo Larochelle. A meta-learning perspective on cold-start recommendations for items. Advances in neural information processing ...

    2017

  4. [24]

    Cold-start sequential recommendation via meta learner

    [Zheng et al.(2021)Zheng, Liu, Li, and Wu] Yujia Zheng, Siyi Liu, Zekun Li, and Shu Wu. Cold-start sequential recommendation via meta learner. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 4706–4713,

    2021

  5. [26]

    Approaches and algorithms to mitigate cold start problems in recommender systems: a systematic literature review

    [Panda and Ray(2022)] Deepak Kumar Panda and Sanjog Ray. Approaches and algorithms to mitigate cold start problems in recommender systems: a systematic literature review. Journal of Intelligent Information Systems, 59 (2):341–366,

    2022

  6. [27]

    Using confidence bounds for exploitation-exploration trade-offs

    [Auer(2002)] Peter Auer. Using confidence bounds for exploitation-exploration trade-offs. Journal of Machine Learning Research, 3(Nov):397–422,

    2002

  7. [28]

    An empirical evaluation of thompson sampling

    [Chapelle and Li(2011)] Olivier Chapelle and Lihong Li. An empirical evaluation of thompson sampling. Advances in neural information processing systems, 24,

    2011

  8. [29]

    Accurately interpreting clickthrough data as implicit feedback

    [Joachims et al.(2017)Joachims, Granka, Pan, Hembrooke, and Gay] Thorsten Joachims, Laura Granka, Bing Pan, Helene Hembrooke, and Geri Gay. Accurately interpreting clickthrough data as implicit feedback. In Acm Sigir Forum, volume 51, pages 4–11. Acm New York, NY , USA,

    2017

  9. [30]

    Click models for web search

    13 arXiv Template A PREPRINT [Chuklin et al.(2022)Chuklin, Markov, and De Rijke] Aleksandr Chuklin, Ilya Markov, and Maarten De Rijke. Click models for web search. Springer Nature,

    2022

  10. [31]

    A measure of asymptotic efficiency for tests of a hypothesis based on the sum of observations

    [Chernoff(1952)] Herman Chernoff. A measure of asymptotic efficiency for tests of a hypothesis based on the sum of observations. The Annals of Mathematical Statistics, pages 493–507,

    1952

  11. [34]

    Tight regret bounds for stochastic combinatorial semi-bandits

    [Kveton et al.(2015)Kveton, Wen, Ashkan, and Szepesvari] Branislav Kveton, Zheng Wen, Azin Ashkan, and Csaba Szepesvari. Tight regret bounds for stochastic combinatorial semi-bandits. In Artificial Intelligence and Statistics, pages 535–543. PMLR,

    2015

  12. [35]

    We denote by Bt,k = q δ ln t Nk(t) the UCB-exploration bonus and by B+ t,k = q δ ln T Nk(t) an upper bound of this bonus

    A Proof of T heorem1 In our regression analysis, we primarily follow the work of [ Lagrée et al.(2016)Lagrée, Vernade, and Cappe] and incorporate additional calculations from [ Combes et al.(2015)Combes, Talebi Mazraeh Shahi, Proutiere, et al., Kveton et al.(2015)Kveton, Wen, ...

    2016

  13. [36]

    PT t=1 ∆a(t)I{ ¯Et, ∆a(t) > 0} ≤PT t=1 ∆a(t)I{Ft}. Proof. Taking action a(t) means that KX k=1 γk · Pak(t) · Uak(t)(t) ≥ KX k=1 γk · Pk · Uk(t) 14 arXiv Template A PREPRINT Under event ¯Et, all UCB’s are above the true parameter θk so we have PK k=1 γk · Pak(t) · (θak (t) + 2B...

    2015

  14. [1952]

    Probability inequalities for sums of bounded random variables

    [Hoeffding(1994)] Wassily Hoeffding. Probability inequalities for sums of bounded random variables. The collected works of Wassily Hoeffding, pages 409–426,

    1994

  15. [1994]

    Combinatorial bandits revisited

    [Combes et al.(2015)Combes, Talebi Mazraeh Shahi, Proutiere, et al.] Richard Combes, Mohammad Sadegh Talebi Mazraeh Shahi, Alexandre Proutiere, et al. Combinatorial bandits revisited. Advances in neural information processing systems, 28,

    2015

  16. [2002]

    Thompson sampling for dynamic multi-armed bandits

    [Gupta et al.(2011)Gupta, Granmo, and Agrawala] Neha Gupta, Ole-Christoffer Granmo, and Ashok Agrawala. Thompson sampling for dynamic multi-armed bandits. In 2011 10th International Conference on Machine Learning and Applications and Workshops, volume 1, pages 484–489. IEEE,

    2011

  17. [2008]

    Bandit learning to rank with position-based click models: Personalized and equal treatments

    [Zhou et al.(2023)Zhou, Liu, Jiao, Dong, Chen, Gao, and Sun] Tianchen Zhou, Jia Liu, Yang Jiao, Chaosheng Dong, Yetian Chen, Yan Gao, and Yi Sun. Bandit learning to rank with position-based click models: Personalized and equal treatments. arXiv preprint arXiv:2311.04528,

    2023 arXiv

  18. [2009]

    Factorization meets the neighborhood: a multifaceted collaborative filtering model

    [Koren(2008)] Yehuda Koren. Factorization meets the neighborhood: a multifaceted collaborative filtering model. In Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 426–434,

    2008

  19. [2011]

    Improved online learning algorithms for ctr prediction in ad auctions

    12 arXiv Template A PREPRINT [Feng et al.(2023)Feng, Liaw, and Zhou] Zhe Feng, Christopher Liaw, and Zixin Zhou. Improved online learning algorithms for ctr prediction in ad auctions. In International Conference on Machine Learning, pages 9921–9937. PMLR,

    2023

  20. [2014]

    Introduction to multi-armed bandits

    [Slivkins et al.(2019)] Aleksandrs Slivkins et al. Introduction to multi-armed bandits. Foundations and Trends® in Machine Learning, 12(1-2):1–286,

    2019

  21. [2015]

    Multi-armed bandits in recommendation systems: A survey of the state-of-the-art and future directions

    [Silva et al.(2022)Silva, Werneck, Silva, Pereira, and Rocha] Nícollas Silva, Heitor Werneck, Thiago Silva, Adri- ano CM Pereira, and Leonardo Rocha. Multi-armed bandits in recommendation systems: A survey of the state-of-the-art and future directions. Expert Systems with Appl...

    2022

  22. [2016]

    Deep & cross network for ad click predictions

    [Wang et al.(2017)Wang, Fu, Fu, and Wang] Ruoxi Wang, Bin Fu, Gang Fu, and Mingliang Wang. Deep & cross network for ad click predictions. In Proceedings of the ADKDD’17, pages 1–7

    2017

  23. [2017]

    Deep interest network for click- through rate prediction

    [Zhou et al.(2018)Zhou, Zhu, Song, Fan, Zhu, Ma, Yan, Jin, Li, and Gai] Guorui Zhou, Xiaoqiang Zhu, Chenru Song, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. Deep interest network for click- through rate prediction. In Proceedings of the 24th ACM SI...

    2018

  24. [2018]

    Deepfm: a factorization-machine based neural network for ctr prediction

    [Guo et al.(2017)Guo, Tang, Ye, Li, and He] Huifeng Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, and Xiuqiang He. Deepfm: a factorization-machine based neural network for ctr prediction. arXiv preprint arXiv:1703.04247,

    2017 arXiv

  25. [2019]

    Addressing cold start in product search via empirical bayes

    [Han et al.(2022)Han, Castells, Gupta, Xu, and Salaka] Cuize Han, Pablo Castells, Parth Gupta, Xu Xu, and Vamsi Salaka. Addressing cold start in product search via empirical bayes. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management, p...

    2022

  26. [2021]

    A perspective view and survey of meta-learning

    [Vilalta and Drissi(2002)] Ricardo Vilalta and Youssef Drissi. A perspective view and survey of meta-learning. Artificial intelligence review, 18:77–95,

    2002

  27. [2022]

    Facing the cold start problem in recommender systems

    [Lika et al.(2014)Lika, Kolomvatsos, and Hadjiefthymiades] Blerina Lika, Kostas Kolomvatsos, and Stathes Had- jiefthymiades. Facing the cold start problem in recommender systems. Expert systems with applications , 41(4):2065–2073,

    2014

  28. [2023]

    Addressing the item cold-start problem by attribute-driven active learning

    [Zhu et al.(2019)Zhu, Lin, He, Wang, Guan, Liu, and Cai] Yu Zhu, Jinghao Lin, Shibi He, Beidou Wang, Ziyu Guan, Haifeng Liu, and Deng Cai. Addressing the item cold-start problem by attribute-driven active learning. IEEE Transactions on Knowledge and Data Engineering, 32(4):631–644,

    2019

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.