Pith. sign in

REVIEW 4 major objections 7 minor 90 references

Accelerated learning from recommender systems using multi-armed bandit

T0 review · 4 major / 7 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A multi-armed bandit layer between offline metrics and A/B tests can cut recommender evaluation from months to weeks.

desk verdict A useful production case study of Thompson sampling as an A/B screen, but the reported A/B test validates the previous campaign winner rather than the current MAB winner, so the central acceleration claim is not empirically supported. read the letter →

arxiv 1908.06158 v1 pith:KOFO57J4 submitted 2019-08-16 cs.IR cs.LG

Meisam Hejazinia , Kyler Eastman , Shuqin Ye , Abbas Amirabadi , Ravi Divvela This is my paper · ORCID
classification cs.IRcs.LG
keywords multi-armedbanditrecommendersystemsThompsonsamplingA/Btestingofflineevaluationclick-throughrateconversionproductionexperimentation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that recommendation teams can evaluate algorithms much faster by inserting a multi-armed bandit (MAB) test between biased offline metrics and slow online A/B tests. In the authors' production vacation-rental recommender, four recommendation variants ran in one MAB campaign, and the bandit shifted traffic to the highest-clicking arm within two weeks. The paper reports that the winner of a previous MAB campaign beat the control in a follow-up A/B test by 10.8 percent on click-through rate, with a non-significant 0.3 percent gain in conversion rate. The authors estimate that this hybrid pipeline can test up to six algorithms in about five weeks, versus about six months for serial A/B tests. The paper's own caveat is that the bandit screen ranks by click-through rate, which the authors say does not necessarily predict conversion.

What carries the argument

The load-bearing mechanism is Thompson sampling on a Bernoulli bandit with a Beta prior: each arm's click probability $p_i$ starts at $\mathrm{Beta}(1,1)$, accumulates successes and failures from the daily reward table, and the posterior $\mathrm{Beta}(S_i(t)+1, F_i(t)+1)$ is sampled to set the next day's traffic split. The paper adds production safeguards around this core: a minimum traffic floor so a cold arm is not starved in a seasonal business, nightly mini-batch aggregation to blunt bot noise, blacklisting for defective variants, and no allocation change when upstream data is absent. This machinery is what lets the authors treat the MAB as a short, roughly one-to-two-week screen rather than a replacement for A/B testing.

What would settle it

Run a MAB campaign whose click-through winner is then A/B tested against a lower-CTR arm with higher offline conversion metrics; if the lower-CTR arm wins the A/B conversion test, the bandit screen ranked arms on the wrong objective for that marketplace.

Watch

Extended reading notes

Core claim

The paper's central claim is that multi-armed bandit tests bridge the gap between quick but biased offline metrics and unbiased but slow online A/B testing, and that a production MAB stage can be built and run as a practical filter before A/B tests. It describes a daily pipeline in which user events are joined into a recommendation dataset, each arm's click probability is modeled as a Beta posterior, Thompson sampling draws 10,000 samples per day, and traffic is reallocated by the fraction of draws each arm wins. In the reported case study, the bandit concentrated all traffic on MAB_ARM-2 within two weeks, matching offline CTR rankings; offline CVR metrics, however, favored MAB_ARM-4, so the authors kept both as candidates for later A/B tests. An A/B test of the previous campaign winner against control showed +10.8 percent CTR and +0.3 percent CVR, the latter not statistically significant. The paper frames this as evidence that the hybrid pipeline accelerates iteration while still reserving unbiased conversion measurement for A/B tests.

Load-bearing premise

The screening step ranks arms by click-through rate and assumes that CTR is a trustworthy short-term stand-in for the conversion metric that the later A/B test measures; the paper itself says CTR does not necessarily predict CVR.

Editorial extensions

If this is right

  • Up to six candidate algorithms can be screened in about five weeks rather than roughly six months of serial A/B tests.
  • A/B tests only receive candidates that have already survived an unbiased short-term screen, so each A/B test compares more cultivated alternatives.
  • Users see fewer sub-optimal recommenders during experimentation because the bandit shifts traffic toward the currently better arm.
  • Offline metrics can be sanity-checked against unbiased traffic in one to two weeks, with divergent metrics such as CTR versus CVR flagging candidates for the next A/B test.
  • MAB testing supplements rather than replaces A/B testing: conversion-level claims still come from the follow-up A/B test.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the short-term screen is run on CTR alone, a marketplace whose revenue tracks conversion more than clicks may systematically promote the wrong arm; a multi-objective reward would be a natural test of this.
  • The five-week versus six-month speedup assumes candidate development can proceed in parallel and one MAB campaign can carry up to six arms; marketplaces with less traffic or longer purchase cycles would need to re-estimate the constants.
  • The paper's template could be reused to benchmark how often offline metric rankings actually predict MAB and A/B winners across many campaigns, which would turn its single case study into a calibration.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper reports an industrial case study in which a Thompson-sampling multi-armed bandit is inserted between offline ranking-metric evaluation and full A/B testing for recommender-system variants at Vrbo/Expedia. Four recommender variants (MAB_ARM-1 through MAB_ARM-4) are compared against a production Control and a previous MAB campaign winner. CTR is used as the bandit reward, while both CTR and CVR are reported as final online metrics. The authors argue that this hybrid workflow lets a team test up to six algorithms in about five weeks rather than six months of sequential A/B tests, by using the bandit to screen out poor candidates before committing to a full A/B test. The paper presents offline ranking metrics, MAB traffic-allocation results, and one A/B test comparing the previous campaign winner with Control.

Significance. If the central claim holds, the paper is a valuable real-world demonstration of MAB-based screening in production. Its strengths include the use of a standard Bayesian Thompson-sampling formulation with daily batched updates, the description of practical engineering safeguards (secured traffic threshold, blacklisting, invariance under missing data), and the rare presentation of offline, MAB, and A/B metrics for the same deployed variants. The theoretical contribution is standard, and the value is primarily as an experience report. The main weakness is that the presented evidence does not directly validate the claimed screening workflow, because the only A/B test reported is for a previous campaign winner rather than for the arm selected by the current MAB.

major comments (4)
  1. [Section 5.2 and Section 6, Table 1] The A/B test reported in Table 1 is not for the arm that the current MAB campaign selected. Section 5.2 states that once a campaign ends, the MAB winner will be A/B tested against the current control, but Table 1 only shows an A/B test of the previous campaign winner against Control; MAB_ARM-2 and the other current arms are not A/B tested. As a result, the central claim that the MAB identifies candidates that will succeed in a full A/B test is not empirically supported. The authors should either provide an A/B test of MAB_ARM-2 (or another current arm) or explicitly state that the current campaign winner's A/B validation is future work and correspondingly soften the abstract and introduction claims.
  2. [Table 1 and Section 6] The online MAB results are reported as point estimates without confidence intervals, posterior credible intervals, or significance tests, and the term 'expected CTR' is not formally defined. Likewise, the claim in Section 6 that traffic was 'allocated all' to MAB_ARM-2 within two weeks is not accompanied by a stopping rule, an analysis of sampling variability, or a demonstration that the allocation was stable rather than an artifact of a particular Thompson-sampling run. Without uncertainty quantification, neither the arm comparison nor the claimed speed-up can be properly evaluated.
  3. [Section 4.2 and Section 7] The bandit reward is CTR only, while the A/B objective used for final decisions includes both CTR and CVR. The paper itself acknowledges that 'CTR does not necessarily predict CVR' and reports that MAB_ARM-4 was best on offline CVR while losing on CTR. This creates a direct threat to the screening logic: if the MAB ranking on CTR diverges from the objective on which A/B winners are assessed, promoting the MAB winner may promote the wrong arm. The authors should address this by explaining the choice of CTR as the bandit reward, by reporting a secondary MAB analysis using CVR as the reward, or by otherwise demonstrating that CTR ranking is a valid screen for CVR outcomes.
  4. [Section 1 and Section 6] The five-week versus six-month acceleration argument is presented only as a workflow illustration; the paper does not measure the actual time or traffic required for the MAB step versus A/B testing, nor does it report sample sizes (visitors, views, and clicks per arm) or the number of users exposed to suboptimal arms. Please report campaign durations, traffic volumes, and, if possible, the regret or opportunity cost of the exploration phase. These quantities are needed to substantiate the central 'accelerated learning' claim.
minor comments (7)
  1. [Abstract] The sentence 'We argue that multi armed bandit (MAB) testing as a solution to these issues' is grammatically incomplete; it should read 'is a solution' or be rephrased.
  2. [Throughout] The word 'arm' is consistently misspelled as 'armed' in several places (e.g., 'each armed' in Section 4.2, 'winner armed' in Section 4.4, and '2nd best armed' in Section 6).
  3. [Section 4.2] The text says 'probability of a click' but writes 'probably of a click'; this typo should be corrected.
  4. [Section 3] The description of MAB_ARM-1 as 'built on a set of top of key item features' is unclear; please rephrase to specify what 'top' refers to.
  5. [Figures 1 and 3] In the submitted text, Figures 1 and 3 are referenced but their images are not visible. If this is a formatting issue, please ensure the figures are included; Figure 3 is essential to the Section 6 claim about traffic re-allocation over time.
  6. [Section 4.4] The configurable parameters (secured traffic threshold, RUDS look-ahead window, and the 10,000 posterior samples) are described qualitatively but their chosen values are not reported; please provide the settings used in this campaign and, if available, a brief sensitivity check.
  7. [Section 3 and References] The Control model is cited to the authors' own prior work [59]; the relation between the Control model used here and that paper should be made explicit so that the reader can assess whether the comparison is a strong baseline.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper is an empirical case study with standard Thompson sampling; the main evidence gap is a missing A/B validation of the current MAB winner, which is a correctness limitation, not a circular derivation.

full rationale

The paper does not contain a derivation chain that reduces to its own inputs. The MAB method is standard Thompson sampling with a Beta-Bernoulli model (Section 4.2), and its CTR-based traffic allocation is explicitly the algorithm's objective, not a hidden prediction fitted to the outcome. The offline metrics, MAB results, and A/B results are reported as separate measurements with acknowledged inconsistencies (Section 7), which indicates the authors are not forcing agreement. The only self-citation is [59], used to describe the production control model's neural-network architecture; this is a deployed baseline, not a load-bearing theoretical premise, and it does not by itself establish the paper's central workflow claim. The main weakness is that the reported A/B test validates the previous campaign winner, not the current MAB-selected arm, so the claim that MAB screening accelerates learning without degrading A/B outcomes is not empirically demonstrated. That is an evidence gap for correctness, not circularity: no equation or fitted parameter is renamed as a prediction, and no conclusion is asserted merely because the authors cited themselves. The paper is self-contained as an engineering report, so the circularity score is 0.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central claim rests on standard Thompson sampling plus several unquantified production assumptions, not on a mathematical derivation. No numbers are fitted to data in the paper, but three hand-set system parameters are not fully disclosed.

free parameters (3)
  • secured traffic threshold = not disclosed
    Configurable lower bound on per-arm traffic introduced in Section 4.4 to prevent zero allocation to temporarily losing arms. The value is chosen by hand and not reported; it modifies the MAB allocation output.
  • RUDS look-ahead window = not disclosed
    Section 4.1 uses a configured look-ahead window to join recommendation events with clicks and purchases. Its length determines reward attribution and is not specified.
  • posterior sample count = 10,000
    Section 4.2 draws 10,000 samples per daily mini-batch to estimate the probability each arm is best. This is a hand-chosen computational parameter, not fitted to data.
assumptions (5)
  • standard math Rewards are Bernoulli with a Beta(1,1) prior and conjugate posterior updates.
    Section 4.2 models clicks as Bernoulli variables with a Beta prior, citing Agrawal and Goyal [72]. This is standard Bayesian Thompson sampling.
  • domain assumption Per-visitor click events after randomization are independent enough for the daily Bernoulli sufficient statistics.
    Section 4.1 states a visitor-based metric is used 'to minimize the potential statistical dependency between observations'; independence is assumed but not tested.
  • domain assumption CTR is a valid short-term target whose ranking predicts the longer-term A/B outcome.
    Sections 5.2 and 7 optimize the MAB on CTR only, while the paper itself says CTR does not necessarily predict CVR and that MAB_ARM-4 had better offline CVR. This premise is load-bearing for the screen-and-promote workflow.
  • domain assumption The lower-bound traffic threshold and nightly mini-batch make the bandit robust to seasonality and bots without breaking randomization.
    Section 4.4 asserts these protections work based on practice ('we have observed...') with no quantitative evidence or formal guarantees.
  • domain assumption Known regret theory for Thompson sampling transfers to this non-stationary, non-IID production setting.
    Section 2.1 notes theory assumes stationary IID environments; the paper relies on ad hoc safeguards rather than theoretical guarantees in the production setting.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Accelerated learning from recommender systems using multi-armed bandit." pith.science (2026). https://pith.science/paper/KOFO57J4

@misc{pith2026190806158,
  author       = {Pith},
  title        = {Pith review of: Accelerated learning from recommender systems using multi-armed bandit},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KOFO57J4}},
  note         = {Machine review of arXiv:1908.06158}
}
read the original abstract

Recommendation systems are a vital component of many online marketplaces, where there are often millions of items to potentially present to users who have a wide variety of wants or needs. Evaluating recommender system algorithms is a hard task, given all the inherent bias in the data, and successful companies must be able to rapidly iterate on their solution to maintain their competitive advantage. The gold standard for evaluating recommendation algorithms has been the A/B test since it is an unbiased way to estimate how well one or more algorithms compare in the real world. However, there are a number of issues with A/B testing that make it impractical to be the sole method of testing, including long lead time, and high cost of exploration. We argue that multi armed bandit (MAB) testing as a solution to these issues. We showcase how we implemented a MAB solution as an extra step between offline and online A/B testing in a production system. We present the result of our experiment and compare all the offline, MAB, and online A/B tests metrics for our use case.

Figures

Figures reproduced from arXiv: 1908.06158 by the authors.

Figure 1
Figure 1. Mult-armed Bandit testing in the larger context of [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Architecture diagram of MAB daily processing. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Multi-armed Bandit Re-Allocation Timeseries [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

90 extracted references · 58 canonical work pages

  1. [1]

    Balashov I. Bar S. Lempel R. Moscovici N. Abensur, D. and I. Orlov. 2019. Produc- tization Challenges of Contextual Multi-Armed Bandits. (2019)

    2019

  2. [36]

    Daniel N Hill, Houssam Nassif, Yi Liu, Anand Iyer, and SVN Vishwanathan

    work page

  3. [70]

    Steven L Scott. 2015. Multi-armed bandit experiments in the online service economy. Applied Stochastic Models in Business and Industry 31, 1 (2015), 37–45

    work page 2015

  4. [59]

    Pavlos Mitsoulis-Ntompos, Meisam Hejazinia, Serena Zhang, and Travis Brady

    work page

  5. [2]

    Alekh Agarwal, Daniel Hsu, Satyen Kale, John Langford, Lihong Li, and Robert Schapire. 2014. Taming the monster: A fast and simple algorithm for contextual bandits. In International Conference on Machine Learning . 1638–1646

    2014

  6. [3]

    Alekh Agarwal, Haipeng Luo, Behnam Neyshabur, and Robert E Schapire. 2016. Corralling a band of bandit algorithms. arXiv preprint arXiv:1612.06246 (2016)

    work page Pith review arXiv 2016

  7. [4]

    Niko-Petteri Ahonen et al. 2017. Applying Bayesian Bandits For Solving Optimal Budget Allocation In Social Media Marketing. (2017)

    2017

  8. [5]

    Sanjeev Arora, Yingyu Liang, and Tengyu Ma. 2016. A simple but tough-to-beat baseline for sentence embeddings. (2016)

    2016

Show all 90 references
  1. [6]

    Jean-Yves Audibert and Sebastien Bubeck. 2010. Regret bounds and minimax policies under partial monitoring. Journal of Machine Learning Research 11, Oct (2010), 2785–2836

    2010

  2. [7]

    Orly Avner and Shie Mannor. 2014. Concurrent bandits and cognitive radio networks. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer, 66–81

    2014

  3. [8]

    Oren Barkan and Noam Koenigstein. 2016. Item2vec: neural item embedding for collaborative filtering. In 2016 IEEE 26th International Workshop on Machine Learning for Signal Processing (MLSP) . IEEE, 1–6

    2016

  4. [9]

    Joeran Beel, Marcel Genzmehr, Stefan Langer, Andreas Nürnberger, and Bela Gipp

  5. [10]

    Lilian Besson and Emilie Kaufmann. 2017. Multi-player bandits revisited. arXiv preprint arXiv:1711.02317 (2017)

    2017 arXiv

  6. [11]

    Veronika Bogina and Tsvi Kuflik. 2017. Incorporating Dwell Time in Session- Based Recommendations with Recurrent Neural Networks.. In RecTemp@ RecSys. 57–59

    2017

  7. [12]

    Etienne Boursier and Vianney Perchet. 2018. SIC-MMAB: synchronisation involves communication in multiplayer multi-armed bandits. arXiv preprint arXiv:1809.08151 (2018)

    2018 arXiv

  8. [13]

    Björn Brodén, Mikael Hammar, Bengt J Nilsson, and Dimitris Paraschakis. 2018. Ensemble recommendations via Thompson sampling: an experimental study within e-Commerce. In 23rd International Conference on Intelligent User Interfaces . ACM, 19–29

    2018

  9. [14]

    Sébastien Bubeck, Nicolo Cesa-Bianchi, et al. 2012. Regret analysis of stochastic and nonstochastic multi-armed bandit problems. Foundations and Trends ® in Machine Learning 5, 1 (2012), 1–122

    2012

  10. [15]

    Yang Cao, Zheng Wen, Branislav Kveton, and Yao Xie. 2018. Nearly Optimal Adaptive Procedure with Change Detection for Piecewise-Stationary Bandit. arXiv preprint arXiv:1802.03692 (2018)

    2018 arXiv

  11. [16]

    Olivier Cappé, Aurélien Garivier, Odalric-Ambrym Maillard, Rémi Munos, Gilles Stoltz, et al. 2013. Kullback–leibler upper confidence bounds for optimal sequen- tial allocation. The Annals of Statistics 41, 3 (2013), 1516–1541

    2013

  12. [17]

    Hugo Caselles-Dupré, Florian Lesaint, and Jimena Royo-Letelier. 2018. Word2vec applied to recommendation: Hyperparameters matter. In Proceedings of the 12th ACM Conference on Recommender Systems . ACM, 352–356

    2018

  13. [18]

    Nicolò Cesa-Bianchi, Claudio Gentile, Gábor Lugosi, and Gergely Neu. 2017. Boltzmann exploration done right. In Advances in Neural Information Processing Systems. 6284–6293

    2017

  14. [19]

    Olivier Chapelle and Lihong Li. 2011. An empirical evaluation of thompson sampling. In Advances in neural information processing systems . 2249–2257

    2011

  15. [20]

    Si Chen, Kristofer-Roy G Reyes, Maneesh K Gupta, Michael C McAlpine, and Warren B Powell. 2015. Optimal learning in experimental design using the knowledge gradient policy with application to characterizing nanoemulsion stability. SIAM/ASA Journal on Uncertainty Quantification...

    2015

  16. [21]

    Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, et al

  17. [22]

    Richard Combes, Stefan Magureanu, and Alexandre Proutiere. 2017. Minimal exploration in structured stochastic bandits. In Advances in Neural Information Processing Systems. 1763–1771

    2017

  18. [23]

    Simen Eide and Ning Zhou. 2018. Deep neural network marketplace recom- menders in online experiments. In Proceedings of the 12th ACM Conference on Recommender Systems. ACM, 387–391

    2018

  19. [24]

    Michael D Ekstrand, John T Riedl, Joseph A Konstan, et al. 2011. Collaborative filtering recommender systems. Foundations and Trends® in Human–Computer Interaction 4, 2 (2011), 81–173

    2011

  20. [25]

    Noyan Evirgen and Alper Kose. 2017. The effect of communication on noncoop- erative multiplayer multi-armed bandit problems. In 2017 16th IEEE International Conference on Machine Learning and Applications (ICMLA) . IEEE, 331–336

    2017

  21. [26]

    Sarah Filippi, Olivier Cappé, and Aurélien Garivier. 2010. Optimism in reinforce- ment learning and Kullback-Leibler divergence. In 2010 48th Annual Allerton Conference on Communication, Control, and Computing (Allerton) . IEEE, 115–122

    2010

  22. [27]

    Xavier Gabaix, David Laibson, Guillermo Moloche, and Stephen Weinberg. 2006. Costly information acquisition: Experimental analysis of a boundedly rational model. American Economic Review 96, 4 (2006), 1043–1068. Accelerated learning from recommender systems using multi-armed b...

    2006

  23. [28]

    Aurélien Garivier and Olivier Cappé. 2011. The KL-UCB algorithm for bounded stochastic bandits and beyond. In Proceedings of the 24th annual conference on learning theory. 359–376

    2011

  24. [29]

    Aurélien Garivier, Hédi Hadiji, Pierre Menard, and Gilles Stoltz. 2018. KL-UCB- switch: optimal regret bounds for stochastic bandits from both a distribution- dependent and a distribution-free viewpoints. arXiv preprint arXiv:1805.05071 (2018)

    2018 arXiv

  25. [30]

    Aurélien Garivier, Tor Lattimore, and Emilie Kaufmann. 2016. On explore-then- commit strategies. InAdvances in Neural Information Processing Systems. 784–792

    2016

  26. [31]

    Carlos A Gomez-Uribe and Neil Hunt. 2016. The netflix recommender system: Algorithms, business value, and innovation. ACM Transactions on Management Information Systems (TMIS) 6, 4 (2016), 13

    2016

  27. [32]

    Mihajlo Grbovic and Haibin Cheng. 2018. Real-time personalization using em- beddings for search ranking at Airbnb. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining . ACM, 311–320

    2018

  28. [33]

    Alois Gruson, Praveen Chandar, Christophe Charbuillet, James McInerney, Samantha Hansen, Damien Tardieu, and Ben Carterette. 2019. Offline Evaluation to Make Decisions About PlaylistRecommendation Algorithms. In Proceedings of the Twelfth ACM International Conference on Web Se...

    2019

  29. [34]

    Romain Guigourès, Yuen King Ho, Evgenii Koriagin, Abdul-Saboor Sheikh, Urs Bergmann, and Reza Shirvany. 2018. A hierarchical bayesian model for size recom- mendation in fashion. In Proceedings of the 12th ACM Conference on Recommender Systems. ACM, 392–396

    2018

  30. [35]

    Asela Gunawardana and Guy Shani. 2015. Evaluating recommender systems. In Recommender systems handbook. Springer, 265–308

    2015

  31. [37]

    FO Isinkaye, YO Folajimi, and BA Ojokoh. 2015. Recommendation systems: Principles, methods and evaluation. Egyptian Informatics Journal 16, 3 (2015), 261–273

    2015

  32. [38]

    Christopher C Johnson. 2014. Logistic matrix factorization for implicit feedback data. Advances in Neural Information Processing Systems 27 (2014)

    2014

  33. [39]

    Dileep Kalathil, Naumaan Nayyar, and Rahul Jain. 2014. Decentralized learning for multiplayer multiarmed bandits. IEEE Transactions on Information Theory 60, 4 (2014), 2331–2345

    2014

  34. [40]

    Baekjin Kim and Ambuj Tewari. 2019. On the Optimality of Perturbations in Stochastic and Adversarial Multi-armed Bandit Problems. arXiv preprint arXiv:1902.00610 (2019)

    2019 arXiv

  35. [41]

    Nakagawa H

    Honda J. Nakagawa H. Komiyama, J. 2019. Optimal Regret Analysis of Thompson Sampling in Stochastic Multi-armed Bandit Problem with Multiple Plays. (2019)

    2019

  36. [42]

    Jay Kreps, Neha Narkhede, Jun Rao, et al. 2011. Kafka: A distributed messaging system for log processing. In Proceedings of the NetDB . 1–7

    2011

  37. [43]

    Branislav Kveton, Csaba Szepesvari, Mohammad Ghavamzadeh, and Craig Boutilier. 2019. Perturbed-History Exploration in Stochastic Multi-Armed Bandits. arXiv preprint arXiv:1902.10089 (2019)

    2019 arXiv

  38. [44]

    Joon Kwon, Vianney Perchet, and Claire Vernade. 2017. Sparse stochastic bandits. arXiv preprint arXiv:1706.01383 (2017)

    2017 arXiv

  39. [45]

    Tor Lattimore. 2015. Optimally confident UCB: Improved regret for finite-armed bandits. arXiv preprint arXiv:1507.07880 (2015)

    2015 arXiv

  40. [46]

    Tor Lattimore. 2016. Regret analysis of the anytime optimally confident UCB algorithm. arXiv preprint arXiv:1603.08661 (2016)

    2016 arXiv

  41. [47]

    Quoc Le and Alexander Smola. 2007. Direct optimization of ranking measures. arXiv preprint arXiv:0704.3359 (2007)

    2007 arXiv

  42. [48]

    Lihong Li, Wei Chu, John Langford, Taesup Moon, and Xuanhui Wang. 2012. An unbiased offline evaluation of contextual bandit algorithms with generalized linear models. In Proceedings of the Workshop on On-line Trading of Exploration and Exploitation 2. 19–36

    2012

  43. [49]

    Lihong Li, Wei Chu, John Langford, and Robert E Schapire. 2010. A contextual- bandit approach to personalized news article recommendation. In Proceedings of the 19th international conference on World wide web . ACM, 661–670

    2010

  44. [50]

    Dawen Liang, Jaan Altosaar, Laurent Charlin, and David M Blei. 2016. Factor- ization meets the item embedding: Regularizing matrix factorization with item co-occurrence. In Proceedings of the 10th ACM conference on recommender systems. ACM, 59–66

    2016

  45. [51]

    Fang Liu, Joohyun Lee, and Ness Shroff. 2018. A change-detection based frame- work for piecewise-stationary multi-armed bandit problem. In Thirty-Second AAAI Conference on Artificial Intelligence

    2018

  46. [52]

    Fang Liu, Sinong Wang, Swapna Buccapatnam, and Ness Shroff. 2018. UCBoost: a boosting approach to tame complexity and optimality for stochastic bandits. arXiv preprint arXiv:1804.05929 (2018)

    2018 arXiv

  47. [53]

    Keqin Liu and Qing Zhao. 2010. Distributed learning in multi-armed bandit with multiple players. IEEE Transactions on Signal Processing 58, 11 (2010), 5667–5681

    2010

  48. [54]

    J Derek Lomas, Jodi Forlizzi, Nikhil Poonwala, Nirmal Patel, Sharan Shodhan, Kis- han Patel, Ken Koedinger, and Emma Brunskill. 2016. Interface design optimiza- tion as a multi-armed bandit problem. In Proceedings of the 2016 CHI Conference on Human Factors in Computing System...

    2016

  49. [55]

    Gábor Lugosi and Abbas Mehrabian. 2018. Multiplayer bandits without observing collision information. arXiv preprint arXiv:1808.08416 (2018)

    2018 arXiv

  50. [56]

    Yizhi Mao, Miao Chen, Abhinav Wagle, Junwei Pan, Michael Natkovich, and Don Matheson. 2018. A Batched Multi-Armed Bandit Approach to News Headline Testing. In 2018 IEEE International Conference on Big Data (Big Data) . IEEE, 1966– 1973

    2018

  51. [57]

    Dale McConachie and Dmitry Berenson. 2017. Bandit-based model selection for deformable object manipulation. arXiv preprint arXiv:1703.10254 (2017)

    2017 arXiv

  52. [58]

    James McInerney, Benjamin Lacker, Samantha Hansen, Karl Higley, Hugues Bouchard, Alois Gruson, and Rishabh Mehrotra. 2018. Explore, exploit, and explain: personalizing explainable recommendations with bandits. In Proceedings of the 12th ACM Conference on Recommender Systems . ...

    2018

  53. [60]

    Andriy Mnih and Ruslan R Salakhutdinov. 2008. Probabilistic matrix factorization. In Advances in neural information processing systems . 1257–1264

    2008

  54. [61]

    Eric Moulines. 1985. 8 On Upper-Confidence Bound Policies for Non-Stationary Bandit Problems. (1985)

    1985

  55. [62]

    Noam Koenigstein Oren Barkan. 2017. Item2Vec: Neural Item Embedding for Collaborative Filtering. (2017)

    2017

  56. [63]

    Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP) . 1532–1543

    2014

  57. [64]

    Ladislav Peska and Peter Vojtas. 2018. Off-line vs. On-line Evaluation of Recom- mender Systems in Small E-commerce. arXiv preprint arXiv:1809.03186 (2018)

    2018 arXiv

  58. [65]

    Vishnu Raj and Sheetal Kalyani. 2017. Taming non-stationary bandits: A Bayesian approach. arXiv preprint arXiv:1707.09727 (2017)

    2017 arXiv

  59. [66]

    Steffen Rendle, Christoph Freudenthaler, and Lars Schmidt-Thieme. 2010. Factor- izing personalized markov chains for next-basket recommendation. InProceedings of the 19th international conference on World wide web . ACM, 811–820

    2010

  60. [67]

    Steffen Rendle, Li Zhang, and Yehuda Koren. 2019. On the Difficulty of Evaluating Baselines: A Study on Recommender Systems. arXiv preprint arXiv:1905.01395 (2019)

    2019 arXiv

  61. [68]

    Jonathan Rosenski, Ohad Shamir, and Liran Szlak. 2016. Multi-player bandits–a musical chairs approach. In International Conference on Machine Learning . 155– 163

    2016

  62. [69]

    Stéphane Ross, Paul Mineiro, and John Langford. 2013. Normalized online learn- ing. arXiv preprint arXiv:1305.6646 (2013)

    2013 arXiv

  63. [71]

    Yevgeny Seldin and Gábor Lugosi. 2017. An improved parametrization and analysis of the EXP3++ algorithm for stochastic and adversarial bandits. arXiv preprint arXiv:1702.06103 (2017)

    2017 arXiv

  64. [72]

    Navin Goyal Shipra Agrawal. 2012. Analysis of Thompson Sampling for the Multi-armed Bandit Problem. In 25th Annual Conference on Learning Theory . 39.1âĂŞ39.26

    2012

  65. [73]

    Herbert A Simon. 1972. Theories of bounded rationality.Decision and organization 1, 1 (1972), 161–176

    1972

  66. [74]

    Adish Singla, Hamed Hassani, and Andreas Krause. 2017. Learning to Use Learners’ Advice. arXiv preprint arXiv:1702.04825 (2017)

    2017 arXiv

  67. [75]

    Pavel Sirotkin. 2013. On search engine evaluation metrics. arXiv preprint arXiv:1302.2318 (2013)

    2013 arXiv

  68. [76]

    Brent Smith and Greg Linden. 2017. Two decades of recommender systems at Amazon. com. Ieee internet computing 21, 3 (2017), 12–18

    2017

  69. [77]

    Liang Tang, Yexi Jiang, Lei Li, and Tao Li. 2014. Ensemble contextual bandits for personalized recommendation. In Proceedings of the 8th ACM Conference on Recommender Systems. ACM, 73–80

    2014

  70. [78]

    Flavian Vasile, Elena Smirnova, and Alexis Conneau. 2016. Meta-prod2vec: Product embeddings using side-information for recommendation. In Proceedings of the 10th ACM Conference on Recommender Systems . ACM, 225–232

    2016

  71. [79]

    Shengxian Wan, Yanyan Lan, Pengfei Wang, Jiafeng Guo, Jun Xu, and Xueqi Cheng. 2015. Next Basket Recommendation with Neural Networks.. In RecSys Posters

    2015

  72. [80]

    Jizhe Wang, Pipei Huang, Huan Zhao, Zhibo Zhang, Binqiang Zhao, and Dik Lun Lee. 2018. Billion-scale commodity embedding for e-commerce recommendation in alibaba. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining . ACM, 839–848

    2018

  73. [81]

    Shoujin Wang, Longbing Cao, and Yan Wang. 2019. A Survey on Session-based Recommender Systems. arXiv preprint arXiv:1902.04864 (2019)

    2019 arXiv

  74. [82]

    Yingfei Wang, Chu Wang, and Warren Powell. 2016. The knowledge gradient for sequential decision making with stochastic binary feedbacks. In International Conference on Machine Learning . 1138–1147

    2016

  75. [83]

    Lai Wei and Vaibhav Srivatsva. 2018. On abruptly-changing and slowly-varying multiarmed bandit problems. In 2018 Annual American Control Conference (ACC). IEEE, 6291–6296. , , Meisam Hejazinia, Kyler Eastman, Shuqin Ye, Abbas Amirabadi, and Ravi Divvela

    2018

  76. [84]

    Shu Wu, Yuyuan Tang, Yanqiao Zhu, Liang Wang, Xing Xie, and Tieniu Tan. 2018. Session-based Recommendation with Graph Neural Networks. arXiv preprint arXiv:1811.00855 (2018)

    2018 arXiv

  77. [85]

    Chunqiu Zeng, Qing Wang, Shekoofeh Mokhtari, and Tao Li. 2016. Online context- aware recommendation with time varying multi-armed bandit. In Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 2025–2034

    2016

  78. [86]

    Julian Zimmert and Yevgeny Seldin. 2018. An optimal algorithm for stochastic and adversarial bandits. arXiv preprint arXiv:1807.07623 (2018)

    2018 arXiv

  79. [2013]

    InProceedings of the international workshop on reproducibility and replication in recommender systems evaluation

    A comparative analysis of offline and online evaluations and discussion of research paper recommender system evaluation. InProceedings of the international workshop on reproducibility and replication in recommender systems evaluation . ACM, 7–14

  80. [2016]

    In Proceedings of the 1st workshop on deep learning for recommender systems

    Wide & deep learning for recommender systems. In Proceedings of the 1st workshop on deep learning for recommender systems . ACM, 7–10

  81. [2017]

    In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

    An efficient bandit algorithm for realtime multivariate optimization. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining . ACM, 1813–1821

  82. [2019]

    arXiv preprint arXiv:1906.11336 (2019)

    A Simple Deep Personalized Recommendation System. arXiv preprint arXiv:1906.11336 (2019)

    2019 arXiv

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.