A Batched Multi-Armed Bandit Approach to News Headline Testing

Abhinav Wagle; Don Matheson; Junwei Pan; Miao Chen; Michael Natkovich; Yizhi Mao

REVIEW 3 major objections 4 minor 26 references

A Batched Multi-Armed Bandit Approach to News Headline Testing

T0 review · 3 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that batched Thompson Sampling gathers 3.69% more clicks than equal-allocation headline testing while converging to the optimal headline within about 30 minutes.

desk verdict Solid industrial bandit application with an honest limitations section; the headline 3.69% gain is a simulated estimate without error bars and depends on stationary CTRs that the paper's own data suggest are violated. read the letter →

arxiv 1908.06256 v2 pith:JCKAPWCF submitted 2019-08-17 cs.LG stat.ML

Yizhi Mao , Miao Chen , Abhinav Wagle , Junwei Pan , Michael Natkovich , Don Matheson This is my paper · ORCID

classification cs.LGstat.ML

keywords multi-armedbanditThompsonSamplingbatchedupdatesheadlinetestingBernoulliclick-throughratetest-rolloutstrategynewsoptimization

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that replacing the common test-rollout headline experiment with batched Thompson Sampling (bTS) increases clicks. In the authors' simulation, bTS gathers 3.69% more total clicks over an article's lifespan, with a 13.54% gain in the first hour, while converging to the best headline in about 30 minutes for 80% of articles. If true, publishers can optimize headlines while testing them instead of sacrificing the first hour to equal allocation. The method is a multi-armed bandit where each headline is an arm, clicks are Bernoulli rewards, and posterior distributions update every five minutes.

What carries the argument

The central object is batched Thompson Sampling (bTS) on a Bernoulli bandit. Each headline is an arm with an unknown click probability $\theta_k$; the algorithm maintains a $\text{Beta}(\alpha_k,\beta_k)$ posterior for each arm, draws one sample per arm at every incoming view, displays the arm with the largest sample, and aggregates clicks and impressions over fixed five-minute intervals to update the posterior with the summation rule. The batch mechanism is what makes the method feasible for high-volume traffic while preserving Thompson Sampling's exploration-exploitation trade-off.

What would settle it

Run both bTS and test-rollout on the same set of articles while continuing to record clicks for every headline variant beyond the first hour, and compare total clicks using the recorded performance; if the measured gain falls materially below 3.69% when real post-first-hour click-through rates are used, the simulation's assumption of stationary click rates is the cause.

Watch

Extended reading notes

Core claim

The central discovery is that a multi-armed bandit treatment of headline testing, using Thompson Sampling with batched updates, can harvest clicks during the exploration phase without sacrificing accuracy. Treating each headline variant as a Bernoulli arm with a Beta prior, the algorithm reallocates traffic toward the currently best-seeming headline while retaining enough exploration to identify the true optimum. The authors tune three factors: a 48-hour stopping point chosen from the empirical distribution of article active lifespans, a summation update rule that adds raw click and non-click counts to the Beta parameters, and a five-minute update interval that balances infrastructure cost against marginal click gain. In their simulation, which uses real impression counts and user clicks simulated from first-hour empirical click-through rates, the method outperforms the test-rollout baseline by 3.69% in total clicks, converges correctly on 99.25% of articles, and shows an 80th-percentile optimization time under 30 minutes.

Load-bearing premise

User responses in the evaluation are simulated from fixed Bernoulli distributions whose success probabilities are the click-through rates observed in the first hour, and those probabilities are assumed constant over the whole article lifespan.

Editorial extensions

If this is right

Publishers can shift traffic toward winning headlines within minutes of publication instead of after a fixed test hour, so the early-life traffic peak is no longer spent on inferior headlines.
Because bTS keeps testing across the article's active lifespan, it can detect and self-correct from an unfavorable initial allocation; in the stress test, 80% of articles recovered within 33 minutes.
The observed 71.53% reduction in impressions on sub-optimal headlines implies a direct improvement in user experience, since fewer readers see headlines that perform poorly.
The algorithm's tuned parameters (48 hours, five-minute batches, summation update) transfer to other high-volume settings where optimization and testing must happen simultaneously, such as marketing campaigns or signup funnels.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A consequence the paper leaves implicit is that the 3.69% gain is probably a lower bound: if headline click rates drift after the first hour, bTS can reallocate while test-rollout cannot, so the real-world gap may be larger.
A consequence the paper leaves implicit is that its baseline is intentionally lenient toward test-rollout, since it grants the baseline knowledge of the true best arm 97% of the time in the post-testing period; a baseline using only the first-hour winner would likely enlarge bTS's advantage.
An untested extension is whether the five-minute batch interval stays optimal when traffic volume varies sharply across articles, since batch size is determined by observed impressions rather than a fixed event count.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Worth a look if you care about applied bandits. The paper does not invent batched Thompson Sampling, but it does something useful: it spells out a real deployment problem (news headline testing at Yahoo Front Page), compares summation versus normalization batch updates, and gives a sensible engineering recipe (5-minute updates, 48-hour horizon, summation). The writeup is clear and the tuning study, while simple, is a legitimate contribution for practitioners. The authors also deserve credit for telling you what they don't know. Section V explicitly says the stationary-CTR assumption is unassessed and that real CTRs can drift. That is honest and it makes the paper easy to read as an engineering report rather than an overclaiming methods paper. The soft spot is the one the stress-test note identifies, and it is real. The central claim—3.69% more clicks than test-rollout—comes from a simulation in which each arm's true CTR is fixed at its testing-period value. The baseline in equations (1)-(4) makes the same assumption. But Section II-B reports a 12% discrepancy between testing and post-testing CTRs, so the stationarity assumption is not just untested; the paper's own data suggest it is violated. If CTRs drift or reorder, the simulated gain could shrink, grow, or even change sign. There are also no error bars, no number of seeds, and no released code, so we cannot tell whether the 13.54% first-hour gain or the 1% post-testing gain is a stable result. The 1% post-testing gain is especially fragile because it is computed under the same fixed-CTR assumption that guarantees the baseline is optimal whenever it wins. That said, this is not a load-bearing flaw in the sense of invalidating the approach. The paper's structural point—bTS can reduce traffic to suboptimal headlines during the testing hour and may adapt to later drift—is sound. The flaw is in the magnitude claim. Nobody should cite the 3.69% as an established effect size. The paper would be improved by uncertainty quantification, a non-stationary sensitivity analysis, and a clearer statement that the gain is an upper/lower bound only under stationarity. Who gets value? Practitioners running headline or creative tests, and researchers who want a concrete example of batched TS in production. The math is straightforward and the citation pattern is appropriate. It deserves a serious referee; the method is sensible and the limitations are acknowledged. My recommendation: send it to review, but require a revised quantitative section before acceptance. I would not cite the 3.69% in my own work, but I might cite the batch-update comparison.

Referee Report

3 major / 4 minor

Summary. The paper formulates headline testing at Yahoo Front Page as a Bernoulli multi-armed bandit problem and proposes batched Thompson Sampling (bTS) with three tunable factors: the algorithm stopping point, the update method, and the update frequency. Using minute-level empirical impressions and clicks from production data, the authors simulate user responses from the empirical testing-period CTR of each headline variant, and compare bTS against the existing test-rollout strategy. They report that bTS converges correctly for 99.25% of articles, reaches stable optimal-arm allocation for 80% of articles within 30 minutes, self-corrects from an adversarial initial allocation within 33 minutes for 80% of articles, and produces 3.69% more total clicks than the test-rollout baseline (13.54% in the first hour and 1.00% in the remaining hours).

Significance. If the results held, the paper would offer a practical, deployment-oriented recipe for batched Thompson Sampling in high-volume content testing, with clearly specified algorithms and a transparent baseline definition. The strengths include the use of production impression and click data, the explicit comparison of summation and normalization update rules, and the honest acknowledgment in Section V that the stationarity assumption has not been assessed. However, the central quantitative claims rest entirely on a stationary simulation whose key assumption is in tension with the paper's own reported 12% testing/post-testing CTR discrepancy, and none of the simulation statistics are accompanied by uncertainty quantification. The contribution is therefore conditional on additional non-stationary analysis and more rigorous statistical reporting, rather than being established by the current evidence.

major comments (3)

[Section IV-D, Eqs. (1)-(4), Table IV] The headline 3.69% gain is computed under a fully stationary model: in the test-rollout baseline, Eq. (3) sets the post-testing click probability equal to the testing-period CTR of the winner, and in Section III-B2 the bTS simulation draws from Bernoulli(theta-hat_k) with theta-hat_k equal to the same fixed testing-period empirical CTRs for all 48 hours. This is in tension with the paper's own observation in Section II-B that testing-period and post-testing-period CTRs differ by 12% on average, and Section V states that the stationarity assumption remains untested. The simulation therefore cannot quantify the failure mode that motivated the paper, and the assertion in Section IV-D that 'the actual gain after implementation is likely to be larger' is not supported by the presented evidence. Please add non-stationary simulations calibrated to the reported 12% discrepancy, including drift and rank-reversal scenarios, and report click gains and convergence metrics under those scenarios alongside the stationary results.
[Section IV, Tables II-IV and Figures 2-4] All headline statistics are point estimates from stochastic simulations with no uncertainty quantification. The paper does not state the number of simulation runs or random seeds, and no confidence intervals or standard errors are reported anywhere. For example, the 3.69% total gain and the 1.00% post-testing gain in Table IV could easily be within Monte Carlo noise given the stochastic Bernoulli process used to generate user responses. Please report multi-seed means with standard errors or confidence intervals, and specify the number of simulation runs used for every reported quantity.
[Sections III-B2 through III-E and Section IV] The three algorithm factors (48-hour stopping point, summation update, and 5-minute update frequency) are selected using the same simulation practice and empirical data on which the final performance is then reported. Section III-B2 describes the simulation used to 'determine and tune the factors,' and Section IV uses the same simulation to report the method's click gain, with no held-out articles or separate validation phase described. This creates a risk that the reported 3.69% gain reflects in-sample selection of these factors. Please provide an out-of-sample evaluation, a sensitivity analysis around the chosen factor values, or both.

minor comments (4)

[Section III-D, Algorithm 2] The normalization update computes S_k/(S_k+F_k), which is undefined when an arm receives zero impressions in a batch; please specify the convention for this case.
[Section II-B] The '12% discrepancy in their CTR' is ambiguous; please state the metric (for example, mean absolute percentage difference or difference in percentage points) and report the distribution of the discrepancy across articles.
[Section IV-A] The definition of false convergence relies on the phrase 'allocate most traffic to the optimal arm' without a threshold; please define this criterion precisely so that the 99.25% rate is reproducible.
[Section III-B1] Please report the number of articles, the number of headline variants per article, and the range of empirical CTRs in the dataset; these quantities affect convergence speed and the magnitude of the reported gains.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the headline 3.69% click-gain figure is a Monte Carlo simulation outcome computed under an explicit stationary-arm model, not a fitted parameter or a self-citation chain.

full rationale

The paper's claimed derivation chain is not circular. The 3.69% gain is not algebraically forced by the inputs: Section IV-D defines the test-rollout baseline with equations (1)-(4) using Binomial samples with testing-period CTRs, and Section III-B2 says bTS user responses are simulated from Bernoulli(theta-hat_k) with the same empirical testing-period CTRs. Both algorithms face the same generative model, so the comparison is a controlled simulation evaluation rather than an identity; the click totals depend on each algorithm's stochastic allocation and are not equal to the CTR inputs by construction. There are no self-citations of the authors' own prior work, so no load-bearing self-citation chain or imported uniqueness theorem is present. The paper's tuning of update method and frequency on the same simulated data is an in-sample model-selection issue, but the differences across the frequency grid are small (Table III) and no equation reduces the reported gain to the chosen hyperparameter. The acknowledged stationarity limitation (Section V: 'The current assumption is that theta_k... is constant over time') is a correctness/validity caveat, not a circularity, because it concerns whether the simulation generalizes to real drifting CTRs, not whether the reported number is definitionally equivalent to its inputs. The claims about false convergence, speed, and self-correction are likewise empirical evaluations against the simulated ground truth, which is the standard role of a simulation. Therefore the derivation is self-contained and the appropriate score is 0.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The central simulation and tuning rest on a stationary Bernoulli reward model, empirical first-hour CTRs used as true click probabilities, and hyperparameters selected from the same data used for evaluation. No new physical or mathematical entities are introduced.

free parameters (4)

Update frequency (batch interval) = 5 minutes
Chosen in Section III-E as a trade-off between click gain (Table III shows 1-minute updates give the most clicks, 5-minute updates are 0.06% worse) and infrastructure cost, evaluated on the same simulated data.
Algorithm stopping point = 48 hours
Set in Section III-C because 95% of articles reach 95% of their impressions within 48 hours, based on the empirical dataset used for evaluation.
Update method = summation update
Selected in Section III-D because Table II shows summation update consistently outperforms normalization update across all update frequencies on the same simulated data.
Self-correction criterion = 5 consecutive batches
In Section IV-C, self-correction is defined as the optimal arm having the highest likelihood to be displayed for five consecutive batches, a hand-chosen threshold used to measure stress-test recovery time.

assumptions (3)

domain assumption Each headline has a fixed, unknown Bernoulli click probability theta_k that does not change over the article lifespan.
Section III-A1 models rewards as Bernoulli(theta_k) and the evaluation treats theta_k as constant. Section V acknowledges that this stationarity assumption should be assessed after implementation.
domain assumption The one-hour testing-period empirical CTR is an unbiased estimate of theta_k and is used as the true success probability in the simulation.
Section III-B2 simulates clicks from Bernoulli(hat theta_k) using testing-period CTRs, ignoring estimation uncertainty and any post-testing change in CTR.
domain assumption The three-day empirical impressions and clicks dataset is representative of article traffic patterns at Yahoo Front Page.
Section III-B1 uses two weekdays and one weekend day, with no analysis of seasonal, category, or time-of-day variation, so the distribution of article lifespans and CTRs may not generalize.

how reviews work

0 comments

Cite this review

Pith. "Pith review of A Batched Multi-Armed Bandit Approach to News Headline Testing." pith.science (2026). https://pith.science/paper/JCKAPWCF

@misc{pith2026190806256,
  author       = {Pith},
  title        = {Pith review of: A Batched Multi-Armed Bandit Approach to News Headline Testing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JCKAPWCF}},
  note         = {Machine review of arXiv:1908.06256}
}

read the original abstract

Optimizing news headlines is important for publishers and media sites. A compelling headline will increase readership, user engagement and social shares. At Yahoo Front Page, headline testing is carried out using a test-rollout strategy: we first allocate equal proportion of the traffic to each headline variation for a defined testing period, and then shift all future traffic to the best-performing variation. In this paper, we introduce a multi-armed bandit (MAB) approach with batched Thompson Sampling (bTS) to dynamically test headlines for news articles. This method is able to gradually allocate traffic towards optimal headlines while testing. We evaluate the bTS method based on empirical impressions/clicks data and simulated user responses. The result shows that the bTS method is robust, converges accurately and quickly to the optimal headline, and outperforms the test-rollout strategy by 3.69% in terms of clicks.

Figures

Figures reproduced from arXiv: 1908.06256 by the authors.

**Figure 1.** Histogram of article active lifespan which can only be calculated during the testing period. After the simulation is completed on all articles, we quantify the performance of a headline testing algorithm by total clicks summed across all articles over their lifespans. C. Factor 1: Algorithm Stopping Point Due to the observed CTR discrepancy between the testing and post-testing period illustrated in Section II-B, we … view at source ↗

**Figure 2.** Sample articles with correct convergence: traffic proportion allocated [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

**Figure 3.** Histogram of time to optimize. All articles that takes over 60 minutes [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: Histogram of time needed for self-correction. All articles that takes [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

26 extracted references · 20 canonical work pages

[1]

Headline testing page at optimizely

“Headline testing page at optimizely.” [Online]. Available: https: //www.optimizely.com/optimization-glossary/headline-testing/

work page
[2]

Customer acquisition via display advertising using multi-armed bandit experiments,

E. M. Schwartz, E. T. Bradlow, and P. S. Fader, “Customer acquisition via display advertising using multi-armed bandit experiments,” Market- ing Science, vol. 36, no. 4, pp. 500–522, 2017

work page 2017
[3]

Predicting the shape and peak time of news article views,

Y . Keneshloo, S. Wang, E.-H. S. Han, and N. Ramakrishnan, “Predicting the shape and peak time of news article views,” in Big Data (Big Data), 2016 IEEE International Conference on . IEEE, 2016, pp. 2400–2409

work page 2016
[4]

Analysis of thompson sampling for the multi- armed bandit problem,

S. Agrawal and N. Goyal, “Analysis of thompson sampling for the multi- armed bandit problem,” in Conference on Learning Theory , 2012, pp. 39–1

2012
[5]

Algorithms for multi-armed bandit prob- lems,

V . Kuleshov and D. Precup, “Algorithms for multi-armed bandit prob- lems,” arXiv preprint arXiv:1402.6028 , 2014

arXiv 2014
[6]

Finite-time analysis of the multiarmed bandit problem,

P. Auer, N. Cesa-Bianchi, and P. Fischer, “Finite-time analysis of the multiarmed bandit problem,” Machine learning , vol. 47, no. 2-3, pp. 235–256, 2002

work page 2002
[7]

Exploration–exploitation tradeoff using variance estimates in multi-armed bandits,

J.-Y . Audibert, R. Munos, and C. Szepesv ´ari, “Exploration–exploitation tradeoff using variance estimates in multi-armed bandits,” Theoretical Computer Science, vol. 410, no. 19, pp. 1876–1902, 2009

work page 1902
[8]

Regret bounds and minimax policies under partial monitoring,

J.-Y . Audibert and S. Bubeck, “Regret bounds and minimax policies under partial monitoring,” Journal of Machine Learning Research , vol. 11, no. Oct, pp. 2785–2836, 2010

work page 2010

Show all 26 references

[9]

On the likelihood that one unknown probability exceeds another in view of the evidence of two samples,

W. R. Thompson, “On the likelihood that one unknown probability exceeds another in view of the evidence of two samples,” Biometrika, vol. 25, no. 3/4, pp. 285–294, 1933

1933
[10]

Gittins, K

J. Gittins, K. Glazebrook, and R. Weber, Multi-armed bandit allocation indices. John Wiley & Sons, 2011

2011
[11]

An empirical evaluation of thompson sampling,

O. Chapelle and L. Li, “An empirical evaluation of thompson sampling,” in Advances in neural information processing systems , 2011, pp. 2249– 2257

2011
[12]

A modern bayesian look at the multi-armed bandit,

S. L. Scott, “A modern bayesian look at the multi-armed bandit,” Applied Stochastic Models in Business and Industry, vol. 26, no. 6, pp. 639–658, 2010

2010
[13]

A bayesian learning automaton for solving two-armed bernoulli bandit problems,

O.-C. Granmo, “A bayesian learning automaton for solving two-armed bernoulli bandit problems,” in 2008 Seventh International Conference on Machine Learning and Applications . IEEE, 2008, pp. 23–30

2008
[14]

Optimistic bayesian sampling in contextual-bandit problems,

B. C. May, N. Korda, A. Lee, and D. S. Leslie, “Optimistic bayesian sampling in contextual-bandit problems,” Journal of Machine Learning Research, vol. 13, no. Jun, pp. 2069–2106, 2012

2012
[15]

Thompson sampling: An asymptotically optimal ﬁnite-time analysis,

E. Kaufmann, N. Korda, and R. Munos, “Thompson sampling: An asymptotically optimal ﬁnite-time analysis,” in International Conference on Algorithmic Learning Theory . Springer, 2012, pp. 199–213

2012
[16]

Web-scale bayesian click-through rate prediction for sponsored search advertising in microsoft’s bing search engine

T. Graepel, J. Q. Candela, T. Borchert, and R. Herbrich, “Web-scale bayesian click-through rate prediction for sponsored search advertising in microsoft’s bing search engine.” Omnipress, 2010

2010
[17]

Computational advertising: the linkedin way,

D. Agarwal, “Computational advertising: the linkedin way,” in Pro- ceedings of the 22nd ACM international conference on Conference on information & knowledge management . ACM, 2013, pp. 1585–1586

2013
[18]

Laser: A scalable response prediction platform for online advertising,

D. Agarwal, B. Long, J. Traupman, D. Xin, and L. Zhang, “Laser: A scalable response prediction platform for online advertising,” in Proceedings of the 7th ACM international conference on Web search and data mining . ACM, 2014, pp. 173–182

2014
[19]

Multi-armed bandit experiments in the online service economy,

S. L. Scott, “Multi-armed bandit experiments in the online service economy,”Applied Stochastic Models in Business and Industry , vol. 31, no. 1, pp. 37–45, 2015

2015
[20]

An efﬁcient bandit algorithm for realtime multivariate optimization,

D. N. Hill, H. Nassif, Y . Liu, A. Iyer, and S. Vishwanathan, “An efﬁcient bandit algorithm for realtime multivariate optimization,” in Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining . ACM, 2017, pp. 1813–1821

2017
[21]

Overview of content experiments: Multi-armed bandit experiments,

S. L. Scott, “Overview of content experiments: Multi-armed bandit experiments,” 2014. [Online]. Available: https://support.google.com/ analytics/answer/2844870

2014
[22]

Randomized allocation with nonparametric estimation for a multi-armed bandit problem with covariates,

Y . Yang, D. Zhu et al. , “Randomized allocation with nonparametric estimation for a multi-armed bandit problem with covariates,” The Annals of Statistics , vol. 30, no. 1, pp. 100–121, 2002

2002
[23]

Asymptotically efﬁcient adaptive allocation rules,

T. L. Lai and H. Robbins, “Asymptotically efﬁcient adaptive allocation rules,” Advances in applied mathematics , vol. 6, no. 1, pp. 4–22, 1985

1985
[24]

Online network revenue management using thompson sampling,

K. Ferreira, D. Simchi-Levi, and H. Wang, “Online network revenue management using thompson sampling,” 2017

2017
[25]

Efﬁcient thompson sampling for online matrix-factorization recommen- dation,

J. Kawale, H. H. Bui, B. Kveton, L. Tran-Thanh, and S. Chawla, “Efﬁcient thompson sampling for online matrix-factorization recommen- dation,” in Advances in neural information processing systems, 2015, pp. 1297–1305

2015
[26]

Stop clickbait: Detecting and preventing clickbaits in online news media,

A. Chakraborty, B. Paranjape, S. Kakarla, and N. Ganguly, “Stop clickbait: Detecting and preventing clickbaits in online news media,” in Proceedings of the 2016 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining . IEEE Press, 2016, pp. 9–16

2016

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

Headline testing page at optimizely

“Headline testing page at optimizely.” [Online]. Available: https: //www.optimizely.com/optimization-glossary/headline-testing/

work page

[2] [2]

Customer acquisition via display advertising using multi-armed bandit experiments,

E. M. Schwartz, E. T. Bradlow, and P. S. Fader, “Customer acquisition via display advertising using multi-armed bandit experiments,” Market- ing Science, vol. 36, no. 4, pp. 500–522, 2017

work page 2017

[3] [3]

Predicting the shape and peak time of news article views,

Y . Keneshloo, S. Wang, E.-H. S. Han, and N. Ramakrishnan, “Predicting the shape and peak time of news article views,” in Big Data (Big Data), 2016 IEEE International Conference on . IEEE, 2016, pp. 2400–2409

work page 2016

[4] [4]

Analysis of thompson sampling for the multi- armed bandit problem,

S. Agrawal and N. Goyal, “Analysis of thompson sampling for the multi- armed bandit problem,” in Conference on Learning Theory , 2012, pp. 39–1

2012

[5] [5]

Algorithms for multi-armed bandit prob- lems,

V . Kuleshov and D. Precup, “Algorithms for multi-armed bandit prob- lems,” arXiv preprint arXiv:1402.6028 , 2014

arXiv 2014

[6] [6]

Finite-time analysis of the multiarmed bandit problem,

P. Auer, N. Cesa-Bianchi, and P. Fischer, “Finite-time analysis of the multiarmed bandit problem,” Machine learning , vol. 47, no. 2-3, pp. 235–256, 2002

work page 2002

[7] [7]

Exploration–exploitation tradeoff using variance estimates in multi-armed bandits,

J.-Y . Audibert, R. Munos, and C. Szepesv ´ari, “Exploration–exploitation tradeoff using variance estimates in multi-armed bandits,” Theoretical Computer Science, vol. 410, no. 19, pp. 1876–1902, 2009

work page 1902

[8] [8]

Regret bounds and minimax policies under partial monitoring,

J.-Y . Audibert and S. Bubeck, “Regret bounds and minimax policies under partial monitoring,” Journal of Machine Learning Research , vol. 11, no. Oct, pp. 2785–2836, 2010

work page 2010