Thompson Sampling with Approximate Inference

Justin Domke; My Phan; Yasin Abbasi-Yadkori

REVIEW 2 major objections 4 minor 24 references

Thompson Sampling with Approximate Inference

T0 review · 2 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper shows that even a small constant error in the approximate posterior, measured by alpha-divergence, can make Thompson sampling incur linear regret, and that for alpha <= 0 a small amount of forced exploration restores sublinear…

desk verdict A clean existence result showing constant α-divergence error can break Thompson sampling; the forced-exploration fix for α≤0 is new, but Lemma 2's proof needs a patch for unbounded parameter spaces. read the letter →

arxiv 1908.04970 v2 pith:SQ3LR5H2 submitted 2019-08-14 cs.LG stat.ML

My Phan , Yasin Abbasi-Yadkori , Justin Domke This is my paper · ORCID

classification cs.LGstat.ML

keywords Thompsonsamplingapproximateinferencealpha-divergencemulti-armedbanditsregretboundsforcedexplorationposteriorconcentrationexploration-exploitation

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper studies Thompson sampling when the posterior distribution is only approximated, with the error measured by alpha-divergence. It establishes that for any positive error threshold, there are approximate posteriors within that threshold that cause linear regret: over-exploration when alpha > 0 and under-exploration when alpha < 1. For alpha <= 0, it proves that adding a forced-exploration step—choosing an arm uniformly at random with probability p_t that decays to zero but sums to infinity—restores sublinear regret even when the approximation error is a large constant. These results matter because exact posterior inference is rarely available in practice, so knowing which approximation errors are harmless and which are fatal guides the design of practical Thompson sampling variants.

What carries the argument

The key object is the $\alpha$-divergence D_alpha(P,Q) = (1 - integral p^$\alpha$ q^(1-$\alpha$) dx) / ($\alpha$(1-$\alpha$)), a family that includes reverse KL ($\alpha$ to 0), forward KL ($\alpha$ to 1), Hellinger ($\alpha$ = 1/2), and chi-squared ($\alpha$ = 2). The sign of $\alpha$ determines which region of the posterior the approximation preserves: small $\alpha$ fits the dominant mode, while large $\alpha$ covers the whole support. The paper constructs explicit approximate distributions that shift probability mass from the region where the best arm is identified to the suboptimal-arm region; for $\alpha$ > 0 this creates over-exploration, while for $\alpha$ < 1 the posterior fails to concentrate because pulling the suboptimal arm reveals nothing about the ranking (under the independence assumption). For $\alpha$ <= 0, the proof leverages a posterior-concentration lemma: if every arm is chosen infinitely often, the posterior probability of the true best arm tends to 1, and the divergence bound then forces the approximation to concentrate as well.

What would settle it

Run a two-armed Gaussian bandit with a prior where M2 and M1-M2 are dependent, and use the paper's under-exploring approximation that always puts mass on the suboptimal arm. If the alpha-divergence to the true posterior grows with time and the regret becomes sublinear, the independence condition is essential to the construction; if the regret stays linear, the phenomenon is broader than the stated assumptions.

Watch

Extended reading notes

Core claim

The core discovery is that a constant bound on the alpha-divergence between the true posterior and the approximate posterior is, by itself, insufficient to guarantee good regret: for any epsilon > 0 and any alpha, there exist approximate posteriors satisfying D_alpha(Pi_t, Q_t) < epsilon at every step that make Thompson sampling choose the wrong arm a constant fraction of the time (for alpha > 0) or always (for alpha < 1 under an independence condition), yielding linear frequentist and Bayesian regret. Conversely, for alpha <= 0, the same divergence constraint is benign once the algorithm is augmented with a uniform-random action with probability p_t = o(1) and sum_t p_t = infinity; then the posterior concentrates and the regret becomes o(T).

Load-bearing premise

The under-exploration negative result assumes that, under the prior, the suboptimal arm's mean is independent of the difference between the two means, so pulling the wrong arm teaches the algorithm nothing about which arm is better.

Editorial extensions

If this is right

Any practical approximate inference scheme for Thompson sampling must either keep the alpha-divergence error below a shrinking threshold for all but o(T) steps, or pair the algorithm with forced exploration when alpha <= 0.
The negative result for alpha > 0 cannot be repaired by forced exploration; methods whose approximation errors fall in that regime (for example, those minimizing forward KL or chi-squared divergence) need a different safeguard.
For alpha <= 0, a simple uniform-random action with decaying probability converts a linearly-regretting approximate sampler into one with o(T) regret, provided the underlying posterior-concentration lemma applies.
The paper shows that methods such as ensemble sampling and mean-field variational inference, which are known to under-explore, can be improved by this forced-exploration step in theory and in simulations.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A reader might infer that the alpha-divergence value alone is not the right diagnostic for approximate Thompson sampling; what matters for regret is whether the approximation preserves the probability mass assigned to the optimal-arm region, so benchmarking algorithms solely by divergence error is insufficient.
The independence condition in the under-exploration theorem is probably not sharp: with dependent means, alternative approximation constructions might still block posterior concentration, but the paper's explicit construction would need modification.
The same divergence-parameterized analysis could extend to other Bayesian sequential decision problems, such as Bayesian optimization or active learning, where the analogue of forced exploration is an informative query rather than a uniform one.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: this is a solid theory paper. It shows that a constant α-divergence bound between posterior and approximation is not enough to guarantee sublinear regret for Thompson sampling, and that for α≤0 a small forced-exploration rate restores sublinearity. The counterexamples in Theorems 1 and 2 are constructed, not fitted, and the proofs are mostly checkable. That is the real contribution; I haven't seen this characterization elsewhere.

The positive result (Theorem 3) is also new and practically useful: add p_t=o(1) exploration with divergent sum, and under the Russo/Qin assumptions you get o(T) regret even with a large constant error. The application to Ensemble sampling with Θ(log T) models in Appendix E is a nice touch.

Soft spots, in order: 1. Lemma 2, which transfers a parameter-space divergence bound to the arm-marginal divergence, is proved via a Jensen step over regions Ω_i with a uniform density 1/|Ω_i|. Under the Gaussian prior (Assumption 2) those regions have infinite volume, so the proof as written doesn't apply. The inequality itself is just the data-processing inequality for f-divergences, so the math is almost certainly true, but the proof needs a limiting argument or a direct DPI citation. Since Theorem 3's applicability to Gaussian priors relies on this transfer, the paper should fix it before publication. 2. Theorem 2's independence assumption (M2 ⊥ M1−M2) is doing a lot of work. The authors state it plainly, but it means the under-exploration lower bound only applies to a restricted class of priors. Fine as a lower bound, just don't over-generalize. 3. No code for the simulations. Not a correctness problem, and the simulations are illustrative, but reproducibility would be better with code.

The citation pattern looks honest; Lu & Van Roy is the right comparison, and the authors are clear about what is new and what depends on external lemmas. The 'deterministic mapping f' existence results are carefully worded so as not to overclaim.

Bottom line: this deserves a serious refereeing. The main theorems are likely correct, the Lemma 2 proof gap is real but patchable, and the paper gives a clear takeaway: with α>0 constant inference error can be fatal; with α≤0 forced exploration rescues you. I'd send it to review.

Referee Report

2 major / 4 minor

Summary. The paper studies Thompson sampling with approximate inference in k-armed bandits, measuring inference error by alpha-divergence. It shows that if the approximate posterior is only required to have small constant alpha-divergence from the true posterior at every round, then Thompson sampling can suffer linear frequentist and Bayesian regret: for alpha>0 through an over-exploring construction (Theorem 1, Corollary 1), and for alpha<1 through an under-exploring construction under an independence condition on the prior (Theorem 2, Corollary 2). Conversely, for alpha<=0, the paper proposes adding forced exploration with probability p_t=o(1) and sum p_t=infinity, and proves sublinear regret (Theorem 3). The negative constructions are explicit and checkable; the positive result depends on an external posterior-concentration lemma from Russo (2016) and on Lemma 2, which transfers a divergence bound from parameter space to the distribution over the best arm.

Significance. If the results stand, the paper delivers a clean message: an alpha-divergence error bound alone is not a sufficient statistic for the exploration behavior of Thompson sampling, and forced exploration can repair the problem for alpha<=0. The constructive lower bounds are built from explicit posterior reweightings rather than fitted parameters, which makes Theorems 1 and 2 robust and easy to verify. The paper also connects the discussion to Ensemble sampling and mean-field variational inference, and its simulations support the main intuition. The main open point for the positive result is the rigor of Lemma 2 for unbounded Gaussian parameter spaces, which is true but requires a corrected proof.

major comments (2)

[Lemma 2 (main text, proof in Appendix C)] The proof of Lemma 2 applies Lemma 9 with the uniform density 1/|S| on each best-arm region S=Omega_i, which requires |Omega_i| to be finite. Under Assumption 2 (uncorrelated multivariate Gaussian prior) the parameter space is R^k and the regions Omega_i={m:m_i=max_j m_j} have infinite volume, so the Jensen argument as written is undefined. This matters because Theorem 3 is stated for Assumption 2 and its proof needs the arm-level bound D_alpha(Pi_t,Q_t)<epsilon to follow from the parameter-level bound, making Lemma 2 load-bearing. The inequality itself is the data-processing inequality for f-divergences under the deterministic map m -> argmax_i m_i and is true; please replace the proof with an explicit data-processing argument or a limiting argument over finite-volume truncations of Omega_i.
[Section 5.1, Theorem 3, and Lemma 2] The paper overloads the symbols Pi_t and Q_t: immediately before Theorem 3 they are defined as distributions on the best arm, while in the problem statement and in Lemma 2 they are distributions on the parameter vector. Theorem 3 states D_alpha(Pi_t,Q_t)<epsilon without specifying which pair, and the paragraph after Lemma 2 says 'whenever the condition D_alpha(Pi_t,Q_t)<epsilon is satisfied, the condition D_alpha(Pi_t,Q_t)<epsilon will be satisfied' with identical notation for both. This ambiguity makes the logical chain from the parameter-space guarantee to Lemma 10 hard to verify. Please introduce distinct symbols for the arm-level and parameter-level distributions and rewrite the affected statements.

minor comments (4)

[Appendix A, Lemma 5] In the alpha=1 entry, the factor 1-1/e^epsilon is correct but can be read as a misprint; writing e^{-epsilon} would be clearer.
[Section B.4, Figure 5 captions] The captions read 'epsilon as a function z' and should be 'epsilon as a function of z'.
[Section E] The claim that Lemma 10 applies when the KL bound holds only on a subset of times T0 is informal; Lemma 10 as stated requires the bound at all times. The intended subsequence argument should be made explicit if this application is meant to be more than heuristic.
[Section 2.3, Eq. (3)] When alpha=0 or alpha=1, D_alpha is defined by its limiting value; stating this explicitly next to Eq. (3) would help the reader.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: negative results are explicit adversarial constructions; positive results rely on external concentration lemmas.

full rationale

The paper's negative results (Theorems 1-2 and Corollaries 1-2) are existence proofs, not fitted predictions. The approximate distributions Q_t are explicitly constructed from the posterior Π_t (Eq. 6 for α>0 and Eq. 10 for α<1), the α-divergence is computed in closed form (Eq. 7 and Lemma 8), and linear regret follows directly from the construction. No parameter is fitted to data and then renamed as a prediction. The positive result (Theorem 3) takes D_α(Π_t,Q_t)<ε as an assumption, invokes the external posterior-concentration Lemma 1 from Russo (2016) and Qin et al. (2017) to obtain concentration under forced exploration, and uses Lemma 2, a data-processing inequality, to transfer divergence from the parameter posterior to the arm-selection distribution. The cited results are independent prior work, not self-citations of the authors, and no load-bearing step reduces by the paper's own equations to its inputs. The only notable concern, a possible proof gap in Lemma 2 for unbounded Gaussian regions because Lemma 9 assumes finite |Ω_i|, is a correctness issue rather than a circularity issue, so it does not affect the circularity score.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The central claims are existence results, so the listed free parameters are existential constants chosen to satisfy divergence constraints, not values fitted to data. No new physical or mathematical entities are introduced. The proofs rely on standard tools, an external posterior-concentration lemma, and one prior-independence assumption used to build the under-exploration counterexample.

free parameters (2)

r (Theorem 1 reweighting factor) = r > 1, chosen so that D_alpha(Pi_t,Q_t) < epsilon
Introduced ad hoc in Eq. 6 to cap the probability of the optimal region at 1/r. It is an existential constant depending on epsilon and alpha, not fitted to data. The average regret per step is c*Delta*(1-1/r).
z (prior probability of the wrong arm in Theorem 2) = z close to 1, chosen so that D_alpha(Pi_t,Q_t) < epsilon
Introduced ad hoc to keep the posterior probability of the wrong arm constant. The error threshold epsilon is a function of z (Appendix B.4), and z is chosen to make the divergence small.

assumptions (5)

standard math Standard real analysis and probability tools: Jensen's inequality, dominated convergence, posterior update formulas for exponential families.
Used throughout the proofs, e.g., Lemma 9 for the coarsening inequality and Lemma 12 for sub-linearity.
domain assumption The problem setup assumes an approximate distribution Q_t exists with D_alpha(Pi_t,Q_t) < epsilon at every time step.
This is the premise of the problem statement (Section 2.4); the theorems characterize what follows from this bound alone.
domain assumption Theorem 3 relies on Lemma 1 from Russo (2016) / Qin et al. (2017): if every arm is chosen infinitely often, the posterior probability of the best arm converges to 1.
Required under Assumption 1 (bounded canonical exponential family) or Assumption 2 (Gaussian prior with known variance), stated in Section 5.1.
ad hoc to paper In Theorem 2, the prior satisfies M2 independent of M1-M2.
This independence is introduced specifically to keep the posterior from concentrating when only arm 2 is observed; without it the under-exploration construction fails.
standard math The alpha-divergence is jointly convex, and the regions used in Lemma 9 have finite volume (bounded parameter support).
Convexity is cited from Cichocki and Amari (2010); the finite-volume normalization in Lemma 9 assumes bounded parameter support, which the main theorems do not explicitly state.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Thompson Sampling with Approximate Inference." pith.science (2026). https://pith.science/paper/SQ3LR5H2

@misc{pith2026190804970,
  author       = {Pith},
  title        = {Pith review of: Thompson Sampling with Approximate Inference},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SQ3LR5H2}},
  note         = {Machine review of arXiv:1908.04970}
}

abstract

We study the effects of approximate inference on the performance of Thompson sampling in the $k$-armed bandit problems. Thompson sampling is a successful algorithm for online decision-making but requires posterior inference, which often must be approximated in practice. We show that even small constant inference error (in $\alpha$-divergence) can lead to poor performance (linear regret) due to under-exploration (for $\alpha<1$) or over-exploration (for $\alpha>0$) by the approximation. While for $\alpha > 0$ this is unavoidable, for $\alpha \leq 0$ the regret can be improved by adding a small amount of forced exploration even when the inference error is a large constant.

Figures

Figures reproduced from arXiv: 1908.04970 by the authors.

**Figure 1.** The Gaussian Q which minimizes Dα(P, Q) for different values of α where the target distribution P is a mixture of two Gaussians. Based on [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗

**Figure 2.** Approximation Qt (with high variance) and approximation Zt (with small variance) are defined in Section 3 where D1(Πt, Qt) = 2 and D0(Πt, Zt) = 1.5. Arm 1 is the true best arm. posterior (Figure 2a). In Section 4 we show that when α > 0 the approximation can incur this type of error, leading to linear regret. On the other hand, the regret of Zt is larger because Zt explores less than the exact Thompson sampling algo… view at source ↗

**Figure 3.** Updating the posterior by exact Thompson sampling or adding forced exploration does not [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Figures from the paper (2 more)

**Figure 4.** Figure 4: Lower bound of regret per time-step as a function of [PITH_FULL_IMAGE:figures/full_fig_p014_4.png]

**Figure 5.** Figure 5: as a function z that makes the regret linear for different values of α for the example constructed in the proof of Theorem 2. C Proof of Lemma 2 To convert between Dα(Πt, Qt) and Dα(Πt, Qt) we first prove the following lemma: Lemma 9 (Jensen’s Inequality). Let f : R2…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

24 extracted references · 19 canonical work pages

[1]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...
[2]

and Goyal, N

Agrawal, S. and Goyal, N. Further optimal regret bounds for T hompson sampling. In Proceedings of the Sixteenth International Conference on Artificial Intelligence and Statistics (AISTATS 2013), volume 31 of Proceedings of Machine Learning Research, pp.\ 99--107. PMLR, 2013

work page 2013
[3]

Andrieu, C., de Freitas, N., Doucet, A., and Jordan, M. I. An introduction to MCMC for machine learning. Machine Learning, 50 0 (1): 0 5--43, 2003. ISSN 1573-0565. doi:10.1023/A:1020281327116

doi:10.1023/a:1020281327116 2003
[4]

M., Kucukelbir, A., and McAuliffe, J

Blei, D. M., Kucukelbir, A., and McAuliffe, J. D. Variational inference: A review for statisticians. Journal of the American Statistical Association, 112 0 (518): 0 859--877, 2017. doi:10.1080/01621459.2017.1285773

arXiv 2017
[5]

Stan: A probabilistic programming language

Carpenter, B., Gelman, A., Hoffman, M., Lee, D., Goodrich, B., Betancourt, M., Brubaker, M., Guo, J., Li, P., and Riddell, A. Stan: A probabilistic programming language. Journal of Statistical Software, 76 0 (1), 2017

work page 2017
[6]

and Amari, S

Cichocki, A. and Amari, S. Families of alpha- beta- and gamma- divergences: Flexible and robust measures of similarities. Entropy, 12: 0 1532--1568, 2010

work page 2010
[7]

and Johansen, A

Doucet, A. and Johansen, A. A tutorial on particle filtering and smoothing: Fifteen years later. Handbook of Nonlinear Filtering, 12: 0 656--704, 2009

work page 2009
[8]

Thompson sampling for complex online problems

Gopalan, A., Mannor, S., and Mansour, Y. Thompson sampling for complex online problems. In Proceedings of the 31st International Conference on Machine Learning, volume 32 of Proceedings of Machine Learning Research, pp.\ 100--108, Bejing, China, 22--24 Jun 2014. PMLR

work page 2014

Show all 24 references

[9]

H., Kveton, B., Tran-Thanh, L., and Chawla, S

Kawale, J., Bui, H. H., Kveton, B., Tran-Thanh, L., and Chawla, S. Efficient T hompson sampling for online matrix-factorization recommendation. In Advances in Neural Information Processing Systems 28 (NIPS 2015), pp.\ 1297--1305. Curran Associates, Inc., 2015

2015
[10]

Garbage in, reward out: Bootstrapping exploration in multi-armed bandits

Kveton, B., Szepesvari, C., Vaswani, S., Wen, Z., Lattimore, T., and Ghavamzadeh, M. Garbage in, reward out: Bootstrapping exploration in multi-armed bandits. In Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning ...

2019
[11]

and Li, L

Liu, C.-Y. and Li, L. On the prior sensitivity of thompson sampling. In Algorithmic Learning Theory, pp.\ 321--336, Cham, 2016. Springer International Publishing. ISBN 978-3-319-46379-7

2016
[12]

and Van Roy, B

Lu, X. and Van Roy, B. Ensemble sampling. In Advances in Neural Information Processing Systems 30 (NIPS 2017), pp.\ 3260--3268. Curran Associates, Inc., 2017

2017
[13]

Divergence measures and message passing

Minka, T. Divergence measures and message passing. Technical Report MSR-TR-2005-173, January 2005

2005
[14]

Minka, T., Winn, J., Guiver, J., Zaykov, Y., Fabian, D., and Bronskill, J. / Infer. NET 0.3 , 2018. Microsoft Research Cambridge. http://dotnet.github.io/infer

2018
[15]

and Weinberger, M

Ordentlich, E. and Weinberger, M. J. A distribution dependent refinement of P insker's inequality. International Symposium on Information Theory, 2004. ISIT 2004. Proceedings., pp.\ 29--, 2004

2004
[16]

Generalization and exploration via randomized value functions

Osband, I., Van Roy, B., and Wen, Z. Generalization and exploration via randomized value functions. In Proceedings of the 33rd International Conference on International Conference on Machine Learning - Volume 48, ICML'16, pp.\ 2377--2386. JMLR.org, 2016

2016
[17]

Improving the expected improvement algorithm

Qin, C., Klabjan, D., and Russo, D. Improving the expected improvement algorithm. In Advances in Neural Information Processing Systems 30, pp.\ 5381--5391. Curran Associates, Inc., 2017

2017
[18]

Deep B ayesian bandits showdown: An empirical comparison of bayesian deep networks for thompson sampling

Riquelme, C., Tucker, G., and Snoek, J. Deep B ayesian bandits showdown: An empirical comparison of bayesian deep networks for thompson sampling. In International Conference on Learning Representations (ICLR 2018), 2018

2018
[19]

Simple B ayesian algorithms for best arm identification

Russo, D. Simple B ayesian algorithms for best arm identification. In 29th Annual Conference on Learning Theory (COLT 2016), volume 49 of Proceedings of Machine Learning Research, pp.\ 1417--1418. PMLR, 2016

2016
[20]

and Roy, B

Russo, D. and Roy, B. V. An information-theoretic analysis of T hompson sampling. Journal of Machine Learning Research, 17 0 (68): 0 1--30, 2016

2016
[21]

J., Roy, B

Russo, D. J., Roy, B. V., Kazerouni, A., Osband, I., and Wen, Z. A tutorial on T hompson sampling. Foundations and Trends® in Machine Learning, 11 0 (1): 0 1--96, 2018. ISSN 1935-8237. doi:10.1561/2200000070

2018 doi
[22]

V., and Fonnesbeck, C

Salvatier, J., Wiecki, T. V., and Fonnesbeck, C. Probabilistic programming in Python using PyMC3 . PeerJ Computer Science, 2: 0 e55, 2016

2016
[23]

B., Rudolph, M., Liang, D., and Blei, D

Tran, D., Kucukelbir, A., Dieng, A. B., Rudolph, M., Liang, D., and Blei, D. M. Edward: A library for probabilistic modeling, inference, and criticism . arXiv preprint arXiv:1610.09787, 2016

2016 arXiv
[24]

and Wiggins, C

Urteaga, I. and Wiggins, C. Variational inference for the multi-armed contextual bandit. In Proceedings of the Twenty-First International Conference on Artificial Intelligence and Statistics (AISTATS 2018), volume 84 of Proceedings of Machine Learning Research, pp.\ 698--706. ...

2018

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

[2] [2]

and Goyal, N

Agrawal, S. and Goyal, N. Further optimal regret bounds for T hompson sampling. In Proceedings of the Sixteenth International Conference on Artificial Intelligence and Statistics (AISTATS 2013), volume 31 of Proceedings of Machine Learning Research, pp.\ 99--107. PMLR, 2013

work page 2013

[3] [3]

Andrieu, C., de Freitas, N., Doucet, A., and Jordan, M. I. An introduction to MCMC for machine learning. Machine Learning, 50 0 (1): 0 5--43, 2003. ISSN 1573-0565. doi:10.1023/A:1020281327116

doi:10.1023/a:1020281327116 2003

[4] [4]

M., Kucukelbir, A., and McAuliffe, J

Blei, D. M., Kucukelbir, A., and McAuliffe, J. D. Variational inference: A review for statisticians. Journal of the American Statistical Association, 112 0 (518): 0 859--877, 2017. doi:10.1080/01621459.2017.1285773

arXiv 2017

[5] [5]

Stan: A probabilistic programming language

Carpenter, B., Gelman, A., Hoffman, M., Lee, D., Goodrich, B., Betancourt, M., Brubaker, M., Guo, J., Li, P., and Riddell, A. Stan: A probabilistic programming language. Journal of Statistical Software, 76 0 (1), 2017

work page 2017

[6] [6]

and Amari, S

Cichocki, A. and Amari, S. Families of alpha- beta- and gamma- divergences: Flexible and robust measures of similarities. Entropy, 12: 0 1532--1568, 2010

work page 2010

[7] [7]

and Johansen, A

Doucet, A. and Johansen, A. A tutorial on particle filtering and smoothing: Fifteen years later. Handbook of Nonlinear Filtering, 12: 0 656--704, 2009

work page 2009

[8] [8]

Thompson sampling for complex online problems

Gopalan, A., Mannor, S., and Mansour, Y. Thompson sampling for complex online problems. In Proceedings of the 31st International Conference on Machine Learning, volume 32 of Proceedings of Machine Learning Research, pp.\ 100--108, Bejing, China, 22--24 Jun 2014. PMLR

work page 2014