Pith. sign in

REVIEW 1 major objections 5 minor 300 references

Optimal and Efficient Contextual Combinatorial Semi-bandits with General Function Approximation

T0 review · 1 major / 5 minor · reviewed 2026-08-02 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper introduces SquareCB.Comb, a provably efficient algorithm for contextual combinatorial semi-bandits that reaches a minimax-optimal regret of O(√(mAT log|F|)), matching the best possible rate up to logarithmic factors—the first suc

desk verdict A solid, likely correct resolution of the minimax rate for contextual combinatorial semi-bandits with general function approximation; two addressable gaps in the write-up, not in the core proof. read the letter →

arxiv 2607.13686 v1 pith:JQ7PXXHC submitted 2026-07-15 cs.LG

Hao Qin , Chicheng Zhang This is my paper · ORCID
classification cs.LG
keywords contextualcombinatorialsemi-banditsminimaxoptimalregretgeneralfunctionapproximationsemi-banditfeedbacklog-barrierexplorationdecision-estimationcoefficientboundsregressionoracle
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tackles contextual combinatorial semi-bandits—choosing a subset of arms each round while seeing only the rewards of the chosen arms—when rewards are modeled by an arbitrary finite function class, not just linear functions. It proposes SquareCB.Comb, which each round solves a single A-dimensional convex program with a log-barrier regularizer to obtain marginal inclusion probabilities, then draws a feasible action matching those marginals. The paper proves expected regret O(√(mAT log|F|)) under realizability and a batch square-loss regression oracle, and proves a matching lower bound up to log factors. If right, this is the first minimax-optimal CCSB guarantee beyond linear rewards and it removes the extra √m factor paid by reducing CCSB to full-bandit learning.

What carries the argument

The key mechanism is the participation vector: for a distribution over combinatorial actions, the vector of marginal selection probabilities per arm. SquareCB.Comb alternates a batch square-loss regression oracle for reward estimates with a log-barrier-regularized convex program over participation vectors—maximize estimated reward plus (1/γ)Σ log p̄(a)—whose solution certifies a small value of the combinatorial square-loss decision-estimation coefficient (CS-DEC), a complexity measure adapted from interactive decision making to semi-bandit feedback. The crucial reduction is that the CS-DEC minimization, which naively runs over exponentially many actions, depends only on marginals and therefo

What would settle it

Take a DAG shortest-path instance and add an edge that lies on no s-t path, then run SquareCB.Comb's line-5 optimization. The log p̄(a) term for that edge is -∞, so the arg max does not exist and the algorithm cannot execute. Alternatively, if a variant runs after explicitly removing unreachable arms, its regret should be reported on the pruned instance, not the original.

Watch

Extended reading notes

Core claim

The central claim is that for any finite reward function class F, a realizable CCSB instance with A arms, action size at most m, and horizon T can be learned with expected regret O(√(mAT log|F|)) by a computationally tractable algorithm, and no algorithm can do better than Ω̃(√(mAT log|F|)) in the worst case. This matches the best known rates for linear CCSB and for slate-recommendation policy search, while covering arbitrary combinatorial constraints such as paths, slates, and matchings. The paper also shows that the standard reduction from CCSB to contextual bandits suffers an extra factor of m, so SquareCB.Comb's arm-level use of semi-bandit feedback is essential to closing that gap.

Load-bearing premise

The load-bearing premise is that Eq. (2)'s log-barrier objective has a maximizer: there must be at least one feasible action distribution whose marginal probability is positive for every arm—an unstated reachability condition that fails for action families containing an arm no feasible action ever selects.

Editorial extensions

If this is right

  • Achieves O(√(mAT log|F|)) regret while matching lower bound Ω̃(√(mAT log|F|)), so the rate is minimax-optimal up to log factors.
  • Closes the √m gap over the full-bandit reduction, which only gives m√(AT log|F|).
  • Instantiates to linear reward classes with Õ(√(mAdT)) regret, matching known linear CCSB rates.
  • For shortest-path constraints on a DAG, yields Õ(√(NMT log|F|)) regret, improving on a reduction to contextual MDPs.
  • For m=1, recovers the standard contextual-bandit regret √(AT log|F|).

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: the log-barrier step implicitly requires every arm to be reachable—there must be some feasible action distribution with positive marginal probability for every arm. For action families containing arms that never appear in any feasible action, the optimization is undefined, so a practical implementation would need to restrict to reachable sub-arms or add a smoothing term.
  • Editorial extension: the participation-vector and CS-DEC certificate is likely adaptable to other structured observation models, such as probabilistically triggered arms or cascading feedback, wherever a coverage-type quantity can replace the squared-loss term.
  • Editorial extension: the paper conjectures that under independent arms the minimax rate may be strictly lower than Õ(√(mAT log|F|)); if true, the m under the square root is not inherent to all CCSB structures but only to correlated worst cases.
  • Editorial extension: a direct empirical test of the √m separation is to compare SquareCB.Comb against the full-bandit reduction on action sets where the optimal action is rare; the theory predicts the regret gap grows with m.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

1 major / 5 minor

Summary. The paper studies contextual combinatorial semi-bandits (CCSB) with a general finite reward function class F. The proposed algorithm, SquareCB.Comb, maintains a reward estimate from a batch online square-loss regression oracle; at each round it computes a participation vector by maximizing a log-barrier-regularized linear objective over the convex hull of the feasible action set, samples a combinatorial action with matching marginals, and observes semi-bandit feedback. The main theoretical claims are: (i) Theorem 1 gives a regret bound in terms of the oracle's square-loss regret, and Corollary 2 specializes it to O(sqrt(mAT log|F|)) under realizability and bounded action size m; (ii) Theorem 4 provides a matching eOmega(sqrt(mAT log N)) lower bound for a realizable finite-class CCSB instance. The paper also contains an offline-oracle-efficient epoch variant and experiments on learning-to-rank corpora.

Significance. If the advertised guarantees hold as stated, this is a substantial contribution: it would be the first minimax-optimal bound for CCSB with general function approximation, it removes the sqrt(m) gap over the naive full-bandit reduction, and it replaces policy enumeration with regression-oracle calls. The main proof chain—log-barrier certificate, surrogate min-max, CS-DEC, and reduction to batch regression—is coherent, and the lower bound uses a standard interval/product construction. The central caveat is an implicit strict-positivity requirement on the participation vector in Eq. (2); once that is made explicit or repaired by preprocessing, the core result appears defensible.

major comments (1)
  1. [Section 4, Eq. (2); Lemmas 5 and 14] The optimization in Eq. (2) is undefined when no feasible participation vector has all coordinates strictly positive, because the log-barrier term sum_a log pbar(a) is -infinity on the boundary. The paper assumes only Assumption 2 (||s||_1 <= m) and advertises 'no structural assumptions on the action set beyond a cardinality bound,' but for any S containing a single action of size m<A, the only pbar in Sbar has zeros off that action, so Eq. (2) has no feasible maximizer and Algorithm 1 is undefined. The same issue arises in the contextual shortest-path example when the DAG contains an edge on no s-t path. The subsequent analysis relies on strict positivity: Lemma 5 uses the first-order condition with 1/hat-p, and Lemma 14 explicitly assumes pbar(a)>0 for all a. This is a load-bearing gap in the claimed scope. The paper must either add an explicit reachability assumption (e.g., every arm
minor comments (5)
  1. [Section 4.1, Theorem 1] Theorem 1 sets gamma = sqrt(AT/Reg_batch(T)), but Reg_batch(T) is a realized quantity that depends on the algorithm's own data and is not available at run time. The theorem should be phrased as: if the regression oracle satisfies Reg_batch(T) <= R, then with gamma = sqrt(AT/R) the regret is O(sqrt(AT R)). Corollary 2 is explicit and fine, but the statement of Theorem 1 as written is not an implementable algorithmic guarantee.
  2. [Appendix D.1, proof of Lemma 5] The step from <1/hat-p, qbar> <= A/gamma to <1/hat-p, qbar^2> <= A/gamma is correct only because qbar(a) in [0,1], but the '=>' arrow hides this. Please add a one-line justification for clarity.
  3. [Section 5, Theorem 4] The lower bound is stated only for A/m integer and for T divisible by M. For a clean minimax-optimality claim over all parameter values, note that the construction can be extended to non-divisible A by grouping floor(A/m) disjoint paths and treating leftover arms as never selected; this yields the same rate up to constants. Please state this extension explicitly or acknowledge the restriction.
  4. [Section 4.1, extensions paragraph] The misspecification extension says the regret is O(sqrt(mAT log|F|) + epsilon m sqrt(AT)) and then mentions an 'additive epsilon T factor' as unavoidable. These are inconsistent: the displayed additive term is sublinear in T, not epsilon T. Please clarify whether the final bound has an epsilon T term or an epsilon sqrt(T) term.
  5. [Appendix H, lower-bound construction] The constructed environment sets r_t(a)=w_{t,j} for all a in p_j, making arm rewards within a path perfectly correlated. This is allowed by the CCSB model (which fixes only marginals), but it should be stated explicitly so that readers do not assume the Kveton et al. proposition is being invoked under independent-arm assumptions.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the main regret bound is derived from the batch regression oracle and an independent lower bound; one self-citation appears only in a non-central offline-oracle variant.

full rationale

The central derivation is not circular. Theorem 1 is proved from Lemma 8, which in turn follows from Lemmas 5-7: these lemmas show that the log-barrier solution of Eq. (2) certifies a surrogate min-max objective and hence certifies a small CS-DEC value, and the instantaneous regret is then bounded by A/gamma plus a square-loss estimation term. Summing over rounds and applying Assumption 3 gives E[Regret(T)] <= O(sqrt(A T Regbatch(T))); with Assumption 3's batch-size m the oracle regret is m log|F|, yielding Corollary 2's sqrt(m A T log|F|) bound. No parameter is fitted to data in a way that forces the bound, and the target regret does not appear as an input. The lower bound in Theorem 4 is an independent construction built on Kveton et al. (2015) and Agarwal et al. (2012), so it does not assume the upper bound. The only self-citation is Theorem 10, quoted from the authors' own Qin and Zhang (2026) preprint and used only in Section E to analyze the OE2D.Comb offline-oracle variant; it is not needed for the paper's main SquareCB.Comb guarantee or for the minimax lower bound. That self-citation is therefore minor and not load-bearing. I also note that Eq. (2)'s log-barrier regularizer requires pbar(a)>0 for every arm, an unstated reachability condition; this is a correctness/assumption gap rather than a circularity.

Assumptions & free parameters 2 free parameters · 8 assumptions · 0 invented entities

No new physical or ontological entities are introduced. CS-DEC and Coverage are new mathematical definitions used for analysis; they carry no independent empirical predictions of their own.

free parameters (2)
  • gamma (exploration parameter) = sqrt(A T / (m log|F|)) in Corollary 2; gamma_t = gamma0 * sqrt(A t / m) in experiments
    Set by known horizon and class size in the theory; not fitted to reward data. In experiments, the scaling constant gamma0 is tuned on Stage-1 seeds.
  • gamma0 (experiment) = selected from {1,2,5}*10^k, k=-1,0,1, plus 10
    Experimental exploration scaling constant; chosen by maximizing mean realized reward on tuning seeds. It affects the empirical comparison but not the theoretical regret bound.
assumptions (8)
  • domain assumption Realizability (Assumption 1): f* in F.
    Used throughout; the regret bound and the regression-oracle reduction need the true reward function to be in the provided class, and the paper cites Foster-Rakhlin and Lattimore et al. for necessity.
  • domain assumption Bounded action size (Assumption 2): ||s||_1 <= m for every s in S.
    Defines m in the regret rate and is used in Lemma 6 and to bound the batch size of the regression oracle.
  • domain assumption Batch-mode online regression oracle (Assumption 3): Reg_batch(T) <= O(B log|G|).
    The regret theorem is stated in terms of Reg_batch; EWA plus Mesterharm's delayed-feedback reduction is cited to justify it, but it is an assumed oracle input.
  • domain assumption Finite function class F.
    The main theorem and lower bound count |F|; infinite-class extensions are only sketched via sequential Rademacher complexity.
  • ad hoc to paper Reachability/positivity: S-bar contains a participation vector with strictly positive entries for every coordinate.
    Needed for the log-barrier in Eq. (2) to have a solution; not stated in Assumptions 1-2 or in the 'no structural assumptions' claim.
  • domain assumption Exact marginal sampling oracle O_sample: for each pbar in S-bar it returns s in S with E[s] = pbar.
    Algorithm 1 line 6 requires exact marginals; existence follows from the definition of S-bar, but computational efficiency is problem-specific.
  • domain assumption Non-contextual m-path lower bound (Kveton et al. 2015, Prop. 2): Omega(sqrt(Am tau)).
    Used as the building block of the CCSB lower bound in Theorem 4; it is an external theorem, not proved in this paper.
  • standard math EWA square-loss regret and Mesterharm delayed-feedback reduction.
    Used to justify Assumption 3; cited to Cesa-Bianchi-Lugosi, Rakhlin-Sridharan, and Mesterharm.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Optimal and Efficient Contextual Combinatorial Semi-bandits with General Function Approximation." pith.science (2026). https://pith.science/paper/JQ7PXXHC

@misc{pith2026260713686,
  author       = {Pith},
  title        = {Pith review of: Optimal and Efficient Contextual Combinatorial Semi-bandits with General Function Approximation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JQ7PXXHC}},
  note         = {Machine review of arXiv:2607.13686}
}
abstract

We study the contextual combinatorial semi-bandit (CCSB) problem with general reward function approximation. At each round, the learner observes a context, selects a combinatorial action consisting of a subset of basic arms, and receives the reward of each selected arm; the goal is to maximize the cumulative reward over time. We propose SquareCB.Comb, a computationally efficient algorithm that, at each round, solves a convex optimization problem to sample a combinatorial action that balances exploration and exploitation. SquareCB.Comb scales to large arm sets and imposes no structural assumptions on the action set beyond a cardinality bound of $m$ on each combinatorial action. We prove that SquareCB.Comb achieves a minimax optimal regret bound of $O(\sqrt{m A T \log |\mathcal{F}|})$, where $A$ is the number of arms, $m$ is the maximum number of arms in a combinatorial action, $T$ is the time horizon, and $\mathcal{F}$ is the reward function class. In the realizable setting, this bound matches the state-of-the-art regret guarantees achieved by policy search-based algorithms in the more restricted slate recommendation settings, while simultaneously generalizing to arbitrary combinatorial action structures and general reward function approximation.

Figures

Figures reproduced from arXiv: 2607.13686 by the authors.

Figure 1
Figure 1. Per-round average reward on MSLR-WEB30k (left) and Yahoo! LTR Set 1 (right), with each online [PITH_FULL_IMAGE:figures/full_fig_p010_1.png] view at source ↗
Figure 2
Figure 2. Per-round average reward by regression oracle: rows are the two corpora (MSLR-WEB30k, top; [PITH_FULL_IMAGE:figures/full_fig_p027_2.png] view at source ↗
Figure 3
Figure 3. Contextual m-path construction. Each m × (A/m) grid depicts the shared arm set [A], with cell (j, k) being the k-th node of the j-th path. The optimal path is shaded pink and yields expected reward 1/2; the remaining A/m − 1 paths are shaded beige and yield reward 1/2 − ∆. Different contexts induce different optimal paths, so the reward distributions across contexts are independent. For each f ∈ F, define the enviro… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

300 extracted references · 6 canonical work pages

  1. [1]

    Kaufmann, Emilie and Capp

  2. [2]

    1933 , journal =

    On the likelihood that one unknown probability exceeds another in view of the evidence of two samples , author =. 1933 , journal =

    1933

  3. [3]

    , year =

    Thompson, William R. , year =. Biometrika , volume =

  4. [4]

    1936 , journal =

    Design of experiments , author =. 1936 , journal =

    1936

  5. [5]

    Ville, Jean , year =. Bull. Amer. Math. Soc , volume =

  6. [6]

    1952 , journal =

    A Generalization of Sampling Without Replacement from a Finite Universe , author =. 1952 , journal =

    1952

  7. [7]

    Journal of the American statistical Association , publisher =

    Horvitz, Daniel G and Thompson, Donovan J , year =. Journal of the American statistical Association , publisher =

  8. [8]

    Biometrika , publisher =

    Bartlett, Maurice S , year =. Biometrika , publisher =

Show all 300 references
  1. [9]

    Biometrika , publisher =

    Bartlett, M S , year =. Biometrika , publisher =

  2. [10]

    The Bell System Technical Journal , volume =

    Kelly, J L , year =. The Bell System Technical Journal , volume =

  3. [11]

    Biometrics , volume =

    Bechhofer, Robert E , year =. Biometrics , volume =

  4. [12]

    The Annals of Mathematical Statistics , publisher =

    Chernoff, Herman , year =. The Annals of Mathematical Statistics , publisher =

  5. [13]

    Canadian Journal of Mathematics , volume =

    Kiefer, Jack and Wolfowitz, Jacob , year =. Canadian Journal of Mathematics , volume =

  6. [14]

    Journal of the American Statistical Association , volume =

    Hoeffding, Wassily , year =. Journal of the American Statistical Association , volume =

  7. [15]

    Annals of Mathematical Statistics , volume =

    Paulson, Edward , year =. Annals of Mathematical Statistics , volume =

  8. [16]

    Cover, Thomas M , year =

  9. [17]

    The Computer Journal , publisher =

    Nelder, J A and Mead, R , year =. The Computer Journal , publisher =. doi:10.1093/comjnl/7.4.308 , issn =

    doi

  10. [18]

    Journal of the Royal Statistical Society: Series B (Methodological) , publisher =

    Cox, David R and Snell, E Joyce , year =. Journal of the Royal Statistical Society: Series B (Methodological) , publisher =

  11. [19]

    IEEE Trans

    Krichevsky, Raphail E and Trofimov, Victor K , year =. IEEE Trans. Information Theory , volume =

  12. [20]

    McCullagh, Peter and Nelder, John A , year =

  13. [21]

    1985 , journal =

    Asymptotically efficient adaptive allocation rules , author =. 1985 , journal =

    1985

  14. [22]

    Advances in Applied Mathematics , volume =

    Lai, T L and Robbins, Herbert , year =. Advances in Applied Mathematics , volume =

  15. [23]

    Automatic Control, IEEE Transactions on , volume =

    Anantharam, V and Varaiya, P and Walrand, J , year =. Automatic Control, IEEE Transactions on , volume =

  16. [24]

    Angluin, Dana , year =. Mach. Learn. , volume =

  17. [25]

    Agrawal, Rajeev and Teneketzis, Demosthenis and Anantharam, Venkatachalam , year =

  18. [26]

    Proceedings of the Conference on Learning Theory (COLT) , pages =

    Littlestone, Nick , year =. Proceedings of the Conference on Learning Theory (COLT) , pages =

  19. [27]

    McCullagh, P and Nelder, J A , year =

  20. [28]

    1990 , journal =

    Expected-outcome: A general model of static evaluation , author =. 1990 , journal =

    1990

  21. [29]

    IEEE Transactions on Pattern Analysis and Machine Intelligence , publisher =

    Abramson, Bruce , year =. IEEE Transactions on Pattern Analysis and Machine Intelligence , publisher =

  22. [30]

    Advances in Neural Information Processing Systems (NeurIPS) , publisher =

    Atlas, Les E and Cohn, David A and Ladner, Richard E , year =. Advances in Neural Information Processing Systems (NeurIPS) , publisher =

  23. [31]

    NSF-CBMS regional conference series in probability and statistics , pages =

    Pollard, David , year =. NSF-CBMS regional conference series in probability and statistics , pages =

  24. [32]

    Stewart, G. W. and guang Sun, Ji , year =

  25. [33]

    Proceedings of the International Conference on Machine Learning (ICML) , publisher =

    Sutton, Richard S , year =. Proceedings of the International Conference on Machine Learning (ICML) , publisher =

  26. [34]

    Journal of the Royal Statistical Society: Series B (Methodological) , publisher =

    Cordeiro, Gauss M and McCullagh, Peter , year =. Journal of the Royal Statistical Society: Series B (Methodological) , publisher =

  27. [35]

    Discrete

    Betke, Ulrich and Henk, Martin , year =. Discrete

  28. [36]

    1993 , journal =

    Improving generalization for temporal difference learning: The successor representation , author =. 1993 , journal =

    1993

  29. [37]

    Biometrika , publisher =

    Firth, David , year =. Biometrika , publisher =

  30. [38]

    The Annals of Statistics , publisher =

    Foster, Dean P and George, Edward I , year =. The Annals of Statistics , publisher =

  31. [39]

    1994 , month =

    Advances in Neural Information Processing Systems (NeurIPS) , publisher =. 1994 , month =

    1994

  32. [40]

    1994 , journal =

    Some alphabetic optimal designs for the logistic regression model , author =. 1994 , journal =

    1994

  33. [41]

    SIAM journal on control and optimization , publisher =

    Agrawal, Rajeev , year =. SIAM journal on control and optimization , publisher =

  34. [42]

    Advances in Applied Probability , publisher =

    Agrawal, Rajeev , year =. Advances in Applied Probability , publisher =

  35. [43]

    Journal of the American Statistical Association , publisher =

    Robins, James M and Rotnitzky, Andrea , year =. Journal of the American Statistical Association , publisher =

  36. [44]

    Advances in Applied Mathematics , publisher =

    Burnetas, Apostolos N and Katehakis, Michael N , year =. Advances in Applied Mathematics , publisher =

  37. [45]

    Journal of artificial intelligence research , volume =

    Kaelbling, Leslie Pack and Littman, Michael L and Moore, Andrew W , year =. Journal of artificial intelligence research , volume =

  38. [46]

    Mathematics of Operations Research , publisher =

    Khachiyan, Leonid G , year =. Mathematics of Operations Research , publisher =

  39. [47]

    Journal of the Royal Statistical Society: Series B (Methodological) , publisher =

    Tibshirani, Robert , year =. Journal of the Royal Statistical Society: Series B (Methodological) , publisher =

  40. [48]

    Machine Learning , volume =

    Blum, Avrim , year =. Machine Learning , volume =

  41. [49]

    Proceedings of the ACM symposium on Theory of computing (STOC) , volume =

    Freund, Yoav and Schapire, Robert E and Singer, Yoram and Warmuth, Manfred K , year =. Proceedings of the ACM symposium on Theory of computing (STOC) , volume =. doi:10.1145/258533.258616 , isbn =

  42. [50]

    Graves, Todd L and Lai, Tze Leung , year =. SIAM J. Control Optim. , publisher =

  43. [51]

    Artificial Intelligence Review , volume =

    Maron, Oded and Moore, Andrew W , year =. Artificial Intelligence Review , volume =

  44. [52]

    Herbster, Mark and Warmuth, Manfred K , year =. Mach. Learn. , volume =

  45. [53]

    IEEE Transactions on Circuits and Systems for Video Technology , volume =

    Rui, Yong and Huang, T S and Ortega, M and Mehrotra, S , year =. IEEE Transactions on Circuits and Systems for Video Technology , volume =

  46. [54]

    1998 , booktitle =

    Learning agents for uncertain environments , author =. 1998 , booktitle =

    1998

  47. [55]

    1998 , publisher =

    Reinforcement learning: An introduction , author =. 1998 , publisher =

    1998

  48. [56]

    1999 , booktitle =

    Associative reinforcement learning using linear probabilistic concepts , author =. 1999 , booktitle =

    1999

  49. [57]

    , year =

    Abe, Naoki and Long, Philip M. , year =. Proceedings of the International Conference on Machine Learning (ICML) , pages =

  50. [58]

    1999 , journal =

    Design of experiments for locally weighted regression , author =. 1999 , journal =

    1999

  51. [59]

    and Gallagher, Hanna Alix , year =

    Golde, Chris M. and Gallagher, Hanna Alix , year =. Ecosystems , doi =

  52. [60]

    The Annals of Statistics , publisher =

    Mammen, Enno and Tsybakov, Alexandre B and Others , year =. The Annals of Statistics , publisher =

  53. [61]

    1999 , booktitle =

    Policy invariance under reward transformations: Theory and application to reward shaping , author =. 1999 , booktitle =

    1999

  54. [62]

    , year =

    Vovk, V. , year =. Machine Learning , volume =. doi:10.1023/A:1007595032382 , isbn =

    doi

  55. [63]

    Duda, R O and Hart, P E and Stork, D G , year =

  56. [64]

    , author =

    Algorithms for inverse reinforcement learning. , author =. 2000 , booktitle =

    2000

  57. [65]

    Proceedings of The International Conference on Machine Learning (ICML) , pages =

    Precup, Doina , year =. Proceedings of The International Conference on Machine Learning (ICML) , pages =

  58. [66]

    Proceedings of the International Conference on Image Processing (ICIP) , pages =

    Chen, Yunqiang and Zhou, Xiang Sean and Huang, Thomas S , year =. Proceedings of the International Conference on Image Processing (ICIP) , pages =

  59. [67]

    Warmuth, Manfred K. and R. 2001 , booktitle =

    2001

  60. [68]

    Journal of Machine Learning Research , volume =

    Auer, Peter , year =. Journal of Machine Learning Research , volume =

  61. [69]

    2002 , journal =

    Finite-time analysis of the multiarmed bandit problem , author =. 2002 , journal =

    2002

  62. [70]

    2002 , journal =

    The nonstochastic multiarmed bandit problem , author =. 2002 , journal =

    2002

  63. [71]

    Proceedings of the Conference on Learning Theory (COLT) , pages =

    Even-dar, Eyal and Mannor, Shie and Mansour, Yishay , year =. Proceedings of the Conference on Learning Theory (COLT) , pages =

  64. [72]

    2002 , booktitle =

    PAC bounds for multi-armed bandit and Markov decision processes , author =. 2002 , booktitle =

    2002

  65. [73]

    Statistics in medicine , publisher =

    Heinze, Georg and Schemper, Michael , year =. Statistics in medicine , publisher =

  66. [74]

    Machine learning , volume =

    Kearns, Michael and Mansour, Yishay and Ng, Andrew Y , year =. Machine learning , volume =

  67. [75]

    2002 , journal =

    A sparse sampling algorithm for near-optimal planning in large Markov decision processes , author =. 2002 , journal =

    2002

  68. [76]

    2002 , publisher =

    Matou. 2002 , publisher =

    2002

  69. [77]

    British Journal of Developmental Psychology , volume =

    Pellegrini, A D and Long, Jeffrey D , year =. British Journal of Developmental Psychology , volume =

  70. [78]

    Warmuth, Manfred K. and R. 2002 , booktitle =

    2002

  71. [79]

    IEEE Transactions on Information Theory , volume =

    Weinberger, Marcelo J and Ordentlich, Erik , year =. IEEE Transactions on Information Theory , volume =

  72. [80]

    Auer, Peter and Cesa-Bianchi, Nicol. SIAM J. Comput. , publisher =. 2003 , month =. doi:10.1137/S0097539701398375 , issn =

    2003 doi

  73. [81]

    Mathematical Programming, Series B , doi =

    Burer, Samuel and Monteiro, Renato D C , year =. Mathematical Programming, Series B , doi =

  74. [82]

    Multimedia systems , publisher =

    Zhou, Xiang Sean and Huang, Thomas S , year =. Multimedia systems , publisher =

  75. [83]

    ICML workshop on The Continuum from Labeled to Unlabeled Data in Machine Learning and Data Mining , pages =

    Zhu, Xiaojin and Lafferty, John and Ghahramani, Zoubin , year =. ICML workshop on The Continuum from Labeled to Unlabeled Data in Machine Learning and Data Mining , pages =

  76. [84]

    Proceedings of the International Conference on Machine Learning (ICML) , pages =

    Zhu, Xiaojin and Ghahramani, Zoubin and Lafferty, John , year =. Proceedings of the International Conference on Machine Learning (ICML) , pages =

  77. [85]

    Machine Learning , volume =

    Zinkevich, Martin , year =. Machine Learning , volume =. doi:10.1.1.10.9960 , isbn =

  78. [86]

    Proceedings of the ACM Symposium on Theory of Computing (STOC) , pages =

    Awerbuch, Baruch and Kleinberg, Robert D , year =. Proceedings of the ACM Symposium on Theory of Computing (STOC) , pages =

  79. [87]

    Proceedings of the Twentieth Annual Symposium on Computational Geometry , pages =

    Datar, Mayur and Immorlica, Nicole and Indyk, Piotr and Mirrokni, Vahab S , year =. Proceedings of the Twentieth Annual Symposium on Computational Geometry , pages =

  80. [88]

    The Annals of Statistics , publisher =

    Tsybakov, Alexander B , year =. The Annals of Statistics , publisher =

  81. [89]

    and Bousquet, Olivier and Mendelson, Shahar , year =

    Bartlett, Peter L. and Bousquet, Olivier and Mendelson, Shahar , year =

  82. [90]

    Social Choice and Welfare , publisher =

    Erlander, Sven , year =. Social Choice and Welfare , publisher =

  83. [91]

    Advances in Neural Information Processing Systems (NeurIPS) , pages =

    Kleinberg, Robert D , year =. Advances in Neural Information Processing Systems (NeurIPS) , pages =

  84. [92]

    Journal of Optimization Theory and applications , publisher =

    Kumar, Piyush and Yildirim, E Alper , year =. Journal of Optimization Theory and applications , publisher =

  85. [93]

    2005 , booktitle =

    Learning structured prediction models: A large margin approach , author =. 2005 , booktitle =

    2005

  86. [94]

    Zhu, Xiaojin , year =

  87. [95]

    2006 , booktitle =

    Allenberg, Chamy and Auer, Peter and Gy. 2006 , booktitle =

    2006

  88. [96]

    2006 , journal =

    , author =. 2006 , journal =

    2006

  89. [97]

    Cesa-Bianchi, Nicolo and Lugosi, Gabor , year =

  90. [98]

    Journal of Machine Learning Research , volume =

    Even-Dar, Eyal and Mannor, Shie and Mansour, Yishay , year =. Journal of Machine Learning Research , volume =

  91. [99]

    2006 , booktitle =

    Kocsis, Levente and Szepesv. 2006 , booktitle =

    2006

  92. [100]

    2006 , booktitle =

    Bandit based monte-carlo planning , author =. 2006 , booktitle =

    2006

  93. [101]

    2006 , publisher =

    Theory of point estimation , author =. 2006 , publisher =

    2006

  94. [102]

    2006 , publisher =

    Optimal design of experiments , author =. 2006 , publisher =

    2006

  95. [103]

    2006 , booktitle =

    Maximum margin planning , author =. 2006 , booktitle =

    2006

  96. [104]

    Michigan State Universiy , volume =

    Yang, Liu and Jin, Rong , year =. Michigan State Universiy , volume =

  97. [105]

    RPI Comp Sci Dept TR , pages =

    Civril, Ali and Magdon-Ismail, Malik , year =. RPI Comp Sci Dept TR , pages =

  98. [106]

    2007 , booktitle =

    Coquelin, Pierre-Arnaud and Munos, R. 2007 , booktitle =

    2007

  99. [107]

    2007 , journal =

    Gy. 2007 , journal =

    2007

  100. [108]

    2007 , booktitle =

    A bound on the label complexity of agnostic active learning , author =. 2007 , booktitle =

    2007

  101. [109]

    IBM Research Report , volume =

    Hazan, Elad and Seshadhri, Comandur , year =. IBM Research Report , volume =

  102. [110]

    Hazan, Elad and Agarwal, Amit and Kale, Satyen , year =. Mach. Learn. , volume =

  103. [111]

    2007 , journal =

    The epoch-greedy algorithm for multi-armed bandits with side information , author =. 2007 , journal =

    2007

  104. [112]

    2007 , journal =

    The epoch-greedy algorithm for contextual multi-armed bandits , author =. 2007 , journal =

    2007

  105. [113]

    Pattern recognition , publisher =

    Liu, Ying and Zhang, Dengsheng and Lu, Guojun and Ma, Wei-Ying , year =. Pattern recognition , publisher =

  106. [114]

    , author =

    Bayesian Inverse Reinforcement Learning. , author =. 2007 , booktitle =

    2007

  107. [115]

    2007 , journal =

    A game-theoretic approach to apprenticeship learning , author =. 2007 , journal =

    2007

  108. [116]

    Proceedings of the Conference on Uncertainty in Artificial Intelligence (UAI) , publisher =

    Yang, Liu and Jin, Rong and Sukthankar, Rahul , year =. Proceedings of the Conference on Uncertainty in Artificial Intelligence (UAI) , publisher =

  109. [117]

    Abernethy, Jacob and Bartlett, Peter and Rakhlin, Alexander and Tewari, Ambuj , year =

  110. [118]

    2008 , journal =

    Near-optimal regret bounds for reinforcement learning , author =. 2008 , journal =

    2008

  111. [119]

    2008 , journal =

    Progressive strategies for Monte-Carlo tree search , author =. 2008 , journal =

    2008

  112. [120]

    Proceedings of the 39th ACM Technical Symposium on Computer Science Education (SIGCSE) , doi =

    Dahlberg, Teresa and Barnes, Tiffany and Rorrer, Audrey and Powell, Eve and Cairco, Lauren , year =. Proceedings of the 39th ACM Technical Symposium on Computer Science Education (SIGCSE) , doi =

  113. [121]

    Proceedings of the Conference on Learning Theory (COLT) , pages =

    Dani, Varsha and Hayes, Thomas P and Kakade, Sham M , year =. Proceedings of the Conference on Learning Theory (COLT) , pages =

  114. [122]

    ACM Computing Surveys (Csur) , publisher =

    Datta, Ritendra and Joshi, Dhiraj and Li, Jia and Wang, James Z , year =. ACM Computing Surveys (Csur) , publisher =

  115. [123]

    Nature , publisher =

    Hao, Linhui and Sakurai, Akira and Watanabe, Tokiko and Sorensen, Ericka and Nidom, Chairul A and Newton, Michael A and Ahlquist, Paul and Kawaoka, Yoshihiro , year =. Nature , publisher =

  116. [124]

    IEEE INFOCOM 2008 - The 27th Conference on Computer Communications , pages =

    Huang, Senhua and Liu, Xin and Ding, Zhi , year =. IEEE INFOCOM 2008 - The 27th Conference on Computer Communications , pages =

    2008

  117. [125]

    Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR) , pages =

    Jain, P and Kulis, B and Grauman, K , year =. Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR) , pages =

  118. [126]

    2008 , booktitle =

    J. 2008 , booktitle =

    2008

  119. [127]

    Proceedings of the fortieth annual ACM symposium on Theory of computing , pages =

    Kleinberg, Robert and Slivkins, Aleksandrs and Upfal, Eli , year =. Proceedings of the fortieth annual ACM symposium on Theory of computing , pages =

  120. [128]

    2008 , booktitle =

    Multi-armed bandits in metric spaces , author =. 2008 , booktitle =

    2008

  121. [129]

    and King, Irwin , year =

    Ma, Hao and Yang, Haixuan and Lyu, Michael R. and King, Irwin , year =

  122. [130]

    2008 , booktitle =

    Mnih, Volodymyr and Szepesv. 2008 , booktitle =

    2008

  123. [131]

    2008 , journal =

    An analysis of model-based Interval Estimation for Markov Decision Processes , author =. 2008 , journal =. doi:https://doi.org/10.1016/j.jcss.2007.08.009 , issn =

    2008 doi

  124. [132]

    2008 , booktitle =

    Apprenticeship learning using linear programming , author =. 2008 , booktitle =

    2008

  125. [133]

    Wainwright, Martin J and Jordan, Michael I , year =. Found. Trends Mach. Learn. , volume =

  126. [134]

    , author =

    Maximum entropy inverse reinforcement learning. , author =. 2008 , booktitle =

    2008

  127. [135]

    2009 , journal =

    Exploration--exploitation tradeoff using variance estimates in multi-armed bandits , author =. 2009 , journal =

    2009

  128. [136]

    , author =

    Minimax Policies for Adversarial and Stochastic Bandits. , author =. 2009 , booktitle =

    2009

  129. [137]

    2009 , booktitle =

    Abbasi-Yadkori, Yasin and Antos, Andr. 2009 , booktitle =

    2009

  130. [138]

    2009 , booktitle =

    Bubeck, S. 2009 , booktitle =

    2009

  131. [139]

    Frazier, Peter I , year =

  132. [140]

    Advances in Neural Information Processing Systems (NeurIPS) , pages =

    Guillory, Andrew and Bilmes, Jeff A , year =. Advances in Neural Information Processing Systems (NeurIPS) , pages =

  133. [141]

    , year =

    Hazan, Elad and Seshadri, C. , year =. Proceedings of the International Conference on Machine Learning (ICML) , doi =

  134. [142]

    Proceedings of the International Conference on Machine Learning (ICML) , pages =

    Heidrich-Meisner, V and Igel, Christian , year =. Proceedings of the International Conference on Machine Learning (ICML) , pages =

  135. [143]

    Kalai, Adam Tauman and Sastry, Ravi , year =

  136. [144]

    Foundations and Trends

    Kannan, Ravindran and Vempala, Santosh and Others , year =. Foundations and Trends

  137. [145]

    2009 , journal =

    Estimation of the warfarin dose with clinical and pharmacogenetic data (vol 360, pg 753, 2009) , author =. 2009 , journal =

    2009

  138. [146]

    Computer , publisher =

    Koren, Yehuda and Bell, Robert and Volinsky, Chris , year =. Computer , publisher =

  139. [147]

    IEEE Transactions on Pattern Analysis and Machine Intelligence , volume =

    Kulis, Brian and Jain, Prateek and Grauman, Kristen , year =. IEEE Transactions on Pattern Analysis and Machine Intelligence , volume =. doi:10.1109/TPAMI.2009.151 , isbn =

    2009 doi

  140. [148]

    Proceedings of the IEEE International Conference on Computer Vision (ICCV) , pages =

    Kulis, Brian and Grauman, Kristen , year =. Proceedings of the IEEE International Conference on Computer Vision (ICCV) , pages =

  141. [149]

    McMahan, HB and Streeter, MJ , year =

  142. [150]

    Colt , url =

    McMahan, HB and Streeter, MJ , year =. Colt , url =

  143. [151]

    IEEE Transactions on Automatic Control , publisher =

    Mersereau, Adam J and Rusmevichientong, Paat and Tsitsiklis, John N , year =. IEEE Transactions on Automatic Control , publisher =

  144. [152]

    2009 , journal =

    Unifying the named natural exponential families and their relatives , author =. 2009 , journal =

    2009

  145. [153]

    Niculescu-Mizil, Alexandru , year =

  146. [154]

    2009 , school =

    Learning to Search , author =. 2009 , school =

    2009

  147. [155]

    2009 , publisher =

    Variational analysis , author =. 2009 , publisher =

    2009

  148. [156]

    Advances in artificial intelligence , publisher =

    Su, Xiaoyuan and Khoshgoftaar, Taghi M , year =. Advances in artificial intelligence , publisher =

  149. [157]

    Proceedings of the Twentieth Annual ACM-SIAM Symposium on Discrete Algorithms , pages =

    Tropp, Joel A , year =. Proceedings of the Twentieth Annual ACM-SIAM Symposium on Discrete Algorithms , pages =

  150. [158]

    2009 , booktitle =

    Walsh, Thomas J and Szita, Istv. 2009 , booktitle =

    2009

  151. [159]

    Proceedings of the International Conference on Machine Learning (ICML) , pages =

    Yue, Yisong and Joachims, Thorsten , year =. Proceedings of the International Conference on Machine Learning (ICML) , pages =

  152. [160]

    2010 , journal =

    Antos, Andr. 2010 , journal =

    2010

  153. [161]

    2010 , booktitle =

    Audibert, Jean-Yves and Bubeck, S. 2010 , booktitle =

    2010

  154. [162]

    2010 , booktitle =

    Best arm identification in multi-armed bandits , author =. 2010 , booktitle =

    2010

  155. [163]

    2010 , booktitle =

    Bubeck, S. 2010 , booktitle =

    2010

  156. [164]

    Chernov, A and Vovk, V , year =. Proc. of the Conference on Uncertainty in Artificial Intelligence (UAI) , pages =

  157. [165]

    Dekel, Ofer and Gentile, Claudio and Sridharan, Karthik , year =

  158. [166]

    2010 , booktitle =

    Filippi, Sarah and Cappe, Olivier and Garivier, Aur. 2010 , booktitle =

    2010

  159. [167]

    2010 , booktitle =

    Parametric bandits: The generalized linear case , author =. 2010 , booktitle =

    2010

  160. [168]

    Machine Learning , volume =

    Hazan, Elad and Kale, Satyen , year =. Machine Learning , volume =

  161. [169]

    COLT , pages =

    Honda, Junya and Takemura, Akimichi , year =. COLT , pages =

  162. [170]

    Honda, Junya and Takemura, Akimichi , year =. An. Annual

  163. [171]

    Advances in Neural Information Processing Systems (NeurIPS) , pages =

    Jain, Prateek and Vijayanarasimhan, Sudheendra and Grauman, Kristen , year =. Advances in Neural Information Processing Systems (NeurIPS) , pages =

  164. [172]

    Proceedings of the International Conference on Machine Learning (ICML) , pages =

    Kalyanakrishnan, Shivaram and Stone, Peter , year =. Proceedings of the International Conference on Machine Learning (ICML) , pages =

  165. [173]

    Keshavan, Raghunandan H and Montanari, Andrea and Oh, Sewoong , year =. J. Mach. Learn. Res. , volume =

  166. [174]

    2010 , booktitle =

    A contextual-bandit approach to personalized news article recommendation , author =. 2010 , booktitle =

    2010

  167. [175]

    2010 , booktitle =

    Contextual multi-armed bandits , author =. 2010 , booktitle =

    2010

  168. [176]

    Mohan, Karthik and Fazel, Maryam , year =

  169. [177]

    SIAM Rev

    Recht, Benjamin and Fazel, Maryam and Parrilo, Pablo A , year =. SIAM Rev. , publisher =

  170. [178]

    Rusmevichientong, Paat and Tsitsiklis, John N , year =. Math. Oper. Res. , volume =

  171. [179]

    and Seeger, Matthias , year =

    Srinivas, Niranjan and Krause, Andreas and Kakade, Sham M. and Seeger, Matthias , year =. Proceedings of the International Conference on Machine Learning (ICML) , pages =

  172. [180]

    and Gibson, B.R

    Zhu, X. and Gibson, B.R. and Jun, K.-S. and Rogers, T.T. and Harrison, J. and Kalish, C. , year =. Proceedings of the International Conference on Machine Learning (ICML) , isbn =

  173. [181]

    Advances in Neural Information Processing Systems (NeurIPS) , pages =

    Agarwal, Alekh and Foster, Dean P and Hsu, Daniel J and Kakade, Sham M and Rakhlin, Alexander , year =. Advances in Neural Information Processing Systems (NeurIPS) , pages =

  174. [182]

    Advances in Neural Information Processing Systems (NeurIPS) , pages =

    Abbasi-Yadkori, Yasin and Pal, David and Szepesvari, Csaba , year =. Advances in Neural Information Processing Systems (NeurIPS) , pages =

  175. [183]

    Proceedings of the International Conference on Artificial Intelligence and Statistics (AISTATS) , pages =

    Beygelzimer, Alina and Langford, John and Li, Lihong and Reyzin, Lev and Schapire, Robert , year =. Proceedings of the International Conference on Artificial Intelligence and Statistics (AISTATS) , pages =

  176. [184]

    ICGA Journal , volume =

    Browne, Cameron , year =. ICGA Journal , volume =

  177. [185]

    Theoretical Computer Science , publisher =

    Bubeck, S. Theoretical Computer Science , publisher =. 2011 , month =

    2011

  178. [186]

    2011 , journal =

    Bubeck, S. 2011 , journal =

    2011

  179. [187]

    2011 , booktitle =

    Lipschitz bandits without the lipschitz constant , author =. 2011 , booktitle =

    2011

  180. [188]

    2011 , journal =

    An empirical evaluation of thompson sampling , author =. 2011 , journal =

    2011

  181. [189]

    Proceedings of the International Conference on Artificial Intelligence and Statistics (AISTATS) , volume =

    Chu, Wei and Li, Lihong and Reyzin, Lev and Schapire, Robert E , year =. Proceedings of the International Conference on Artificial Intelligence and Statistics (AISTATS) , volume =

  182. [190]

    2011 , booktitle =

    Continuous upper confidence trees , author =. 2011 , booktitle =

    2011

  183. [191]

    2011 , booktitle =

    Dud. 2011 , booktitle =

    2011

  184. [192]

    2011 , journal =

    Efficient optimal learning for contextual bandits , author =. 2011 , journal =

    2011

  185. [193]

    2011 , booktitle =

    Garivier, Aur. 2011 , booktitle =

    2011

  186. [194]

    2011 , booktitle =

    The KL-UCB algorithm for bounded stochastic bandits and beyond , author =. 2011 , booktitle =

    2011

  187. [195]

    Garnett, Roman and Krishnamurthy, Yamuna and Wang, Donghan and Schneider, Jeff and Mann, Richard , year =

  188. [196]

    2011 , booktitle =

    Bayesian optimal active search on graphs , author =. 2011 , booktitle =

    2011

  189. [197]

    Artificial Intelligence , publisher =

    Gelly, Sylvain and Silver, David , year =. Artificial Intelligence , publisher =

  190. [198]

    IEEE Transactions on Pattern Analysis and Machine Intelligence , volume =

    He, Xiaofei and Ji, Ming and Zhang, Chiyuan and Bao, Hujun , year =. IEEE Transactions on Pattern Analysis and Machine Intelligence , volume =. doi:10.1109/TPAMI.2011.44 , isbn =

    2011 doi

  191. [199]

    Hofmann, Katja and Whiteson, Shimon and de Rijke, Maarten , year =

  192. [200]

    2011 , journal =

    An asymptotically optimal policy for finite support models in the multiarmed bandit problem , author =. 2011 , journal =

    2011

  193. [201]

    Kalyanakrishnan, Shivaram , year =

  194. [202]

    Proceedings of the ACM international conference on Web search and data mining , pages =

    Li, Lihong and Chu, Wei and Langford, John and Wang, Xuanhui , year =. Proceedings of the ACM international conference on Web search and data mining , pages =

  195. [203]

    APPRENTISSAGE S

    Maillard, Odalric-Ambrym , year =. APPRENTISSAGE S

  196. [204]

    2011 , booktitle =

    Orabona, Francesco and Cesa-Bianchi, Nicol. 2011 , booktitle =

    2011

  197. [205]

    Annals of Mathematics and Artificial Intelligence , volume =

    Rosin, Christopher D , year =. Annals of Mathematics and Artificial Intelligence , volume =

  198. [206]

    2011 , journal =

    Multi-armed bandits with episode context , author =. 2011 , journal =

    2011

  199. [207]

    2011 , booktitle =

    Contextual bandits with similarity information , author =. 2011 , booktitle =

    2011

  200. [208]

    Advances in Neural Information Processing Systems (NeurIPS) , pages =

    Srebro, Nati and Sridharan, Karthik and Tewari, Ambuj , year =. Advances in Neural Information Processing Systems (NeurIPS) , pages =

  201. [209]

    Proceedings of the 28th International Conference on International Conference on Machine Learning , pages =

    Yu, Jia Yuan and Mannor, Shie , year =. Proceedings of the 28th International Conference on International Conference on Machine Learning , pages =

  202. [210]

    Proceedings of the International Conference on Algorithmic Learning Theory (ALT) , pages =

    Adamskiy, Dmitry and Koolen, Wouter M and Chernov, Alexey and Vovk, Vladimir , year =. Proceedings of the International Conference on Algorithmic Learning Theory (ALT) , pages =

  203. [211]

    Artificial Intelligence and Statistics , pages =

    Agarwal, Alekh and Dud ' ik, Miroslav and Kale, Satyen and Langford, John and Schapire, Robert , year =. Artificial Intelligence and Statistics , pages =

  204. [212]

    IEEE Transactions on Information Theory , volume =

    Agarwal, Alekh and Bartlett, Peter L and Ravikumar, Pradeep and Wainwright, Martin J , year =. IEEE Transactions on Information Theory , volume =

  205. [213]

    Proceedings of the Conference on Learning Theory (COLT) , volume =

    Agrawal, Shipra and Goyal, Navin , year =. Proceedings of the Conference on Learning Theory (COLT) , volume =

  206. [214]

    CoRR , volume =

    Agrawal, Shipra and Goyal, Navin , year =. CoRR , volume =

  207. [215]

    Abbasi-Yadkori, Yasin and Pal, David and Szepesvari, Csaba , year =

  208. [216]

    2012 , journal =

    REGAL: A regularization based algorithm for reinforcement learning in weakly communicating MDPs , author =. 2012 , journal =

    2012

  209. [217]

    2012 , publisher =

    Dynamic programming and optimal control: Volume I , author =. 2012 , publisher =

    2012

  210. [218]

    IEEE Transactions on Computational Intelligence and AI in games , volume =

    Browne, Cameron B and Powley, Edward and Whitehouse, Daniel and Lucas, Simon M and Cowling, Peter I and Rohlfshagen, Philipp and Tavener, Stephen and Perez, Diego and Samothrakis, Spyridon and Colton, Simon , year =. IEEE Transactions on Computational Intelligence and AI in ga...

  211. [219]

    2012 , journal =

    A survey of monte carlo tree search methods , author =. 2012 , journal =

    2012

  212. [220]

    2012 , journal =

    Bubeck, S. 2012 , journal =

    2012

  213. [221]

    2012 , booktitle =

    Busoniu, Lucian and Munos, R. 2012 , booktitle =

    2012

  214. [222]

    2012 , journal =

    Cesa-Bianchi, Nicolo and Lugosi, G. 2012 , journal =

    2012

  215. [223]

    2012 , booktitle =

    Cesa-Bianchi, Nicolo and Gaillard, Pierre and Lugosi, G. 2012 , booktitle =

    2012

  216. [224]

    Jmlr , pages =

    Cesa-Bianchi, N and Gaillard, Pierre and Lugosi, G and Stoltz, Gilles , year =. Jmlr , pages =

  217. [225]

    Chapelle, Olivier and Li, Lihong , year =

  218. [226]

    2012 , journal =

    Further Optimal Regret Bounds for Thompson Sampling , author =. 2012 , journal =. 1209.3353 , timestamp =

    2012 arXiv

  219. [227]

    Journal of Machine Learning Research , volume =

    Dekel, Ofer and Gentile, Claudio and Sridharan, Karthik , year =. Journal of Machine Learning Research , volume =

  220. [228]

    Annual Allerton Conference on Communication, Control, and Computing, Allerton , pages =

    Deshpande, Yash and Montanari, Andrea , year =. Annual Allerton Conference on Communication, Control, and Computing, Allerton , pages =. doi:10.1109/Allerton.2012.6483433 , isbn =

    2012

  221. [229]

    Engineering, Electronic and Szepesv, U K Csaba and Science, Computing , year =

  222. [230]

    Advances in Neural Information Processing Systems (NeurIPS) , pages =

    Gabillon, Victor and Ghavamzadeh, Mohammad and Lazaric, Alessandro , year =. Advances in Neural Information Processing Systems (NeurIPS) , pages =

  223. [231]

    Garnett, Roman and Krishnamurthy, Yamuna and Xiong, Xuehan and Schneider, Jeff and Mann, Richard , year =

  224. [232]

    2012 , journal =

    Bayesian optimal active search and surveying , author =. 2012 , journal =

    2012

  225. [233]

    2012 , booktitle =

    Are we ready for autonomous driving? the kitti vision benchmark suite , author =. 2012 , booktitle =

    2012

  226. [234]

    Advances in Neural Information Processing Systems (NeurIPS) , publisher =

    Gong, Yunchao and Kumar, Sanjiv and Verma, Vishal and Lazebnik, Svetlana , year =. Advances in Neural Information Processing Systems (NeurIPS) , publisher =

  227. [235]

    Proceedings - IEEE International Conference on Data Mining (ICDM) , pages =

    Gu, Quanquan and Han, Jiawei , year =. Proceedings - IEEE International Conference on Data Mining (ICDM) , pages =. doi:10.1109/ICDM.2012.72 , isbn =

    2012 doi

  228. [236]

    2012 , journal =

    Gy. 2012 , journal =

    2012

  229. [237]

    2012 , journal =

    Finite-time regret bound of a bandit algorithm for the semi-bounded support model , author =. 2012 , journal =

    2012

  230. [238]

    Electronic Communications in Probability , publisher =

    Hsu, Daniel and Kakade, Sham and Zhang, Tong and Others , year =. Electronic Communications in Probability , publisher =

  231. [239]

    Theory of Computing , volume =

    Har-Peled, Sariel and Indyk, Piotr and Motwani, Rajeev , year =. Theory of Computing , volume =

  232. [240]

    Proceedings of the International Conference on Artificial Intelligence and Statistics (AISTATS) , pages =

    Ji, Ming and Han, Jiawei , year =. Proceedings of the International Conference on Artificial Intelligence and Statistics (AISTATS) , pages =

  233. [241]

    Proceedings of the International Conference on Machine Learning (ICML) , pages =

    Kalyanakrishnan, Shivaram and Tewari, Ambuj and Auer, Peter and Stone, Peter , year =. Proceedings of the International Conference on Machine Learning (ICML) , pages =

  234. [242]

    2012 , booktitle =

    Kaufmann, Emilie and Korda, Nathaniel and Munos, R. 2012 , booktitle =

    2012

  235. [243]

    2012 , journal =

    Inverse reinforcement learning through structured classification , author =. 2012 , journal =

    2012

  236. [244]

    Proceedings of the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Database (ECML/PKDD) , pages =

    Kunapuli, Gautam and Shavlik, Jude , year =. Proceedings of the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Database (ECML/PKDD) , pages =. doi:10.1007/978-3-642-33460-3_60 , isbn =

    doi

  237. [245]

    Proceedings of the Workshop on On-line Trading of Exploration and Exploitation 2 , volume =

    Li, Lihong and Chu, Wei and Langford, John and Moon, Taesup and Wang, Xuanhui , year =. Proceedings of the Workshop on On-line Trading of Exploration and Exploitation 2 , volume =

  238. [246]

    , year =

    Lyall, Catherine and Meagher, Laura R. , year =. Futures , volume =. doi:10.1016/j.futures.2012.03.011 , issn =

    2012 doi

  239. [247]

    , year =

    Lyall, Catherine and Meagher, Laura R. , year =. Futures , doi =

  240. [248]

    Advances in Neural information processing systems (NeurIPS) , pages =

    Mcmahan, Brendan and Streeter, Matthew , year =. Advances in Neural information processing systems (NeurIPS) , pages =

  241. [249]

    Proceedings of the International Conference on Artificial Intelligence and Statistics (AISTATS) , volume =

    Orabona, Francesco and Cesa-Bianchi, Nicolo and Gentile, Claudio , year =. Proceedings of the International Conference on Artificial Intelligence and Statistics (AISTATS) , volume =

  242. [250]

    IEEE Transactions on Information Theory , publisher =

    Plan, Yaniv and Vershynin, Roman , year =. IEEE Transactions on Information Theory , publisher =

  243. [251]

    Multimedia Tools and Applications , publisher =

    Rotter, Pawel , year =. Multimedia Tools and Applications , publisher =

  244. [252]

    2012 , booktitle =

    Reconstruction from anisotropic random measurements , author =. 2012 , booktitle =

    2012

  245. [253]

    Proceedings of the IEEE , volume =

    Slaney, Malcolm and Lifshits, Yury and He, Junfeng , year =. Proceedings of the IEEE , volume =. doi:10.1109/JPROC.2012.2193849 , issn =

    2012

  246. [254]

    Advances in Neural Information Processing Systems (NeurIPS) , pages =

    Snoek, Jasper and Larochelle, Hugo and Adams, Ryan P , year =. Advances in Neural Information Processing Systems (NeurIPS) , pages =

  247. [255]

    2012 , booktitle =

    Partially observable Markov decision processes , author =. 2012 , booktitle =

    2012

  248. [256]

    Shalev-Shwartz, Shai , year =. Found. Trends Mach. Learn. , volume =

  249. [257]

    International Journal of Multimedia Information Retrieval , publisher =

    Thomee, Bart and Lew, Michael S , year =. International Journal of Multimedia Information Retrieval , publisher =

  250. [258]

    2012 , booktitle =

    Mujoco: A physics engine for model-based control , author =. 2012 , booktitle =

    2012

  251. [259]

    Proceedings of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL HLT) , publisher =

    Xu, Jun-Ming and Jun, Kwang-Sung and Zhu, Xiaojin and Bellmore, Amy , year =. Proceedings of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL HLT) , publisher =

  252. [260]

    and Jun, K.-S

    Xu, J.-M. and Jun, K.-S. and Zhu, X. and Bellmore, A. , year =. North American Chapter of the Association for Computational Linguistics - Human Language Technologies (NAACL HLT) , isbn =

  253. [261]

    Proceedings of the International Conference on Machine Learning (ICML) , pages =

    Yue, Yisong and Hong, Sue Ann Sa and Guestrin, Carlos , year =. Proceedings of the International Conference on Machine Learning (ICML) , pages =

  254. [262]

    Yue, Yisong and Broder, Josef and Kleinberg, Robert and Joachims, Thorsten , year =

  255. [263]

    Proceedings of the International Conference on Machine Learning (ICML) , pages =

    Agrawal, Shipra and Goyal, Navin , year =. Proceedings of the International Conference on Machine Learning (ICML) , pages =

  256. [264]

    Artificial intelligence and statistics , pages =

    Agrawal, Shipra and Goyal, Navin , year =. Artificial intelligence and statistics , pages =

  257. [265]

    2013 , booktitle =

    Continuous upper confidence trees with polynomial exploration--consistency , author =. 2013 , booktitle =

    2013

  258. [266]

    IEEE Trans

    Bengio, Yoshua and Courville, Aaron and Vincent, Pascal , year =. IEEE Trans. Pattern Anal. Mach. Intell. , volume =

  259. [267]

    2013 , booktitle =

    Bubeck, S. 2013 , booktitle =

    2013

  260. [268]

    2013 , journal =

    Kullback-Leibler upper confidence bounds for optimal sequential allocation , author =. 2013 , journal =

    2013

  261. [269]

    2013 , booktitle =

    Cesa-Bianchi, Nicol. 2013 , booktitle =

    2013

  262. [270]

    IEEE Communications Letters , publisher =

    Chatzigeorgiou, I , year =. IEEE Communications Letters , publisher =

  263. [271]

    Crammer, Koby and Gentile, Claudio , year =. Mach. Learn. , volume =

  264. [272]

    2013 , booktitle =

    An optimistic posterior sampling strategy for bayesian reinforcement learning , author =. 2013 , booktitle =

    2013

  265. [273]

    Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD , publisher =

    Gunasekar, Suriya and Acharya, Ayan and Gaur, Neeraj and Ghosh, Joydeep , year =. Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD , publisher =

  266. [274]

    PLoS Computational Biology , publisher =

    Hao, Linhui and He, Quiling and Wang, Zhishi and Craven, Mark and Newton, Michael A and Ahlquist, Paul , year =. PLoS Computational Biology , publisher =

  267. [275]

    Advances in Neural Information Processing Systems (NeurIPS) , publisher =

    Hillel, Eshcar and Karnin, Zohar S and Koren, Tomer and Lempel, Ronny and Somekh, Oren , year =. Advances in Neural Information Processing Systems (NeurIPS) , publisher =

  268. [276]

    Proceedings of the Annual ACM Symposium on Theory of Computing , publisher =

    Jain, Prateek and Netrapalli, Praneeth and Sanghavi, Sujay , year =. Proceedings of the Annual ACM Symposium on Theory of Computing , publisher =. doi:10.1145/2488608.2488693 , isbn =

  269. [277]

    2013 , booktitle =

    Joulani, Pooria and Gy. 2013 , booktitle =

    2013

  270. [278]

    and Zhu, X

    Jun, K.-S. and Zhu, X. and Settles, B. and Rogers, T.T. , year =

  271. [279]

    Proceedings of the International Conference on Machine Learning (ICML) , pages =

    Karnin, Zohar and Koren, Tomer and Somekh, Oren , year =. Proceedings of the International Conference on Machine Learning (ICML) , pages =

  272. [280]

    Proceedings of the Conference on Learning Theory (COLT) , pages =

    Kaufmann, Emilie and Kalyanakrishnan, Shivaram , year =. Proceedings of the Conference on Learning Theory (COLT) , pages =

  273. [281]

    Konyushkova, Ksenia and Glowacka, Dorota , year =

  274. [282]

    Koolen, Wouter M. and. 2013 , journal =. doi:10.1109/TIT.2013.2273353 , issn =

    2013

  275. [283]

    2013 , journal =

    Thompson sampling for 1-dimensional exponential family bandits , author =. 2013 , journal =

    2013

  276. [284]

    Ma, Yifei and Garnett, Roman and Schneider, Jeff , year =

  277. [285]

    Advances in Neural Information Processing Systems (NeurIPS) , pages =

    McMahan, Brendan and Abernethy, Jacob , year =. Advances in Neural Information Processing Systems (NeurIPS) , pages =

  278. [286]

    2013 , journal =

    Nickl, Richard and. 2013 , journal =. doi:10.1214/13-AOS1170 , issn =

    2013 doi

  279. [287]

    and Persson, Kristin A

    Ong, Shyue Ping and Richards, William Davidson and Jain, Anubhav and Hautier, Geoffroy and Kocher, Michael and Cholia, Shreyas and Gunter, Dan and Chevrier, Vincent L. and Persson, Kristin A. and Ceder, Gerbrand , year =. Computational Materials Science , doi =

  280. [288]

    Advances in Neural Information Processing Systems (NeurIPS) , pages =

    Orabona, F , year =. Advances in Neural Information Processing Systems (NeurIPS) , pages =

  281. [289]

    2013 , journal =

    (More) efficient reinforcement learning via posterior sampling , author =. 2013 , journal =

    2013

  282. [290]

    2013 , booktitle =

    Russo, Daniel and. 2013 , booktitle =

    2013

  283. [291]

    2013 , booktitle =

    Urvoy, Tanguy and Clerot, Fabrice and F. 2013 , booktitle =

    2013

  284. [292]

    Asilomar Conference on Signals, Systems and Computers , pages =

    Vakili, S and Zhao, Q , year =. Asilomar Conference on Signals, Systems and Computers , pages =

  285. [293]

    2013 , booktitle =

    Valko, Michal and Korda, Nathan and Munos, R. 2013 , booktitle =

    2013

  286. [294]

    Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD) , pages =

    Wang, Xuezhi and Garnett, Roman and Schneider, Jeff , year =. Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD) , pages =

  287. [295]

    Proceedings of the International Conference on Machine Learning (ICML) , pages =

    Agarwal, Alekh and Hsu, Daniel and Kale, Satyen and Langford, John and Li, Lihong and Schapire, Robert , year =. Proceedings of the International Conference on Machine Learning (ICML) , pages =

  288. [296]

    2014 , booktitle =

    Taming the monster: A fast and simple algorithm for contextual bandits , author =. 2014 , booktitle =

    2014

  289. [297]

    2014 , journal =

    Audibert, Jean-Yves and Bubeck, S. 2014 , journal =

    2014

  290. [298]

    Advances in Neural Information Processing Systems (NeurIPS) , pages =

    Bresler, Guy and Chen, George H and Shah, Devavrat , year =. Advances in Neural Information Processing Systems (NeurIPS) , pages =

  291. [299]

    IEEE Transactions on Computational Intelligence and AI in Games , volume =

    Cazenave, Tristan , year =. IEEE Transactions on Computational Intelligence and AI in Games , volume =

  292. [300]

    2014 , journal =

    Sequential halving applied to trees , author =. 2014 , journal =

    2014

Pith tools

Reviewed August 2, 2026 · model on record in the stance chip above.