Pith. sign in

REVIEW 4 major objections 5 minor 37 references

For large action spaces, off-policy learning succeeds or fails mainly on the optimization landscape of the objective, and simple weighted log-likelihood objectives consistently beat sophisticated value-estimator objectives.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Reward-weighted log-likelihood objectives outperform complex off-policy estimators in large action spaces because their optimization landscapes are much easier to navigate.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection The optimization-vs-estimation claim is plausible and worth airing, but the empirical support is under-specified and the theory is partly deferred. the 4 major comments →

arxiv 2509.03456 v2 pith:7C6GE53M submitted 2025-09-03 stat.ML cs.LG

Off-Policy Learning in Large Action Spaces: Optimization Matters More Than Estimation

Imad Aouali , Otmane Sakhi This is my paper
classification stat.ML cs.LG
keywords offline contextual banditsoff-policy learningoff-policy evaluationlarge action spacesweighted log-likelihoodoptimization landscapesoftmax policies
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that in off-policy learning with very large action spaces, the main obstacle is not statistical estimation error but the optimization landscape of the objective being maximized. It shows that standard off-policy evaluation (OPE) objectives such as IPS, DR, MIPS, OffCEM, and POTEC are non-concave and can trap gradient descent in suboptimal regions, with these pitfalls growing with the number of actions. In contrast, policy-weighted log-likelihood (PWLL) objectives, which reweight the log-likelihood of logged actions, are strongly concave for softmax policies and avoid these traps while still reaching reward-aligned solutions. Empirically, on recommendation datasets with up to one million actions, these simpler objectives are more stable and recover policies that match or beat the OPE-based state of the art. A reader should care because the paper challenges the standard assumption that better value estimation inherently yields better learned policies.

Core claim

The paper's central claim is that the estimator-centric paradigm in off-policy learning breaks down for large action spaces: optimizing a statistically accurate off-policy evaluation estimator is not the most reliable route to a good policy, because the optimization landscape of these objectives becomes severely non-concave as the action set grows. The authors show that simpler weighted log-likelihood objectives, despite being poor value estimators, are strongly concave for L2-regularized linear softmax policies and therefore have a unique global maximum reachable by gradient methods. Their experiments on MovieLens (60K actions), Twitch (200K actions), and GoodReads (1M actions) show that PW

What carries the argument

The central object is the policy-weighted log-likelihood (PWLL) objective, a weighted average of the log-likelihood of each logged action: U^g_n(π) = (1/n) Σ g(r_i, π0(a_i|x_i)) log π(a_i|x_i). The weighting function g selects the algorithm: LPI uses g=r, cLPI uses g=r/max(p0,τ), and RegKL uses g=exp(r/β)-1. Its role is to make learning a surrogate objective that is strongly concave for L2-regularized linear softmax policies (Proposition 3.1), guaranteeing a unique global maximum that gradient methods can find reliably. The paper contrasts this with OPE-based objectives, which are linear in the policy and, when combined with softmax parametrizations, can have exponentially many local maxima

Load-bearing premise

The conclusion that optimization, not estimation, is the bottleneck assumes the OPE baselines were given a fair and neutral hyperparameter sweep; if additional tuning, multiple seeds, or a different policy architecture stabilizes them, the optimization explanation loses its force.

What would settle it

A decisive falsifier would be a controlled sweep: for each dataset, fix a per-method tuning budget of dozens of random trials over batch size, learning-rate schedule, and model size, run multiple seeds, and check whether any OPE-based method, especially POTEC, matches or exceeds cLPI's validation reward. Alternatively, a synthetic bandit with a known globally optimal policy could measure, as K grows, how frequently gradient runs converge to suboptimal local maxima for each objective.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • OPE-based off-policy learning in large action spaces can fail for optimization reasons independent of statistical efficiency; improving estimator accuracy does not automatically translate into better learned policies.
  • Treating the OPE objective as the final learning objective is often the wrong design choice; objectives chosen for a well-behaved, concave landscape, such as PWLL, can match or outperform far more sophisticated estimators.
  • Policy parametrization matters mostly through trainability: lightweight architectures and lower-dimensional embeddings train faster and often reach higher final reward than heavyweight ones.
  • PWLL methods' poor performance as OPE estimators does not hurt their performance as learning objectives, decoupling OPE accuracy from OPL success.
  • The success of cLPI over POTEC suggests that even carefully decomposed, estimator-aware parametrizations cannot fully overcome the optimization barrier inherent to OPE-style objectives.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A direct extension the authors leave implicit is that the same optimization-first lens likely applies to any policy class with a poorly behaved softmax landscape, not only the discrete bandit setting; concave or easily optimizable surrogates may matter more than variance reduction in large-scale recommendation and language-model policy learning.
  • A concrete test beyond the paper's scope would give each OPE method the same tuning budget as the PWLL methods, including random hyperparameter searches and multiple seeds; if any OPE method then closes the gap, the claimed optimization advantage would be an artifact of the comparison protocol.
  • Because the asymptotic PWLL solutions are closed-form reward-weighted distributions, using them as warm starts before fine-tuning an unbiased OPE objective could combine the stable landscape with asymptotic unbiasedness, a combination the paper does not test.
  • Section 2.2 states Propositions 2.1 and 2.2 with proofs deferred to the full version and adapted from prior work; as written, the theoretical content is an adaptation rather than a self-contained proof, so the strongest version of the claim—that the failure is inherent to the OPE objective class—needs the full proof before extending beyond linear estimators.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper contrasts two paradigms for off-policy learning (OPL) in large action spaces: optimizing OPE estimators (IPS, cIPS, DR, MIPS, OffCEM, POTEC) versus optimizing policy-weighted log-likelihood (PWLL) objectives (LPI, cLPI, RegKL, BPR). It claims that OPE-based objectives have poor optimization landscapes that worsen with action-space size, that PWLL objectives are strongly concave for softmax policies, and that empirical comparisons on MovieLens, Twitch, and GoodReads show PWLL methods are more robust and yield better final policies. Theoretical support is given as asymptotic characterizations of the solutions of several estimators and as three propositions whose proofs are deferred. The experiments vary batch size and learning-rate schedule (Fig. 1) and policy capacity (Fig. 3).

Significance. If the central claim holds, it is practically important: it would indicate that estimator-centric OPE/OPL research should be complemented by optimization-aware objective design, and that simple log-likelihood objectives can beat sophisticated estimators at scale. The paper identifies a concrete, testable phenomenon and is clearly written. However, the current evidence is not conclusive. The theoretical propositions are unproved, one asymptotic formula is internally inconsistent, and the empirical benchmark is underspecified (no seeds or error bars, no method-specific tuning, no code). These issues must be resolved before the claim 'optimization matters more than estimation' is established.

major comments (4)
  1. [Section 2.1, DR equation] The displayed asymptotic solution for DR mixes indices. The expression inside the argmax uses π_0(a|x) in the numerator and π_0(a_i|x_i) in the denominator while maximizing over a′; it should involve π_0(a′|x)/max{π_0(a′|x), τ}. As written, the propensity factor is constant with respect to a′, so the argmax reduces to \rhat(x,a′) when the factor is positive, which is not the claimed bias-corrected solution and contradicts the text. This undermines the theoretical analysis of OPE-based objectives and must be corrected and derived.
  2. [Sections 2.2 and 3.1, Propositions 2.1, 2.2, 3.1] These propositions are central to the paper's theoretical argument but are stated without proof, with a note that proofs will appear in a full version and that they are 'adapted from [6,18]'. The reduction is not shown; it is not obvious that the cited results apply verbatim to arbitrary linear-in-π OPE estimators or to the PWLL objective with all considered weightings. Please include the proofs or a precise statement of the provenance and the conditions under which the cited results transfer.
  3. [Section 4.1 and Fig. 1] The experimental comparison is the main empirical support for the central claim, but it is underspecified. Only batch size and LR schedule are varied; the clipping threshold τ, the clustering function and number of clusters for MIPS/OffCEM/POTEC, the reward model for DR/OffCEM/POTEC, the L2 coefficient, the number of epochs, and the embedding dimension are not reported. The figure has no error bars or seeds. Without per-method hyperparameter tuning or a sensitivity analysis, the observed poor performance of OPE methods could be an artifact of undertuning. This is a load-bearing gap.
  4. [Section 4.2 and Fig. 3] The claim that lightweight policy parametrizations converge faster and often yield higher final reward is not supported by reported numbers, error bars, or a precise definition of 'heavyweight' vs 'lightweight' (embedding sizes, hidden units). Since this claim is used to reinforce the optimization-centric message, it needs quantitative support.
minor comments (5)
  1. [Sections 2.1 and 3] The notation 1[a = argmax_{a′} ...] is understandable but should be cleaned up; the argmax variable should appear consistently.
  2. [Section 3] BPR is listed as a PWLL method in the experiments but its weighting function g is not defined in the text. Please add a brief description.
  3. [Throughout] There are several typos, e.g., 'This study can be extended to other estimators [2,23,28] but its omitted for conciseness' and 'Conjunct effect model' for OffCEM. Please proofread.
  4. [References] References [16] and [17] are duplicates of the same paper; merge them.
  5. [Figures] Figures 1 and 3 lack error bars and a note on the number of random seeds. At minimum, provide these in a supplementary material or state that only one seed was used.

Circularity Check

0 steps flagged

No significant circularity: the central comparison is empirical on external datasets, and the theoretical propositions are either borrowed from external work or standard math.

full rationale

The paper's central claim—that OPE-based OPL objectives are harder to optimize than PWLL objectives in large action spaces—is supported by (i) Propositions 2.1–2.2, which are explicitly adapted from external works [6,18] and concern a generic class of estimators linear in the policy, not the paper's own definitions; (ii) Proposition 3.1, which asserts strong concavity of L2-regularized log-likelihood for linear softmax policies, a standard mathematical fact; and (iii) an empirical evaluation on public datasets (MovieLens, Twitch, GoodReads) comparing established baselines. The conclusions follow directly from the reported reward comparisons in Figs. 1–3. No parameter is fitted to a subset of data and then used to 'predict' a closely related quantity; no central premise is justified solely by self-citation. Self-references (e.g., [2,28]) appear only as baseline methods or background, not as load-bearing evidence. The omitted proofs for Propositions 2.1–2.2 are a completeness/rigor concern, not a circularity. No step reduces by construction to its own inputs.

Axiom & Free-Parameter Ledger

7 free parameters · 5 axioms · 0 invented entities

The paper's derivations depend chiefly on standard bandit assumptions, the softmax policy class, and borrowed optimization theorems. No new entities are introduced. The main user-chosen or fitted quantities are the clipping threshold, RegKL temperature, reward model, clustering function, and optimization hyperparameters, none of which are fitted to the central conclusion.

free parameters (7)
  • Clipping threshold tau = not reported; user-chosen
    Controls the bias-variance tradeoff in cIPS, cLPI, DR, MIPS, and OffCEM; the asymptotic solutions in Eqs. (2) and (5) depend on it.
  • RegKL temperature beta = not reported; user-chosen
    In Eq. (6), beta interpolates between behavior cloning (beta to infinity) and greedy reward pursuit (beta to 0).
  • L2 regularization coefficient = not reported
    Proposition 3.1 (strong concavity of the PWLL objective) holds for an L2-regularized linear softmax policy, but the coefficient value is not given.
  • Reward model for DR/OffCEM/POTEC = estimated from data
    Double-robust style objectives depend on a learned reward model; its accuracy directly influences the asymptotic policies in Section 2.1.
  • Clustering function and number of clusters = designed by user; not reported
    MIPS, OffCEM, and POTEC require an action clustering function phi; the asymptotic solutions are cluster-dependent.
  • Optimization hyperparameters (batch size, LR schedule, epochs) = grid not fully reported
    Figure 1 shows that the central empirical result is differential sensitivity to these choices; the exact grid and settings are not given.
  • Policy capacity (embedding dimension) = not reported
    Figure 3 compares heavyweight vs lightweight policies without specifying the exact dimensions or architecture details.
axioms (5)
  • domain assumption The contextual bandit tuples (x_i, a_i, r_i) are i.i.d. and the logging policy pi0 is known.
    This is the standard offline bandit setup stated in Section 1; all asymptotic solutions assume it.
  • domain assumption The reward function r(x,a) exists and the infinite-data argmax limits are well-defined.
    The asymptotic solutions in Eqs. (1)-(6) require finite expectations and a well-defined population argmax.
  • domain assumption The policy class is softmax with linear inner-product parametrization.
    Propositions 2.1, 2.2, and 3.1 are stated for this class; generalization to other policy classes is not shown.
  • ad hoc to paper Propositions 2.1 and 2.2, adapted from [6,18], carry over to the OPE estimators considered here.
    The paper states the propositions are adapted and defers proofs; the transfer of conditions to IPS, DR, MIPS, OffCEM, and POTEC is not established.
  • domain assumption The reward model and clustering function used by OPE baselines are reasonable and fairly tuned.
    DR, OffCEM, and POTEC performance depends on these fitted components, but the paper does not describe how they are trained or tuned.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Off-Policy Learning in Large Action Spaces: Optimization Matters More Than Estimation." pith.science (2026). https://pith.science/paper/7C6GE53M

@misc{pith2026250903456,
  author       = {Pith},
  title        = {Pith review of: Off-Policy Learning in Large Action Spaces: Optimization Matters More Than Estimation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7C6GE53M}},
  note         = {Machine review of arXiv:2509.03456}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Off-policy evaluation (OPE) and off-policy learning (OPL) are foundational for decision-making in offline contextual bandits. Recent advances in OPL primarily optimize OPE estimators with improved statistical properties, assuming that better estimators inherently yield superior policies. Although theoretically justified, this estimator-centric approach neglects a critical practical obstacle: challenging optimization landscapes. In this paper, we provide theoretical insights and empirical evidence showing that current OPL methods encounter severe optimization issues, particularly as the action space grows. We show that estimator-aware policy parametrization can mitigate, but not fully resolve, optimization challenges. Building on this, we explore simpler weighted log-likelihood objectives and demonstrate that they enjoy substantially better optimization properties and still recover competitive, often superior, learned policies. Our findings emphasize the necessity of explicitly addressing optimization considerations in the development of OPL algorithms for large action spaces.

Figures

Figures reproduced from arXiv: 2509.03456 by Imad Aouali, Otmane Sakhi.

Figure 2
Figure 2. Figure 2: Average MSE by dataset and method. 4.2 Lightweight Policy Parametrization Helps We also compare lightweight and heavyweight policy parametriza￾tions (e.g., smaller architectures, lower-dimensional embeddings). As shown in [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Training progress over 10 epochs on three datasets, [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figure 1
Figure 1. Figure 1: Effect of batch size and learning rate schedule on [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

37 extracted references · 31 canonical work pages · 4 internal anchors

  1. [1]

    Imad Aouali, Amine Benhalloum, Martin Bompaire, Achraf Ait Sidi Hammou, Sergey Ivanov, Benjamin Heymann, David Rohde, Otmane Sakhi, Flavian Vasile, and Maxime Vono. 2022. Reward optimizing recommendation using deep learning and fast maximum inner product search. In proceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining . 4772–4773

    work page 2022

  2. [2]

    Imad Aouali, Victor-Emmanuel Brunel, David Rohde, and Anna Korba. 2023. Exponential smoothing for off-policy learning. In International Conference on Machine Learning. PMLR, 984–1017

    work page 2023

  3. [3]

    Imad Aouali, Victor-Emmanuel Brunel, David Rohde, and Anna Korba. 2024. Bayesian Off-Policy Evaluation and Learning for Large Action Spaces. arXiv preprint arXiv:2402.14664 (2024)

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  4. [4]

    Imad Aouali, Achraf Ait Sidi Hammou, Sergey Ivanov, Otmane Sakhi, David Rohde, and Flavian Vasile. 2022. Probabilistic Rank and Reward: A Scalable Model for Slate Recommendation. arXiv:2208.06263 [cs.IR]

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  5. [5]

    Léon Bottou, Jonas Peters, Joaquin Quiñonero-Candela, Denis X Charles, D Max Chickering, Elon Portugaly, Dipankar Ray, Patrice Simard, and Ed Snelson. 2013. Counterfactual Reasoning and Learning Systems: The Example of Computational Advertising. Journal of Machine Learning Research 14, 11 (2013)

    work page 2013

  6. [6]

    Minmin Chen, Ramki Gummadi, Chris Harris, and Dale Schuurmans. 2019. Sur- rogate Objectives for Batch Policy Optimization in One-step Decision Making. In Advances in Neural Information Processing Systems , H. Wallach, H. Larochelle, A. Beygelzimer, F. d 'Alché-Buc, E. Fox, and R. Garnett (Eds.), Vol. 32. Cur- ran Associates, Inc. https://proceedings.neuri...

    work page 2019

  7. [7]

    Matej Cief, Jacek Golebiowski, Philipp Schmidt, Ziawasch Abedjan, and Artur Bekasov. 2024. Learning action embeddings for off-policy evaluation. InEuropean Conference on Information Retrieval . Springer, 108–122

    work page 2024

  8. [8]

    Miroslav Dudík, Dumitru Erhan, John Langford, and Lihong Li. 2012. Sample- Efficient Nonstationary Policy Evaluation for Contextual Bandits. In Proceedings of the Twenty-Eighth Conference on Uncertainty in Artificial Intelligence (Catalina Island, CA) (UAI’12). AUAI Press, Arlington, Virginia, USA, 247–254

    work page 2012

  9. [9]

    Miroslav Dudik, Dumitru Erhan, John Langford, and Lihong Li. 2014. Doubly Robust Policy Evaluation and Optimization. Statist. Sci. 29, 4 (2014), 485–511

    work page 2014

  10. [10]

    Miroslav Dudík, John Langford, and Lihong Li. 2011. Doubly robust policy evaluation and learning. International Conference on Machine Learning (2011)

    work page 2011

  11. [11]

    Mehrdad Farajtabar, Yinlam Chow, and Mohammad Ghavamzadeh. 2018. More robust doubly robust off-policy evaluation. InInternational Conference on Machine Learning. PMLR, 1447–1456

    work page 2018

  12. [12]

    Daniel G Horvitz and Donovan J Thompson. 1952. A generalization of sampling without replacement from a finite universe. Journal of the American statistical Association 47, 260 (1952), 663–685

    work page 1952

  13. [13]

    Olivier Jeunen and Bart Goethals. 2021. Pessimistic reward models for off- policy learning in recommendation. InFifteenth ACM Conference on Recommender Systems. 63–74

    work page 2021

  14. [14]

    Ilja Kuzborskij, Claire Vernade, Andras Gyorgy, and Csaba Szepesvári. 2021. Confident off-policy evaluation and selection through self-normalized importance weighting. In International Conference on Artificial Intelligence and Statistics . PMLR, 640–648

    work page 2021

  15. [15]

    Shyong Lam and Jon Herlocker. 2016. MovieLens Dataset. http://grouplens.org/datasets/movielens/

    work page 2016

  16. [17]

    Dawen Liang and Nikos Vlassis. 2023. Local Policy Improvement for Recom- mender Systems. arXiv:2212.11431 [cs.LG] https://arxiv.org/abs/2212.11431

    Pith/arXiv arXiv 2023

  17. [18]

    Jincheng Mei, Chenjun Xiao, Bo Dai, Lihong Li, Csaba Szepesvari, and Dale Schuurmans. 2020. Escaping the Gravitational Pull of Softmax. In Advances in Neural Information Processing Systems , H. Larochelle, M. Ran- zato, R. Hadsell, M.F. Balcan, and H. Lin (Eds.), Vol. 33. Curran Associates, Inc., 21130–21140. https://proceedings.neurips.cc/paper_files/pap...

    work page 2020

  18. [19]

    Alberto Maria Metelli, Alessio Russo, and Marcello Restelli. 2021. Subgaussian and differentiable importance sampling for off-policy evaluation and learning. Advances in Neural Information Processing Systems 34 (2021), 8119–8132

    work page 2021

  19. [20]

    Jie Peng, Hao Zou, Jiashuo Liu, Shaoming Li, Yibao Jiang, Jian Pei, and Peng Cui. 2023. Offline policy evaluation in large action spaces via outcome-oriented action grouping. In Proceedings of the ACM Web Conference 2023 . 1220–1230

    work page 2023

  20. [21]

    2021.Recommendation on Live- Streaming Platforms: Dynamic A vailability and Repeat Consumption

    Jérémie Rappaz, Julian McAuley, and Karl Aberer. 2021.Recommendation on Live- Streaming Platforms: Dynamic A vailability and Repeat Consumption. Association for Computing Machinery, New York, NY, USA, 390–399. https://doi.org/10. 1145/3460231.3474267

    arXiv 2021

  21. [22]

    Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme

  22. [23]

    Noveen Sachdeva, Lequn Wang, Dawen Liang, Nathan Kallus, and Julian McAuley

    work page

  23. [24]

    Yuta Saito and Thorsten Joachims. 2022. Off-Policy Evaluation for Large Action Spaces via Embeddings. arXiv preprint arXiv:2202.06317 (2022)

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  24. [25]

    Yuta Saito, Qingyang Ren, and Thorsten Joachims. 2023. Off-policy evaluation for large action spaces via conjunct effect modeling. In international conference on Machine learning. PMLR, 29734–29759

    work page 2023

  25. [26]

    Yuta Saito, Jihan Yao, and Thorsten Joachims. 2025. POTEC: Off-Policy Contextual Bandits for Large Action Spaces via Policy Decomposition. In The Thirteenth International Conference on Learning Representations . https://openreview.net/ forum?id=LXftdR11io

    work page 2025

  26. [27]

    Otmane Sakhi, Pierre Alquier, and Nicolas Chopin. 2023. PAC-Bayesian Of- fline Contextual Bandits With Guarantees. In Proceedings of the 40th Interna- tional Conference on Machine Learning (Proceedings of Machine Learning Re- search, Vol. 202), Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (Eds.). P...

    work page 2023

  27. [28]

    Otmane Sakhi, Imad Aouali, Pierre Alquier, and Nicolas Chopin. 2024. Logarith- mic Smoothing for Pessimistic Off-Policy Evaluation, Selection and Learning. In Advances in Neural Information Processing Systems , A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang (Eds.), Vol. 37. Curran Associates, Inc., 80706–80755. https://p...

    work page 2024

  28. [29]

    Otmane Sakhi, Stephen Bonner, David Rohde, and Flavian Vasile. 2020. Blob: A probabilistic model for recommendation that combines organic and bandit signals. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining . 783–793

    work page 2020

  29. [30]

    Yi Su, Maria Dimakopoulou, Akshay Krishnamurthy, and Miroslav Dudík. 2020. Doubly robust off-policy evaluation with shrinkage. In International Conference on Machine Learning. PMLR, 9167–9176

    work page 2020

  30. [31]

    Yi Su, Lequn Wang, Michele Santacatterina, and Thorsten Joachims. 2019. Cab: Continuous adaptive blending for policy evaluation and learning. InInternational Conference on Machine Learning . PMLR, 6005–6014

    work page 2019

  31. [32]

    Adith Swaminathan and Thorsten Joachims. 2015. Batch learning from logged bandit feedback through counterfactual risk minimization.The Journal of Machine Learning Research 16, 1 (2015), 1731–1755

    work page 2015

  32. [33]

    Muhammad Faaiz Taufiq, Arnaud Doucet, Rob Cornish, and Jean-Francois Ton

    work page

  33. [34]

    Mengting Wan, Rishabh Misra, Ndapa Nakashole, and Julian J. McAuley. 2019. Fine-Grained Spoiler Detection from Large-Scale Review Corpora. In Proceed- ings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers , Anna Korho- nen, David R. Traum, and Lluís Màrquez (...

    doi:10.18653/v1/p19-1248 2019

  34. [35]

    Yu-Xiang Wang, Alekh Agarwal, and Miroslav Dudık. 2017. Optimal and adaptive off-policy evaluation in contextual bandits. InInternational Conference on Machine Learning. PMLR, 3589–3597

    2017

  35. [2012]

    arXiv preprint arXiv:1205.2618 (2012)

    BPR: Bayesian personalized ranking from implicit feedback. arXiv preprint arXiv:1205.2618 (2012)

    Pith/arXiv arXiv 2012

  36. [2023]

    Off-Policy Evaluation for Large Action Spaces via Policy Convolution

    Off-policy evaluation for large action spaces via policy convolution. arXiv preprint arXiv:2310.15433 (2023)

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  37. [2024]

    Advances in Neural Information Processing Systems 36 (2024)

    Marginal Density Ratio for Off-Policy Evaluation in Contextual Bandits. Advances in Neural Information Processing Systems 36 (2024)

    work page 2024

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.