Off-Policy Evaluation of Ranking Policies via Embedding-Space User Behavior Modeling

Chihiro Maru; Hiroko Shoji; Tatsuki Takahashi

REVIEW 4 major objections 4 minor 16 references

Off-Policy Evaluation of Ranking Policies via Embedding-Space User Behavior Modeling

T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Ranking policies can be evaluated from logged bandit data with far less variance by reweighting over embedding-space user behavior models, and the paper proves that the resulting GMIPS estimator is unbiased under common ranking-embedding…

desk verdict A useful ranking-OPE extension with a load-bearing flaw in the main unbiasedness proof, but the fix is small and the empirical side is solid. read the letter →

arxiv 2506.00446 v1 pith:QODGU4LQ submitted 2025-05-31 stat.ML cs.LG

Tatsuki Takahashi , Chihiro Maru , Hiroko Shoji This is my paper

classification stat.MLcs.LG

keywords off-policyevaluationrankingpoliciesinversepropensityscoreactionembeddingsmarginalizedimportanceweightscascadeuserbehaviorrecommendersystemsbanditfeedback

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Off-policy evaluation of ranking policies asks how well a new recommender would perform using only bandit logs collected by an older policy. The difficulty is that the number of possible rankings grows exponentially in both catalog size and list length, so standard inverse-propensity estimators have enormous variance. This paper proposes to compress the action space by working with ranking embeddings, the observable features of each item in each position, and to reweight over subsets of these embeddings under assumed user behavior models. It introduces the generalized marginalized IPS (GMIPS) family, claims unbiasedness under common support on the embedding space plus a user-behavior assumption, and proves that GMIPS has lower variance than the corresponding action-space estimator. Experiments on synthetic and real-world data show the cascade-on-embeddings variant MRIPS achieving the lowest MSE, even when the assumptions are violated.

What carries the argument

The load-bearing object is the generalized marginal importance weight over a subset of ranking embeddings, $w_{\Phi_k}(x,e) = p(\Phi_k(e) \mid x, \pi) \,/\, p(\Phi_k(e) \mid x, \pi_0)$, where $\Phi_k(e)$ is the part of the embedding vector that the user behavior model says controls the reward at position $k$. This is the ranking analogue of the single-action marginalized IPS weight, and it is what shrinks the effective action space from $\Pi(\mathcal{A})$ to $\Pi(\mathcal{E})$. The companion assumptions are no direct effect of ranking actions on rewards given embeddings (Assumption 3.2) and a user behavior model on ranking embeddings (Assumption 3.3); GMIPS specializes to MSIPS with the full embedding vector, to MIIPS with position-wise independent weights, and to MRIPS with cascade weights, each choice trading bias against variance.

What would settle it

Take a synthetic dataset with two distinct actions that have identical embeddings at every position but different expected rewards at the same position, fit a logging policy that covers the embedding marginals, and run GMIPS on logged data. If the estimate differs from the true policy value by roughly the first term of Theorem 3.8, $\mathbb{E}\left[(w_{\Phi^c_k}^{-1}(x,e)-1)\,q_k(x,a,e)\right]$, then Proposition 3.6's unbiasedness claim cannot hold under Assumptions 3.1 and 3.3 alone.

Watch

Extended reading notes

Core claim

The paper's central claim is that measuring policy divergence on the ranking embedding space instead of the ranking action space removes the dominant source of variance while preserving unbiasedness. Concretely, Proposition 3.6 asserts that for any target policy $\pi$, the position-wise GMIPS estimate is unbiased, $\mathbb{E}_{\mathcal{D}}[\hat{V}_{\mathrm{GMIPS}}^{(k)}(\pi;\mathcal{D})] = V^{(k)}(\pi)$, whenever the logging and target policies have common support on the marginal ranking-embedding distribution (Assumption 3.1) and the reward follows the stated user behavior model on ranking embeddings (Assumption 3.3). Theorem 3.7 then gives a closed-form variance reduction over the action-space GIPS estimator, proportional to the variance of the action-level importance weights and the expected squared reward, so the gain grows with the number of unique actions and the length of the ranking. The paper also derives an explicit bias expression when the no-direct-effect assumption fails, showing that the bias decomposes into misspecification of the behavior model and residual action effects, and it argues that the MRIPS variant balances these two errors best.

Load-bearing premise

The argument silently needs the condition that once the embedding vector is fixed, the exact identities of the chosen items carry no extra information about the reward; without that no-direct-effect condition, the step replacing $q_k(x,a,e)$ by $q_k(x,\Phi_k(e))$ is not justified, even though the unbiasedness proposition is stated under a weaker-looking embedding-support assumption.

Editorial extensions

If this is right

If the unbiasedness claim holds, ranking OPE no longer requires the raw action-level common support condition; logs from policies that never selected certain rankings can still evaluate policies that do, as long as the embedding-level marginal support overlaps.
The variance gap in Theorem 3.7 grows with the variance of action-space importance weights, so GMIPS is most valuable precisely in the large-catalog, long-ranking regimes that break existing estimators.
Because MIIPS coincides with per-position MIPS, the ranking setting inherits existing single-action embedding estimators as a special case.
With a data-driven embedding-dimension selector such as SLOPE, the MRIPS variant keeps low MSE in realistic settings where the no-direct-effect assumption fails, giving a practical prescription for deploying the estimator.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The proof of Proposition 3.6 replaces $q_k(x,a,e)$ by $q_k(x,\Phi_k(e))$ at the point where Assumption 3.3 is invoked; if this replacement is read strictly, it also requires the no-direct-effect condition, which is stronger than the stated Assumption 3.3. A reader testing GMIPS should check whether unbiasedness holds in a simulation where two actions share every embedding but have different expect
The bias decomposition in Theorem 3.8 suggests a practical diagnostic: the gap between MRIPS and MSIPS estimates can be used to learn which embedding dimensions carry reward-relevant information before running a new A/B test.
The same embedding-marginalization scheme can be extended to slate-bandit settings with scalar rewards, since the bottleneck is the same exponential growth of the action space; the paper does not develop this extension.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Two things to know about arXiv:2506.00446. First, the paper has a genuinely new piece: MRIPS, a doubly marginalized estimator for ranking OPE that assumes a cascade model on ranking embeddings, plus a family that nests Saito & Joachims's MIPS. Second, the main theorem that supposedly gives GMIPS unbiasedness under Assumptions 3.1 and 3.3 is wrong as stated. The proof replaces qk(x,a,e) with qk(x,Phi_k(e)) by citing Assumption 3.3, but Assumption 3.3 only equates E[r|x,e] with E[r|x,Phi_k(e)], not the conditional expectation given a. Proposition 3.4 claims A3.3 implies A3.2, which fails for K=1 where Phi_1(e)=e and A3.3 is tautological. A simple counterexample satisfies the stated assumptions and gives biased GMIPS. The same unjustified substitution appears in Theorem 3.7 and Theorem C.1; the latter also needs a second-moment version of A3.3 that is not stated.

What the paper does well: it identifies a real gap—embedding-based OPE has only been done for single actions, not rankings—and the MRIPS estimator is a sensible extension of cascade behavior to embedding spaces. The synthetic and semi-synthetic experiments are thorough, with code, and the SLOPE-based embedding dimension selection is a nice practical touch. The bias–variance decomposition in Theorem 3.8 looks like the right way to think about the trade-off, even if I haven't verified every algebra step.

The soft spot is the one above, and it is load-bearing: without a corrected unbiasedness statement, the paper's central theoretical claim does not hold under the assumptions it lists. The fix is straightforward—add Assumption 3.2 (no direct effect of actions given embeddings) to Proposition 3.6 and Theorem 3.7, and add the second-moment condition for variance claims. The empirical results may still show practical value for MRIPS, but they cannot be justified by the current theory.

This paper is for researchers working on off-policy evaluation in recommender systems, particularly those interested in embedding-based methods. It deserves a serious referee, not a desk reject, because the MRIPS idea and the experiments are worth engaging with, and the theoretical error is repairable. I'd urge the editor to send it to review with a clear request to fix the assumptions.

Referee Report

4 major / 4 minor

Summary. The paper considers off-policy evaluation (OPE) of ranking policies when the ranking action space is large. It augments logged bandit data with ranking embeddings and introduces two structural assumptions: no direct effect of ranking actions on rewards given embeddings (Assumption 3.2) and a user-behavior model on embedding spaces (Assumption 3.3). Under these assumptions it defines the GMIPS estimator and its doubly marginalized variants MIIPS and MRIPS, and claims unbiasedness (Proposition 3.6), variance reduction relative to GIPS (Theorem 3.7), and a bias decomposition (Theorem 3.8). The experimental sections compare the proposed estimators with existing IPS-family estimators on synthetic and semi-synthetic data, including an estimator-selection step via SLOPE.

Significance. If the theoretical claims held, the paper would make a useful contribution to ranking OPE by extending embedding-based marginalization to ranking policies and by proposing a cascade-based doubly marginalized estimator. The paper's strengths include a clearly stated setup, reproducible code, and a broad set of experiments. However, the central unbiasedness theorem is false as stated, and the variance theorems rely on unstated second-moment assumptions; the contribution therefore needs substantial correction before its significance can be assessed.

major comments (4)

[Section 3.3, Proposition 3.6 and Appendix C.1] The proof's step '= ... q_k(x, Phi_k(e)) ... because of Assumption 3.3' incorrectly replaces q_k(x,a,e) with q_k(x,Phi_k(e)). Assumption 3.3 only equates E[r|x,e] and E[r|x,Phi_k(e)]; it says nothing about E[r|x,a,e]. A concrete counterexample is K=1, Phi_1(e)=e, p(e|x,a)=p(e|x), a independent of e, and r=e+gamma*a. Then Assumption 3.1 and 3.3 hold (3.3 is tautological), but E_D[hat V_GMIPS] = E[e] + gamma E_{pi0}[a] while V(pi) = E[e] + gamma E_pi[a], giving bias gamma(E_pi[a]-E_pi0[a]). The fix is to add Assumption 3.2 to Proposition 3.6 (and Theorem 3.7), or to redefine Assumption 3.3 as q_k(x,a,e)=q_k(x,Phi_k(e)) for all a; with either correction the proof can be repaired, but the statement as written fails.
[Section 3.2, Proposition 3.4 and the preceding claim] Proposition 3.4 is false: Assumption 3.3 does not imply Assumption 3.2. The same counterexample as above has Assumption 3.3 holding trivially while the action has a direct effect on the reward through gamma*a. The accompanying sentence that Assumption 3.2 is a necessary and sufficient condition for the validity of Assumption 3.3 is also incorrect: Assumption 3.2 alone does not imply the behavior-model equality E[r|x,e]=E[r|x,Phi_k(e)]. This invalidates the claim that the GMIPS has only the two bias patterns listed after Proposition 3.4.
[Theorem 3.7 and Theorem C.1, Appendices C.2 and C.3] The variance proofs use 'because of Assumption 3.3' to replace E[r(k)^2|x,a,e] with E[r(k)^2|x,Phi_k(e)]. Assumption 3.3 is a first-moment condition only, and even together with Assumption 3.2 it does not imply the required equality for the second moment. A second-moment assumption such as E[r(k)^2|x,e]=E[r(k)^2|x,Phi_k(e)] and E[r(k)^2|x,a,e]=E[r(k)^2|x,e] is needed but is not stated anywhere. As written, the variance-reduction claims in Theorem 3.7 and Theorem C.1 are unproved.
[Section 4.3 and Appendix D.1] The experiments labeled 'under Assumption 3.3' generate rewards from q_k(x,e), i.e. p(r|x,e), so Assumption 3.2 (no direct effect of actions on rewards given embeddings) holds by construction. These experiments therefore do not test Proposition 3.6 under only Assumption 3.3; they are consistent only with the corrected version of the theorem. The paper should state this explicitly, or include a setting where the reward depends on a even after conditioning on e, if the stronger claim is intended.

minor comments (4)

[Section 5] In the first sentence of Section 5, 'read-world scenarios' should be 'real-world scenarios'.
[Equation (14)] Equation (14) is missing a closing parenthesis in the denominator: the softmax expression should be pi_0(a(k)|x) = exp(beta * qbar(x,a(k))) / sum_{a in A_k} exp(beta * qbar(x,a)), with the sum fully in the denominator.
[Appendix D.2] The definitions of c_random0 and c_random1 contain the fragment 'Beseed=0' and 'Beseed=1'; this appears to be a typo for a Bernoulli draw with a fixed seed and should be clarified.
[Theorem 3.7 display] The expression E_r[r(k)^2] in the statement of Theorem 3.7 does not specify the conditioning distribution; the proof later uses E_{p(r|x,Phi_k(e))}[r(k)^2], which should be stated in the theorem.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the GMIPS unbiasedness and variance theorems are attempted algebraic consequences of the paper's explicitly stated modeling assumptions, with no fitted parameter or self-citation chain doing the work.

full rationale

The paper's derivation chain is self-contained in the sense relevant to circularity. The GMIPS estimator is defined directly as an importance-weighted sum over ranking-embedding subsets, and Proposition 3.6 derives unbiasedness by marginalizing the logging distribution over actions; the variance comparison in Theorem 3.7 and the bias decomposition in Theorem 3.8 are also attempted by direct calculation from the stated assumptions. No parameter is fitted to a subset of data and then reported as a prediction of the same quantity: the SLOPE procedure is used only for embedding-dimension selection from logged data, which is standard hyperparameter selection rather than a fitted input renamed as a result. The paper's citations to prior work, including Saito and Joachims (2022) for MIPS, are contextual and are not load-bearing: the embedding assumption and estimator are defined in the present paper, and the equivalence of MIIPS to MIPS is explicitly acknowledged rather than disguised. There is, however, a genuine correctness gap that should not be counted as circularity: in Appendix C.1, the proof of Proposition 3.6 replaces qk(x,a,e) by qk(x,Phi_k(e)) with the annotation 'because of Assumption 3.3', but Assumption 3.3 as written only asserts E[r|x,e] = E[r|x,Phi_k(e)], which does not control E[r|x,a,e]. The unbiasedness claim therefore requires either Assumption 3.2 (no direct effect) plus a first-moment matching condition, or a restated Assumption 3.3 of the form qk(x,a,e)=qk(x,Phi_k(e)); Proposition 3.4's claimed implication from Assumption 3.3 to Assumption 3.2 is also not implied by the written assumption. These are proof and specification defects, not instances of the paper's conclusions reducing to its inputs by definition or by self-citation.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central claims rest on three domain assumptions about embedding support and reward structure. The stated unbiasedness theorem additionally needs Assumption 3.2 (no direct effect), which is missing from its statement. This is a logical error, not an extra postulate, and no invented entities are introduced.

free parameters (1)

Embedding dimension D = 3 per position (synthetic), 15 (real), selected by SLOPE in some runs
Controls the bias-variance trade-off in experiments; the theoretical statements treat the embedding space as given, so this is an experimental hyperparameter, not a fitted constant in the derivation.

assumptions (4)

domain assumption Assumption 3.1: Common Ranking Embedding Support, p(e|x,pi) > 0 implies p(e|x,pi0) > 0.
Needed for the importance weight over embeddings to be well-defined and for the unbiasedness proof.
domain assumption Assumption 3.2: No Direct Effect on Rankings, a ⊥ r | x, e.
Required in the proof of Proposition 3.6 and Theorem 3.7, though omitted from their stated hypotheses.
domain assumption Assumption 3.3: User Behavior Model on Ranking Embedding Spaces, E[r|x,e] = E[r|x,Phi_k(e)].
Central assumption that the reward depends only on the chosen embedding subset, but insufficient alone for the claimed unbiasedness.
domain assumption Assumption 2.1: Common Support on ranking actions, used in Theorem 3.7.
Used to ensure GIPS is unbiased so that the variance comparison in Theorem 3.7 is meaningful.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Off-Policy Evaluation of Ranking Policies via Embedding-Space User Behavior Modeling." pith.science (2026). https://pith.science/paper/QODGU4LQ

@misc{pith2026250600446,
  author       = {Pith},
  title        = {Pith review of: Off-Policy Evaluation of Ranking Policies via Embedding-Space User Behavior Modeling},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QODGU4LQ}},
  note         = {Machine review of arXiv:2506.00446}
}

read the original abstract

Off-policy evaluation (OPE) in ranking settings with large ranking action spaces, which stems from an increase in both the number of unique actions and length of the ranking, is essential for assessing new recommender policies using only logged bandit data from previous versions. To address the high variance issues associated with existing estimators, we introduce two new assumptions: no direct effect on rankings and user behavior model on ranking embedding spaces. We then propose the generalized marginalized inverse propensity score (GMIPS) estimator with statistically desirable properties compared to existing ones. Finally, we demonstrate that the GMIPS achieves the lowest MSE. Notably, among GMIPS variants, the marginalized reward interaction IPS (MRIPS) incorporates a doubly marginalized importance weight based on a cascade behavior assumption on ranking embeddings. MRIPS effectively balances the trade-off between bias and variance, even as the ranking action spaces increase and the above assumptions may not hold, as evidenced by our experiments.

Figures

Figures reproduced from arXiv: 2506.00446 by the authors.

**Figure 1.** Assumption of no direct effect in ranking settings. Ranking actions do not have a causal effect on rewards. which assumes that embeddings are observed independently at each position (i.e., p(e|x, a) = QK k=1 p(e(k)|x, a(k))). For instance, this is observed when the category (e.g., movie genre) of unique actions for each position is treated as ranking embeddings. We can also assume dependencies, where the embedding… view at source ↗

**Figure 2.** The trade-off between bias and variance in the GMIPS Theorem 3.8 indicates that the bias of the GMIPS is the sum of violations of both Assumptions 3.2 (the second term) and 3.3 (the first term). The first term includes w −1 Φc k (x, e) − 1, which represents the policy deviation of the importance weights for complement set Φ c k (e) of Φk(e), which are not considered owing to their double marginalization, where wΦc … view at source ↗

**Figure 3.** Comparison of MSE normalized by V (π) when Assumption 3.3 holds. The solid lines represent existing estimators, whereas the dashed lines indicate our proposed estimators. The colors vary according to the user behavior model assumed for each space. Note that all MSE values are presented on log-scale. mator selection methods such as SLOPE (Su et al., 2020b), which is based on Lepski’s principle (Lepski & Spokoiny, 199… view at source ↗

Figures from the paper (6 more)

**Figure 4.** Figure 4: MSE, bias, and variance normalized by V (π) with varying number of unobserved dimensions in the ranking embeddings. Note that these are all log-scale. to the no(standard) assumption as candidates. Regarding the logging and target policies, we establish an anti-optimal …

**Figure 5.** Figure 5: MSE normalized by V (π) with varying sample sizes on EUR-Lex4K (left) and RCV1-2K (right) datasets. Note that all MSE values are presented on log-scale. The computations are performed 1000 times using logged data replicated with different random seeds. leverages double…

**Figure 6.** Figure 6: Comparison of MSE normalized by V (π) under data where the assumption of user behavior model on ranking embeddings are valid. The solid lines are existing estimators and the dashed lines are our proposed estimators. The colors are changed according to the user behavior…

**Figure 7.** Figure 7: MSE, bias, and variance normalized by V (π) with varying number of deficient unique actions under Assumptions 3.1 and 3.2. Note that these are all log-scale [PITH_FULL_IMAGE:figures/full_fig_p025_7.png]

**Figure 8.** Figure 8: MSE, bias, and variance normalized by V (π) with varying complexity of the user behavior under which Assumption 3.3 does not hold while Assumptions 3.1 and 3.2 hold. Note that these are all linear-scale [PITH_FULL_IMAGE:figures/full_fig_p025_8.png]

**Figure 9.** Figure 9: MSE, bias, and variance normalized by V (π) with variation in sample sizes of logged data. Comparison of MSIPS, MRIPS without SLOPE, and MSIPS(w/SLOPE), MRIPS(w/SLOPE). Note that these are all log-scale. selecting the embedding dimension using only logged data, the MSI…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

16 extracted references · 14 canonical work pages

[1]

• Inverse Cascade: cinverse cascade(k, l) = 1, ∀l ≥ k, and cinverse cascade(k, l) = 0, otherwise

≤ k ≤ min(K, k+ 1), l= k, and cneighbor 1(k, l) = 0, otherwise. • Inverse Cascade: cinverse cascade(k, l) = 1, ∀l ≥ k, and cinverse cascade(k, l) = 0, otherwise. • Random 0: ( crandom 0 ∼ Beseed=0, or crandom 0 = 1, ∀l = k), and crandom 0(k, l) = 0, otherwise. • Random 1: ( crandom 1 ∼ Beseed=1, or crandom 1 = 1, ∀l = k), and crandom 1(k, l) = 0, otherwis...

work page 2021
[2]

Note that for the RCV1-2K dataset, the top 1600 training and top 4000 test samples were extracted for the experiment

Statistics for the EUR-Lex4K and RCV1-2K datasets from the Extreme Classification Repository (Bhatia et al., 2016). Note that for the RCV1-2K dataset, the top 1600 training and top 4000 test samples were extracted for the experiment. train size ntrain test size ntest dimensions of features number of labels EUR-Lex4K 15,539 3,809 5,000 3,993 RCV1-2K 623,84...

work page 2016
[3]

We utilized SLOPE to determine the optimal number of dimensions to extract from the first dimension of the embedding

≤ min m∈[M ] Bias[ ˆV (π; D, θm)] + CNF[ ˆV (π; D, θm)] For detailed explanations, please refer to (Tucker & Lee, 2021). We utilized SLOPE to determine the optimal number of dimensions to extract from the first dimension of the embedding

work page 2021
[5]

Long-term off-policy evaluation and learning

Saito, Y ., Abdollahpouri, H., Anderton, J., Carterette, B., and Lalmas, M. Long-term off-policy evaluation and learning. In Proceedings of the ACM on Web Conference 2024, pp. 3432–3443, 2024a. Saito, Y ., Yao, J., and Joachims, T. Potec: Off-policy learn- ing for large action spaces via two-stage policy decompo- sition. arXiv preprint arXiv:2402.06151, 2...

arXiv 2024
[6]

Related Works Here, we provide an overview of the related works pertinent to our study

10 Off-Policy Evaluation of Ranking Policies via Embedding-Space User Behavior Modeling A. Related Works Here, we provide an overview of the related works pertinent to our study. A.1. Off-Policy Evaluation (OPE) As mentioned in the main text, OPE aims to accurately estimate the policy value of the target policy to make decisions using only logged bandit d...

work page 2011
[8]

However, selecting estimators using only logged data is challenging, as their biases are influenced by the true policy value

or stem from different estimator classes (Udagawa et al., 2023). However, selecting estimators using only logged data is challenging, as their biases are influenced by the true policy value. Regarding the tuning of estimators with hyperparameters, MSE surrogates have been proposed, which utilize unbiased estimators, such as IPS and DR (Su et al., 2020a). ...

work page 2023
[9]

proposed the policy adaptive estimator selection via importance fitting (PAS-IF). This method facilitates the selection of different estimators by learning a pseudo-policy, which enables the pseudo-online experimental performance of a target policy based on samples taken from logged data. In relation to our study, we employed the SLOPE procedure to determ...

work page 2020
[15]

We observed that the MRIPS achieved the lowest MSE in most cases, although the cascade model on ranking embeddings does not hold. Despite snAIPS (w/UBT) adjusting the optimal importance weights according to user context, the MSE deteriorates as user behavior becomes more complex and diverse, which is primarily owing to bias and variance issues associated ...

work page 2000

Show all 16 references

[16]

Comparison of MSIPS, MRIPS without SLOPE, and MSIPS(w/SLOPE), MRIPS(w/SLOPE)

MSE, bias, and variance normalized by V (π) with variation in sample sizes of logged data. Comparison of MSIPS, MRIPS without SLOPE, and MSIPS(w/SLOPE), MRIPS(w/SLOPE). Note that these are all log-scale. selecting the embedding dimension using only logged data, the MSIPS(w/SLO...

2022
[1975]

are often developed to prevent the duplication of unique actions across positions. Additionally, a policy learning method that considers fairness among items has been proposed to enable a greater number of items to engage the user compared to single-action decisions (Singh & J...

2019
[2014]

5 that can be reproduced on any operating system. D.1. Data Generation Process of Synthetic Experiments We generated synthetic data based on the Open Bandit Pipeline (OBP) 6 (Saito et al., 2020), and synthetic experimental settings of previous studies (Saito & Joachims, 2022; ...

2020
[2017]

However, if this assumption is not met, the PI exhibits significant bias (Kiyohara et al., 2024)

demonstrated that the pseudo inverse (PI) estimator, as the sum of importance weights per slot, is valid under the assumption of linearity in rewards. However, if this assumption is not met, the PI exhibits significant bias (Kiyohara et al., 2024). To address the bias of the P...

2024
[2020]

Off-policy evaluation for large action spaces via pol- icy convolution

Sachdeva, N., Wang, L., Liang, D., Kallus, N., and McAuley, J. Off-policy evaluation for large action spaces via pol- icy convolution. In Proceedings of the ACM on Web Conference 2024, pp. 3576–3585,

2024
[2021]

Offline policy evaluation in large action spaces via outcome-oriented action grouping

Peng, J., Zou, H., Liu, J., Li, S., Jiang, Y ., Pei, J., and Cui, P. Offline policy evaluation in large action spaces via outcome-oriented action grouping. In Proceedings of the ACM Web Conference 2023, pp. 1220–1230,

2023
[2022]

Open bandit dataset and pipeline: Towards realistic and reproducible off-policy evaluation

Saito, Y ., Aihara, S., Matsutani, M., and Narita, Y . Open bandit dataset and pipeline: Towards realistic and reproducible off-policy evaluation. arXiv preprint arXiv:2008.07146,

2008 arXiv
[2023]

Off-policy evalu- ation of slate bandit policies via optimizing abstraction

Kiyohara, H., Nomura, M., and Saito, Y . Off-policy evalu- ation of slate bandit policies via optimizing abstraction. In Proceedings of the ACM on Web Conference 2024, pp. 3150–3161,

2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

• Inverse Cascade: cinverse cascade(k, l) = 1, ∀l ≥ k, and cinverse cascade(k, l) = 0, otherwise

≤ k ≤ min(K, k+ 1), l= k, and cneighbor 1(k, l) = 0, otherwise. • Inverse Cascade: cinverse cascade(k, l) = 1, ∀l ≥ k, and cinverse cascade(k, l) = 0, otherwise. • Random 0: ( crandom 0 ∼ Beseed=0, or crandom 0 = 1, ∀l = k), and crandom 0(k, l) = 0, otherwise. • Random 1: ( crandom 1 ∼ Beseed=1, or crandom 1 = 1, ∀l = k), and crandom 1(k, l) = 0, otherwis...

work page 2021

[2] [2]

Note that for the RCV1-2K dataset, the top 1600 training and top 4000 test samples were extracted for the experiment

Statistics for the EUR-Lex4K and RCV1-2K datasets from the Extreme Classification Repository (Bhatia et al., 2016). Note that for the RCV1-2K dataset, the top 1600 training and top 4000 test samples were extracted for the experiment. train size ntrain test size ntest dimensions of features number of labels EUR-Lex4K 15,539 3,809 5,000 3,993 RCV1-2K 623,84...

work page 2016

[3] [3]

We utilized SLOPE to determine the optimal number of dimensions to extract from the first dimension of the embedding

≤ min m∈[M ] Bias[ ˆV (π; D, θm)] + CNF[ ˆV (π; D, θm)] For detailed explanations, please refer to (Tucker & Lee, 2021). We utilized SLOPE to determine the optimal number of dimensions to extract from the first dimension of the embedding

work page 2021

[4] [5]

Long-term off-policy evaluation and learning

Saito, Y ., Abdollahpouri, H., Anderton, J., Carterette, B., and Lalmas, M. Long-term off-policy evaluation and learning. In Proceedings of the ACM on Web Conference 2024, pp. 3432–3443, 2024a. Saito, Y ., Yao, J., and Joachims, T. Potec: Off-policy learn- ing for large action spaces via two-stage policy decompo- sition. arXiv preprint arXiv:2402.06151, 2...

arXiv 2024

[5] [6]

Related Works Here, we provide an overview of the related works pertinent to our study

10 Off-Policy Evaluation of Ranking Policies via Embedding-Space User Behavior Modeling A. Related Works Here, we provide an overview of the related works pertinent to our study. A.1. Off-Policy Evaluation (OPE) As mentioned in the main text, OPE aims to accurately estimate the policy value of the target policy to make decisions using only logged bandit d...

work page 2011

[6] [8]

However, selecting estimators using only logged data is challenging, as their biases are influenced by the true policy value

or stem from different estimator classes (Udagawa et al., 2023). However, selecting estimators using only logged data is challenging, as their biases are influenced by the true policy value. Regarding the tuning of estimators with hyperparameters, MSE surrogates have been proposed, which utilize unbiased estimators, such as IPS and DR (Su et al., 2020a). ...

work page 2023

[7] [9]

proposed the policy adaptive estimator selection via importance fitting (PAS-IF). This method facilitates the selection of different estimators by learning a pseudo-policy, which enables the pseudo-online experimental performance of a target policy based on samples taken from logged data. In relation to our study, we employed the SLOPE procedure to determ...

work page 2020

[8] [15]

We observed that the MRIPS achieved the lowest MSE in most cases, although the cascade model on ranking embeddings does not hold. Despite snAIPS (w/UBT) adjusting the optimal importance weights according to user context, the MSE deteriorates as user behavior becomes more complex and diverse, which is primarily owing to bias and variance issues associated ...

work page 2000