{"id":"008c789f-7e58-4ddc-b323-21977d3276f5","arxiv_id":"2502.08993","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Using both action-selection and reward-observation propensities gives an unbiased off-policy value estimator under position-based missingness and the no-direct-effect assumption.","lead":"This paper shows that standard off-policy evaluation for recommendations is biased when not every displayed item receives a reward, and it proposes a new estimator that also divides by the reward-observation probability. A generalist reader should care because real recommendation logs combine logging-policy bias with position-based missingness.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The generative model in §3.1 is MAR, not MNAR, so the unbiasedness proof does not cover the paper's headline claim of handling missing-not-at-random rewards.","rationale":"The reader's weakest_assumption already identifies the same issue: the generative model makes o and r independent given x, which is MAR, not MNAR. My stress-test agrees with that identification and sharpens the consequence: the unbiasedness proof of MIPS(w/ROIPS) in Eq. (5) depends critically on this conditional independence, so the paper does not actually establish unbiasedness for missing-not-at-random rewards. The paper deserves credit for a transparent derivation, a clear statement of Assumption 1, and released code, but the central claim is overstated relative to the model. The proposed estimator is likely a valid correction for position-dependent missingness (MAR), and the synthetic experiments are consistent with that, but the title and abstract claim MNAR. Since this concern was already central to the reader's conditional verdict, my read does not change the verdict: the paper should be accepted only if the claims are revised to MAR/position-bias or a genuine MNAR analysis is added. I recommend UNCHANGED because the conditional verdict already captures the need for this revision.","tokens_in":5773,"tokens_out":4094,"duration_ms":44568,"concrete_test":"Simulate genuine MNAR data with the same setup as Section 4 but draw o_k from a distribution that depends on the unobserved reward, e.g., θ(o_k=1|x,r) = sigmoid(β·r_k + γ·position_k), with β ≠ 0. Use the oracle marginal P(o_k=1|x) (computed by Monte Carlo averaging over r) as the ROIPS in Eq. (5), and average the estimator over at least 100 seeds with n = 1000. If the averaged estimate does not match V(π) within Monte Carlo error, the Proposition in §3.2 fails for true MNAR. Also repeat with θ(o_k|x) estimated heuristically as in the paper; report the bias separately.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that MIPS(w/ROIPS) is unbiased when rewards are missing not at random (MNAR). However, the data-generation model in Section 3.1 is written as (x,a,e_a,o,r) ~ p(x)π0(a|x)θ(o|x)p(r|x,a). This factorization makes the observation indicator o conditionally independent of reward r given x (and a). That is a missing-at-random (MAR) mechanism with respect to the observed context, not MNAR. In true MNAR, the observation probability depends on the unobserved reward, i.e., p(o|x,r) ≠ p(o|x). The proof of the Theorem and the Proposition both replace E[r_k 1{o_k=1} | x,a] with E[r_k|x,a]·P(o_k=1|x), which is only valid under this conditional independence. If missingness actually depends on r_k, then E[r_k / θ(o_k|x) · 1{o_k=1}] = E[ r_k · θ(o_k|x,r_k)/θ(o_k|x) ], which does not generally equal E[r_k] and can be severely biased. Thus the unbiasedness proposition does not hold for the MNAR setting advertised in the title and abstract; it holds only for position-dependent MAR missingness, which is a substantially weaker claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies off-policy evaluation (OPE) for recommender systems when logged data contain both logging-policy bias and display-position-induced missing rewards. It defines a position-wise policy value, reviews the marginalized IPS (MIPS) estimator, and introduces an observation indicator o into the data-generating process. The paper proves that MIPS is biased when rewards are not always observed, proposes a new estimator, MIPS(w/ROIPS), that additionally divides by the reward-observation propensity, and proves unbiasedness under a no-direct-effect assumption. Synthetic experiments compare the proposed estimator with several baselines as the strength of position bias varies, and the paper reports that the version using a heuristic observation propensity achieves the lowest mean squared error.","tokens_in":6043,"tokens_out":4923,"duration_ms":52054,"significance":"If the result holds for the advertised setting, the paper addresses a practically important gap: standard OPE estimators implicitly assume that all rewards are observed, which is unrealistic when display position governs visibility. The unbiasedness proof is a direct and correct calculation under the stated generative model, and the proposed estimator is a natural combination of MIPS and an observation-propensity correction. A strength is that the implementation is provided. However, the paper's headline claim of handling missing-not-at-random rewards is not supported by the assumptions actually used, because the generative model makes the observation indicator independent of the reward given the context. The experimental section also lacks error bars, so the empirical superiority is not substantiated. With an honest reframing to position-dependent missingness, the contribution is useful but incremental.","major_comments":[{"comment":"The generative model is written as (x, a, e_a, o, r) ~ p(x)π0(a|x)θ(o|x)p(r|x,a), which makes o and r conditionally independent given x (and a). This is a missing-at-random type mechanism with respect to the observed context, not a missing-not-at-random mechanism. The proof of the Proposition relies on the identity E[r_k 1{o_k=1} | x,a] = E[r_k|x,a]·P(o_k=1|x), which is valid only under that conditional independence. Under genuine MNAR, where p(o_k|x,r_k) differs from p(o_k|x), the proof fails: E[r_k 1{o_k=1}/θ(o_k|x)] = E[r_k θ(o_k|x,r_k)/θ(o_k|x)], which is not generally equal to E[r_k]. Thus the theorem and proposition do not establish unbiasedness for the MNAR setting advertised in the title and abstract; they establish unbiasedness for position-dependent missingness that depends only on the observed context. The authors should either revise the title, abstract, and terminology to describe position-dependent MAR missingness, or extend the estimator and analysis to observation mechanisms that depend on the unobserved reward.","section":"§3.1, Proposition proof and Eq. (5)"},{"comment":"The main experimental conclusion that MIPS(w/heuristic ROIPS) achieves the lowest MSE is reported without error bars, confidence intervals, or significance tests across the S=100 logged-data replicates. Each plotted MSE is itself a random quantity estimated from a finite number of seeds, so the reader cannot determine whether the observed differences, such as the reported 88.4% improvement over MIPS at α=3.0, are reliable. The authors should report standard errors or interquartile ranges and, ideally, paired comparisons between estimators.","section":"§4.3 and Figure 2"}],"minor_comments":[{"comment":"The sentence 'when both bias exits in the logged data' contains a grammatical error; it should be 'when both biases exist in the logged data'.","section":"§1"},{"comment":"The unbiasedness Proposition applies to the estimator using the true observation propensity θ(o_k|x). The experiments, however, emphasize MIPS(w/heuristic ROIPS), which uses a context-free estimate from the logged data. The paper should state explicitly that the theoretical unbiasedness result does not apply to the heuristic version and that its performance is only empirically assessed.","section":"§3.2"},{"comment":"The notation e_{a,k} and e_{a_k} are used inconsistently for the embedding of the action at position k; a single consistent notation would improve readability.","section":"Notation throughout"},{"comment":"The phrase 'although Assumption 1 holds when rewards are MNAR' is grammatically ambiguous. It should be rephrased to say that the bias arises even when Assumption 1 holds and rewards are missing due to display position.","section":"§3.1, Theorem statement"}],"recommendation":"major_revision","confidential_remarks":"The central issue is the mismatch between the paper's advertised MNAR setting and the actual MAR generative model used in the proof. This is not a mere terminology problem: the unbiasedness claim fails if missingness depends on the unobserved reward. The paper can be repaired by reframing the contribution as a correction for position-dependent missing rewards that are conditionally independent of rewards given the context. If the authors are unwilling to change the title and abstract accordingly, the paper should not be accepted in its current form. With the reframing, the contribution is sound but modest; the lack of error bars in the experiments also needs to be addressed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The thing to know: this paper derives a clean bias formula for MIPS when rewards are only observed with probability θ(o|x), and proposes a corrected estimator (MIPS with ROIPS) that divides by that observation propensity. The math is right for the model it defines, and the heuristic version works in their synthetic experiments. But the paper sells this as handling missing-not-at-random rewards, and that is not what the model assumes. The generative model in §3.1 factorizes as p(x)π0(a|x)θ(o|x)p(r|x,a), which makes o conditionally independent of r given x. That is missing-at-random with respect to the observed context, not MNAR. Under true MNAR, where p(o|x,r) depends on the unobserved reward, the unbiasedness proof collapses: E[r/θ(o|x)·1{o=1}] becomes E[r·θ(o|x,r)/θ(o|x)], which is not generally E[r]. So the central claim as advertised is not established.\n\nWhat is genuinely useful: the bias formula in Theorem 1 is a clear, correct statement of how position-dependent missingness biases MIPS, and the ROIPS correction is a sensible fusion of known URL and MIPS techniques. The proof of the proposition is a direct expectation calculation and holds under the stated assumptions. They also ship code, which is good practice. The synthetic experiments show the heuristic estimator can reduce MSE substantially when the true θ is unknown, which is the practically relevant scenario.\n\nThe soft spots, in proportion: (1) The MNAR terminology is the load-bearing issue. If the paper were repositioned as handling MAR position bias, the contribution is modest but honest. (2) The experiments report no error bars around the MSE, and the heuristic ROIPS is a context-free per-position average, which is crude; it works in their setup, but the claim that it balances bias and variance would be stronger with variance bands. (3) The 'no direct effect' assumption is strong and they do acknowledge it, though it is not the main problem.\n\nWho should read this: people working on OPE for recommender logs with position-based missingness, especially those building on Saito and Joachims' MIPS. It is a useful footnote that deserves to exist.\n\nRecommendation: send it to peer review, but ask the authors to fix the MNAR claim or the model. If the model stays MAR, the title and abstract need to change. The estimator itself is worth refereeing.","headline":"A correct estimator for position-dependent missingness, but the MNAR label oversells it: the model is MAR, so the proof does not cover the headline claim.","tokens_in":6510,"tokens_out":1760,"would_cite":true,"duration_ms":18649,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Dividing logged rewards by their observation probability restores unbiased off-policy evaluation when missing rewards depend on display position.","keywords":["off-policy evaluation","missing not at random","position bias","inverse propensity score","recommender systems","marginalized importance sampling","unbiased recommender learning"],"falsifier":"Generate logged data in which the observation probability is a function of the reward itself, for example $\\theta(o_k=1|x,r_k)$ decreasing for low $r_k$, while keeping every other part of the model identical; if the resulting MIPS(w/ROIPS) estimate differs systematically from the true policy value across many replications, the conditional independence that carries the proof is violated.","tokens_in":5600,"feed_emoji":"🎯","tokens_out":7944,"duration_ms":75463,"temperature":0.7,"pith_summary":"This paper tackles a realistic failure mode in off-policy evaluation for recommender systems: the data used to estimate a new policy's value are logged under a known recommendation policy, but they also have position-dependent missing rewards, because items shown lower on a page are less often observed. The paper first proves that the standard marginalized inverse propensity score (MIPS) estimator, which corrects only for the logging policy, is biased under this kind of missing-not-at-random reward, with per-position bias equal to the expected reward times the probability that the position's reward is unobserved. It then proposes MIPS(w/ROIPS), which multiplies the usual logging-policy correction by a second inverse propensity score for reward observation, and proves this estimator is unbiased for the policy value whenever the no-direct-effect assumption holds. A practical consequence demonstrated in synthetic experiments is that an observation propensity estimated from the logged data alone can be plugged in and still reduce mean squared error, because the bias introduced by a heuristic correction is outweighed by a large variance reduction.","feed_headline":"One extra division fixes position-biased off-policy evaluation","feed_subtitle":"A second inverse-propensity weight cancels the display-position bias in logged rewards.","key_machinery":"The load-bearing object is the double propensity weight $$w(x_i,e_{a_i,k})/\\$\\theta$(o_k|x_i) = \\frac{\\pi(e_{a_k}|x)}{\\pi_0(e_{a_k}|x)}\\cdot\\frac{1}{\\$\\theta$(o_k|x_i)},$$ applied only to logged positions with $o_{i,k}=1$. The first factor, the marginalized importance weight, removes the logging-policy bias; the second, the inverse reward-observation propensity, removes the position bias by cancelling the probability that the position is observed. The estimator's unbiasedness rests on the identity $\\mathbb{E}[I\\{o_k=1\\}|x]=\\theta(o_k|x)$ together with the generative model in which $o$ and $r$ are independent given $x$.","core_discovery":"The central claim is a bias formula and its cure. Under Assumption 1 (no direct effect), meaning $E[r_k|x,a]=E[r_k|x,e_{a_k}]$, when logged rewards are observed through a position-dependent mechanism $\\theta(o|x)$, the MIPS estimator has bias $$\\mathrm{Bias}(\\hat{V}^{(k)}_{\\mathrm{MIPS}}(\\pi;D)) = \\mathbb{E}_{p(x)\\pi(e_{a_k}|x)}[q_k(x,e_{a_k})(1-\\$\\theta$(o_k|x))].$$ The proposed estimator, $$\\hat{V}^{(k)}_{\\mathrm{MIPS(w/ROIPS)}}(\\pi;D) = \\frac{1}{n}\\sum_{i:o_{i,k}=1} w(x_i,e_{a_i,k})\\frac{r_{i,k}}{\\$\\theta$(o_k|x_i)},$$ with $w(x,e_{a_k})=\\pi(e_{a_k}|x)/\\pi_0(e_{a_k}|x)$, is then proved to satisfy $E_D[\\hat{V}^{(k)}_{\\mathrm{MIPS(w/ROIPS)}}]=V^{(k)}(\\pi)$. The proof is a cancellation: the inverse observation propensity removes the factor $\\theta(o_k|x)$ introduced by conditioning on $o_{i,k}=1$.","pith_inferences":["If the observation propensity truly depends on the reward value itself, the conditional-independence premise fails and the proof no longer holds; a natural extension is to model $\\theta(o_k|x,r_k)$ and quantify the residual bias.","The same double-propensity correction can be paired with a doubly robust estimator or control variate to reduce the variance that the extra inverse weight adds, especially when true propensities are used.","Plugging MIPS(w/ROIPS) into policy learning objectives would give an unbiased gradient estimator under the same assumptions, extending the correction from evaluation to learning.","Estimating $\\theta$ with a context-dependent model rather than a position-only rate is a testable upgrade: it should reduce the heuristic estimator's bias at the cost of higher variance."],"forward_implications":["Plain MIPS systematically undervalues policies that place rewards in positions with low observation probabilities, because the expectation of the estimator is $V^{(k)}(\\pi)$ minus the bias term.","With the true observation propensity, MIPS(w/ROIPS) is unbiased per position, so the sum over positions estimates the full policy value without position-bias error.","When the true propensity is unknown, a heuristic position-only estimate from logged data keeps the estimator nearly unbiased in experiments and lowers variance enough to reduce MSE by about 88.4% at the strongest position bias tested.","The bias theorem applies to all basic OPE/L estimators that implicitly assume full observation of rewards, so the missingness correction is not specific to MIPS.","Because the correction uses only logged observation indicators, it can be applied in real recommendation logs without additional data collection."],"supporting_citations":[{"why":"Supplies the MIPS estimator and the no-direct-effect assumption that the paper modifies.","marker":"[8]"},{"why":"Supplies the reward-observation propensity formulation for position-biased implicit feedback that ROIPS extends.","marker":"[6]"},{"why":"Supplies the inverse-propensity weighting principle for debiasing observable rewards.","marker":"[5]"},{"why":"Supplies the synthetic bandit logging pipeline used in the experiments.","marker":"[1]"}],"fun_headline_variants":["Two propensities beat one for biased off-policy evaluation","Inverse observation propensity cancels position bias in OPE","Fix MNAR reward bias with a second propensity score","Position-biased rewards? Divide by observation probability"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The argument holds only if reward depends on the full ranking action just through its category embedding (the no-direct-effect assumption) and whether a reward is observed is conditionally independent of the reward value once the context is fixed.","fun_headline_variants_meta":{"raw":{"variants":["Two propensities beat one for biased off-policy evaluation","Inverse observation propensity cancels position bias in OPE","Fix MNAR reward bias with a second propensity score","Position-biased rewards? Divide by observation probability"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000238,"raw_usage":{"total_tokens":1494,"prompt_tokens":911,"completion_tokens":583,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":527,"completion_tokens_details":{"reasoning_tokens":518}},"tokens_in":527,"tokens_out":583,"duration_ms":6102,"temperature":1.0,"reasoning_tokens":518,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T22:59:13.116103+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Generate logged data in which the observation probability is a function of the reward itself, for example $\\theta(o_k=1|x,r_k)$ decreasing for low $r_k$, while keeping every other part of the model identical; if the resulting MIPS(w/ROIPS) estimate differs systematically from the true policy value across many replications, the conditional independence that carries the proof is violated.","supporting_citations":[{"cited_title":"Off-Policy Eval- uation for Large Action Spaces via Embeddings","cited_arxiv_id":null,"evidence_quote":"Supplies the MIPS estimator and the no-direct-effect assumption that the paper modifies."},{"cited_title":"Unbiased recommender learning from missing-not-at-random implicit feedback","cited_arxiv_id":null,"evidence_quote":"Supplies the reward-observation propensity formulation for position-biased implicit feedback that ROIPS extends."},{"cited_title":"Recommendations as treat- ments: Debiasing learning and evaluation","cited_arxiv_id":null,"evidence_quote":"Supplies the inverse-propensity weighting principle for debiasing observable rewards."}],"review_version":1}