Pith. sign in

REVIEW 3 major objections 5 minor 45 references

Unidentified and Confounded? Understanding Two-Tower Models for Unbiased Learning to Rank (Extended Abstract)

T0 review · 3 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper argues that additive two-tower ranking models are unaffected by logging policy strength once they are well-specified and identifiable, and that the observed degradation under strong policies comes from unidentifiability or from m

desk verdict A valuable reframing of two-tower degradation as identifiability plus misspecification, but the central misspecification lemma is asserted rather than derived, and the τ experiments do not isolate the mechanism. read the letter →

arxiv 2508.21698 v1 pith:L6ATTXFU submitted 2025-08-29 cs.IR

Philipp Hager , Onno Zoeter , Maarten de Rijke This is my paper
classification cs.IR
keywords two-towermodelsunbiasedlearningtorankidentifiabilitypositionbiasloggingpolicymodelmisspecificationinversepropensityweightingclick
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to explain why two-tower learning-to-rank models degrade when trained on clicks from strong production ranking policies, replacing two competing explanations—confounding versus unidentifiability—with a single sharper picture. It shows that additive two-tower models are identifiable without randomized document swaps whenever the feature distributions of query-document pairs overlap across ranks and the relevance tower is continuous; when they are identifiable and well-specified, the logging policy drops out of the optimality equations entirely. The remaining failure mode is model misspecification: if the model cannot reproduce the true click probabilities and its residual errors correlate with how often documents appear in each position, a strong policy amplifies estimation bias. The paper therefore concludes that logging policies affect two-tower models only through misspecification, and proposes residual monitoring, randomized data collection, and inverse-propensity sample weighting as practical checks and remedies.

What carries the argument

The additive two-tower model, which predicts clicks as a sigmoid of a position-bias logit θ_k plus a relevance logit γ_{q,d} (or r(x_{q,d})). Identification is carried by a graph whose vertices are ranks and whose edges connect ranks with overlapping feature support; connectivity of that graph makes the parameters recoverable up to an additive constant after normalization. The policy analysis rewrites the expected negative log-likelihood as a policy-weighted sum, leading to Lemma 3.1 (the policy cancels for well-specified identifiable models) and Lemma 3.2 (misspecified models must satisfy policy-weighted residual conditions, so residual-policy correlation can amplify bias).

What would settle it

Simulate non-linear user behavior so a linear two-tower model is misspecified, but construct the logging policy score from features that are independent of the model's residual error, so ε(q,d,k) has zero correlation with π(d,k|q); train under a deterministic strength-α=1 policy. If position-bias estimates still degrade despite zero residual-policy correlation, the Lemma 3.2 mechanism is incomplete; if they remain accurate, the correlation premise is confirmed.

Watch

Extended reading notes

Core claim

The central claim is that the widely reported drop in two-tower ranking performance under strong logging policies is neither a direct confounding effect nor an architecture-specific quirk. For an additive two-tower model P(C=1|q,d,k)=σ(θ_k+γ_{q,d}), identifiability requires that the same query-document pairs appear across positions (via swaps) or that feature supports overlap across ranks with a continuous relevance tower; once those conditions hold, a well-specified model's gradient equations are satisfied independently of the logging policy. Bias enters only when the model is misspecified: the optimizer must drive policy-weighted residual errors to zero, and if the residual error ε(q,d,k)

Load-bearing premise

The load-bearing premise is that a misspecified two-tower model has residual click-prediction errors that are systematically correlated with the logging policy's display probabilities across ranks; if real residuals are independent of document placement, the paper's claim that policies act only through misspecification is not supported.

Editorial extensions

If this is right

  • Click logs from deterministic production rankings are sufficient to identify feature-based two-tower models, as long as the features of documents shown at different ranks overlap; randomization is only strictly needed for per-pair or embedding models.
  • For well-specified and identifiable models, the strength of the logging policy is irrelevant to the estimated parameters, so observed degradation under strong policies signals misspecification rather than policy confounding per se.
  • Residual errors that correlate with position indicate vulnerability; monitoring click residuals for correlation with display probabilities can flag when training data will amplify bias.
  • Weighting each logged interaction inversely to its display propensity dampens policy amplification whenever every document has nonzero probability of appearing in other ranks; under deterministic policies with zero propensities, it cannot help.
  • Simulating strong logging policies by sorting directly on expert labels manufactures omitted-variable bias and should be avoided; sorting by model predictions does not reproduce the degradation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A sharper operationalisation of the paper's residual-monitoring advice: estimate the residual vector of a fitted two-tower model and compute its correlation with position-specific display probabilities; when that correlation is near zero, policy strength should not degrade parameter recovery, giving teams a cheap pre-training check.
  • The overlap condition can be assessed directly from logged data by estimating feature-support overlap across ranks; if overlap is vanishing, collecting even a small fraction of randomized swaps may be more valuable than any loss-function change.
  • The argument implies a division of labor in future work: misspecification fixes target the relevance tower's capacity and feature set, while identifiability fixes target data collection; both failure modes produce similar symptom curves but need different remedies.
  • The same logic extends beyond clicks to any two-tower decomposition of a propensity and an outcome: if the outcome model is correct, exposure policy is irrelevant; if not, exposure bias amplifies error.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper analyzes additive two-tower models for unbiased learning to rank, seeking to explain why training on clicks from strong logging policies degrades ranking performance. It makes two theoretical claims: (i) two-tower models can be identified without document swaps when feature distributions overlap across ranks (Theorem A.1, proved only in approximate form), and (ii) logging policies have no effect on well-specified, identifiable models (Lemma 3.1) but can amplify estimation bias in misspecified models when residual errors correlate with the policy's display probabilities (Lemma 3.2). The paper reports simulation experiments on MSLR30K that compare well-specified and misspecified models under varying policy randomness, and proposes an inverse-propensity sample-weighting scheme for the misspecified case. A central conclusion is that 'logging policies affect two-tower models only through model misspecification.'

Significance. If the claims are correct, the paper would provide a useful resolution of the confounding-versus-identifiability debate for two-tower ULTR models, with practical guidance on when swaps or randomization are needed and when policy-induced degradation should be attributed to model misspecification. The authors' controlled simulation framework, which directly compares recovered bias parameters against ground truth, is a strength, as is the explicit release of code, data, and results. The identifiability extension to feature-based models and the demonstration that well-specified feature-based models can recover position bias without swaps are valuable. However, the central theoretical support for the misspecification mechanism is incomplete, and the experiments do not isolate the posited residual-policy correlation mechanism from ordinary covariate shift.

major comments (3)
  1. [Section 3, Lemma 3.2] The proof of Lemma 3.2 does not derive the claimed bias-amplification mechanism. Equation (10) states only the stationary conditions of the weighted loss: the policy-weighted residual averages vanish across positions and across items. The proof then asserts that when residuals are 'systematically correlated' with the policy's display probabilities, 'the optimizer must shift model parameters from their true values.' Under misspecification there is no unambiguous 'true' value, and the proof does not compare the minimizer to the uniform-policy minimizer or quantify any shift. The claim that residual-policy correlation amplifies bias is an assertion, not a consequence of the displayed equations. Since the paper's central dichotomy—'logging policies affect two-tower models only through model misspecification'—and the residual-monitoring heuristic in Section 5 depend on this lemma, this is a l
  2. [Section 2.2 and Appendix A, Theorem A.1] The main text states identifiability as a binary condition via Eq. (4) and the abstract says model parameters can be 'recovered,' but Theorem A.1 establishes only 'approximately identifiable up to an additive constant,' with an error bound 2L||x1-x2||_2 in Eq. (14). The proof uses x1≈x2 rather than exact equality. If the supports intersect at an exact feature vector, exact equality follows, but for empirical feature vectors there need not be exact matches, so the paper's main claim is stronger than the theorem. Moreover, the text says 'identifying additive two-tower models requires overlap in distributions across ranks,' but the theorem only proves sufficiency of overlap, not necessity. The authors should state the approximation explicitly in the main text and either prove necessity or soften the language from 'requires' to 'is guaranteed by.' This is a central contribution, so the overs
  3. [Section 4, Figures 2 and 3] The experiments vary the temperature parameter τ, but τ changes the marginal feature distribution P(x|k) itself: at τ=0 each position receives a deterministic document, while at τ=1 all positions receive uniform random documents. For any misspecified model, the minimizer of the weighted loss changes with these marginals even if there is no 'systematic correlation between residual errors and policy display probabilities.' Thus the observed bias amplification in Fig. 3 is equally consistent with ordinary covariate shift under misspecification, and the experiments do not isolate the mechanism posited in Lemma 3.2. The conclusions about 'bias amplification' and the residual-monitoring recommendation would be better supported by a control that keeps P(x|k) fixed while varying policy exposure, or by directly measuring the residual-policy correlation and showing it mediates the effect.
minor comments (5)
  1. [Section 3, Eq. (5)] The loss is written with P(C=1|q,d,k) inside the expectation, but the convention in the rest of the paper uses the model prediction σ(θ_k+γ_qd) for probabilities. Clarify that Eq. (5) is the expected negative log-likelihood over clicks, i.e., the expectation of C log σ(...) + (1-C) log(1-σ(...)) under the true click distribution.
  2. [Section 3, Lemma 3.1] The proof states that for gradients to vanish at the optimal parameters, Eq. (8) must hold for every pair with π(d,k|q)>0. This is true at the true parameters in the well-specified case, but the statement as written is not a general property of minimizers of the weighted loss; it would be clearer to say 'at the true parameters, the residual is zero, so the gradients vanish and, by identifiability, this is the unique optimum.'
  3. [Figures 3 and 5] The word 'Ommitted' should be 'Omitted' in the figure captions/axes.
  4. [Section 5 and Appendix E] The proposed sample-weighting scheme in Eq. (18) is inverse propensity weighting. It would be helpful to note explicitly the connection to existing IPS estimators and to state the additional assumption that propensities are known or reliably estimated; the paper only says they are estimated by counting displays, which may be unstable for rare position-item pairs.
  5. [Section 2.2 and Appendix B] Appendix B advises using fewer features or shallower networks to improve overlap and continuity, but Section 3 warns that misspecification can cause policy bias. These two pieces of advice could conflict; the paper should acknowledge the trade-off directly in the main text rather than only in the appendix.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the identifiability theorem extends an external result with its own proof, the experiments benchmark against synthetic ground truth, and self-citations are contextual.

full rationale

The derivation chain is not circular. The identifiability result (Theorem A.1, Eqs. 11–14) is an explicit extension of Chen et al. [8, Theorem 1]; the proof supplies a continuity/Lipschitz argument, and the overlap condition is defined from feature supports, not from the conclusion that overlap is needed. Lemma 3.1 is a direct algebraic consequence of stationarity: if the model is well-specified and identifiable, Eq. (8) holds for every displayed pair, so the policy π(d,k|q) does not alter the parameter solution. Lemma 3.2 is weaker than its conclusion: Eq. (10) only restates weighted residual balance conditions, and the claim that residual–policy correlation shifts parameters is asserted rather than derived. That is an evidentiary gap in the proof, not a circular reduction: no equation defines the bias in terms of the conclusion, and no fitted quantity is later renamed a prediction. The simulations are external benchmarks: position-bias ground truth θ̂_k = −ln(k) is chosen independently (Appendix C), and the models succeed or fail at recovering it; the τ sweep changes the marginal feature distribution, which is a confound for the misspecification hypothesis but again not a self-referential construction. The self-citations ([15], [19], [20]) are contextual—a simulation convention, a comparison method, and a pointer to the full paper—and are not used to prove the paper's central claims. No step in the paper reduces by construction to its own inputs, so the honest finding is no circularity.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The theoretical claims are parameter-free; experimental hyperparameters such as policy strength alpha, temperature tau, noise sigma=0.2, truncation to 25 documents, learning rate, and weight decay are simulation controls rather than fitted values that produce the conclusions. The axioms above are the premises the reader must grant for the central claims to bind.

assumptions (5)
  • domain assumption Relevance tower r and any alternative r' are continuous and share a Lipschitz constant L.
    Appendix A uses this to equate position offsets as feature support overlaps; exact identifiability requires the approximation error ||x1-x2||2 to go to zero, which is not guaranteed in practice.
  • domain assumption Feature support overlaps across positions and the position graph G is connected.
    Eq. 4 and Theorem A.1. Overlap is fragile in high dimensions, and deterministic strong policies can make position-specific supports effectively disjoint.
  • domain assumption The two-tower model is well-specified, able to perfectly model true click probabilities on all displayed pairs.
    Lemma 3.1 depends on this. Section 3 concedes that real-world models are misspecified, so the policy-invariance result is conditional.
  • domain assumption Clicks are generated by an additive two-tower model with position logits theta_k = -ln(k).
    Appendix C click simulation. User-model mismatch is left to future work, so the empirical confirmation is limited to the matched case.
  • ad hoc to paper Residual errors are systematically correlated with policy display probabilities.
    Lemma 3.2 introduces the correlation premise to explain bias amplification; no direct measurement of this correlation is reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Unidentified and Confounded? Understanding Two-Tower Models for Unbiased Learning to Rank (Extended Abstract)." pith.science (2026). https://pith.science/paper/L6ATTXFU

@misc{pith2026250821698,
  author       = {Pith},
  title        = {Pith review of: Unidentified and Confounded? Understanding Two-Tower Models for Unbiased Learning to Rank (Extended Abstract)},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/L6ATTXFU}},
  note         = {Machine review of arXiv:2508.21698}
}
read the original abstract

Additive two-tower models are popular learning-to-rank methods for handling biased user feedback in industry settings. Recent studies, however, report a concerning phenomenon: training two-tower models on clicks collected by well-performing production systems leads to decreased ranking performance. This paper investigates two recent explanations for this observation: confounding effects from logging policies and model identifiability issues. We theoretically analyze the identifiability conditions of two-tower models, showing that either document swaps across positions or overlapping feature distributions are required to recover model parameters from clicks. We also investigate the effect of logging policies on two-tower models, finding that they introduce no bias when models perfectly capture user behavior. However, logging policies can amplify biases when models imperfectly capture user behavior, particularly when prediction errors correlate with document placement across positions. We propose a sample weighting technique to mitigate these effects and provide actionable insights for researchers and practitioners using two-tower models.

Figures

Figures reproduced from arXiv: 2508.21698 by the authors.

Figure 1
Figure 1. Two-tower models trained on deterministic logging [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Evaluating position bias for three well-specified models matching the simulated user behavior. All models converge [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Evaluating position bias under model misspecification: (left) training a linear relevance model on non-linear user [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Two-tower models trained on deterministic logging [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Reweighting samples during training inversely to the propensity of a policy placing an item in a given position. IPS [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

45 extracted references · 42 canonical work pages

  1. [42]

    Yunan Zhang, Le Yan, Zhen Qin, Honglei Zhuang, Jiaming Shen, Xuanhui Wang, Michael Bendersky, and Marc Najork. 2023. Towards Disentangling Relevance and Bias in Unbiased Learning to Rank. In The 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD)

    work page 2023

  2. [31]

    Dan Luo, Lixin Zou, Qingyao Ai, Zhiyu Chen, Chenliang Li, Dawei Yin, and Brian D. Davison. 2024. Unbiased Learning-to-Rank Needs Unconfounded Propen- sity Estimation. In The 47th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR)

    work page 2024

  3. [1]

    Aman Agarwal, Ivan Zaitsev, Xuanhui Wang, Cheng Li, Marc Najork, and Thorsten Joachims. 2019. Estimating Position Bias without Intrusive Interven- tions. In The 12th ACM International Conference on Web Search and Data Mining (WSDM)

    work page 2019

  4. [2]

    Bruce Croft

    Qingyao Ai, Keping Bi, Cheng Luo, Jiafeng Guo, and W. Bruce Croft. 2018. Un- biased Learning to Rank with Unbiased Propensity Estimation. In The 41st In- ternational ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR)

    work page 2018

  5. [3]

    Jessa Bekker, Pieter Robberechts, and Jesse Davis. 2019. Beyond the Selected Completely at Random Assumption for Learning from Positive and Unlabeled Data. In Machine Learning and Knowledge Discovery in Databases: European Conference (ECML PKDD)

    work page 2019

  6. [4]

    Giuseppe Di Benedetto, Alexander Buchholz, Ben London, Matej Jakimov, Yan- nik Stein, Jan Malte Lichtenberg, Vito Bellini, and Matteo Ruffini. 2023. Con- textual Position Bias Estimation Using a Single Stochastic Logging Policy. (2023). https://www.amazon.science/publications/contextual-position-bias- estimation-using-a-single-stochastic-logging-policy

    work page 2023

  7. [5]

    James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman-Milne, and Qiao Zhang. 2018. JAX: Composable Transformations of Python+NumPy Programs. http://github.com/google/jax

    work page 2018

  8. [6]

    Sebastian Bruch, Xuanhui Wang, Michael Bendersky, and Marc Najork. 2019. An Analysis of the Softmax Cross Entropy Loss for Learning-to-Rank with Binary Relevance. In Proceedings of the 2019 ACM SIGIR International Conference on Theory of Information Retrieval (ICTIR)

    work page 2019

Show all 45 references
  1. [7]

    Olivier Chapelle and Yi Chang. 2011. Yahoo! Learning to Rank Challenge Overview. Journal of Machine Learning Research (JMLR) 14 (2011), 1–24

    2011

  2. [8]

    Mouxiang Chen, Chenghao Liu, Zemin Liu, Zhuo Li, and Jianling Sun. 2024. Identifiability Matters: Revealing the Hidden Recoverable Condition in Unbiased Learning to Rank. In The 41st International Conference on Machine Learning (ICML)

    2024

  3. [9]

    Xiaoshu Chen, Xiangsheng Li, Kunliang Wei, Bin Hu, Lei Jiang, Zeqian Huang, and Zhanhui Kang. 2023. Multi-Feature Integration for Perception-Dependent Examination-Bias Estimation. In Proceedings of The Sixteen ACM International Conference on Web Search and Data Mining (WSDM)

    2023

  4. [10]

    Aleksandr Chuklin, Ilya Markov, and Maarten de Rijke. 2015. Click Models for Web Search . Morgan & Claypool. https://doi.org/10.2200/ S00654ED1V01Y201507ICR043

    2015

  5. [11]

    Nick Craswell, Onno Zoeter, Michael Taylor, and Bill Ramsey. 2008. An Exper- imental Comparison of Click Position-bias Models. In Proceedings of the 2008 International Conference on Web Search and Data Mining (WSDM)

    2008

  6. [12]

    Alexander D’Amour, Peng Ding, Avi Feller, Lihua Lei, and Jasjeet Sekhon. 2021. Overlap in Observational Studies with High-dimensional Covariates. Journal of Econometrics 221, 2 (2021), 644–654. https://doi.org/10.1016/j.jeconom.2019.10. 014

    2021 doi

  7. [13]

    Domenico Dato, Claudio Lucchese, Franco Maria Nardini, Salvatore Orlando, Raffaele Perego, Nicola Tonellotto, and Rossano Venturini. 2016. Fast Ranking with Additive Ensembles of Oblivious and Non-Oblivious Regression Trees.ACM Transactions on Information Systems (TOIS) 35, 2,...

    2016

  8. [14]

    Domenico Dato, Sean MacAvaney, Franco Maria Nardini, Raffaele Perego, and Nicola Tonellotto. 2022. The Istella22 Dataset: Bridging Traditional and Neural Learning to Rank Evaluation. In The 45th International ACM SIGIR Conference on Research and Development in Information Retr...

    2022

  9. [15]

    Romain Deffayet, Philipp Hager, Jean-Michel Renders, and Maarten de Rijke

  10. [16]

    Romain Deffayet, Jean-Michel Renders, and Maarten De Rijke. 2023. Evaluating the Robustness of Click Models to Policy Distributional Shift. ACM Transactions on Information Systems (TOIS) 41, 4, Article 84 (2023)

    2023

  11. [17]

    Zhichong Fang, Aman Agarwal, and Thorsten Joachims. 2019. Intervention Harvesting for Context-Dependent Examination-Bias Estimation. In The 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR)

    2019

  12. [18]

    Huifeng Guo, Jinkai Yu, Qing Liu, Ruiming Tang, and Yuzhou Zhang. 2019. PAL: A Position-bias Aware Learning Framework for CTR Prediction in Live Recom- mender Systems. In Proceedings of the 13th ACM Conference on Recommender Systems (RecSys)

    2019

  13. [19]

    Philipp Hager, Maarten de Rijke, and Onno Zoeter. 2023. Contrasting Neural Click Models and Pointwise IPS Rankers. In The 45th European Conference on Information Retrieval (ECIR)

    2023

  14. [20]

    Philipp Hager, Onno Zoeter, and Maarten de Rijke. 2025. Unidentified and Confounded? Understanding Two-Tower Models for Unbiased Learning to Rank. In The International ACM SIGIR Conference on Innovative Concepts and Theories in Information Retrieval (ICTIR)

    2025

  15. [21]

    Malay Haldar, Prashant Ramanathan, Tyler Sax, Mustafa Abdool, Lanbo Zhang, Aamir Mansawala, Shulin Yang, Bradley Turnbull, and Junshuo Liao. 2020. Im- proving Deep Learning for Airbnb Search. InThe 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (WSDM)

    2020

  16. [22]

    Jonathan Heek, Anselm Levskaya, Avital Oliver, Marvin Ritter, Bertrand Ronde- pierre, Andreas Steiner, and Marc van Zee. 2024. Flax: A Neural Network Library and Ecosystem for JAX . http://github.com/google/flax

    2024

  17. [23]

    Rolf Jagerman, Harrie Oosterhuis, and Maarten de Rijke. 2019. To Model or to Intervene: A Comparison of Counterfactual and Online Learning to Rank from User Interactions. In The International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR)

    2019

  18. [24]

    Rolf Jagerman, Xuanhui Wang, Honglei Zhuang, Zhen Qin, Michael Bendersky, and Marc Najork. 2022. Rax: Composable Learning-to-Rank Using JAX. In Pro- ceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD)

    2022

  19. [25]

    Thorsten Joachims, Adith Swaminathan, and Tobias Schnabel. 2017. Unbiased Learning-to-Rank with Biased Feedback. In Proceedings of the Tenth ACM Inter- national Conference on Web Search and Data Mining (WSDM)

    2017

  20. [26]

    Kirill Khrylchenko and Alexander Fritzler. 2023. Personalized Transformer- based Ranking for e-Commerce at Yandex. arXiv:2310.03481 [cs.IR] https: //arxiv.org/abs/2310.03481

    2023 arXiv

  21. [27]

    Daphne Koller and Nir Friedman. 2009. Probabilistic Graphical Models: Principles and Techniques. MIT press

    2009

  22. [28]

    Arthur Lewbel. 2019. The Identification Zoo: Meanings of Identification in Econometrics. Journal of Economic Literature 57, 4 (2019), 835–903

    2019

  23. [29]

    Muthukrishnan, Vishwa Vinay, and Zheng Wen

    Shuai Li, Yasin Abbasi-Yadkori, Branislav Kveton, S. Muthukrishnan, Vishwa Vinay, and Zheng Wen. 2018. Offline Evaluation of Ranking Policies with Click Models. In The 24th ACM SIGKDD International Conference on Knowledge Discov- ery and Data Mining (KDD)

    2018

  24. [30]

    Ilya Loshchilov and Frank Hutter. 2019. Decoupled Weight Decay Regularization. (2019). arXiv:1711.05101 [cs.LG]

    2019 arXiv

  25. [32]

    Maddison, Andriy Mnih, and Yee Whye Teh

    Chris J. Maddison, Andriy Mnih, and Yee Whye Teh. 2017. The Concrete Distribu- tion: A Continuous Relaxation of Discrete Random Variables. InThe International Conference on Learning Representations (ICLR)

    2017

  26. [33]

    Harrie Oosterhuis. 2022. Reaching the End of Unbiasedness: Uncovering Implicit Limitations of Click-Based Learning to Rank. In Proceedings of the 2022 ACM SIGIR International Conference on Theory of Information Retrieval (SIGIR)

    2022

  27. [34]

    Harrie Oosterhuis and Maarten de Rijke. 2020. Policy-Aware Unbiased Learning to Rank for Top-k Rankings. In The International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR)

    2020

  28. [35]

    Tao Qin and Tie-Yan Liu. 2013. Introducing LETOR 4.0 Datasets. arXiv:1306.2597 [cs.IR]

    2013 arXiv

  29. [36]

    Zhen Qin, Le Yan, Honglei Zhuang, Yi Tay, Rama Kumar Pasumarthi, Xuanhui Wang, Mike Bendersky, and Marc Najork. 2021. Are Neural Rankers still Out- performed by Gradient Boosted Decision Trees?. In International Conference on Learning Representations (ICLR)

    2021

  30. [37]

    Filip Radlinski and Thorsten Joachims. 2006. Minimally Invasive Randomization for Collecting Unbiased Preferences from Clickthrough Logs. InThe 21st National Conference on Artificial Intelligence (AAAI)

    2006

  31. [38]

    Yuta Saito, Suguru Yaginuma, Yuta Nishino, Hayato Sakata, and Kazuhide Nakata

  32. [39]

    1989.Learning from Delayed Rewards

    Christopher John Cornish Hellaby Watkins. 1989.Learning from Delayed Rewards. Ph. D. Dissertation. University of Cambridge

    1989

  33. [40]

    Rafael Wilms, Eveline Mäthner, Lothar Winnen, and Ralf Lanwehr. 2021. Omitted Variable Bias: A Threat to Estimating Causal Relationships.Methods in Psychology 5 (2021), 100075. https://doi.org/10.1016/j.metip.2021.100075

    2021

  34. [41]

    Le Yan, Zhen Qin, Honglei Zhuang, Xuanhui Wang, Michael Bendersky, and Marc Najork. 2022. Revisiting Two-tower Models for Unbiased Learning to Rank. In Proceedings of the 45th International ACM SIGIR Conference on Research and Unidentified and Confounded? Understanding Two-Tow...

    2022

  35. [43]

    Zhe Zhao, Lichan Hong, Li Wei, Jilin Chen, Aniruddh Nath, Shawn Andrews, Aditee Kumthekar, Maheswaran Sathiamoorthy, Xinyang Yi, and Ed Chi. 2019. Recommending What Video to Watch Next: A Multitask Ranking System. In Proceedings of the 13th ACM Conference on Recommender System...

    2019

  36. [2020]

    In International Conference on Web Search and Data Mining (WSDM)

    Unbiased Recommender Learning from Missing-Not-At-Random Implicit Feedback. In International Conference on Web Search and Data Mining (WSDM)

  37. [2023]

    In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR)

    An Offline Metric for the Debiasedness of Click Models. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR)

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.