Pith. sign in

REVIEW 4 major objections 4 minor 65 references

The paper argues that an exposure-based, auto-differentiable estimator makes reinforcement learning for ranking faster, stabler, and easier to use than existing custom-gradient methods—while an appendix retracts the estimator's derivation a

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

An exposure-based policy-gradient estimator for ranking is proposed; the appendix retracts the key derivation, and the reported experiments were not rerun with the corrected estimator.

T0 review reviewed 2026-08-01 challenge →

load-bearing objection The paper retracts its own main estimator in an appendix, so the headline experiments are invalidated, but the marginalization and baseline-correction ideas have real merit. the 4 major comments →

arxiv 2607.18689 v1 pith:XG7ARARO submitted 2026-07-21 cs.LG cs.IR

Exposure-Based Reinforcement Learning to Rank

Harrie Oosterhuis , Rolf Jagerman , Zhen Qin , Xuanhui Wang This is my paper
classification cs.LG cs.IR
keywords Learning to RankReinforcement LearningPolicy GradientExposurePlackett-LuceBaseline CorrectionMarginalizationAuto-Differentiation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The authors set out to make reinforcement learning for learning-to-rank practical: instead of writing custom gradients, they estimate the gradient of a document-exposure distribution from sampled Plackett-Luce rankings, using leave-one-out baselines and partial marginalization to cut variance. Their headline claim is that this exposure-based estimator converges much faster and to higher NDCG@10 than the existing PL-Rank custom gradient, and that it works in a plug-and-play way with auto-differentiation for any exposure-based loss. The paper's own appendix, however, retracts the key derivation: subtracting a per-document baseline from each position weight inside the sum over positions is not unbiased, because a document may be absent from the ranking. The experiments in the main text use the uncorrected estimator and were not rerun, so the empirical claims rest on a known-incorrect gradient.

Core claim

On its own terms, the paper claims the policy-gradient problem for ranking can be pushed behind the exposure distribution: estimate each document's expected exposure by marginalizing over sampled prefixes, correct with leave-one-out baselines, and let the chain rule hand any differentiable exposure-based loss its gradient. The authors report that this exposure-based estimator (Eqs. 33–34) reaches significantly higher NDCG@10 at convergence than PL-Rank and other RL-for-LTR estimators for N≥5, converges in about a third of the epochs, and stays stable where PL-Rank's custom gradient diverges under 32-bit floats. Appendix A retracts Eq. 33: since Σ_k P(y_k=d|π) ≤ 1, the per-position baseline d

What carries the argument

The central object is the exposure-based gradient estimator (Eqs. 33–34): for each sampled ranking, it sums over positions k the baseline-corrected position weight θ_k−b_d multiplied by the derivative of the placement probability plus the placement probability times the log-prefix derivative. This is wrapped in a function that returns the estimated exposure distribution, so any loss written as a differentiable function of exposure (relevance utility, fairness ratios, KL-divergence distillation) gets an automatic gradient through the chain rule. The abstraction's correctness collapses on the baseline: the retraction shows the exposure estimate has K+1 outcomes—placement in one of K positions,

Load-bearing premise

That the leave-one-out baseline b_d cancels inside the per-position sum, leaving an unbiased gradient of expected exposure; the appendix itself shows this fails because a document can be unplaced, so Σ_k P(y_k=d|π) ≤ 1.

What would settle it

For a small Plackett-Luce model (e.g., 3 documents, K=2), compute the exact exposure gradient by finite differences and compare it to the gradient from Eq. 33; the mismatch equals the omitted unplaced-document term identified in Eq. 36.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If the estimator were unbiased, any exposure-based ranking objective could be optimized in a few lines of differentiable code, removing the need for custom-gradient implementations.
  • The reported results imply that with N≥5 samples, the exposure-based estimator matches or beats standard and placement estimators in speed and converged NDCG@10, and with N≥100 on MSLR the gains are statistically significant.
  • The paper claims PL-Rank's unnormalized intermediate quantities make it numerically unstable when run for many epochs at 32-bit precision, a failure mode not visible in earlier short runs.
  • The corrected estimator preserves the plug-and-play exposure abstraction and adds an unplaced-document probability term, but its empirical performance is unverified in this paper.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A natural next step is to rerun the NDCG@10 comparisons with the corrected estimator; there is no evidence yet that the reported speed and convergence advantages survive.
  • The same baseline-vs-absence failure pattern may affect any estimator that treats exposure as a sum over positions without accounting for non-placement, suggesting a general caution for exposure-based RL formulations.
  • If the PL-Rank instability claim holds, previously published CPU-era comparisons of RL-to-rank estimators may need re-examination under GPU and 32-bit float conditions.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes several policy-gradient estimators for learning-to-rank (LTR) with Plackett-Luce ranking models, focusing on variance reduction via baseline corrections and partial marginalization, and introduces an 'exposure-based' estimator that places gradient estimation behind an exposure distribution to enable plug-and-play auto-differentiation. The authors claim that this exposure-based estimator converges faster and achieves significantly higher NDCG@10 at convergence than PL-Rank and other RL-for-LTR estimators on two benchmark datasets, and that it simplifies optimization of exposure-based losses. The paper also reports instability of PL-Rank under 32-bit floating point and provides implementation listings.

Significance. If the central claim were correct, the exposure-based estimator would be a practical and important contribution: it would let practitioners optimize any differentiable exposure-based loss without custom gradient implementations, while remaining computationally competitive on GPUs. The paper is generally well written and transparent, and it provides code listings and publicly available implementations, which are strengths. However, the exposure-based estimator as presented in the main text (Eq. 33) is explicitly retracted in Appendix A, and the main experiments were run with that incorrect estimator. The corrected estimator was not evaluated, so the headline experimental results do not support the paper's central claim. The ancillary contributions — baseline corrections and marginalization for utility-based estimators — appear sound, but they do not rescue the exposure-based claim. The Appendix's explicit acknowledgment that 'the experiments could not be repeated with the corrected estimator' is a decisive missing-support element.

major comments (4)
  1. [Appendix A, Eq. 36] The paper itself states that Eq. 33 is incorrect: the leave-one-out baseline b_d does not cancel because the probability of a document not being placed is omitted (Eq. 36 shows Σ_k P(y_k=d|π) ≤ 1). Since Eq. 33 is the estimator used in all main experiments (Section 7, Listing 1), the reported NDCG@10 improvements in Tables 1–2 and Figures 2–5 are based on a biased gradient estimator. This directly invalidates the central claim in Section 7.3 that 'the exposure-based estimator provides significant improvements in performance at convergence and learning speed over other RL for LTR estimators, as long as N≥5.'
  2. [Section 7.3 / Conclusion] The headline conclusion rests on experiments that were not rerun with the corrected estimator. Appendix A provides a corrected formula (Eq. 39) and implementation (Listing 3), but states that the experiments could not be repeated before the deadline. Without experiments using the corrected estimator, there is no empirical evidence that the exposure-based estimator outperforms the other methods; the published learning curves and significance tests are not trustworthy evidence for the proposed method.
  3. [Section 5.1, Eq. 33-34] The derivation of the exposure-based gradient estimator is load-bearing for the entire method. The chain rule (Eq. 34) requires an unbiased estimate of ∂θ_d/∂f. Appendix A shows that the proposed estimator is biased. The corrected estimator introduces an additional term for the document not being placed in any position. Consequently, the claimed 'seamless plug-and-play integration with auto-differentiation' (Abstract, Section 5.2) is not achieved by the method evaluated in the paper.
  4. [Listing 1 vs Listing 3] Listing 1 implements the incorrect estimator (Eq. 33) and is the one used in the experiments. Listing 3 implements the corrected estimator (Eq. 39). The paper never reports results with Listing 3. The gap between the submitted main text and its own appendix is an explicit limitation: the central empirical evidence is retracted but the main-text claims are not revised. A revised manuscript would need new experiments, not just a rewording.
minor comments (4)
  1. [Section 7.5] The text says 'when we consider the loss values reached in Table 1' but should refer to Table 2, which reports the non-traditional loss values.
  2. [Section 5.1, Eq. 35] The notation 'δθ_d/δf' uses δ both as functional derivative and as partial derivative; consider standard partial-derivative notation to avoid ambiguity.
  3. [Section 7.1, Fig. 2] The caption says 'without baseline corrections' but PL-Rank never has baseline corrections; it might be clearer to state 'with b=0 for all estimators' explicitly.
  4. [Section 4.4] The phrase 'for each sampled ranking of length K, D rankings of length one are considered...' is slightly confusing because the initial prefix is length zero; clarify that the marginalization enumerates placements from each prefix.

Circularity Check

0 steps flagged

No circularity found; the Appendix A retraction of Eq. 33 is a correctness error, not a circular reduction.

full rationale

The derivation chain is self-contained: Eqs. 22-35 build the estimators from standard REINFORCE/log-derivative identities and Plackett-Luce sampling. Baseline corrections are standard leave-one-out estimates (Eq. 20), and the marginalization estimators are derived by replacing sampled placements with exact sums over π(d|prefix) (Eqs. 26-30). The exposure-based estimator is transparently constructed as a surrogate: Listing 1 explicitly states the 'trick: value is expected exposure but gradient comes from loss', so no fitted parameter or target quantity is renamed as a prediction. Self-citations to Oosterhuis [37,38] are to a prior method that the paper directly compares against and criticizes; no load-bearing claim is imported from those citations. Appendix A does state that Eq. 33 is incorrect: 'In contrast with the utility estimates in Section 4 where one can substract the baseline per position placement, for exposure estimates this is incorrect', and later says 'the experiments could not be repeated with the corrected estimator.' That is a serious correctness/validity problem for the headline empirical claims, but it is not circularity: the flawed step is an algebraic error in an unbiasedness proof, not a definitional identity, fitted-input-as-prediction, self-citation chain, imported uniqueness, or renaming that forces the output to equal its inputs by construction. Therefore the circularity score is 0.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 0 invented entities

No new physical or ontological entities are postulated; the exposure abstraction is a computational reparameterization of existing expected-exposure quantities. The flawed estimator is a mathematical error, not an invented entity.

free parameters (3)
  • PL-Rank network size (2-layer, 32 units, sigmoid) = 32 units per hidden layer; sigmoid activation
    Section 6: 'except for PL-Rank which reached better results with a two hidden-layer network with 32 units each and sigmoid activations.' This hand-chosen architecture is a confound in the PL-Rank stability comparison.
  • Float precision = 32-bit floats on GPU
    Section 7.1: original PL-Rank used 64-bit floats on CPU; the authors chose 32-bit, which they believe exacerbated PL-Rank's instability. This is a deliberate experimental condition, not a fitted estimator parameter.
  • Smoothing epsilon for L_frac-fair = 0.01
    Section 6: 'we add 0.01 to zero relevance labels to avoid divisions by zero'; minor preprocessing constant for the fairness experiment.
axioms (4)
  • domain assumption Rankings are generated by a Plackett-Luce model, and sampling from it gives unbiased estimates of expectations.
    Section 3.3 and 4: every estimator is built on PL sampling; this is standard in LTR but is a modeling assumption the analysis depends on.
  • ad hoc to paper The exposure-based gradient estimator (Eq. 33) is an unbiased estimate of δθ_d/δf.
    Section 5.1 derives it; Appendix A retracts it. Eq. 36 shows the baseline does not cancel because the document may not be placed in the top K. This is the load-bearing premise, now known false.
  • domain assumption Leave-one-out baseline corrections are unbiased and reduce variance without changing the optimum.
    Section 3.4 cites Kool et al. and standard RL; the paper's standard, placement, and marginalization estimators rely on it.
  • standard math Exposure-based losses are differentiable functions of the exposure vector, so the multivariable chain rule (Eq. 34) applies.
    Section 5.1 invokes the chain rule for L(θπ|q); this is standard calculus for the smooth losses considered.

reviewed 2026-08-01 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Exposure-Based Reinforcement Learning to Rank." pith.science (2026). https://pith.science/paper/XG7ARARO

@misc{pith2026260718689,
  author       = {Pith},
  title        = {Pith review of: Exposure-Based Reinforcement Learning to Rank},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XG7ARARO}},
  note         = {Machine review of arXiv:2607.18689}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Reinforcement learning (RL) methods for learning-to-rank (LTR) can optimize (almost) any ranking goal, e.g., from precision or discounted cumulative gain to fairness-of-exposure or ranking distillation. However, standard RL is ineffective and computationally costly due to the enormous action space in LTR settings. Existing methods reach computational efficiency through custom gradient computation algorithms, but they are very complex to implement and often clash with auto-differentiation. Consequently, existing RL for LTR is not attractive to many practitioners. We reconsider RL for LTR while actively avoiding reliance on custom gradients. Contrary to the existing approaches, we focus on variance reduction and GPU computation. In doing so, we discover that high sample-efficiency can be reached through baseline corrections and partial marginalization. Furthermore, we propose an abstraction that places gradient estimation behind a document-exposure distribution, this enables seamless plug-and-play integration with auto-differentiation. Thereby, one only has to implement a loss as a differentiable function of exposure and RL for LTR can optimize it using auto-differentiation. Our experimental results reveal that our new exposure-based RL for LTR approach converges considerably faster and at significantly higher ranking performance than existing custom gradients, with no additional costs in computation time when using GPUs. In contrast, existing custom gradients result in severe stability issues when converging over many epochs, which never occur for our methods. Thus, we considerably improve RL for LTR methodology by increasing its effectiveness, efficiency, and ease of application.

Figures

Figures reproduced from arXiv: 2607.18689 by Harrie Oosterhuis, Rolf Jagerman, Xuanhui Wang, Zhen Qin.

Figure 1
Figure 1. Figure 1: Example of a sampled ranking of a five-document [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Performance of different estimators without baseline corrections (i.e., 𝑏 = 0) for varying numbers of samples (𝑁) per gradient estimate. Y-axis: NDCG@10 on test set; X-axis: training epochs. Top-row: MSLR-Web30k; Bottom-row: Istella-S. N = 2 N = 10 N = 100 N = 1000 MSLR - NDCG@10 0.465 0.470 0.475 0.480 Istella - NDCG@10 0 2500 5000 7500 10000 12500 15000 0.73 0.74 0 2500 5000 7500 10000 12500 15000 0 2500… view at source ↗
Figure 3
Figure 3. Figure 3: Performance of different estimators with baseline corrections for varying numbers of samples (𝑁) per estimate. Y-axis: NDCG@10 on test set; X-axis: training epochs. Top-row: MSLR-Web30k; Bottom-row: Istella-S. the other estimators has stability issues while also being constrained to 32 bit floats. Importantly, PL-Rank stores many unnormalized quantities in its intermediate steps, whereas the other methods … view at source ↗
Figure 4
Figure 4. Figure 4: Seconds (Y-axis) used per training epoch by different [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Comparison of exposure-based estimator and baseline estimator for optimizing various non-traditional exposure [PITH_FULL_IMAGE:figures/full_fig_p009_5.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

65 extracted references · 4 linked inside Pith

  1. [1]

    Martín Abadi. 2016. TensorFlow: learning functions at scale. InProceedings of the 21st ACM SIGPLAN international conference on functional programming. 1–1

    2016

  2. [2]

    Aman Agarwal, Kenta Takatsu, Ivan Zaitsev, and Thorsten Joachims. 2019. A general framework for counterfactual learning-to-rank. InProceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. 5–14

    2019

  3. [3]

    Brian Bartell, Garrison W Cottrell, and Richard Belew. 1995. Learning to retrieve information. InProceedings of the Swedish Conference on Connectionism. 27

    1995

  4. [4]

    Biega, Krishna P

    Asia J. Biega, Krishna P. Gummadi, and Gerhard Weikum. 2018. Equity of Attention: Amortizing Individual Fairness in Rankings. InThe 41st International ACM SIGIR Conference on Research & Development in Information Retrieval(Ann Arbor, MI, USA)(SIGIR ’18). Association for Computing Machinery, New York, NY, USA, 405–414. doi:10.1145/3209978.3210063

    arXiv 2018

  5. [5]

    Pierre Blanchard, Desmond J Higham, and Nicholas J Higham. 2020. Ac- curately computing the log-sum-exp and softmax functions.IMA J. Numer. Anal.41, 4 (08 2020), 2311–2330. arXiv:https://academic.oup.com/imajna/article- pdf/41/4/2311/40758053/draa038.pdf doi:10.1093/imanum/draa038

    doi:10.1093/imanum/draa038 2020

  6. [6]

    James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman-Milne, et al. 2018. JAX: composable transformations of Python+ NumPy programs. (2018)

    2018

  7. [7]

    Sebastian Bruch, Shuguang Han, Michael Bendersky, and Marc Najork. 2020. A Stochastic Treatment of Learning to Rank Scoring Functions. InProceedings of the 13th International Conference on Web Search and Data Mining(Houston, TX, USA)(WSDM ’20). Association for Computing Machinery, New York, NY, USA, 61–69. doi:10.1145/3336191.3371844

    arXiv 2020

  8. [8]

    Sebastian Bruch, Masrour Zoghi, Michael Bendersky, and Marc Najork. 2019. Revisiting Approximate Metric Optimization in the Age of Deep Neural Networks. InProceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval(Paris, France)(SIGIR’19). Association for Computing Machinery, New York, NY, USA, 1241–124...

    doi:10.1145/3331184 2019

  9. [9]

    Christopher Burges, Robert Ragno, and Quoc Le. 2006. Learning to rank with nonsmooth cost functions.Advances in neural information processing systems19 (2006)

    2006

  10. [10]

    Chris Burges, Tal Shaked, Erin Renshaw, Ari Lazier, Matt Deeds, Nicole Hamilton, and Greg Hullender. 2005. Learning to rank using gradient descent. InProceedings of the 22nd International Conference on Machine Learning(Bonn, Germany)(ICML ’05). Association for Computing Machinery, New York, NY, USA, 89–96. doi:10. 1145/1102351.1102363

    arXiv 2005

  11. [11]

    Christopher JC Burges. 2010. From ranknet to lambdarank to lambdamart: An overview.Learning11, 23-581 (2010), 81

    2010

  12. [12]

    Zhe Cao, Tao Qin, Tie-Yan Liu, Ming-Feng Tsai, and Hang Li. 2007. Learning to rank: from pairwise approach to listwise approach. InProceedings of the 24th International Conference on Machine Learning(Corvalis, Oregon, USA)(ICML ’07). Association for Computing Machinery, New York, NY, USA, 129–136. doi:10. 1145/1273496.1273513

    arXiv 2007

  13. [13]

    Wei Chu, Zoubin Ghahramani, and Christopher KI Williams. 2005. Gaussian processes for ordinal regression.Journal of machine learning research6, 7 (2005)

    2005

  14. [14]

    Koby Crammer and Yoram Singer. 2001. Pranking with ranking.Advances in neural information processing systems14 (2001)

    2001

  15. [15]

    Peter Dayan. 1991. Reinforcement comparison. InConnectionist Models. Elsevier, 45–51

    1991

  16. [16]

    Fernando Diaz, Bhaskar Mitra, Michael D Ekstrand, Asia J Biega, and Ben Carterette. 2020. Evaluating stochastic rankings with expected exposure. In Proceedings of the 29th ACM international conference on information & knowledge management. 275–284

    2020

  17. [17]

    Norbert Fuhr. 1989. Optimum polynomial retrieval functions based on the probability ranking principle.ACM Trans. Inf. Syst.7, 3 (July 1989), 183–204. doi:10.1145/65943.65944

    arXiv 1989

  18. [18]

    Evan Greensmith, Peter L Bartlett, and Jonathan Baxter. 2004. Variance reduction techniques for gradient estimates in reinforcement learning.Journal of Machine Learning Research5, Nov (2004), 1471–1530

    2004

  19. [19]

    Aditya Grover, Eric Wang, Aaron Zweig, and Stefano Ermon. 2019. Stochastic optimization of sorting networks via continuous relaxations.arXiv preprint arXiv:1903.08850(2019)

    Pith/arXiv arXiv 2019

  20. [20]

    Shashank Gupta, Olivier Jeunen, Harrie Oosterhuis, and Maarten de Rijke. 2024. Optimal Baseline Corrections for Off-Policy Contextual Bandits. InProceedings of the 18th ACM Conference on Recommender Systems(Bari, Italy)(RecSys ’24). Association for Computing Machinery, New York, NY, USA, 722–732. doi:10. 1145/3640457.3688105

    arXiv 2024

  21. [21]

    Kazuyuki Hara, Daisuke Saito, and Hayaru Shouno. 2015. Analysis of function of rectified linear unit used in deep learning. In2015 International Joint Conference on Neural Networks (IJCNN). 1–8. doi:10.1109/IJCNN.2015.7280578

    arXiv 2015

  22. [22]

    Katja Hofmann, Shimon Whiteson, and Maarten de Rijke. 2011. A probabilistic method for inferring preferences from clicks. InProceedings of the 20th ACM International Conference on Information and Knowledge Management(Glasgow, Scotland, UK)(CIKM ’11). Association for Computing Machinery, New York, NY, USA, 249–258. doi:10.1145/2063576.2063618

    arXiv 2011

  23. [23]

    Deborah Hughes-Hallett, Andrew M Gleason, and William G McCallum. 2020. Calculus: Single and multivariable. John Wiley & Sons

    2020

  24. [24]

    Rolf Jagerman, Xuanhui Wang, Honglei Zhuang, Zhen Qin, Michael Bendersky, and Marc Najork. 2022. Rax: composable learning-to-rank using Jax. InPro- ceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 3051–3060

    2022

  25. [25]

    Kalervo Järvelin and Jaana Kekäläinen. 2002. Cumulated gain-based evaluation of IR techniques.ACM Transactions on Information Systems (TOIS)20, 4 (2002), 422–446

    2002

  26. [26]

    Thorsten Joachims. 2002. Optimizing search engines using clickthrough data. In Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining(Edmonton, Alberta, Canada)(KDD ’02). Association for Computing Machinery, New York, NY, USA, 133–142. doi:10.1145/775047. 775067

    doi:10.1145/775047 2002

  27. [27]

    Thorsten Joachims, Adith Swaminathan, and Tobias Schnabel. 2017. Unbiased Learning-to-Rank with Biased Feedback. InProceedings of the Tenth ACM Interna- tional Conference on Web Search and Data Mining(Cambridge, United Kingdom) (WSDM ’17). Association for Computing Machinery, New York, NY, USA, 781–789. doi:10.1145/3018661.3018699

    arXiv 2017

  28. [28]

    Alexandros Karatzoglou, Linas Baltrunas, and Yue Shi. 2013. Learning to rank for recommender systems. InProceedings of the 7th ACM Conference on Recommender Systems. 493–494

    2013

  29. [29]

    Diederik P Kingma. 2014. Adam: A method for stochastic optimization.arXiv preprint arXiv:1412.6980(2014)

    Pith/arXiv arXiv 2014

  30. [30]

    Ömer Kırnap, Fernando Diaz, Asia Biega, Michael Ekstrand, Ben Carterette, and Emine Yilmaz. 2021. Estimation of Fair Ranking Metrics with Incomplete Judgments. InProceedings of the Web Conference 2021(Ljubljana, Slovenia)(WWW ’21). Association for Computing Machinery, New York, NY, USA, 1065–1075. doi:10.1145/3442381.3450080

    arXiv 2021

  31. [31]

    Wouter Kool, Herke van Hoof, and Max Welling. 2019. Buy 4 REINFORCE Samples, Get a Baseline for Free!. InDeepRLStructPred@ICLR. https://api. semanticscholar.org/CorpusID:198489118

    2019

  32. [32]

    John Langford and Tong Zhang. 2007. The epoch-greedy algorithm for contextual multi-armed bandits.Advances in neural information processing systems20, 1 (2007), 96–1

    2007

  33. [33]

    Tie-Yan Liu et al. 2009. Learning to rank for information retrieval.Foundations and Trends®in Information Retrieval3, 3 (2009), 225–331

    2009

  34. [34]

    Claudio Lucchese, Franco Maria Nardini, Salvatore Orlando, Raffaele Perego, Fabrizio Silvestri, and Salvatore Trani. 2016. Post-Learning Optimization of Tree Ensembles for Efficient Ranking. InProceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval(Pisa, Italy)(SIGIR ’16). Association for Computing M...

    arXiv 2016

  35. [35]

    R Duncan Luce. 2012. Individual choice behavior: A theoretical analysis. (2012)

    2012

  36. [36]

    2020.Learning from User Interactions with Rankings: A Uni- fication of the Field.Ph

    Harrie Oosterhuis. 2020.Learning from User Interactions with Rankings: A Uni- fication of the Field.Ph. D. Dissertation. Informatics Institute, University of Amsterdam. https://harrieo.github.io//publication/2021-phd-thesis

    2020

  37. [37]

    Harrie Oosterhuis. 2021. Computationally Efficient Optimization of Plackett-Luce Ranking Models for Relevance and Fairness. InProceedings of the 44th Interna- tional ACM SIGIR Conference on Research and Development in Information Retrieval (Virtual Event, Canada)(SIGIR ’21). Association for Computing Machinery, New York, NY, USA, 1023–1032. doi:10.1145/34...

    arXiv 2021

  38. [38]

    Harrie Oosterhuis. 2022. Learning-to-Rank at the Speed of Sampling: Plackett- Luce Gradient Estimation with Minimal Computational Complexity. InProceed- ings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval(Madrid, Spain)(SIGIR ’22). Association for Computing Machinery, New York, NY, USA, 2266–2271. doi:1...

    arXiv 2022

  39. [39]

    Harrie Oosterhuis. 2023. Doubly Robust Estimation for Correcting Position Bias in Click Feedback for Unbiased Learning to Rank.ACM Trans. Inf. Syst.41, 3, Article 61 (Feb. 2023), 33 pages. doi:10.1145/3569453

    doi:10.1145/3569453 2023

  40. [40]

    Harrie Oosterhuis and Maarten de Rijke. 2018. Differentiable Unbiased Online Learning to Rank. InProceedings of the 27th ACM International Conference on Information and Knowledge Management(Torino, Italy)(CIKM ’18). Association for Computing Machinery, New York, NY, USA, 1293–1302. doi:10.1145/3269206. 3271686

    doi:10.1145/3269206 2018

  41. [41]

    Harrie Oosterhuis and Maarten de Rijke. 2018. Ranking for Relevance and Display Preferences in Complex Presentation Layouts. InThe 41st International ACM SIGIR Conference on Research & Development in Information Retrieval(Ann Arbor, MI, USA)(SIGIR ’18). Association for Computing Machinery, New York, NY, USA, 845–854. doi:10.1145/3209978.3209992

    arXiv 2018

  42. [42]

    Harrie Oosterhuis and Maarten de Rijke. 2020. Taking the Counterfactual Online: Efficient and Unbiased Online Evaluation for Ranking. InProceedings of the 2020 ACM SIGIR on International Conference on Theory of Information Retrieval(Virtual Event, Norway)(ICTIR ’20). Association for Computing Machinery, New York, NY, USA, 137–144. doi:10.1145/3409256.3409...

    arXiv 2020

  43. [43]

    Rama Kumar Pasumarthi, Sebastian Bruch, Xuanhui Wang, Cheng Li, Michael Bendersky, Marc Najork, Jan Pfeifer, Nadav Golbandi, Rohan Anil, and Stephan Wolf. 2019. Tf-ranking: Scalable tensorflow library for learning-to-rank. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2970–2978

    2019

  44. [44]

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. 2019. Pytorch: An imperative style, high-performance deep learning library.Advances in neural information processing systems32 (2019)

    2019

  45. [45]

    Robin L Plackett. 1975. The analysis of permutations.Journal of the Royal Statistical Society Series C: Applied Statistics24, 2 (1975), 193–202

    1975

  46. [46]

    Przemysław Pobrotyn and Radosław Białobrzeski. 2021. Neuralndcg: Direct optimisation of a ranking metric via differentiable relaxation of sorting.arXiv preprint arXiv:2102.07831(2021)

    Pith/arXiv arXiv 2021

  47. [47]

    Tao Qin and Tie-Yan Liu. 2013. Introducing LETOR 4.0 datasets.arXiv preprint arXiv:1306.2597(2013)

    Pith/arXiv arXiv 2013

  48. [48]

    Tao Qin, Tie-Yan Liu, and Hang Li. 2010. A general approximation framework for direct optimization of information retrieval measures.Information retrieval 13 (2010), 375–397

    2010

  49. [49]

    Zhen Qin, Rolf Jagerman, Rama Kumar Pasumarthi, Honglei Zhuang, He Zhang, Aijun Bai, Kai Hui, Le Yan, and Xuanhui Wang. 2023. RD-Suite: A Benchmark for Ranking Distillation. InAdvances in Neu- ral Information Processing Systems, A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (Eds.), Vol. 36. Curran Associates, Inc., 35748–35760. https...

    2023

  50. [50]

    Sashank Reddi, Rama Kumar Pasumarthi, Aditya Menon, Ankit Singh Rawat, Felix Yu, Seungyeon Kim, Andreas Veit, and Sanjiv Kumar. 2021. RankDistil: Knowledge Distillation for Ranking. InProceedings of The 24th International Conference on Artificial Intelligence and Statistics (Proceedings of Machine Learning Research, Vol. 130), Arindam Banerjee and Kenji F...

    2021

  51. [51]

    Anne Schuth, Robert-Jan Bruintjes, Fritjof Buüttner, Joost van Doorn, Carla Groenland, Harrie Oosterhuis, Cong-Nguyen Tran, Bas Veeling, Jos van der Velde, Roger Wechsler, David Woudenberg, and Maarten de Rijke. 2015. Probabilistic Multileave for Online Retrieval Evaluation. InProceedings of the 38th International ACM SIGIR Conference on Research and Deve...

    arXiv 2015

  52. [52]

    Ashudeep Singh and Thorsten Joachims. 2018. Fairness of exposure in rankings. InProceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining. 2219–2228

    2018

  53. [53]

    Ashudeep Singh and Thorsten Joachims. 2019. Policy learning for fairness in ranking.Advances in neural information processing systems32 (2019)

    2019

  54. [54]

    Richard S Sutton, Andrew G Barto, et al. 1999. Reinforcement learning.Journal of Cognitive Neuroscience11, 1 (1999), 126–134

    1999

  55. [55]

    Jiaxi Tang and Ke Wang. 2018. Ranking Distillation: Learning Compact Ranking Models With High Performance for Recommender System. InProceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Min- ing(London, United Kingdom)(KDD ’18). Association for Computing Machinery, New York, NY, USA, 2289–2298. doi:10.1145/3219819.3220021

    arXiv 2018

  56. [56]

    Michael Taylor, John Guiver, Stephen Robertson, and Tom Minka. 2008. SoftRank: optimizing non-smooth rank metrics. InProceedings of the 2008 International Conference on Web Search and Data Mining(Palo Alto, California, USA)(WSDM ’08). Association for Computing Machinery, New York, NY, USA, 77–86. doi:10. 1145/1341531.1341544

    arXiv 2008

  57. [57]

    Aleksei Ustimenko and Liudmila Prokhorenkova. 2020. StochasticRank: Global Optimization of Scale-Free Discrete Functions. InProceedings of the 37th In- ternational Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 119), Hal Daumé III and Aarti Singh (Eds.). PMLR, 9669–9679. https://proceedings.mlr.press/v119/ustimenko20a.html

    2020

  58. [58]

    Vladimir Vapnik. 1991. Principles of risk minimization for learning theory. Advances in neural information processing systems4 (1991)

    1991

  59. [59]

    Xuanhui Wang, Cheng Li, Nadav Golbandi, Michael Bendersky, and Marc Najork

  60. [60]

    Ronald J Williams. 1992. Simple statistical gradient-following algorithms for connectionist reinforcement learning.Machine learning8 (1992), 229–256

    1992

  61. [61]

    RJ Willianms. 1988. Toward a theory of reinforcement-learning connectionist systems.Technical Report(1988)

    1988

  62. [62]

    Mingrui Wu, Yi Chang, Zhaohui Zheng, and Hongyuan Zha. 2009. Smoothing DCG for learning to rank: A novel approach using smoothed hinge functions. In Proceedings of the 18th ACM conference on Information and knowledge management. 1923–1926

    2009

  63. [63]

    Fen Xia, Tie-Yan Liu, Jue Wang, Wensheng Zhang, and Hang Li. 2008. Listwise approach to learning to rank: theory and algorithm. InProceedings of the 25th International Conference on Machine Learning(Helsinki, Finland)(ICML ’08). Association for Computing Machinery, New York, NY, USA, 1192–1199. doi:10. 1145/1390156.1390306 12

    arXiv 2008

  64. [2018]

    InProceedings of the 27th ACM international conference on information and knowledge manage- ment

    The lambdaloss framework for ranking metric optimization. InProceedings of the 27th ACM international conference on information and knowledge manage- ment. 1313–1322

  65. [2376]

    https://proceedings.mlr.press/v130/reddi21a.html

This paper was first reviewed by deepseek-v4-flash on August 1, 2026.