Beyond Pairwise Learning-To-Rank At Airbnb

Daochen Zha; Huiji Gao; Liwei He; Malay Haldar; Sanjeev Katariya

REVIEW 4 major objections 7 minor 35 references

Beyond Pairwise Learning-To-Rank At Airbnb

T0 review · 4 major / 7 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A ranking model that compares every listing with every other listing beat pairwise ranking in Airbnb online tests.

desk verdict A real industrial deployment with a novel all-pairwise architecture and honest reporting, but the headline online gains bundle two changes at once (architecture + loss), so the causal claim needs an ablation. read the letter →

arxiv 2505.09795 v3 pith:LAAYEA3K submitted 2025-05-14 cs.IR

Malay Haldar , Daochen Zha , Huiji Gao , Liwei He , Sanjeev Katariya This is my paper · ORCID

classification cs.IR

keywords learning-to-rankpairwiseLTRall-pairwiseBradley-Terrymodelpermutationinvariancesearchrankingmulti-objectiveoptimizationAirbnb

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that pairwise learning-to-rank, the standard approach used in industrial search, is accurate only within a one-dimensional pairwise-logit space: it cannot represent situations where a searcher must choose between two nearly identical listings while also weighing a third, very different listing. To go beyond that limit, the paper introduces true-pairwise LTR, in which a bivariate model compares two listings and is forced to be anti-commutative, and then all-pairwise LTR, in which each listing is scored through superiority and similarity features computed against every other listing. It reports that the more listing-to-listing interaction a model captures, the higher its offline NDCG, and that this accuracy gap cannot be closed by scaling up model parameters. Deployed as a second-stage re-ranker after a pairwise LTR first stage, all-pairwise LTR improved uncanceled bookings by 0.61% to 1.20% ($p<10^{-5}$) in online experiments at Airbnb and now serves all of its search traffic. If these results are right, interaction-aware ranking can be made practical without giving up latency budgets or a deterministic total order.

What carries the argument

The machinery is the all-pairwise feature network (APFN) feeding the all-pairwise logit network (APLN). For each listing $l_i$, APFN builds superiority features by passing pairwise differences of first-stage pairwise logits through a sigmoid, and similarity features by taking softmax-normalized dot products of listing embeddings; its input layer makes the neuron output $x_i \phi(l_i)+b$ depend on the item identity through an MLP $\phi(\cdot)$, which is what makes the representation invariant to the order of the other listings. The APLN maps these features to the final logit, trained as a residual added to the stable first-stage logit. Two supporting identities carry the argument: anti-commutativity $g(l_a,l_b)=-g(l_b,l_a)$ from Equation 8 guarantees three of the four total-order axioms, and the generalized Bradley-Terry aggregation of Equation 9 converts the resulting pairwise scores into one scalar per listing, which can be sorted with standard algorithms.

What would settle it

Take three real listings $A,B,C$ from search logs and run the trained comparator; if it says $A$ beats $B$, $B$ beats $C$, and $C$ beats $A$, then the generalized Bradley-Terry score must contradict at least one of those pairwise preferences when it imposes a total order. Showing that such cycles occur at a non-trivial rate would demonstrate that the scalar ranking is not actually representing the pairwise signal it was trained on.

Watch

Extended reading notes

Core claim

The central claim is that ranking accuracy is bounded by collinearity of pairwise logits: $\mathrm{logit}(l_a,l_b)+\mathrm{logit}(l_b,l_c)=\mathrm{logit}(l_a,l_c)$, so all pairwise comparisons for one query sit on a single line. The paper constructs a bivariate comparator $g(l_a,l_b)=h(l_a,l_b)-h(l_b,l_a)$, which is anti-commutative, and restores a total order by aggregating the $N-1$ comparisons of each listing with the generalized Bradley-Terry score $\mathrm{score}(l_i)=1/(1+\sum_{j\neq i} e^{-g(l_i,l_j)})$. It then extends this to all-pairwise LTR, where a permutation-invariant network takes, for each listing, superiority features (sigmoid-transformed first-stage logit differences) and similarity features (softmax-normalized embedding dot products) against all other listings, and produces the final logit as a residual on top of the first-stage pairwise logit. In offline experiments all-pairwise LTR outperforms true-pairwise LTR, which outperforms pairwise LTR, and in online tests the deployed model increased uncanceled bookings by 0.61%, uncanceled nights by 0.80%, and uncanceled booking value by 1.20% relative to the production ranker, with $p<10^{-5}$.

Load-bearing premise

The whole construction assumes that pairwise preferences between listings are approximately transitive, so one real number per listing can faithfully represent all pairwise comparisons; the model enforces anti-commutativity but never enforces transitivity.

Editorial extensions

If this is right

Ranking teams that currently use pairwise LTR can expect a consistent accuracy hierarchy: all-pairwise LTR above true-pairwise LTR above pairwise LTR, with parameter scaling alone insufficient to close the gap.
A two-stage architecture, pairwise first stage plus all-pairwise second stage reranking roughly the top 60 listings, captures most of the NDCG benefit while keeping latency under control; reranking more listings gives diminishing returns.
Modeling listing similarity as part of ranking not only improves booking-based metrics but also increases diversity, measured as price variance on the first page, which aligns ranking with the diversity objective from earlier work.
Multi-objective goals such as trip quality can be folded into ranking by weighting pairwise losses, and the resulting single model beat the production multi-model system on uncanceled bookings, nights, value, and trip rating.
A residual connection to the first-stage univariate logit cuts unstable flips caused by small map-bound changes by about 75 percent, making interaction-aware ranking usable in production.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A direct measurement of cyclic preference triples in real search logs would show how much ranking accuracy is being sacrificed by the generalized Bradley-Terry total order; if cycles are common, the scalar score is breaking genuine pairwise signals rather than representing them.
Because superiority features are built from first-stage pairwise logits, all-pairwise LTR inherits any blind spots of the first-stage ranker; an ablation with a deliberately weak first-stage model would reveal whether the second stage can compensate or amplifies bias.
The permutation-invariant construction that weights each input by a function of the item identity is one instance of a broader class of set-based ranking functions; replacing that item-specific weighting with learned attention over the set could trade some stability for additional modeling power, although the paper does not explore this.
The SAT theorem, if accepted, predicts that any attempt to train a single end-to-end model to be fully interaction-aware, scalable, and total-ordered will hit the same wall; the practical lesson generalizes beyond vacation rentals to any marketplace where item utility depends on the comparison set.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The paper is worth a read: it describes a genuine attempt to move beyond pairwise LTR by scoring each item against all others, with a clean residual architecture and a real deployment at Airbnb. The new pieces are the superiority/similarity feature construction, the permutation-invariant weighting layer, and the two-stage design that keeps the quadratic model on the last 60 items. The writing is honest: they report epistemic uncertainty, note that NDCG gains don't map uniformly to bookings, and list open questions. That is more candid than most industrial papers.

The main soft spot is the evidence for the headline online gains. Table 2 compares "all-pairwise-apn trained with the loss of Equation 11" against a production system that used separate models for bookings and trip quality. That changes two things at once: the architecture and the loss/reweighting scheme. No ablation isolates the all-pairwise contribution. The stress-test note is right about this. Figure 9 is described as a direct test of all-pairwise against pairwise, but the numbers and confidence intervals are not in the text, so the reader can't verify the magnitude. Offline Table 1 is clean in direction, but the authors themselves warn that NDCG improvements don't reliably convert to bookings, so it doesn't close the gap.

Two more issues. The SAT theorem is asserted without proof, and it's doing rhetorical work; either prove it or soften it to a conjecture. The generalized Bradley-Terry aggregation (Eq. 9) produces real-valued scores that are sortable, but the induced order is only meaningful if the pairwise relation is approximately transitive, which the anti-commutative construction doesn't guarantee. That's a theoretical gap, though it may be fine in practice.

None of this kills the paper. The architecture and deployment are real, the offline comparison among the four variants is informative, and the engineering lessons (stability, rerank count, parameter scaling) are useful. The reader's conditional verdict is fair. I'd send this to review, but ask for an ablation that separates the architecture from the loss change, and for the actual numbers behind Figure 9.

Referee Report

4 major / 7 minor

Summary. This paper introduces a family of learning-to-rank models that go beyond standard pairwise LTR by modeling interactions between items. It proposes a 'true-pairwise' model using a bivariate scoring function g and an 'all-pairwise' model that scores each item using superiority and similarity features relative to all other items, aggregated by an MLP. A 'SAT theorem' is asserted, stating that no ranking algorithm can simultaneously scale, be accurate, and produce a total order, which motivates a two-stage hybrid system. The paper reports offline NDCG comparisons, diversity and latency experiments, and online A/B tests at Airbnb, including a 0.61%-1.20% lift in uncanceled bookings from a multi-objective all-pairwise model. The final system was deployed to all Airbnb searchers in early 2025.

Significance. If the claims are substantiated, the paper provides a valuable industrial case study and a practical architecture for context-aware ranking: the all-pairwise model is permutation-invariant, captures listing-to-listing interactions, and was deployed at scale. Its strengths are the clear comparison of several design choices (true-pairwise-avg vs. true-pairwise-gbt, APFN vs. attention) and the honest acknowledgment that offline NDCG does not fully transfer to online outcomes. However, the central attribution of the online booking gains to the all-pairwise architecture is currently under-verified, and the theoretical scaffolding (the SAT theorem and the generalized Bradley-Terry aggregation) is under-specified. With additional rigor on these points, the contribution would be a strong addition to the applied LTR literature.

major comments (4)

[Section 2] Section 2 states a 'SAT theorem' asserting that a ranking algorithm can only fulfill two of the three requirements of scalability, accuracy, and total order, but the statement is not formalized and no proof is given. Since this theorem is used to motivate the hybrid two-stage architecture and to frame the paper's contribution, the authors should either provide a precise statement with a proof or clearly label it as an empirical observation supported by the examples in Section 3. As written, calling it a theorem overstates the evidence.
[Section 4.1, Eq. (9)] Section 4.1 claims that applying the generalized Bradley-Terry model in Eq. (9) restores total order. The scores defined by Eq. (9) are real numbers and hence induce a total order, but the paper does not establish that this total order respects the pairwise preferences encoded by g. In particular, g is only required to be anti-commutative (Eq. 8), not transitive; without an additional condition or a proof, the ranking produced by Eq. (9) may disagree with the pairwise comparisons the true-pairwise model was trained to predict. The authors should either prove a consistency property (or cite the relevant theorem from [12]) or state that the aggregation is a heuristic and justify it empirically.
[Section 8, Table 2 and Section 9] The online A/B test in Table 2 compares 'all-pairwise-apfn trained with the loss of Equation 11' against Airbnb's active production ranker, which used separate models for bookings and trip quality. The treatment therefore differs from the baseline in both the architecture (Section 5) and the loss function (Section 6). The reported 0.61%-1.20% uncanceled booking gains cannot be unambiguously attributed to the all-pairwise architecture. The only online test that isolates the architecture (Figure 9) is presented without numerical values or confidence intervals in the text. To support the paper's central claim, the authors should provide the numerical results behind Figure 9 or run an ablation that separates the loss change from the architecture change.
[Section 8, Table 1 and Figures 6-8] The offline results are reported without confidence intervals or significance tests. For example, Table 1 reports NDCG means and standard deviations over eight seeds, but no p-values or effect sizes; the difference between pairwise LTR (I: 0.6776) and all-pairwise-apfn (IV: 0.6867) is large relative to the reported standard deviations, but this is not stated formally. Similarly, Figures 6-8 show point estimates without error bars or confidence bands. Since the paper's hierarchy claim ('pairwise < true-pairwise < all-pairwise') is central, the authors should add uncertainty quantification to these figures and/or significance tests to the Table 1 data.

minor comments (7)

[Section 8 (stability experiment)] The phrase 'unstable flips are reduced by -75%' is a double negative; it should be rephrased to 'flips are reduced by 75%' or 'the residual architecture produces 75% fewer flips'.
[Section 8 and Table 2] The naming is inconsistent: the model is called 'all-pairwise-apfn' in the text and 'all-pairwise-apn' in the heading of Table 2; please standardize the abbreviation throughout.
[Section 2] The acronym 'SAT' is never defined. The reader must infer that it stands for Scalability, Accuracy, and Total order; please state this explicitly.
[References] Reference [5] is described as the paper that introduced the Bradley-Terry-Luce model and analyzed pairwise LTR, but the cited Ford (1957) paper appears to be about solving a ranking problem from binary comparisons, not the BTL model itself; the citation should be checked and corrected if necessary.
[Section 4.1] The claim that anti-commutativity guarantees reflexivity, antisymmetry, and strong connectivity would be clearer if the text explicitly noted that these properties follow for real-valued anti-commutative functions g(l_a,l_b)=-g(l_b,l_a).
[Section 4.1, Eq. (9)] Equation (9) is described as a 'simpler version' of the generalized Bradley-Terry model from [12]; a brief explanation of how it simplifies the model of [12] would help readers assess its validity.
[Section 8] The stability experiment is described as introducing 'a small jitter to the map bounds associated with a search query'; please provide details on the jitter magnitude and the precise definition of a 'flip'.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the all-pairwise derivation is a feature-stacking architecture trained on booked/not-booked labels, with no prediction reducing to its own inputs by construction.

full rationale

The paper's derivation chain is self-contained: Equations 1 through 11 are explicit supervised-learning objectives and fixed aggregation formulas. The all-pairwise model uses first-stage pairwise LTR logits as superiority features, but this is feature stacking, not circular reasoning, because the target remains booked/not-booked labels and the baseline remains the first-stage ranker itself; no fitted parameter is renamed as a prediction. The generalized Bradley-Terry aggregation in Equation 9 is a fixed external formula from Huang et al. (2006), not fitted to the data, and the multi-objective weight in Equations 10-11 is computed directly from trip-rating labels. Self-citations to prior Airbnb work are historical or describe similarity features, but the paper itself specifies the similarity construction (embedding mapping, dot product, softmax) in Section 5, so no load-bearing premise is imported solely from the authors' own prior work. The online comparison in Table 2 changes both the architecture and the loss function relative to the production baseline, so the empirical attribution is underdetermined, but that is an experimental confound rather than a circular derivation. No step in the paper equates a prediction to its input by definition or by fitted equivalence. Therefore the circularity score is 0.

Assumptions & free parameters 3 free parameters · 4 assumptions · 1 invented entities

The central claim rests on standard learning-to-rank machinery, with one unspecified weighting function and the chosen rerank size as the main free parameters. The SAT theorem is an invented framing, not a formal result. The paper's own prior work supplies similarity features and the multi-task context, but these are used as inputs rather than as circular validations.

free parameters (3)

N_rerank = 60
Number of listings reranked by the second-stage ranker, chosen from the latency versus NDCG trade-off in Figure 7.
omega_tripQuality(l) = unspecified
Weighting function in Equation 10 that adjusts loss based on trip rating; its functional form and calibration are not specified in the paper.
Model hyperparameters = not specified
Embedding sizes, MLP widths, and learning rates for phi, psi, and the APLN are not given, yet the reported results depend on them.

assumptions (4)

domain assumption Tversky's choice theory: item utility depends on the entire choice set
Invoked in Section 2 to justify the SAT theorem and the need for interaction-aware ranking.
standard math Bradley-Terry model connects pairwise and pointwise probabilities
Used in Equation 4 to argue that pairwise LTR reduces to pointwise ordering.
standard math Generalized Bradley-Terry model (Huang et al.) converts pairwise comparisons to scores with performance bounds
Cited in Section 4.1 to justify Equation 9 as the aggregation step; the bound is not stated in the paper.
domain assumption Permutation invariance can be achieved by making input-layer weights depend on the associated item
The construction in Section 5 (x_i * phi(l_i) + b) assumes the resulting network still learns useful interactions and that phi can be learned reliably.

invented entities (1)

SAT theorem
purpose: Conceptual framing stating no single ranking algorithm can achieve scalability, accuracy, and total order simultaneously.
The paper asserts this theorem without formal proof; it is an organizing analogy like the CAP theorem rather than a testable entity.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Beyond Pairwise Learning-To-Rank At Airbnb." pith.science (2026). https://pith.science/paper/LAAYEA3K

@misc{pith2026250509795,
  author       = {Pith},
  title        = {Pith review of: Beyond Pairwise Learning-To-Rank At Airbnb},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LAAYEA3K}},
  note         = {Machine review of arXiv:2505.09795}
}

read the original abstract

There are three fundamental asks from a ranking algorithm: it should scale to handle a large number of items, sort items accurately by their utility, and impose a total order on the items for logical consistency. But here's the catch-no algorithm can achieve all three at the same time. We call this limitation the SAT theorem for ranking algorithms. Given the dilemma, how can we design a practical system that meets user needs? Our current work at Airbnb provides an answer, with a working solution deployed at scale. We start with pairwise learning-to-rank (LTR) models-the bedrock of search ranking tech stacks today. They scale linearly with the number of items ranked and perform strongly on metrics like NDCG by learning from pairwise comparisons. They are at a sweet spot of performance vs. cost, making them an ideal choice for several industrial applications. However, they have a drawback-by ignoring interactions between items, they compromise on accuracy. To improve accuracy, we create a "true" pairwise LTR model-one that captures interactions between items during pairwise comparisons. But accuracy comes at the expense of scalability and total order, and we discuss strategies to counter these challenges. For greater accuracy, we take each item in the search result, and compare it against the rest of the items along two dimensions: (1) Superiority: How strongly do searchers prefer the given item over the remaining ones? (2) Similarity: How similar is the given item to all the other items? This forms the basis of our "all-pairwise" LTR framework, which factors in interactions across all items at once. Looking at items on the search result page all together-superiority and similarity combined-gives us a deeper understanding of what searchers truly want. We quantify the resulting improvements in searcher experience through offline and online experiments at Airbnb.

Figures

Figures reproduced from arXiv: 2505.09795 by the authors.

**Figure 1.** The relation logit(𝑙𝑥,𝑙𝑦) ≫ 0 ∧ logit(𝑙𝑥,𝑙𝑧 ) ≈ logit(𝑙𝑦,𝑙𝑧 ) cannot be captured in a 1D space. It is represented by the triangle inequality logit(𝑙𝑥,𝑙𝑦) + logit(𝑙𝑦,𝑙𝑧 ) > logit(𝑙𝑥,𝑙𝑧 ). This restriction on the logit space prevents pairwise LTR from accurately modeling booking probabilities for users. Let’s break it down with an example. Imagine two listings, 𝑙𝑥 and 𝑙𝑦, that are identical apartments in the same bui… view at source ↗

**Figure 2.** True-pairwise LTR [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** Anatomy of a multi-stage ranker. An alternative way to achieve permutation invariance, proposed in [1], involves generating and aggregating permutations of item subsets. In our approach, we avoid the computational complexity of creating permutations. The superiority-similarity features, along with the layer imposing input permutation invariance, make up the all-pairwise feature network (APFN). Construction of superi… view at source ↗

Figures from the paper (5 more)

**Figure 4.** Figure 4: Superiority and similarity features of APFN. [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]

**Figure 5.** Figure 5: The all-pairwise architecture. Superiority features refers to Figure 4a. Similarity features refers to Figure 4b. Together they form the all-pairwise feature network (APFN). The APFN feeds into the all-pairwise logit network (APLN), an MLP. To produce the final logit, …

**Figure 6.** Figure 6: X-axis: Number of model parameters. Y-axis: Model NDCG. [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]

**Figure 8.** Figure 8: X-axis: Number of listings reranked. Y-axis: Normalized variance of [PITH_FULL_IMAGE:figures/full_fig_p006_8.png]

**Figure 9.** Figure 9: X-axis: Treatment models. Y-axis: % gain in uncancelled bookings. [PITH_FULL_IMAGE:figures/full_fig_p007_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

35 extracted references · 19 canonical work pages

[12]

Tzu-Kuo Huang, Ruby C Weng, Chih-Jen Lin, and Greg Ridgeway. 2006. Gener- alized Bradley-Terry Models and Multi-Class Probability Estimates. Journal of Machine Learning Research 7, 1 (2006)

work page 2006
[1]

Qingyao Ai, Xuanhui Wang, Sebastian Bruch, Nadav Golbandi, Michael Bender- sky, and Marc Najork. 2019. Learning Groupwise Multivariate Scoring Functions Using Deep Neural Networks. In Proceedings of the 2019 ACM SIGIR Interna- tional Conference on Theory of Information Retrieval (Santa Clara, CA, USA) (IC- TIR ’19) . Association for Computing Machinery, N...

arXiv 2019
[2]

Irwan Bello, Sayali Kulkarni, Sagar Jain, Craig Boutilier, Ed Chi, Elad Eban, Xiyang Luo, Alan Mackey, and Ofer Meshi. 2019. Seq2Slate: Re-ranking and Slate Optimization with RNNs. Technical Report. https://arxiv.org/abs/1810.02019

arXiv 2019
[3]

Jane Bromley, Isabelle Guyon, Yann LeCun, Eduard Säckinger, and Roopak Shah
[4]

Zhe Cao, Tao Qin, Tie-Yan Liu, Ming-Feng Tsai, and Hang Li. 2007. Learning to Rank: From Pairwise Approach to Listwise Approach. In Proceedings of the 24th International Conference on Machine Learning (Corvalis, Oregon, USA) (ICML ’07). 129–136

work page 2007
[5]

L. R. Ford. 1957. Solution of a Ranking Problem from Binary Comparisons. The American Mathematical Monthly 64, 8 (1957), 28–33. http://www.jstor.org/stable/ 2308513

work page 1957
[6]

Aditya Grover, Eric Wang, Aaron Zweig, and Stefano Ermon. 2019. Stochastic Opti- mization of Sorting Networks via Continuous Relaxations. InInternational Confer- ence on Learning Representations. https://openreview.net/forum?id=H1eSS3CcKX

work page 2019
[7]

Malay Haldar, Mustafa Abdool, Liwei He, Dillon Davis, Huiji Gao, and Sanjeev Katariya. 2023. Learning To Rank Diversely At Airbnb. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management (Birm- ingham, United Kingdom,) (CIKM ’23). Association for Computing Machinery, New York, NY, USA, 4609–4615. https://doi.org/10....

arXiv 2023

Show all 35 references

[8]

Turnbull, Bren- dan M

Malay Haldar, Mustafa Abdool, Prashant Ramanathan, Tao Xu, Shulin Yang, Huizhong Duan, Qing Zhang, Nick Barrow-Williams, Bradley C. Turnbull, Bren- dan M. Collins, and Thomas Legrand. 2019. Applying Deep Learning to Airbnb Search. In Proceedings of the 25th ACM SIGKDD Internat...

2019
[9]

Malay Haldar, Prashant Ramanathan, Tyler Sax, Mustafa Abdool, Lanbo Zhang, Aamir Mansawala, Shulin Yang, Bradley Turnbull, and Junshuo Liao. 2020. Im- proving Deep Learning for Airbnb Search. InProceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery ...

2020
[10]

Malay Haldar, Hongwei Zhang, Kedar Bellare, Sherry Chen, Soumyadip Banerjee, Xiaotang Wang, Mustafa Abdool, Huiji Gao, Pavan Tapadia, Liwei He, and Sanjeev Katariya. 2024. Learning to Rank for Maps at Airbnb. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discov...

2024
[11]

Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al. 2022. Training compute-optimal large language models. arXiv preprint arXiv:2203.15556 (2022)

2022 arXiv
[13]

Rolf Jagerman, Zhen Qin, Xuanhui Wang, Michael Bendersky, and Marc Najork
[14]

Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. 2020. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361 (2020)

2020 arXiv
[15]

Alex Kendall and Yarin Gal. 2017. What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?. In Advances in Neural In- formation Processing Systems , I. Guyon, U. Von Luxburg, S. Bengio, H. Wal- lach, R. Fergus, S. Vishwanathan, and R. Garnett (Eds.), Vol. 3...

2017
[16]

Weiwen Liu, Yunjia Xi, Jiarui Qin, Fei Sun, Bo Chen, Weinan Zhang, Rui Zhang, and Ruiming Tang. 2022. Neural Re-ranking in Multi-stage Recommender Systems: A Review. In Proceedings of the Thirty-First International Joint Con- ference on Artificial Intelligence, IJCAI-22 , Lud ...

2022 doi
[17]

Franco Maria Nardini, Roberto Trani, and Rossano Venturini. [n. d.]. Learning bivariate scoring functions for ranking. Discover Computing 27, 1 ([n. d.])
[18]

Liang Pang, Jun Xu, Qingyao Ai, Yanyan Lan, Xueqi Cheng, and Jirong Wen
[19]

Rama Kumar Pasumarthi, Honglei Zhuang, Xuanhui Wang, Michael Bendersky, and Marc Najork. 2020. Permutation Equivariant Document Interaction Network for Neural Learning to Rank. InProceedings of the 2020 ACM SIGIR on International Conference on Theory of Information Retrieval (...

2020
[20]

Sebastian Prillo and Julian Martin Eisenschlos. 2020. SoftSort: a continuous relax- ation for the argsort operator. In Proceedings of the 37th International Conference on Machine Learning (ICML’20) . JMLR.org, Article 722, 10 pages

2020
[21]

Tao Qin, Tie-Yan Liu, and Hang Li. 2010. A general approximation framework for direct optimization of information retrieval measures. Information retrieval 13 (2010), 375–397

2010
[22]

Yi Ren, Xiao Han, Xu Zhao, Shenzheng Zhang, and Yan Zhang. 2023. Slate-Aware Ranking for Recommendation. In Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining (Singapore, Singapore) (WSDM ’23). Association for Computing Machinery, New York,...

2023
[23]

D Sculley et al. 2009. Large scale learning to rank. In NIPS Workshop on Advances in Ranking, Vol. 3

2009
[24]

Nithish Kannen Senthilkumar, Yao Ma, Gerrit van den Burg, and Jean Baptiste Faddoul. 2024. Efficient pointwise-pairwise learning-to-rank for news recommen- dation. (2024). https://www.amazon.science/publications/efficient-pointwise- pairwise-learning-to-rank-for-news-recommendation

2024
[25]

Amit Sharma and Baoshi Yan. 2013. Pairwise learning in recommendation: experiments with community recommendation on linkedin. In Proceedings of the 7th ACM Conference on Recommender Systems (Hong Kong, China) (Rec- Sys ’13). Association for Computing Machinery, New York, NY, U...

2013
[26]

Chun How Tan, Austin Chan, Malay Haldar, Jie Tang, Xin Liu, Mustafa Abdool, Huiji Gao, Liwei He, and Sanjeev Katariya. 2023. Optimizing Airbnb Search Journey with Multi-Task Learning. In Proceedings of the 29th ACM SIGKDD Con- ference on Knowledge Discovery and Data Mining (Lo...

2023
[27]

Amos Tversky. 1972. Elimination by aspects: A theory of choice. Psychological review 79, 4 (1972), 281

1972
[28]

Jianxiong Wei, Anxiang Zeng, Yueqiu Wu, Peng Guo, Qingsong Hua, and Qingpeng Cai. 2020. Generator and Critic: A Deep Reinforcement Learning Approach for Slate Re-ranking in E-commerce. CoRR abs/2005.12206 (2020). arXiv:2005.12206 https://arxiv.org/abs/2005.12206

2020 arXiv
[29]

Wikipedia. 2022. Bradley–Terry model. https://en.wikipedia.org/wiki/Bradley% E2%80%93Terry_model [Online; accessed 14-August-2022]

2022
[30]

Wikipedia. 2025. CAP theorem. https://en.wikipedia.org/wiki/CAP_theorem [Online; accessed 24-April-2025]

2025
[31]

Wikipedia. 2025. Total order. https://en.wikipedia.org/wiki/Total_order [Online; accessed 24-April-2025]

2025
[32]

Tao Zhuang, Wenwu Ou, and Zhirong Wang. 2018. Globally optimized mutual influence aware ranking in e-commerce search. In Proceedings of the 27th Interna- tional Joint Conference on Artificial Intelligence (Stockholm, Sweden) (IJCAI’18). AAAI Press, 3725–3731

2018
[1993]

In Proceedings of the 7th International Conference on Neural Information Processing Systems (Denver, Colorado) (NIPS’93)

Signature verification using a "Siamese" time delay neural network. In Proceedings of the 7th International Conference on Neural Information Processing Systems (Denver, Colorado) (NIPS’93). Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 737–744
[2020]

Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval (2020)

SetRank: Learning a Permutation-Invariant Ranking Model for Information Retrieval. Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval (2020)

2020
[2022]

In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval

On optimizing top-k metrics for neural ranking models. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2303–2307

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [12]

Tzu-Kuo Huang, Ruby C Weng, Chih-Jen Lin, and Greg Ridgeway. 2006. Gener- alized Bradley-Terry Models and Multi-Class Probability Estimates. Journal of Machine Learning Research 7, 1 (2006)

work page 2006

[2] [1]

Qingyao Ai, Xuanhui Wang, Sebastian Bruch, Nadav Golbandi, Michael Bender- sky, and Marc Najork. 2019. Learning Groupwise Multivariate Scoring Functions Using Deep Neural Networks. In Proceedings of the 2019 ACM SIGIR Interna- tional Conference on Theory of Information Retrieval (Santa Clara, CA, USA) (IC- TIR ’19) . Association for Computing Machinery, N...

arXiv 2019

[3] [2]

Irwan Bello, Sayali Kulkarni, Sagar Jain, Craig Boutilier, Ed Chi, Elad Eban, Xiyang Luo, Alan Mackey, and Ofer Meshi. 2019. Seq2Slate: Re-ranking and Slate Optimization with RNNs. Technical Report. https://arxiv.org/abs/1810.02019

arXiv 2019

[4] [3]

Jane Bromley, Isabelle Guyon, Yann LeCun, Eduard Säckinger, and Roopak Shah

[5] [4]

Zhe Cao, Tao Qin, Tie-Yan Liu, Ming-Feng Tsai, and Hang Li. 2007. Learning to Rank: From Pairwise Approach to Listwise Approach. In Proceedings of the 24th International Conference on Machine Learning (Corvalis, Oregon, USA) (ICML ’07). 129–136

work page 2007

[6] [5]

L. R. Ford. 1957. Solution of a Ranking Problem from Binary Comparisons. The American Mathematical Monthly 64, 8 (1957), 28–33. http://www.jstor.org/stable/ 2308513

work page 1957

[7] [6]

Aditya Grover, Eric Wang, Aaron Zweig, and Stefano Ermon. 2019. Stochastic Opti- mization of Sorting Networks via Continuous Relaxations. InInternational Confer- ence on Learning Representations. https://openreview.net/forum?id=H1eSS3CcKX

work page 2019

[8] [7]

Malay Haldar, Mustafa Abdool, Liwei He, Dillon Davis, Huiji Gao, and Sanjeev Katariya. 2023. Learning To Rank Diversely At Airbnb. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management (Birm- ingham, United Kingdom,) (CIKM ’23). Association for Computing Machinery, New York, NY, USA, 4609–4615. https://doi.org/10....

arXiv 2023