Pith. sign in

REVIEW 3 major objections 5 minor 94 references

Breaking the Top-$K$ Barrier: Advancing Top-$K$ Ranking Metrics Optimization in Recommender Systems

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that a quantile-weighted softmax loss, SL@K, is a provable surrogate for NDCG@K and beats existing recommendation losses by 6.03% on average.

desk verdict SL@K is a plausible practical upgrade to Softmax Loss for NDCG@K, but the loss actually optimized in Algorithm C.1 omits the self-term and is not the loss bounded in Theorem 3.2, so the theoretical guarantee needs a fix or a scoping clarification. read the letter →

arxiv 2508.05673 v1 pith:PPJX5UMT submitted 2025-08-04 cs.IR cs.AIcs.LG

Weiqin Yang , Jiawei Chen , Shengjia Zhang , Peng Wu , Yuegang Sun , Yan Feng , Chun Chen , Can Wang This is my paper · ORCID
classification cs.IRcs.AIcs.LG
keywords recommendersystemsNDCG@Koptimizationtop-Krankingmetricssurrogatelosssoftmaxquantileestimationimplicitfeedbacklearningtorank
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that Softmax Loss, a standard surrogate for full-ranking NDCG, is mismatched with NDCG@K, the metric recommender systems are actually judged by, and that closing this gap improves real performance. It introduces SoftmaxLoss@K (SL@K), a weighted softmax loss in which every positive item receives a quantile-based weight that focuses training on items that can enter the top K. The central claim is that, for any user with more than one top-K hit, SL@K is an upper bound of $-\log \mathrm{DCG}@K$, so minimizing SL@K is consistent with maximizing NDCG@K. The paper reports an average 6.03% improvement over the best baselines across four datasets and three recommendation backbones, with the gains strongest at small K. If correct, this gives recommenders a surrogate that is as cheap to train as Softmax Loss while explicitly modeling truncation.

What carries the argument

The load-bearing object is the top-K quantile $\beta^K_u$, the score threshold below which items fall out of the user's top K. It converts the combinatorial truncation term $\mathbb{I}(\pi_{ui}\le K)$ into the simple comparison $\mathbb{I}(s_{ui}\ge \beta^K_u)$, and smoothing that comparison with a sigmoid produces the per-example weight $w_{ui}=\sigma_w(s_{ui}-\beta^K_u)$ that turns ordinary Softmax Loss into SL@K. The second mechanism is the Monte Carlo quantile estimator: sample a few negatives, add all positives, sort, and take the K-th score; Theorem 3.1 bounds its error through a standard empirical-c.d.f. concentration inequality. Together they give a smooth, sampling-friendly surrogate whose gradient concentrates on positive items near or above the threshold.

What would settle it

Train SL@K with the paper's estimator and, each epoch, compute the exact top-K threshold per user from full scores; if for any user with more than one top-K hit the inequality $-\log \mathrm{DCG}@K(u) \le \mathcal{L}_{\mathrm{SL}@K}(u)$ fails when the estimated threshold is used, then the practical algorithm violates the guarantee claimed in Theorem 3.2.

Watch

Extended reading notes

Core claim

The paper's central discovery is that the top-K truncation in NDCG@K can be handled by a per-user threshold, the top-K quantile $\beta^K_u = \inf\{s_{ui} : \pi_{ui} \le K\}$, which separates items inside the top K from the rest. Replacing the threshold indicator with a sigmoid weight and bounding the discounted rank by a softmax sum yields SL@K: $\mathcal{L}_{\mathrm{SL}@K}(u)=\sum_{i\in P_u}\sigma_w(s_{ui}-\beta^K_u)\log \sum_{j\in I}\exp((s_{uj}-s_{ui})/\tau_d)$. The paper proves that, whenever the number of top-K hits $H^K_u$ exceeds 1, SL@K is an upper bound of $-\log \mathrm{DCG}@K(u)$, and that a Monte Carlo estimator with $N$ sampled scores has quantile error that decays exponentially in $N$ (Theorem 3.1). On this basis the paper claims that optimizing SL@K is a theoretically grounded way to maximize NDCG@K, and the experiments support the claim that the bound transfers into gains on real top-K metrics.

Load-bearing premise

The practical estimator keeps all positive items and samples only negatives, and the paper admits this biases the threshold downward; if that biased threshold marks the wrong positives as top-K, the weighting no longer tracks NDCG@K.

Editorial extensions

If this is right

  • Training with SL@K should improve NDCG@K and Recall@K relative to full-ranking surrogates, with the largest gains at small K because truncation weighs most heavily there.
  • SL@K inherits the ease of Softmax Loss: the only added component is a per-user quantile estimate with $O(|U|N\log N)$ overhead, so existing softmax-based recommender code can adopt it with minimal change.
  • The moderate gradient distribution reported for SL@K implies it can use sampled negatives without the instability that the paper documents for LambdaLoss@K and SONG@K.
  • Because the weighting automatically down-weights low-scoring positives, SL@K should degrade more gracefully than SL when false-positive noise is present in the interaction data.
  • The same loss transfers to other top-K tasks such as learning to rank, sequential recommendation, and link prediction, where the paper reports consistent gains over its baselines.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Given the paper's admission that the practical quantile estimate is biased downward, SL@K is effectively optimizing a slightly larger cutoff than the specified K; an explicit bias-correction term might tighten the match to NDCG@K, especially for small K.
  • The exponential error bound in Theorem 3.1 depends on score density near the true quantile; on datasets where many items tie near the K-th position, the practical N=1000 sample could be far from the stated guarantee, suggesting a density-aware sample size rule.
  • A natural extension, which the paper lists as future work, is incremental quantile updating; if the threshold can be updated online, SL@K becomes usable for streaming and continual recommendation.
  • Because the main bound requires $H^K_u>1$, users with a single top-K hit get only the weaker half-log bound; weighting users by their hit count could make the objective uniformly aligned with NDCG@K.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes SoftmaxLoss@K (SL@K), a weighted softmax loss for optimizing NDCG@K in top-K recommendation. The loss weights each positive instance by a sigmoid of the excess of its score over an estimated top-K quantile, and trains the inner softmax term with exponential activations. The authors derive an upper bound relating -log DCG@K to the new loss (Theorem 3.2), provide a Monte Carlo quantile-estimation guarantee (Theorem 3.1), and report experiments on four recommendation datasets with three backbones, plus extensions to learning-to-rank, sequential recommendation, and link prediction, claiming an average 6.03% improvement over baselines. The manuscript is clearly written and the main theoretical derivation is self-contained given its relaxations, but the algorithm presented in the appendix does not implement the loss analyzed in the main text.

Significance. If the theory/implementation gap is resolved, the paper makes a useful contribution: a computationally cheap modification of softmax loss that explicitly models top-K truncation through a quantile-based weight, with a transparent upper-bound argument and broad empirical evaluation. The strengths are the availability of code, the detailed hyperparameter tables, and the systematic comparison against softmax variants and NDCG@K surrogate losses. The proposed loss is simple and plausible, and the empirical gains, if reproducible for the analyzed loss, would be practically meaningful. The main theoretical claim, however, currently attaches to a different objective than the one specified in the optimization algorithm, which blocks acceptance in its present form.

major comments (3)
  1. [§3.1.2, Eq. (3.5) vs. Algorithm C.1 (line 14)] The loss analyzed in the paper is not the loss implemented in the algorithm. Equation (3.5) defines the inner softmax term as L_SL(u,i) = log Σ_{j∈I} σ_d(d_uij), and the proof of Theorem 3.2 in Appendix B.2 relies on the bound π_ui = Σ_{j∈I} δ(d_uij) ≤ Σ_{j∈I} σ_d(d_uij), which requires the self term j=i contributing σ_d(0)=1. Algorithm C.1, line 14, instead computes L_SL(u,i) = log Σ_{j∈\hat N_u} σ_d(d_uij), where \hat N_u is a set of uniformly sampled negative items only and contains no self term and no positive item. Because sampled negative items typically satisfy d_uij<0, the sum can be smaller than 1 and the log can be negative, while the term it is supposed to bound, log π_ui, is nonnegative. The upper-bound relation of Theorem 3.2 therefore does not hold for the objective actually minimized. If the released code follows Algorithm C.1, the reported 6.03% average improvement is for a different loss than the one covered by the theoretical guarantee. Please correct the algorithm to include the self term (for instance, log(1 + Σ_{j∈\hat N_u} σ_d(d_uij))), or provide a separate theoretical analysis of the negative-only surrogate actually used.
  2. [§3.1.3, Theorem 3.1 and Appendix C.2 (negative sampling trick)] Theorem 3.1 provides a finite-sample guarantee for quantile estimation from i.i.d. samples of the score distribution F_u, but the practical estimator described in Appendix C.2 and used in Algorithm C.1 draws \hat I_u = P_u ∪ \hat N_u and takes the K-th order statistic of this mixed set. These samples are not i.i.d. from F_u, and the appendix explicitly acknowledges that the resulting estimator has a downward bias. The main text’s statement that the Monte Carlo strategy offers theoretical precision guarantees is therefore not supported by Theorem 3.1 for the implemented estimator. Please either extend the theorem to cover the all-positive-plus-sampled-negatives estimator, or clearly state that the practical estimator is a heuristic with empirical support, and discuss how the downward bias and the quantile-update interval T_β affect the condition σ_w(s_ui − β) ≥ 1/2 used in the proof of Theorem 3.2.
  3. [Section 4.2, Table 2 (6.03% claim)] The headline claim of an average 6.03% improvement over the best baseline is computed across 24 Recall@20/NDCG@20 comparisons, but the paper reports no standard deviations, confidence intervals, or significance tests. Some individual improvements are small, for example +1.53% and +2.74% on Gowalla and Book Recall@20 with the MF backbone. Given that the empirical claim is central to the paper, the authors should either report repeated-run statistics or at least state explicitly how the average is aggregated and acknowledge the variability in small-improvement cells.
minor comments (5)
  1. [References] There are duplicate references: He et al. 2017 appears as both [27] and [28], and Cao et al. 2007 appears as both [7] and [8]; these should be consolidated.
  2. [Footnotes 4 and 5] The sentence about the H^K_u > 1 assumption is repeated almost verbatim in the main text and again in Appendix B.2; one version can be shortened.
  3. [Eq. (3.3)] The inequality chain in Eq. (3.3) is dense; in particular the step labeled ③ is correct because 1/H ≤ 1 and log π_ui ≥ 0, but the text could state this explicitly to avoid confusion about the direction of the inequality.
  4. [Algorithm C.1] Line 14 reuses the symbol L_SL(u,i) for a sum over sampled negatives only, whereas Eq. (3.5) uses the same symbol for a sum over the full item set. Different notation would make the discrepancy between theory and implementation immediately visible.
  5. [Table 2 and Figure 1] The caption of Figure 1 reports a 60.4x speedup while the text says “60 times”; please make the figure caption consistent with the text.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: SL@K's upper-bound proof is derived from standard relaxations and the quantile weight is model-score-based, not fitted to NDCG@K.

full rationale

The central claim, Theorem 3.2, is proven self-contained in Appendix B.2 from the relaxations in Equations (3.3a)–(3.3e): log2(π+1)≤π, Jensen's inequality, H_K^u≥1, the exponential function as an upper bound of the Heaviside step, and the sigmoid lower bound σ_w(s−β)≥1/2 for top-K positives. These are standard mathematical inequalities, not fitted constants or self-referential definitions. The quantile β_K^u is defined from the model's own scores (Equation 3.1), and the resulting weight w_ui is computed from those scores; no parameter is fitted to the target NDCG@K metric. The 6.03% improvement is an empirical comparison on four external datasets with validation-set hyperparameter tuning, so it is not forced by construction. Self-citations to SL [82] and PSL [84] appear as baselines and design inspiration, but the upper-bound proof does not import any load-bearing theorem from those papers; it is derived directly. The acknowledged bias in the negative-sampling quantile trick (Appendix C.2) and the potential mismatch between Algorithm C.1's sampled inner sum (line 14, which omits the self term) and the full sum in Equation (3.5) are correctness or implementation gaps, not circular reductions: they do not make the claimed result equivalent to its inputs by definition. Accordingly, no circular step is exhibited.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The method introduces no new physical or model entities. It relies on four hyperparameters (tau_w, tau_d, T_beta, N) tuned on validation data, and on domain assumptions about the training dynamics (positive scores rising above negatives, DCG@K positive, mostly more than one top-K hit). The quantile estimation theory in Theorem 3.1 does not cover the practical biased sampling algorithm, which is an acknowledged gap.

free parameters (4)
  • tau_w (sigmoid temperature for quantile weight) = 0.5 to 3.0 (grid search step 0.25)
    Controls the sharpness of the sigmoid weight that gates positive samples; tuned on validation data, not derived.
  • tau_d (temperature for Softmax Loss term) = Optimal tau from SL baseline (searched in 0.01..0.5)
    Inherited from Softmax Loss; set to the best SL temperature, so it is effectively tuned.
  • T_beta (quantile update interval) = 5 or 20
    Determines how often the estimated top-K quantile is refreshed; tuned on validation data.
  • N (negative sample size) = 1000 (200 for MovieLens)
    Number of sampled negatives for both SL term and quantile estimation; chosen by hand to balance cost and accuracy.
assumptions (4)
  • domain assumption DCG@K(u) > 0 for all users during training
    Stated in Section 3.1.2 footnote 2; justifies dropping -log H and keeping the derivation well-defined. Violated for users with zero top-K hits in early training.
  • domain assumption H^K_u > 1 for the tight upper bound
    Theorem 3.2 requires more than one top-K hit for the unweighted upper bound; when H^K_u=1 the paper only proves -1/2 log DCG@K <= L.
  • domain assumption Positive item scores exceed negative item scores during training
    Used in Section 3.1.3 and Appendix C.2 to justify retaining all positives in the quantile sample and treating the estimated quantile as a valid separator; if false, the biased estimator degrades.
  • standard math Jensen's inequality, DKW inequality, quantile function inversion
    Used in the upper-bound proof and Theorem 3.1; standard results assumed without proof.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Breaking the Top-$K$ Barrier: Advancing Top-$K$ Ranking Metrics Optimization in Recommender Systems." pith.science (2026). https://pith.science/paper/PPJX5UMT

@misc{pith2026250805673,
  author       = {Pith},
  title        = {Pith review of: Breaking the Top-$K$ Barrier: Advancing Top-$K$ Ranking Metrics Optimization in Recommender Systems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PPJX5UMT}},
  note         = {Machine review of arXiv:2508.05673}
}
abstract

In the realm of recommender systems (RS), Top-$K$ ranking metrics such as NDCG@$K$ are the gold standard for evaluating recommendation performance. However, during the training of recommendation models, optimizing NDCG@$K$ poses significant challenges due to its inherent discontinuous nature and the intricate Top-$K$ truncation. Recent efforts to optimize NDCG@$K$ have either overlooked the Top-$K$ truncation or suffered from high computational costs and training instability. To overcome these limitations, we propose SoftmaxLoss@$K$ (SL@$K$), a novel recommendation loss tailored for NDCG@$K$ optimization. Specifically, we integrate the quantile technique to handle Top-$K$ truncation and derive a smooth upper bound for optimizing NDCG@$K$ to address discontinuity. The resulting SL@$K$ loss has several desirable properties, including theoretical guarantees, ease of implementation, computational efficiency, gradient stability, and noise robustness. Extensive experiments on four real-world datasets and three recommendation backbones demonstrate that SL@$K$ outperforms existing losses with a notable average improvement of 6.03%. The code is available at https://github.com/Tiny-Snow/IR-Benchmark.

Figures

Figures reproduced from arXiv: 2508.05673 by the authors.

Figure 1
Figure 1. (a) Inconsistency between NDCG and NDCG@ [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. (a) Quantile distribution. The distributions of ideal [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. NDCG@20 performance of SL@𝐾 compared with SL under varying ratios of imposed false positive instances [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: The lambda weight 𝜇𝑢𝑖𝑗 (≥ 0.005) of Top-20 items in LambdaLoss@5. A Analysis of NDCG@𝐾 Surrogate Losses In this section, we provide additional analysis of NDCG@𝐾 surro￾gate losses, including LambdaLoss@𝐾 [31] and SONG@𝐾 [59]. We investigate their gradient distributions…
Figure 5
Figure 5. Figure 5: Comparison of sample quantile estimation with and without the negative sampling trick. [PITH_FULL_IMAGE:figures/full_fig_p015_5.png]
Figure 6
Figure 6. Figure 6: Comparison of the estimated Top-𝐾 quantile ˆ𝛽 𝐾 𝑢 with the ideal Top-𝐾 quantile 𝛽 𝐾 𝑢 . from all items (cf [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]
Figure 7
Figure 7. Figure 7: Sensitivity analysis of SL@𝐾 on 𝜏𝑤. • 𝜏1, 𝜏2 ∈ {0.01, 0.05, 0.1, 0.2, 0.5}. PSL [84]. PSL is an NDCG surrogate loss that generalizes SL by substituting the exponential function with a more appropriate acti￾vation function. Yang et al. [84] proved that PSL establishes a…
Figure 8
Figure 8. Figure 8: NDCG@20 performance of SL@𝐾 compared with SL under varying ratios of imposed false positive instances. "Noise Ratio" denotes the ratio of false positive instances. "Imp." indicates the improvement of SL@𝐾 over SL [PITH_FULL_IMAGE:figures/full_fig_p022_8.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

94 extracted references · 36 canonical work pages

  1. [1]

    Panagiotis Adamopoulos and Alexander Tuzhilin. 2014. On unexpectedness in recommender systems: Or how to better expect the unexpected.ACM Transactions on Intelligent Systems and Technology (TIST) 5, 4 (2014), 1–32

    2014

  2. [2]

    Peter J Bickel and Kjell A Doksum. 2015. Mathematical statistics: basic ideas and selected topics, volumes I-II package . Chapman and Hall/CRC

    2015

  3. [3]

    Stephen Boyd, Corinna Cortes, Mehryar Mohri, and Ana Radovanovic. 2012. Accuracy at the top. Advances in neural information processing systems 25 (2012)

    2012

  4. [4]

    Sebastian Bruch, Xuanhui Wang, Michael Bendersky, and Marc Najork. 2019. An analysis of the softmax cross entropy loss for learning-to-rank with binary relevance. In Proceedings of the 2019 ACM SIGIR international conference on theory of information retrieval. 75–78

    2019

  5. [5]

    Christopher Burges, Robert Ragno, and Quoc Le. 2006. Learning to rank with nonsmooth cost functions. Advances in neural information processing systems 19 (2006)

    2006

  6. [6]

    Chris Burges, Tal Shaked, Erin Renshaw, Ari Lazier, Matt Deeds, Nicole Hamilton, and Greg Hullender. 2005. Learning to rank using gradient descent. InProceedings of the 22nd international conference on Machine learning . 89–96

    2005

  7. [8]

    Zhe Cao, Tao Qin, Tie-Yan Liu, Ming-Feng Tsai, and Hang Li. 2007. Learning to rank: from pairwise approach to listwise approach. In Proceedings of the 24th international conference on Machine learning . 129–136

    2007

  8. [9]

    George Casella and Roger Berger. 2024. Statistical inference. CRC Press

    2024

Show all 94 references
  1. [10]

    Olivier Chapelle and Mingrui Wu. 2010. Gradient descent optimization of smoothed information retrieval metrics. Information retrieval 13 (2010), 216–235

    2010

  2. [11]

    Jiawei Chen, Hande Dong, Yang Qiu, Xiangnan He, Xin Xin, Liang Chen, Guli Lin, and Keping Yang. 2021. AutoDebias: Learning to debias for recommendation. In Proceedings of the 44th international ACM SIGIR conference on research and development in information retrieval. 21–30

    2021

  3. [12]

    Jiawei Chen, Hande Dong, Xiang Wang, Fuli Feng, Meng Wang, and Xiangnan He. 2023. Bias and debias in recommender system: A survey and future directions. ACM Transactions on Information Systems 41, 3 (2023), 1–39

    2023

  4. [13]

    Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020. A simple framework for contrastive learning of visual representations. In Interna- tional conference on machine learning . PMLR, 1597–1607

    2020

  5. [14]

    Eunjoon Cho, Seth A Myers, and Jure Leskovec. 2011. Friendship and mobility: user movement in location-based social networks. In Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining . 1082–1090

    2011

  6. [15]

    Domenico Dato, Claudio Lucchese, Franco Maria Nardini, Salvatore Orlando, Raffaele Perego, Nicola Tonellotto, and Rossano Venturini. 2016. Fast ranking with additive ensembles of oblivious and non-oblivious regression trees. ACM Transactions on Information Systems (TOIS) 35, 2...

    2016

  7. [16]

    Zeshan Fayyaz, Mahsa Ebrahimian, Dina Nawara, Ahmed Ibrahim, and Rasha Kashef. 2020. Recommendation systems: Algorithms, challenges, metrics, and business opportunities. applied sciences 10, 21 (2020), 7748

    2020

  8. [17]

    Chongming Gao, Kexin Huang, Jiawei Chen, Yuan Zhang, Biao Li, Peng Jiang, Shiqi Wang, Zhong Zhang, and Xiangnan He. 2023. Alleviating matthew effect of offline reinforcement learning in interactive recommendation. In Proceedings of the 46th international ACM SIGIR conference o...

    2023

  9. [18]

    Chongming Gao, Shiqi Wang, Shijun Li, Jiawei Chen, Xiangnan He, Wenqiang Lei, Biao Li, Yuan Zhang, and Peng Jiang. 2023. CIRS: Bursting filter bubbles by coun- terfactual interactive recommender system. ACM Transactions on Information Systems 42, 1 (2023), 1–27

    2023

  10. [19]

    Aditya Grover, Eric Wang, Aaron Zweig, and Stefano Ermon. 2019. Stochastic optimization of sorting networks via continuous relaxations. arXiv preprint arXiv:1903.08850 (2019)

    2019 arXiv

  11. [20]

    Michael U Gutmann and Aapo Hyvärinen. 2012. Noise-contrastive estimation of unnormalized statistical models, with applications to natural image statistics. Journal of machine learning research 13, 2 (2012)

    2012

  12. [21]

    Lingxin Hao and Daniel Q Naiman. 2007. Quantile regression. Number 149. Sage

    2007

  13. [22]

    F Maxwell Harper and Joseph A Konstan. 2015. The movielens datasets: History and context. Acm transactions on interactive intelligent systems (tiis) 5, 4 (2015), 1–19

    2015

  14. [23]

    Ruining He and Julian McAuley. 2016. Ups and downs: Modeling the visual evolution of fashion trends with one-class collaborative filtering. In proceedings of the 25th international conference on world wide web . 507–517

    2016

  15. [24]

    Ruining He and Julian McAuley. 2016. VBPR: visual bayesian personalized ranking from implicit feedback. In Proceedings of the AAAI conference on artificial intelligence, Vol. 30

    2016

  16. [25]

    Xiangnan He and Tat-Seng Chua. 2017. Neural factorization machines for sparse predictive analytics. In Proceedings of the 40th International ACM SIGIR conference on Research and Development in Information Retrieval . 355–364

    2017

  17. [26]

    Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yongdong Zhang, and Meng Wang. 2020. Lightgcn: Simplifying and powering graph convolution network for recommendation. In Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval ...

    2020

  18. [28]

    Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017. Neural collaborative filtering. In Proceedings of the 26th international conference on world wide web . 173–182

    2017

  19. [29]

    Katja Hofmann, Anne Schuth, Alejandro Bellogin, and Maarten De Rijke. 2014. Effects of position bias on click-based recommender evaluation. In Advances in Information Retrieval: 36th European Conference on IR Research, ECIR 2014, Amsterdam, The Netherlands, April 13-16, 2014. ...

    2014

  20. [30]

    Neil Hurley and Mi Zhang. 2011. Novelty and diversity in top-n recommendation– analysis and evaluation. ACM Transactions on Internet Technology (TOIT) 10, 4 (2011), 1–30

    2011

  21. [31]

    Rolf Jagerman, Zhen Qin, Xuanhui Wang, Michael Bendersky, and Marc Najork

  22. [32]

    Ashish Jaiswal, Ashwin Ramesh Babu, Mohammad Zaki Zadeh, Debapriya Baner- jee, and Fillia Makedon. 2020. A survey on contrastive self-supervised learning. Technologies 9, 1 (2020), 2

    2020

  23. [33]

    Kalervo Järvelin and Jaana Kekäläinen. 2017. IR evaluation methods for retrieving highly relevant documents. In ACM SIGIR Forum, Vol. 51. ACM New York, NY, USA, 243–250

    2017

  24. [34]

    Johan Ludwig William Valdemar Jensen. 1906. Sur les fonctions convexes et les inégalités entre les valeurs moyennes. Acta mathematica 30, 1 (1906), 175–193

    1906

  25. [35]

    Xu Ji, Joao F Henriques, and Andrea Vedaldi. 2019. Invariant information cluster- ing for unsupervised image classification and segmentation. In Proceedings of the IEEE/CVF international conference on computer vision . 9865–9874

    2019

  26. [36]

    Wang-Cheng Kang and Julian McAuley. 2018. Self-attentive sequential recom- mendation. In 2018 IEEE international conference on data mining (ICDM) . IEEE, 197–206

    2018

  27. [37]

    Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic opti- mization. arXiv preprint arXiv:1412.6980 (2014)

    2014 arXiv

  28. [38]

    Thomas N Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016)

    2016 arXiv

  29. [39]

    Hyeyoung Ko, Suyeon Lee, Yoonseo Park, and Anna Choi. 2022. A survey of recommendation systems: recommendation models, techniques, and application fields. Electronics 11, 1 (2022), 141

    2022

  30. [40]

    R Koenker. 2005. Quantile Regression Cambridge, UK: Cambridge Univ

    2005

  31. [41]

    Yehuda Koren, Robert Bell, and Chris Volinsky. 2009. Matrix factorization tech- niques for recommender systems. Computer 42, 8 (2009), 30–37

    2009

  32. [42]

    Dong Li, Ruoming Jin, Jing Gao, and Zhi Liu. 2020. On sampling top-k rec- ommendation evaluation. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining . 2114–2124

    2020

  33. [43]

    Juanhui Li, Harry Shomer, Haitao Mao, Shenglai Zeng, Yao Ma, Neil Shah, Jiliang Tang, and Dawei Yin. 2023. Evaluating graph neural networks for link predic- tion: Current pitfalls and new benchmarking. Advances in Neural Information Processing Systems 36 (2023), 3853–3866

    2023

  34. [44]

    Siyi Lin, Chongming Gao, Jiawei Chen, Sheng Zhou, Binbin Hu, Yan Feng, Chun Chen, and Can Wang. 2025. How do recommendation models amplify popularity bias? An analysis from the spectral perspective. In Proceedings of the Eighteenth ACM International Conference on Web Search an...

    2025

  35. [45]

    Tie-Yan Liu et al. 2009. Learning to rank for information retrieval. Foundations and Trends® in Information Retrieval 3, 3 (2009), 225–331

    2009

  36. [46]

    Xiao Liu, Fanjin Zhang, Zhenyu Hou, Li Mian, Zhaoyu Wang, Jing Zhang, and Jie Tang. 2021. Self-supervised learning: Generative or contrastive. IEEE transactions on knowledge and data engineering 35, 1 (2021), 857–876

    2021

  37. [47]

    R Duncan Luce. 1959. Individual choice behavior. Vol. 4. Wiley New York

    1959

  38. [48]

    Bodhisattwa Prasad Majumder, Shuyang Li, Jianmo Ni, and Julian McAuley. 2019. Generating personalized recipes from historical user preferences. arXiv preprint arXiv:1909.00105 (2019)

    2019 arXiv

  39. [49]

    Pascal Massart. 1990. The tight constant in the Dvoretzky-Kiefer-Wolfowitz inequality. The annals of Probability (1990), 1269–1283

    1990

  40. [50]

    Julian McAuley, Christopher Targett, Qinfeng Shi, and Anton Van Den Hengel

  41. [51]

    Nicholas Metropolis, Arianna W Rosenbluth, Marshall N Rosenbluth, Augusta H Teller, and Edward Teller. 1953. Equation of state calculations by fast computing machines. The journal of chemical physics 21, 6 (1953), 1087–1092

    1953

  42. [52]

    Liqiang Nie, Wenjie Wang, Richang Hong, Meng Wang, and Qi Tian. 2019. Multi- modal dialog system: Generating responses via adaptive decoders. In Proceedings of the 27th ACM international conference on multimedia . 1098–1106

    2019

  43. [53]

    Aaron van den Oord, Yazhe Li, and Oriol Vinyals. 2018. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748 (2018). Breaking the Top-𝐾 Barrier: Advancing Top-𝐾 Ranking Metrics Optimization in Recommender Systems KDD ’25, August 3–7, 2025,...

    2018 arXiv

  44. [54]

    Emanuel Parzen. 1962. On estimation of a probability density function and mode. The annals of mathematical statistics 33, 3 (1962), 1065–1076

    1962

  45. [55]

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. 2019. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing system...

    2019

  46. [56]

    Robin L Plackett. 1975. The analysis of permutations. Journal of the Royal Statistical Society Series C: Applied Statistics 24, 2 (1975), 193–202

    1975

  47. [57]

    Przemysław Pobrotyn and Radosław Białobrzeski. 2021. Neuralndcg: Direct optimisation of a ranking metric via differentiable relaxation of sorting. arXiv preprint arXiv:2102.07831 (2021)

    2021 arXiv

  48. [58]

    Tao Qin and Tie-Yan Liu. 2013. Introducing LETOR 4.0 Datasets. CoRR abs/1306.2597 (2013). http://arxiv.org/abs/1306.2597

    2013 arXiv

  49. [59]

    Zi-Hao Qiu, Quanqi Hu, Yongjian Zhong, Lijun Zhang, and Tianbao Yang. 2022. Large-scale stochastic optimization of NDCG surrogates for deep learning with provable convergence. arXiv preprint arXiv:2202.12183 (2022)

    2022 arXiv

  50. [60]

    Ahmed Rashed, Josif Grabocka, and Lars Schmidt-Thieme. 2021. A guided learn- ing approach for item recommendation via surrogate loss learning. InProceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 605–613

    2021

  51. [61]

    Zhaochun Ren, Shangsong Liang, Piji Li, Shuaiqiang Wang, and Maarten de Rijke

  52. [62]

    Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme

  53. [63]

    Herbert Robbins and Sutton Monro. 1951. A stochastic approximation method. The annals of mathematical statistics (1951), 400–407

    1951

  54. [64]

    Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi-Rad. 2008. Collective classification in network data. AI magazine 29, 3 (2008), 93–93

    2008

  55. [65]

    Alexander Shapiro. 2017. Distributionally robust stochastic programming. SIAM Journal on Optimization 27, 4 (2017), 2258–2275

    2017

  56. [66]

    Wentao Shi, Jiawei Chen, Fuli Feng, Jizhi Zhang, Junkang Wu, Chongming Gao, and Xiangnan He. 2023. On the theories behind hard negative sampling for recommendation. In Proceedings of the ACM Web Conference 2023 . 812–822

    2023

  57. [67]

    Wentao Shi, Chenxu Wang, Fuli Feng, Yang Zhang, Wenjie Wang, Junkang Wu, and Xiangnan He. 2024. Lower-Left Partial AUC: An Effective and Efficient Optimization Metric for Recommendation. In Proceedings of the ACM on Web Conference 2024. 3253–3264

    2024

  58. [68]

    Thiago Silveira, Min Zhang, Xiao Lin, Yiqun Liu, and Shaoping Ma. 2019. How good your recommender system is? A survey on evaluations in recommendation. International Journal of Machine Learning and Cybernetics 10 (2019), 813–831

    2019

  59. [69]

    Xiaoyuan Su. 2009. A Survey of Collaborative Filtering Techniques. (2009)

    2009

  60. [70]

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems 30 (2017)

    2017

  61. [71]

    Bohao Wang, Jiawei Chen, Changdong Li, Sheng Zhou, Qihao Shi, Yang Gao, Yan Feng, Chun Chen, and Can Wang. 2024. Distributionally robust graph- based recommendation system. In Proceedings of the ACM Web Conference 2024 . 3777–3788

    2024

  62. [72]

    Bohao Wang, Feng Liu, Jiawei Chen, Xingyu Lou, Changwang Zhang, Jun Wang, Yuegang Sun, Yan Feng, Chun Chen, and Can Wang. 2025. MSL: Not All To- kens Are What You Need for Tuning LLM as a Recommender. arXiv preprint arXiv:2504.04178 (2025)

    2025 arXiv

  63. [73]

    Bohao Wang, Feng Liu, Changwang Zhang, Jiawei Chen, Yudi Wu, Sheng Zhou, Xingyu Lou, Jun Wang, Yan Feng, Chun Chen, et al. 2024. Llm4dsr: Leveraing large language model for denoising sequential recommendation. arXiv preprint arXiv:2408.08208 (2024)

    2024 arXiv

  64. [74]

    Mengdi Wang, Ethan X Fang, and Han Liu. 2017. Stochastic compositional gradient descent: algorithms for minimizing compositions of expected-value functions. Mathematical Programming 161 (2017), 419–449

    2017

  65. [75]

    Wenjie Wang, Fuli Feng, Xiangnan He, Hanwang Zhang, and Tat-Seng Chua

  66. [76]

    Xiang Wang, Xiangnan He, Meng Wang, Fuli Feng, and Tat-Seng Chua. 2019. Neural graph collaborative filtering. In Proceedings of the 42nd international ACM SIGIR conference on Research and development in Information Retrieval . 165–174

    2019

  67. [77]

    Xuanhui Wang, Cheng Li, Nadav Golbandi, Michael Bendersky, and Marc Najork

  68. [78]

    Larry Wasserman. 2004. All of statistics: a concise course in statistical inference . Springer Science & Business Media

    2004

  69. [79]

    Hongyi Wen, Longqi Yang, and Deborah Estrin. 2019. Leveraging post-click feedback for content recommendations. InProceedings of the 13th ACM Conference on Recommender Systems. 278–286

    2019

  70. [80]

    Jason Weston, Samy Bengio, and Nicolas Usunier. 2010. Large scale image annotation: learning to rank with joint word-image embeddings.Machine learning 81 (2010), 21–35

    2010

  71. [81]

    Junkang Wu, Jiawei Chen, Jiancan Wu, Wentao Shi, Jizhi Zhang, and Xiang Wang. 2024. Bsl: Understanding and improving softmax loss for recommendation. In 2024 IEEE 40th International Conference on Data Engineering (ICDE) . IEEE, 816–830

    2024

  72. [82]

    Jiancan Wu, Xiang Wang, Xingyu Gao, Jiawei Chen, Hongcheng Fu, and Tianyu Qiu. 2024. On the effectiveness of sampled softmax loss for item recommendation. ACM Transactions on Information Systems 42, 4 (2024), 1–26

    2024

  73. [83]

    Fen Xia, Tie-Yan Liu, Jue Wang, Wensheng Zhang, and Hang Li. 2008. Listwise approach to learning to rank: theory and algorithm. In Proceedings of the 25th international conference on Machine learning . 1192–1199

    2008

  74. [84]

    Weiqin Yang, Jiawei Chen, Xin Xin, Sheng Zhou, Binbin Hu, Yan Feng, Chun Chen, and Can Wang. 2024. PSL: Rethinking and Improving Softmax Loss from Pairwise Perspective for Recommendation. arXiv preprint arXiv:2411.00163 (2024)

    2024 arXiv

  75. [85]

    Zhilin Yang, William Cohen, and Ruslan Salakhudinov. 2016. Revisiting semi- supervised learning with graph embeddings. In International conference on ma- chine learning. PMLR, 40–48

    2016

  76. [86]

    Junliang Yu, Xin Xia, Tong Chen, Lizhen Cui, Nguyen Quoc Viet Hung, and Hongzhi Yin. 2023. XSimGCL: Towards extremely simple graph contrastive learning for recommendation. IEEE Transactions on Knowledge and Data Engi- neering (2023)

    2023

  77. [87]

    An Zhang, Leheng Sheng, Zhibo Cai, Xiang Wang, and Tat-Seng Chua. 2024. Empowering Collaborative Filtering with Principled Adversarial Contrastive Loss. Advances in Neural Information Processing Systems 36 (2024)

    2024

  78. [88]

    Ziwei Zhu, Jianling Wang, and James Caverlee. 2019. Improving top-k recommen- dation via jointcollaborative autoencoders. In The World Wide Web Conference. 3483–3482. KDD ’25, August 3–7, 2025, Toronto, ON, Canada. Weiqin Yang et al. /uni00000014/uni00000015/uni00000016/uni000...

    2019

  79. [95]

    can also be applied to estimate the ranking position𝜋𝑢𝑖 . Specifi- cally, we can uniformly sample𝑁 negative items ˆN𝑢 ={𝑗𝑘 : 𝑗𝑘 i.i.d.∼ Uniform(N𝑢)}𝑁 𝑘=1, then sort the sampled items𝑖∈ ˆI𝑢 =P𝑢∪ ˆN𝑢 Breaking the Top-𝐾 Barrier: Advancing Top-𝐾 Ranking Metrics Optimization in Rec...

    2025

  80. [96]

    Imp." denotes the improvement of SL@𝐾 over LambdaLoss@𝐾, while

    between the estimated DCG and the true DCG. The score function𝑠𝑢𝑖 in GuidedRec is defined as the cosine similarity between user and item embeddings. The hyperparameter search space for GuidedRec is as follows: • lr∈{ 10−1, 10−2, 10−3}. • wd∈{ 0, 10−4, 10−5, 10−6}. LambdaRank [...

    2025

  81. [2009]

    In Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence

    BPR: Bayesian personalized ranking from implicit feedback. In Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence . 452–461

  82. [2015]

    In Proceedings of the 38th international ACM SIGIR conference on research and development in information retrieval

    Image-based recommendations on styles and substitutes. In Proceedings of the 38th international ACM SIGIR conference on research and development in information retrieval. 43–52

  83. [2017]

    In Proceedings of the tenth ACM international conference on web search and data mining

    Social collaborative viewpoint regression with explainable recommenda- tions. In Proceedings of the tenth ACM international conference on web search and data mining. 485–494

  84. [2018]

    In Proceedings of the 27th ACM international conference on information and knowledge manage- ment

    The lambdaloss framework for ranking metric optimization. In Proceedings of the 27th ACM international conference on information and knowledge manage- ment. 1313–1322

  85. [2021]

    In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval

    Clicks can be cheating: Counterfactual recommendation for mitigating clickbait issue. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval . 1288–1297

  86. [2022]

    In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval

    On optimizing top-k metrics for neural ranking models. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2303–2307

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.