Pith. sign in

REVIEW 8 cited by

NeuralNDCG: Direct Optimisation of a Ranking Metric via Differentiable Relaxation of Sorting

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2102.07831 v2 pith:4HA7MDRV submitted 2021-02-15 cs.IR

Przemys{l}aw Pobrotyn , Rados{l}aw Bia{l}obrzeski This is my paper · ORCID
classification cs.IR
keywords evaluationndcgoptimisationsortingapproximationdifferentiablelossmetrics
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Learning to Rank (LTR) algorithms are usually evaluated using Information Retrieval metrics like Normalised Discounted Cumulative Gain (NDCG) or Mean Average Precision. As these metrics rely on sorting predicted items' scores (and thus, on items' ranks), their derivatives are either undefined or zero everywhere. This makes them unsuitable for gradient-based optimisation, which is the usual method of learning appropriate scoring functions. Commonly used LTR loss functions are only loosely related to the evaluation metrics, causing a mismatch between the optimisation objective and the evaluation criterion. In this paper, we address this mismatch by proposing NeuralNDCG, a novel differentiable approximation to NDCG. Since NDCG relies on the non-differentiable sorting operator, we obtain NeuralNDCG by relaxing that operator using NeuralSort, a differentiable approximation of sorting. As a result, we obtain a new ranking loss function which is an arbitrarily accurate approximation to the evaluation metric, thus closing the gap between the training and the evaluation of LTR models. We introduce two variants of the proposed loss function. Finally, the empirical evaluation shows that our proposed method outperforms previous work aimed at direct optimisation of NDCG and is competitive with the state-of-the-art methods.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 8 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Exposure-Based Reinforcement Learning to Rank

    cs.LG 2026-07 reject novelty 6.0 of 10

    An exposure-based policy-gradient estimator for ranking is proposed; the appendix retracts the key derivation, and the reported experiments were not rerun with the corrected estimator.

  2. Breaking the Top-$K$ Barrier: Advancing Top-$K$ Ranking Metrics Optimization in Recommender Systems

    cs.IR 2025-08 conditional novelty 6.0 of 10

    SoftmaxLoss@K weights each positive sample by how far its score exceeds an estimated top-K quantile, turning Softmax Loss into a smooth surrogate for NDCG@K that outperforms prior losses by about 6% on average.

  3. Risk Assessment Framework for Code LLMs via Leveraging Internal States

    cs.SE 2025-04 conditional novelty 6.0 of 10

    PtTrust pre-trains a sparse autoencoder on LLM hidden states at line boundaries, then trains a small ranker to identify erroneous code lines, reporting state-of-the-art cross-task and cross-language results.

  4. SaFRO: Satisfaction-Aware Fusion via Dual-Relative Policy Optimization for Short-Video Search

    cs.IR 2026-03 conditional novelty 5.0 of 10

    A reinforcement-learning fusion framework using query reformulation, session gaps, and next-day retention as satisfaction rewards improved short-video search ranking and retention in Kuaishou's online A/B test.

  5. Ranking Before Serving: Low-Latency LLM Serving via Pairwise Learning-to-Rank

    cs.LG 2025-09 conditional novelty 5.0 of 10

    A prompt-ranking scheduler that predicts relative output lengths with pairwise learning-to-rank and runs shorter LLM requests first reduces serving latency by up to 7.7x in burst tests.

  6. LLM-Guided Semantic Relational Reasoning for Multimodal Intent Recognition

    cs.MM 2025-09 conditional novelty 5.0 of 10

    LGSRR uses LLM-generated semantic descriptions and rankings to improve multimodal intent recognition, reporting SOTA results on MIntRec2.0 and IEMOCAP-DA with gains around 0.5-1.3%.

  7. Bringing GRACE to Recommendation: Fine-Tuning for Sustainable and Accurate Personalization

    cs.IR 2026-07 reject novelty 4.0 of 10

    A fine-tuning framework uses soft sorting and gradient projection to make pretrained food recommenders greener without retraining or reranking.

  8. SalesLoop: Reinforcement Learning from Performance Feedback for Sales Lead Ranking

    cs.LG 2026-07 conditional novelty 4.0 of 10

    A monthly closed-loop RL system for ranking sales leads, using position- and speed-weighted conversion rewards and a listwise GRPO-style loss, reports +7.9% NDCG@K offline and +4.7%/+8.7% conversion lift in a 160-day ...

Pith tools