Pith. sign in

REVIEW 4 major objections 6 minor 2 cited by

ERank shows a pointwise reranker can match listwise models on reasoning-heavy benchmarks by training with fine-grained scores and a listwise reward.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

ERank trains a pointwise reasoning-LLM reranker with fine-grained integer scoring and a listwise reinforcement-learning reward, reporting nDCG@10 of 40.2 on BRIGHT for its 32B variant with BM25 hybrid scoring.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection A practical pointwise reranker with a novel listwise-derived RL reward—worth refereeing, but the reward/inference score mismatch and overclaim need fixing. the 4 major comments →

arxiv 2509.00520 v1 pith:JCWE3HMK submitted 2025-08-30 cs.IR cs.CL

ERank: Fusing Supervised Fine-Tuning and Reinforcement Learning for Effective and Efficient Text Reranking

Yuzheng Cai , Yanzhao Zhang , Dingkun Long , Mingxin Li , Pengjun Xie , Weiguo Zheng This is my paper
classification cs.IR cs.CL
keywords text rerankingpointwise rerankerfine-grained relevance scoringlistwise rewardreinforcement learningGRPOreasoning-intensive retrievalretrieval-augmented generation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that the long-standing quality gap between pointwise and listwise rerankers is not architectural: a pointwise model that scores each document independently can match or beat listwise models on reasoning-intensive retrieval if it is trained the right way. The proposed two-stage pipeline first fine-tunes a reasoning LLM to emit a fine-grained integer relevance score (0–10) instead of a binary yes/no judgment, then refines it with reinforcement learning using a reward derived from list-level ranks rather than individual pointwise errors. This matters because pointwise models are fast, parallelizable, and practical for latency-sensitive systems like retrieval-augmented generation, while listwise models, though often more accurate, are expensive. On the reasoning-intensive BRIGHT benchmark, the 4B model reaches nDCG@10 of 38.7 and a 32B variant reaches 40.2, exceeding listwise baselines while remaining several times faster. The paper locates the cause of earlier pointwise weakness in overconfident binary judgments from chain-of-thought models and replaces them with a more expressive, generative scoring space.

Core claim

The central claim is that a pointwise reranker built on a reasoning LLM can excel at both semantic and reasoning-intensive relevance, provided it is trained in two stages. In the first stage, supervised fine-tuning abandons binary relevance classification and instead trains the model generatively to produce an integer score from 0 to 10 after a reasoning chain; the final ranking score is the integer weighted by its token probability, giving finer discrimination than yes/no outputs. In the second stage, reinforcement learning with GRPO refines the model using a rule-based listwise reward: for each query, all scores from all documents and all rollouts are sorted together, positive documents re

What carries the argument

The key mechanism is the combination of generative fine-grained scoring with a listwise-derived reward. Fine-grained scoring replaces binary labels with integers 0–10, computed as score times token probability, which prevents the overconfidence collapse seen when reasoning LLMs output yes/no judgments. The reward function, rRR, pools N×G scores across all documents and reinforcement-learning rollouts for a query, sorts them globally, and assigns reciprocal-rank rewards to positive documents while penalizing negatives that rank above positives, thereby transferring list-level ranking signal to a pointwise model that remains efficient at inference.

Load-bearing premise

The fine-grained scores that supervise the first training stage are generated entirely by a single teacher model and averaged, with no human verification; if those scores are biased or miscalibrated for the 0–10 scale, the model inherits the bias and the claimed discrimination gain evaporates.

What would settle it

Re-run the pipeline with the SFT labels produced by human raters or a second independent teacher on the same 0–10 scheme; if ERank's nDCG@10 advantage over binary-label training vanishes or shrinks on BRIGHT, fine-grained scoring is not the driver of the gain.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • A single pointwise reranker can be deployed in latency-sensitive RAG pipelines while still handling reasoning-intensive retrieval.
  • Binary relevance classification is the wrong output layer for reasoning-LLM rerankers; generative integer scoring provides better discrimination and can be adopted by other pointwise models.
  • Listwise training signals can be injected into pointwise inference through RL rewards, so global ranking awareness does not require listwise decoding at serving time.
  • The approach scales with backbone size: using identical training data, the 14B and 32B variants improve BRIGHT performance beyond the 4B model.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A natural next test is whether the same fine-grained scoring plus listwise reward transfers to other pointwise ranking tasks, such as answer selection or code retrieval, where binary relevance labels are also prevalent.
  • The paper's comparison between pointwise and listwise rewards suggests the relative-rank signal, not the teacher's absolute scores, carries much of the RL gain; if so, the method could work with preference data alone, removing the teacher-score term.
  • Because the model still generates a chain of thought before scoring, there is headroom to shorten or distill the reasoning path, which could make the reranker even faster without sacrificing ranking quality.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces ERank, a pointwise reranker built from Qwen3 reasoning LLMs. Training proceeds in two stages: SFT with generative fine-grained integer scoring (0–10) on a teacher-synthesized dataset, followed by GRPO RL with a listwise-derived reward rRR that sorts all generated scores across documents and rollouts. The authors evaluate on BRIGHT, FollowIR, TREC DL, and BEIR, reporting that ERank-4B outperforms several 7B/8B pointwise and listwise baselines, and that ERank-32B achieves nDCG@10 40.2 on BRIGHT with a BM25 hybrid, which they call state of the art. They also report substantially lower latency than listwise rerankers.

Significance. The paper addresses a real and important trade-off: reasoning-intensive reranking has relied on expensive listwise models, while pointwise models have weak score discrimination. The two-stage SFT+RL pipeline is well motivated, and the ablations in Tables 5 and 6 support the contribution of each stage and of a listwise reward over a pointwise one. The latency measurements in Figure 8 are useful and concrete. If the central claim held, the paper would be a meaningful step toward efficient reasoning-aware reranking. However, the current manuscript overstates its empirical case and contains a load-bearing inconsistency between the RL reward and the inference-time scoring rule, so the claimed transfer of listwise awareness is not established.

major comments (4)
  1. [§3.3 vs. §3.2 and Appendix B] The RL reward and the deployed ranking score are computed on different quantities. Section 3.2 defines the final ranking score as si × Pr(token=si), and Appendix B confirms that experiments use this weighted final score. Section 3.3 defines rRR by sorting the raw generated integer scores si (also shown in Figure 7), with no probability weighting. The third reward case even compares raw si to the SFT reference ti. Thus the GRPO stage optimizes raw integer rank while inference ranks by probability-weighted scores. If generation probabilities vary across documents (which CoT generation implies), an 8 with low probability can rank below a 7 with high probability at deployment, so the listwise reward may teach the model the wrong quantity. The paper must either compute rRR on the same si×Pr(si) scores used at inference, or provide evidence that raw scores and weighted scores have the same ran
  2. [Table 3 and Table 13] The abstract and introduction claim 'superior effectiveness and robustness compared to existing approaches,' but this is contradicted by the paper's own semantic benchmark results. On BEIR (full, Table 13), ERank-4B averages nDCG@10 44.8 and ERank-32B averages 47.7, while the 7B listwise baselines Rank-R1 and Rearank both average 49.0. On TREC DL (Table 11), ERank-4B averages 68.9 and ERank-32B 68.1, below Rearank-7B at 72.5. The ERank advantage is concentrated on reasoning-heavy BRIGHT and FollowIR. The claims should be reframed as: a pointwise reranker can be competitive with listwise methods on reasoning-intensive benchmarks while preserving pointwise efficiency. The current wording is not supported by the data.
  3. [Appendix C and Appendix D] There is a potential train/evaluation overlap for the BRIGHT benchmark. The SFT dataset samples 10,000 queries from the 'hard query (HQ) set of ReasonIR,' and Appendix D states that 'the instructions used for ReasonIR hard query (HQ) training set are actually those used in BRIGHT benchmark.' The paper does not state whether these ReasonIR-HQ queries are disjoint from the BRIGHT evaluation queries. If the same or overlapping queries appear in training and evaluation, the BRIGHT results, including the headline 40.2 nDCG@10, are in-domain and the SOTA claim is misleading. The authors should provide an explicit disjointness statement and, if needed, de-duplicate against the BRIGHT test set.
  4. [§3.2 Data Synthesis] The SFT labels are entirely teacher-generated by QwQ-32B, with 'consensus' obtained by averaging multiple independent generations but no human verification. The paper's fine-grained discrimination argument assumes the teacher's integer scores are meaningful and well-calibrated. If the teacher is biased for certain relevance grades, the SFT model inherits that bias, and the claimed improvement of {0,...,10} over {0,...,3} or yes/no may partly reflect teacher label noise rather than true relevance discrimination. Because the same teacher is used for initial SFT, the RL reference scores, and the rSE baseline, this is a central dependency. I would like to see at least a calibration/agreement analysis on a human-labeled sample (e.g., on TREC or BEIR judgments), or a clear external validation of the teacher scores.
minor comments (6)
  1. [§3.2] Typo: 'definded' should be 'defined'.
  2. [Appendix F] Duplicate word in 'Detailed detailed configurations'.
  3. [§4.4 / Table 6] The text says rRR demonstrates 'greater robustness and superior overall performance,' but rnDCG ties rRR on the average (33.8) and is better on FollowIR and BEIR-5, while rRR is better on BRIGHT and TREC DL. Consider tempering this claim or providing a significance analysis.
  4. [Figure 8] The latency experiment reports a single benchmark (TREC DL19) without variance or batch-size details. Please report the parallelism settings, hardware, and whether the timings include CoT generation for all baselines.
  5. [§3.2] The final ranking score si × Pr(token=si) is introduced without justification or reference. A brief motivation or an ablation on the probability multiplier would strengthen the paper.
  6. [Table 4] The SOTA comparison uses a different BM25 hybrid normalization for ERank (Z-score) than for Rank-R1-32B-v0.2 (min-max), and the gains over the published baseline are 0.2 points. Please state whether the reported difference is stable across the benchmark subsets and whether the baselines' online scores were produced under the same candidate sets and evaluation script.

Circularity Check

0 steps flagged

No significant circularity; the central empirical claims are evaluated against external benchmarks and baselines.

full rationale

The paper's derivation chain is not circular. The SFT stage trains a generative integer scorer on synthetic labels produced by QwQ-32B, with consensus averaging; this is data construction, not a prediction from the model itself. The RL stage optimizes a rule-based listwise reward (rRR) under GRPO, and the SFT model serves as the initial policy and KL reference. Using the SFT model as a reference for KL and for the negative-document squared-error term is a stability regularizer, not a definition of the target metric, so it does not make the claimed improvement equivalent to its input. Ablations (Table 5) and reward comparisons (Table 6) are measured on external benchmarks (BRIGHT, FollowIR, BEIR, TREC DL), and the headline BRIGHT numbers are compared against external listwise baselines, including numbers taken from the BRIGHT leaderboard. Citations to Qwen/QwQ are to released base models, not to an unverified theorem or to a result whose conclusion is presupposed. The reviewer-identified mismatch between the RL reward (which sorts raw generated scores) and inference (which ranks si * Pr(si)) is a potential training-inference consistency concern, but it is not an instance of a claim reducing to its own inputs by construction. No circular step can be quoted from the paper, so the honest finding is no significant circularity.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 0 invented entities

No new theoretical entities are introduced; the reward function and scoring scheme are mathematical constructs within the paper. The free parameters listed are load-bearing for the central claims and are set by hand without sensitivity analysis.

free parameters (5)
  • score_scale_integer_range = 0 to 10 (integers)
    Hand-chosen scoring granularity for the generative SFT stage; Table 1 shows 0-10 outperforms binary and 0-3 on the tested benchmarks, but the range is not optimized and is load-bearing for the claimed discrimination gain.
  • consensus_generations = 3
    Number of teacher QwQ-32B generations averaged to produce SFT labels; chosen as a cost/quality tradeoff after a 512-query study showing 10 samples beat 3.
  • hybrid_score_weights = 0.2 x z-scored BM25 + 0.8 x z-scored rerank score
    Used for the headline BRIGHT numbers (38.7 and 40.2 in the abstract). The weights are chosen by the authors (Appendix G) and differ from the min-max weights of Rank-R1-32B-v0.2; no sensitivity analysis is provided.
  • reward_negative_penalty = -1 / Phi_min(DP)
    Penalty for negative documents ranked above any positive in the listwise reward rRR; the scaling relative to the positive reciprocal-rank reward is not justified.
  • rollout_count_G = 5
    Number of RL rollouts per document used to form the N x G global sorted list; affects reward noise and training dynamics, set without a sensitivity study.
axioms (4)
  • domain assumption QwQ-32B teacher generations (reasoning chain + integer score) provide reliable training labels for relevance.
    The entire SFT dataset is synthesized by QwQ-32B with no human verification beyond consensus scoring; if the teacher's relevance judgments are biased, the fine-grained scoring signal is biased. Section 3.2.
  • ad hoc to paper The final ranking score si x Pr(token=si) is a valid monotonic estimate of relevance.
    The product of the generated integer and its token probability is introduced in Section 3.2 without a derivation or ablation vs. using si alone.
  • domain assumption GRPO optimization of the listwise reward rRR transfers to better pointwise ranking at inference time.
    The paper assumes optimizing the rank of positive documents across N x G rollouts improves the score distribution for independent pointwise inference. Section 3.3 and Table 6 support it empirically, but it is not proven.
  • domain assumption BM25 hybrid score ensembling is a fair and necessary part of the reranker evaluation on BRIGHT.
    The SOTA claims rely on GPT-4 expanded queries for retrieval and a post-hoc weighted blend with BM25; this obscures the reranker's standalone contribution. Sections 4.2, Appendix G.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of ERank: Fusing Supervised Fine-Tuning and Reinforcement Learning for Effective and Efficient Text Reranking." pith.science (2026). https://pith.science/paper/JCWE3HMK

@misc{pith2026250900520,
  author       = {Pith},
  title        = {Pith review of: ERank: Fusing Supervised Fine-Tuning and Reinforcement Learning for Effective and Efficient Text Reranking},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JCWE3HMK}},
  note         = {Machine review of arXiv:2509.00520}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Text reranking models are a crucial component in modern systems like Retrieval-Augmented Generation, tasked with selecting the most relevant documents prior to generation. However, current Large Language Models (LLMs) powered rerankers often face a fundamental trade-off. On one hand, Supervised Fine-Tuning based pointwise methods that frame relevance as a binary classification task lack the necessary scoring discrimination, particularly for those built on reasoning LLMs. On the other hand, approaches designed for complex reasoning often employ powerful yet inefficient listwise formulations, rendering them impractical for low latency applications. To resolve this dilemma, we introduce ERank, a highly effective and efficient pointwise reranker built from a reasoning LLM that excels across diverse relevance scenarios. We propose a novel two-stage training pipeline that begins with Supervised Fine-Tuning (SFT). In this stage, we move beyond binary labels and train the model generatively to output fine grained integer scores, which significantly enhances relevance discrimination. The model is then further refined using Reinforcement Learning (RL) with a novel, listwise derived reward. This technique instills global ranking awareness into the efficient pointwise architecture. We evaluate the ERank reranker on the BRIGHT, FollowIR, TREC DL, and BEIR benchmarks, demonstrating superior effectiveness and robustness compared to existing approaches. On the reasoning-intensive BRIGHT benchmark, our ERank-4B achieves an nDCG@10 of 38.7, while a larger 32B variant reaches a state of the art nDCG@10 of 40.2.

Figures

Figures reproduced from arXiv: 2509.00520 by Dingkun Long, Mingxin Li, Pengjun Xie, Weiguo Zheng, Yanzhao Zhang, Yuzheng Cai.

Figure 1
Figure 1. Figure 1: Semantic relevance refers to the traditional under [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: ERANK-4B achieves state-of-the-art performance among pointwise rerankers using candidate documents re￾trieved by BM25 with original queries. Under retrieval set￾tings in Section 4.2, ERANK-4B and 32B further achieve the nDCG@10 of 38.7 and 40.2 on BRIGHT, respectively. 2024; Gupta, Ranjan, and Singh 2024; Wang et al. 2024) and agentic systems (Huang et al. 2024; Li et al. 2024). These paradigms demand capa… view at source ↗
Figure 3
Figure 3. Figure 3: Comparison of different reranking paradigms. [PITH_FULL_IMAGE:figures/full_fig_p002_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Overview of the two-stage fine-tuning pipeline for the pointwise ER [PITH_FULL_IMAGE:figures/full_fig_p003_4.png] view at source ↗
Figure 6
Figure 6. Figure 6: Distributions of normalized probability with non [PITH_FULL_IMAGE:figures/full_fig_p004_6.png] view at source ↗
Figure 5
Figure 5. Figure 5: Prompt for scoring with integers from 0 to 10. [PITH_FULL_IMAGE:figures/full_fig_p004_5.png] view at source ↗
Figure 7
Figure 7. Figure 7: Example for rule-based listwise reward rRR when there are G = 2 rollouts and N = 3 documents for query q. the old policy πold. The policy πθ is then updated by opti￾mizing the GRPO objective. This objective is built around a clipped importance sampling estimator, which evaluates the advantage of each trajectory relative to others in the group. To prevent the policy from deviating too drastically from the r… view at source ↗
Figure 8
Figure 8. Figure 8: Latency for returning the complete reranked list [PITH_FULL_IMAGE:figures/full_fig_p007_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Illustrating examples for semantic relevance and reasoning-intensive relevance. [PITH_FULL_IMAGE:figures/full_fig_p010_9.png] view at source ↗
Figure 11
Figure 11. Figure 11: Prompt for Qwen3-32B using binary outputs. [PITH_FULL_IMAGE:figures/full_fig_p011_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Prompt for QwQ-32B using binary outputs. [PITH_FULL_IMAGE:figures/full_fig_p012_12.png] view at source ↗
Figure 14
Figure 14. Figure 14: Reward curve during GRPO training for ER [PITH_FULL_IMAGE:figures/full_fig_p013_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: Response length curve during GRPO training for [PITH_FULL_IMAGE:figures/full_fig_p014_15.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. A Survey of Reasoning-Intensive Retrieval: Progress and Challenges

    cs.IR 2026-04 unverdicted novelty 6.0

    A survey that categorizes RIR benchmarks by domain and modality, proposes a taxonomy for integrating reasoning into retrieval pipelines, and outlines key challenges.

  2. GroupRank: A Groupwise Paradigm for Effective and Efficient Passage Reranking with LLMs

    cs.IR 2025-11 unverdicted novelty 5.0

    GroupRank uses groupwise LLM reranking with answer-free data synthesis and a group-ranking reward to reach 65.2 NDCG@10 on BRIGHT while providing 6.4x faster inference than listwise baselines.

Reference graph

Works this paper leans on

47 extracted references · 22 canonical work pages · cited by 2 Pith papers · 2 internal anchors

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Bajaj, P.; Campos, D.; Craswell, N.; Deng, L.; Gao, J.; Liu, X.; Majumder, R.; McNamara, A.; Mitra, B.; Nguyen, T.; et al. 2016. MS MARCO : A human generated machine reading comprehension dataset. arXiv preprint arXiv:1611.09268

    Pith/arXiv arXiv 2016

  4. [4]

    Chuang, Y.-N.; Chen, C.-M.; Wang, C.-J.; Tsai, M.-F.; Fang, Y.; and Lim, E.-P. 2020. TPR: Text-aware preference ranking for recommender systems. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management, 215--224

    work page 2020

  5. [5]

    Craswell, N.; Mitra, B.; Yilmaz, E.; and Campos, D. 2021. Overview of the TREC 2020 deep learning track. arXiv:2102.07662

    Pith/arXiv arXiv 2021

  6. [6]

    Craswell, N.; Mitra, B.; Yilmaz, E.; Campos, D.; and Voorhees, E. M. 2020. Overview of the TREC 2019 deep learning track. arXiv:2003.07820

    Pith/arXiv arXiv 2020

  7. [7]

    DeepSeek AI . 2025. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. arXiv:2501.12948

    Pith/arXiv arXiv 2025

  8. [8]

    Gao, J.; Chen, B.; Zhao, X.; Liu, W.; Li, X.; Wang, Y.; Wang, W.; Guo, H.; and Tang, R. 2025. Llm4rerank: Llm-based auto-reranking framework for recommendations. In Proceedings of the ACM on Web Conference 2025, 228--239

    work page 2025

  9. [9]

    Gao, L.; Dai, Z.; and Callan, J. 2021. Rethink training of BERT rerankers in multi-stage retrieval pipeline. In European Conference on Information Retrieval, 280--286. Springer

    work page 2021

  10. [10]

    Gupta, S.; Ranjan, R.; and Singh, S. N. 2024. A comprehensive survey of retrieval-augmented generation (RAG) : Evolution, current landscape and future directions. arXiv preprint arXiv:2410.12837

    Pith/arXiv arXiv 2024

  11. [11]

    J.; Shen, Y.; Wallis, P.; Allen-Zhu, Z.; Li, Y.; Wang, S.; Wang, L.; Chen, W.; et al

    Hu, E. J.; Shen, Y.; Wallis, P.; Allen-Zhu, Z.; Li, Y.; Wang, S.; Wang, L.; Chen, W.; et al. 2022. Lora: Low-rank adaptation of large language models. ICLR, 1(2): 3

    2022

  12. [12]

    Huang, X.; Liu, W.; Chen, X.; Wang, X.; Wang, H.; Lian, D.; Wang, Y.; Tang, R.; and Chen, E. 2024. Understanding the planning of LLM agents: A survey. arXiv preprint arXiv:2402.02716

    Pith/arXiv arXiv 2024

  13. [13]

    ielabgroup. 2025. Rank-R1-32B-v0.2 . https://huggingface.co/ielabgroup/Rank-R1-32B-v0.2. Accessed: 2025-07-24

    work page 2025

  14. [14]

    jataware. 2025. XRR2: Expand Retrieve Rerank Rerank - simple method with strong results on BRIGHT benchmark. https://github.com/jataware/XRR2. Accessed: 2025-07-24

    work page 2025

  15. [15]

    Lee, J.; Yun, S.; Kim, H.; Ko, M.; and Kang, J. 2018. Ranking Paragraphs for Improving Answer Recall in Open-Domain Question Answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, 565--569

    work page 2018

  16. [16]

    Li, X.; Wang, S.; Zeng, S.; Wu, Y.; and Yang, Y. 2024. A survey on LLM-based multi-agent systems: workflow, infrastructure, and challenges. Vicinagearth, 1(1): 9

    work page 2024

  17. [17]

    Liang, P.; Bommasani, R.; Lee, T.; Tsipras, D.; Soylu, D.; Yasunaga, M.; Zhang, Y.; Narayanan, D.; Wu, Y.; Kumar, A.; et al. 2022. Holistic evaluation of language models. arXiv preprint arXiv:2211.09110

    Pith/arXiv arXiv 2022

  18. [18]

    Lin, J.; Ma, X.; Lin, S.-C.; Yang, J.-H.; Pradeep, R.; and Nogueira, R. 2021. Pyserini: A Python toolkit for reproducible information retrieval research with sparse and dense representations. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2356--2362

    work page 2021

  19. [19]

    Lin, J.; Nogueira, R.; and Yates, A. 2022. Pretrained transformers for text ranking: Bert and beyond. Springer Nature

    work page 2022

  20. [20]

    Liu, J.; Ma, Y.; Zhao, R.; Zheng, J.; Ma, Q.; and Kang, Y. 2025. ListConRanker: A Contrastive Text Reranker with Listwise Encoding. arXiv preprint arXiv:2501.07111

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  21. [21]

    Liu, Z.; Zhou, Y.; Zhu, Y.; Lian, J.; Li, C.; Dou, Z.; Lian, D.; and Nie, J.-Y. 2024. Information retrieval meets large language models. In Companion Proceedings of the ACM Web Conference 2024, 1586--1589

    work page 2024

  22. [22]

    Ma, X.; Wang, L.; Yang, N.; Wei, F.; and Lin, J. 2024. Fine-tuning llama for multi-stage text retrieval. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2421--2425

    work page 2024

  23. [23]

    Ma, X.; Zhang, X.; Pradeep, R.; and Lin, J. 2023. Zero-shot listwise document reranking with a large language model. arXiv preprint arXiv:2305.02156

    Pith/arXiv arXiv 2023

  24. [24]

    Niu, T.; Joty, S.; Liu, Y.; Xiong, C.; Zhou, Y.; and Yavuz, S. 2024. JudgeRank: Leveraging Large Language Models for Reasoning-Intensive Reranking. arXiv preprint arXiv:2411.00142

    Pith/arXiv arXiv 2024

  25. [25]

    OpenAI. 2024. OpenAI o1 System Card. arXiv:2412.16720

    Pith/arXiv arXiv 2024

  26. [26]

    Qin, Z.; Jagerman, R.; Hui, K.; Zhuang, H.; Wu, J.; Yan, L.; Shen, J.; Liu, T.; Liu, J.; Metzler, D.; et al. 2024. Large Language Models are Effective Text Rankers with Pairwise Ranking Prompting. In Findings of the Association for Computational Linguistics: NAACL 2024, 1504--1518

    work page 2024

  27. [27]

    Qwen Team . 2025 a . Qwen3 Technical Report. arXiv:2505.09388

    Pith/arXiv arXiv 2025

  28. [28]

    Qwen Team . 2025 b . QwQ -32B: Embracing the Power of Reinforcement Learning

    work page 2025

  29. [29]

    V.; Rus, D.; Low, B

    Shao, R.; Qiao, R.; Kishore, V.; Muennighoff, N.; Lin, X. V.; Rus, D.; Low, B. K. H.; Min, S.; Yih, W.-t.; Koh, P. W.; et al. 2025. ReasonIR : Training Retrievers for Reasoning Tasks. arXiv preprint arXiv:2504.20595

    Pith/arXiv arXiv 2025

  30. [30]

    Shao, Z.; Wang, P.; Zhu, Q.; Xu, R.; Song, J.; Bi, X.; Zhang, H.; Zhang, M.; Li, Y.; Wu, Y.; et al. 2024. DeepSeekMath : Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300

    Pith/arXiv arXiv 2024

  31. [31]

    Sheng, G.; Zhang, C.; Ye, Z.; Wu, X.; Zhang, W.; Zhang, R.; Peng, Y.; Lin, H.; and Wu, C. 2024. HybridFlow : A Flexible and Efficient RLHF Framework. arXiv preprint arXiv: 2409.19256

    Pith/arXiv arXiv 2024

  32. [32]

    S.; Tang, M.; Sun, R.; Yoon, J.; Arik, S

    Su, H.; Yen, H.; Xia, M.; Shi, W.; Muennighoff, N.; Wang, H.-y.; Liu, H.; Shi, Q.; Siegel, Z. S.; Tang, M.; Sun, R.; Yoon, J.; Arik, S. O.; Chen, D.; and Yu, T. 2024. BRIGHT : A Realistic and Challenging Benchmark for Reasoning-Intensive Retrieval

    work page 2024

  33. [33]

    S.; Tang, M.; Sun, R.; Yoon, J.; Arik, S

    Su, H.; Yen, H.; Xia, M.; Shi, W.; Muennighoff, N.; Wang, H.-y.; Liu, H.; Shi, Q.; Siegel, Z. S.; Tang, M.; Sun, R.; Yoon, J.; Arik, S. O.; Chen, D.; and Yu, T. 2025. BRIGHT Benchmark Online Website . https://brightbenchmark.github.io/. Accessed: August 26, 2025

    work page 2025

  34. [34]

    Sun, W.; Yan, L.; Ma, X.; Wang, S.; Ren, P.; Chen, Z.; Yin, D.; and Ren, Z. 2023. Is ChatGPT good at search? investigating large language models as re-ranking agents. arXiv preprint arXiv:2304.09542

    Pith/arXiv arXiv 2023

  35. [35]

    Thakur, N.; Reimers, N.; R \"u ckl \'e , A.; Srivastava, A.; and Gurevych, I. 2021. BEIR : A Heterogeneous Benchmark for Zero-shot Evaluation of Information Retrieval Models. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2)

    work page 2021

  36. [36]

    Wang, X.; Wang, Z.; Gao, X.; Zhang, F.; Wu, Y.; Xu, Z.; Shi, T.; Wang, Z.; Li, S.; Qian, Q.; et al. 2024. Searching for best practices in retrieval-augmented generation. arXiv preprint arXiv:2407.01219

    Pith/arXiv arXiv 2024

  37. [37]

    Weller, O.; Chang, B.; MacAvaney, S.; Lo, K.; Cohan, A.; Van Durme, B.; Lawrie, D.; and Soldaini, L. 2025 a . F ollow IR : Evaluating and Teaching Information Retrieval Models to Follow Instructions. In Chiruzzo, L.; Ritter, A.; and Wang, L., eds., Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computation...

    work page 2025

  38. [38]

    Weller, O.; Ricci, K.; Yang, E.; Yates, A.; Lawrie, D.; and Van Durme, B. 2025 b . Rank1: Test-time compute for reranking in information retrieval. arXiv preprint arXiv:2502.18418

    Pith/arXiv arXiv 2025

  39. [39]

    Weller, O.; Van Durme, B.; Lawrie, D.; Paranjape, A.; Zhang, Y.; and Hessel, J. 2024. Promptriever: Instruction-trained retrievers can be prompted like language models. arXiv preprint arXiv:2409.11136

    Pith/arXiv arXiv 2024

  40. [40]

    Wu, S.; Xiong, Y.; Cui, Y.; Wu, H.; Chen, C.; Yuan, Y.; Huang, L.; Liu, X.; Kuo, T.-W.; Guan, N.; et al. 2024. Retrieval-augmented generation for natural language processing: A survey. arXiv preprint arXiv:2407.13193

    Pith/arXiv arXiv 2024

  41. [41]

    Zhang, J.; Chen, Y.; Liu, C.; Niu, N.; and Wang, Y. 2023. Empirical evaluation of ChatGPT on requirements information retrieval under zero-shot setting. In 2023 International Conference on Intelligent Computing and Next Generation Networks (ICNGN), 1--6. IEEE

    work page 2023

  42. [42]

    Zhang, L.; Wang, B.; Qiu, X.; Reddy, S.; and Agrawal, A. 2025. Rerank: Reasoning Re-ranking Agent via Reinforcement Learning. arXiv preprint arXiv:2505.20046

    Pith/arXiv arXiv 2025

  43. [43]

    Zhang, L.; Zhang, Y.; Long, D.; Xie, P.; Zhang, M.; and Zhang, M. 2024. A Two-Stage Adaptation of Large Language Models for Text Ranking. In ACL (Findings)

    work page 2024

  44. [44]

    Zhang, Y.; Long, D.; Xu, G.; and Xie, P. 2022. HLATR: enhance multi-stage text retrieval with hybrid list aware transformer reranking. arXiv preprint arXiv:2205.10569

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  45. [45]

    Zheng, Y.; Zhang, R.; Zhang, J.; Ye, Y.; Luo, Z.; Feng, Z.; and Ma, Y. 2024. LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations). Bangkok, Thailand: Association for Computational Linguistics

    2024

  46. [46]

    Zhuang, S.; Ma, X.; Koopman, B.; Lin, J.; and Zuccon, G. 2025. Rank-R1 : Enhancing reasoning in llm-based document rerankers via reinforcement learning. arXiv preprint arXiv:2503.06034

    Pith/arXiv arXiv 2025

  47. [47]

    Zhuang, S.; Zhuang, H.; Koopman, B.; and Zuccon, G. 2024. A setwise approach for effective and highly efficient zero-shot ranking with large language models. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, 38--47

    work page 2024

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.