Pith. sign in

REVIEW 3 major objections 6 minor 54 references

A pointwise LLM reranker learns to call tools only when uncertain, matching fast baselines while beating larger closed-book reasoners.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-14 10:50 UTC pith:FRWQNECD

load-bearing objection Solid engineering paper that makes selective tool use work for pointwise LLM reranking; SOTA + throughput numbers are real, but the causal role of the adaptive policy is only modestly isolated. the 3 major comments →

arxiv 2607.10555 v1 pith:FRWQNECD submitted 2026-07-12 cs.IR cs.AIcs.CL

Tool-Adaptive LLM Reranker

Zichuan Liu , Ruijin Hua This is my paper
classification cs.IR cs.AIcs.CL
keywords LLM rerankingtool-augmented agentspointwise rankingcost-aware reinforcement learningGRPOhallucination mitigationinformation retrieval
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Generative LLM rerankers hallucinate on complex or long-tail queries because they are closed-book. Calling a search tool for every document is too slow for production. TALRanker treats each pointwise relevance decision as a short agentic process: the model reasons, then either emits a yes/no score immediately or suspends to retrieve evidence before scoring. Training has two stages. First, a masked hybrid loss teaches binary scoring without erasing the model’s ability to generate reasoning and tool calls. Second, reinforcement learning with an asymmetric cost-aware reward rewards correct direct answers, mildly penalizes tool use, and heavily punishes confident wrong answers that never used a tool. The resulting policy stays as fast as ordinary pointwise scorers on easy queries and selectively grounds itself on hard ones, producing state-of-the-art NDCG on both standard and reasoning-heavy benchmarks.

Core claim

Pointwise relevance scoring can be cast as an agentic Markov decision process and optimized with language-preserving warm-up plus asymmetric cost-aware GRPO so that the policy autonomously bypasses tools when confident and retrieves evidence only when necessary, simultaneously matching non-reasoning throughput and surpassing larger closed-book reasoning models on TREC, BEIR, and BRIGHT.

What carries the argument

The asymmetric accuracy reward: error within threshold γ yields a smooth positive score; error beyond γ with zero tool hops yields a catastrophic −1; the same large error after tool use yields 0. Combined with a small per-hop latency penalty λ, this forces the policy to learn its own epistemic boundary.

Load-bearing premise

The policy’s learned confidence signal, shaped by a fixed reward schedule on a modest MS MARCO subset and one commercial search API, correctly decides when external evidence is needed and that the returned evidence reliably fixes hallucinations without introducing new systematic errors.

What would settle it

Force zero-hop inference (or replace the search tool with noise) on BRIGHT and measure whether the reported NDCG gains and selective invocation rates disappear, especially on knowledge-heavy subsets where the paper claims the largest lifts.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Production search engines can keep pointwise O(1) parallelism while still grounding hard documents without paying tool latency on every pair.
  • Compact models (4B–8B) can outperform larger closed-book reasoners by routing only uncertain cases to external knowledge.
  • The same cost-aware MDP pattern can be reused for any scoring task that must trade accuracy against external API cost.
  • Language-preserving warm-up is a practical recipe for adding agentic actions without collapsing generative capacity.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the reward thresholds are poorly tuned or the search API is noisy, the policy may either over-call tools or silently re-learn closed-book hallucination, so calibration and tool quality become first-class system parameters.
  • The same selective-routing idea could transfer to multimodal or multi-tool agentic rankers once the base model’s tool-use format is stable.
  • Distilling the adaptive routing policy into even smaller models, as the paper’s conclusion suggests, would test how much of the gain is pure knowledge versus learned meta-decision skill.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes TALRanker, which reformulates pointwise LLM relevance scoring as a token-level agentic MDP. The policy generates reasoning trajectories and may emit a special <tool_call> action to retrieve external evidence (via Perplexity) before terminating with a yes/no token whose normalized probability yields the continuous score. Training is two-stage: (1) a language-preserving hybrid warm-up (BCE on the terminal token + masked KL on preceding tokens, Eq. 2) on MS MARCO pairs to install discriminative scoring without erasing generative/tool capacity; (2) GRPO with an asymmetric cost-aware reward (R_format + R_acc + R_cost, Eqs. 3–5) that penalizes every hop by λ while awarding a severe −1 only for high-error zero-hop predictions. Evaluations claim SOTA NDCG@10 on TREC-DL, BEIR (avg. 46.0–46.3) and BRIGHT (avg. 28.2–29.7) for 4B/8B Qwen3 variants, with measured tool-invocation rates ≪2 % (often 0 % on TREC) and single-GPU throughput comparable to non-reasoning pointwise baselines (Fig. 1).

Significance. If the adaptive-routing claim holds, the work supplies a practical solution to the accuracy–latency dilemma that currently forces industrial pointwise rerankers either to stay closed-book (and hallucinate on OOD/reasoning queries) or to pay full tool cost on every pair. The language-preserving warm-up, the explicit asymmetric reward design, the scaling analysis across 0.6B–8B, the KL-ablation showing catastrophic forgetting (Fig. 4), and the hyper-parameter sensitivity plots (Fig. 5) are concrete engineering contributions that other agentic IR systems can reuse. Throughput numbers that match Direct-Point/TFRank while beating larger closed-book reasoners (Rank-R1-14B, ReasonRank-7B) would be immediately useful for production RAG pipelines.

major comments (3)
  1. The central claim that the asymmetric cost-aware GRPO (Eqs. 3–5, λ=0.02, γ∈[0.2,0.3]) is what produces the adaptive policy is only weakly isolated. Table 2 shows 4B/8B invocation rates of 0–1.25 %; Table 6 reports only +2.1–3.1 NDCG from “with-tools” versus an artificial forced-zero-hop suffix constraint. No experiment trains an otherwise identical model under the same GRPO objective with tools permanently disabled, or with a non-adaptive always/never baseline. Consequently it remains under-determined whether the headline SOTA numbers are driven by learned epistemic routing or largely by the language-preserving warm-up (Eq. 2) plus base Qwen3 capacity, with the rare tool calls contributing only a residual. A clean ablation is required for the “tool-adaptive” claim to be load-bearing.
  2. Reproducibility of the external-evidence component is limited. All tool observations come from a commercial black-box API (Perplexity Pro). The 10 k MS MARCO subset used for GRPO is not released or described beyond “randomly select,” and the exact tool-call JSON schema and observation formatting appear only in the appendix prompt. Without a public, deterministic retrieval backend or a frozen snapshot of the returned evidence, independent verification of the BRIGHT gains (especially TheoT./TheoQ./LeetCode) is impossible.
  3. The framework’s dependence on base-model tool-use skill is acknowledged but under-quantified for the claimed generality. Figure 3 and the text show that the 0.6 B model collapses under GRPO (format failures, zero-reward episodes) and that even the 1.7 B model still invokes tools on nearly every BRIGHT query. The paper therefore demonstrates that the MDP + reward works only once the base LLM already possesses reliable <tool_call>/<answer> trajectories; the contribution is closer to “reward shaping for already-capable models” than to a general method that endows arbitrary LLMs with adaptive reranking.
minor comments (6)
  1. Inconsistent naming: the title and abstract use “TALRanker,” while Tables 1–3 and Figure 1 use “TALReranker.” Unify throughout.
  2. Figure 1 x-axis is labeled “Throughput (Queries/Hour)” but the caption and text never state whether this is end-to-end (including tool latency) or pure generation; clarify.
  3. Eq. (1) normalizes only yes/no probabilities; the prompt templates also allow free-form reasoning. State explicitly whether any probability mass on other tokens is discarded or renormalized.
  4. Hyper-parameter table (Table 5) lists GRPO group size G only implicitly; give the concrete value used for the reported runs.
  5. Several baseline numbers are marked “† from official repositories”; for fairness, re-evaluate the strongest competitors (Rank-R1-14B, ReasonRank-7B) under the identical candidate pools and evaluation scripts used for TALRanker.
  6. Appendix Algorithm 1 returns a fallback score of 0.0 on malformed trajectories; report how often this fallback is triggered on the test sets.

Circularity Check

0 steps flagged

No circularity: empirical IR method whose SOTA NDCG/throughput claims are measured on external benchmarks, not reduced by construction to training inputs or self-citations.

full rationale

TALRanker defines an agentic MDP for pointwise scoring (states as context+observations, actions including <tool_call>/<answer>, score s from yes/no terminal probs via Eq. 1), warms up with masked hybrid BCE+KL (Eq. 2) on MS MARCO pairs, then optimizes via GRPO under the asymmetric cost-aware reward R = R_format + R_acc + R_cost (Eqs. 3–5, with piecewise R_acc depending on |s−s_gt| vs γ and N_hops, plus −λ N_hops). The load-bearing claims are empirical: NDCG@10 on held-out TREC-DL/BEIR/BRIGHT (Tables 1, 3), measured tool-invocation rates (Table 2), throughput (Fig. 1), and modest tool-available vs forced-zero-hop deltas (Table 6). None of these quantities is defined by or algebraically forced by the fitted reward hyperparameters (λ=0.02, γ∈[0.2,0.3]) or the MS MARCO training subset; the policy is free to (and does) produce different invocation rates and rankings. No self-definitional loop, no fitted parameter re-labeled as prediction, no uniqueness theorem imported from overlapping authors, and no ansatz smuggled via self-citation. The single overlapping-author baseline citation (ReasonRank) is used only for comparison, not as a premise. The paper is therefore self-contained against external benchmarks; any experimental-isolation weaknesses (e.g., incomplete no-tool ablations) are correctness issues, not circularity.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 3 invented entities

Central performance claims rest on standard IR evaluation practice, public base models, a commercial search tool, and a small set of hand-chosen reward and regularization coefficients that control the accuracy-latency tradeoff. No new physical entities; the invented pieces are algorithmic constructs whose value is measured by external NDCG and throughput.

free parameters (4)
  • latency penalty λ = 0.02
    Scales R_cost = −λ·N_hops; set to 0.02 after ablation; directly controls how often tools are used and therefore the claimed efficiency.
  • accuracy tolerance γ = 0.2
    Threshold in the piecewise R_acc that decides whether an error is catastrophic (−1) or zero when tools were used; set in [0.2, 0.3]; defines the epistemic boundary the policy learns.
  • KL coefficient β = 0.01
    Strength of token-wise KL to the reference model on non-terminal tokens in the warm-up hybrid loss; set to 0.01; required to avoid catastrophic forgetting of generative/tool capacity.
  • GRPO group size G and MS MARCO RL subset size = G unspecified in main text; 10k samples
    Number of sampled trajectories per prompt and the 10k-sample subset used for Stage 2; affect variance of relative advantages and the distribution of learned routing.
axioms (4)
  • domain assumption Pointwise yes/no token-probability normalization yields a usable continuous relevance score for ranking.
    Eq. (1) and standard practice in generative pointwise rerankers; assumed throughout evaluation.
  • domain assumption External search tool observations reduce factual hallucination more than they introduce noise or latency that cannot be offset by the reward.
    Motivation and R_acc design; tool is Perplexity API.
  • domain assumption GRPO with the stated scalar reward is a valid policy optimizer for the token-level MDP.
    Stage 2 methodology; relies on Guo et al. 2025 GRPO without re-deriving convergence.
  • ad hoc to paper Masked KL on non-terminal tokens preserves generative and tool-calling capacity while BCE on the terminal token teaches discrimination.
    Eq. (2) hybrid loss; central to the two-stage claim and the 0.6B collapse ablation.
invented entities (3)
  • TALRanker agentic MDP for pointwise scoring no independent evidence
    purpose: Unify internal reasoning, optional tool hops, and terminal yes/no scoring under one policy.
    Core formulation (S,A,P,R) with special actions <tool_call> and <answer>.
  • Asymmetric cost-aware reward (R_format + R_acc + R_cost) no independent evidence
    purpose: Force bypass when confident and tool use when uncertain by making zero-hop errors catastrophic.
    Eqs. (3)–(5); load-bearing for the adaptive-routing claim.
  • Language-preserving hybrid warm-up loss (BCE + masked KL) no independent evidence
    purpose: Install discriminative scoring without erasing generative/tool trajectories needed for Stage 2.
    Eq. (2); ablation shows collapse without it on small models.

pith-pipeline@v1.1.0-grok45 · 23866 in / 3109 out tokens · 40247 ms · 2026-07-14T10:50:57.878042+00:00 · methodology

0 comments
read the original abstract

Generative Large Language Models (LLMs) have revolutionized information retrieval, yet their strictly parametric nature frequently leads to severe factual hallucinations when confronted with complex queries beyond their epistemic boundaries. While external tool-calling can mitigate this, indiscriminately invoking search tools for every document during reranking incurs prohibitive latency overheads, creating an intractable accuracy-efficiency dilemma. To address this challenge, we propose TALRanker, a novel framework that formalizes pointwise relevance scoring as an agentic Markov decision process. We optimize it via a two-stage training paradigm. An initial warm-up utilizes a language-preserving hybrid loss to prevent the catastrophic forgetting of native generative capacities. Subsequently, an asymmetric cost-aware reward equipped in reinforcement learning forces the policy to autonomously bypass tools for maximum efficiency when confident, while selectively retrieving external evidence to avert severe hallucination penalties when uncertain. Extensive evaluations demonstrate that TALRanker achieves state-of-the-art performance across standard and reasoning-intensive retrieval benchmarks, matching throughput with pointwise rerankers while outperforming parameter-heavy reasoning models.

Figures

Figures reproduced from arXiv: 2607.10555 by Ruijin Hua, Zichuan Liu.

Figure 1
Figure 1. Figure 1: The tradeoff between precision (NDCG@10) and [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: The overall training framework of TALRanker. Stage 1 performs a warm-up utilizing a language-preserving hybrid [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Ablation of training stages across model scales on [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Generative performance on nine QA benchmarks. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Ablation study on the penalty intensity λ and the tolerance threshold γ. The left y-axis shows the average NDCG@10 across the BRIGHT benchmark, while the right y-axis represents the overall tool-calling rate. 4B variant across all BRIGHT subsets [PITH_FULL_IMAGE:figures/full_fig_p011_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: A 1-Hop inference trajectory. The model utilizes the [PITH_FULL_IMAGE:figures/full_fig_p012_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: A 0-Hop inference trajectory. The model confi [PITH_FULL_IMAGE:figures/full_fig_p012_7.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

54 extracted references · 13 linked inside Pith

  1. [1]

    From Exploration to Mastery: Enabling

    Changle Qu and Sunhao Dai and Xiaochi Wei and Hengyi Cai and Shuaiqiang Wang and Dawei Yin and Jun Xu and Ji-Rong Wen , pages =. From Exploration to Mastery: Enabling. International Conference on Learning Representations , year=

  2. [2]

    Erdogan, Lutfi Eren and Lee, Nicholas and Jha, Siddharth and Kim, Sehoon and Tabrizi, Ryan and Moon, Suhong and Hooper, Coleman Richard Charles and Anumanchipalli, Gopala and Keutzer, Kurt and Gholami, Amir , booktitle =

  3. [3]

    Neural Information Processing System , pages =

    Toolformer: Language Models Can Teach Themselves to Use Tools , author=. Neural Information Processing System , pages =

  4. [4]

    International Conference on Learning Representations , year=

    ReAct: Synergizing Reasoning and Acting in Language Models , author=. International Conference on Learning Representations , year=

  5. [5]

    arXiv preprint arXiv:2601.14192 , year=

    Toward Efficient Agents: Memory, Tool learning, and Planning , author=. arXiv preprint arXiv:2601.14192 , year=

    Pith/arXiv arXiv

  6. [6]

    International Conference on Learning Representations , pages=

    Efficient Tool Use with Chain-of-Abstraction Reasoning , author=. International Conference on Learning Representations , pages=

  7. [7]

    arXiv preprint arXiv:2504.14870 , year=

    Acting Less is Reasoning More! Teaching Language Model to Act Efficiently , author=. arXiv preprint arXiv:2504.14870 , year=

    Pith/arXiv arXiv

  8. [8]

    arXiv preprint arXiv:2511.21689 , year=

    ToolOrchestra: Elevating Intelligence via Efficient Model and Tool Orchestration , author=. arXiv preprint arXiv:2511.21689 , year=

    arXiv

  9. [9]

    Findings of the Association for Computational Linguistics , pages =

    Qian, Cheng and Acikgoz, Emre Can and Wang, Hongru and Chen, Xiusi and Sil, Avirup and Hakkani-T. Findings of the Association for Computational Linguistics , pages =

  10. [10]

    Neural Information Processing System , pages =

    AcuRank: Uncertainty-Aware Adaptive Computation for Listwise Reranking , author=. Neural Information Processing System , pages =

  11. [11]

    arXiv preprint arXiv:2503.06034 , year=

    Shengyao Zhuang and Xueguang Ma and Bevan Koopman and Jimmy Lin and Guido Zuccon , title=. arXiv preprint arXiv:2503.06034 , year=

    Pith/arXiv arXiv

  12. [12]

    Empirical Methods in Natural Language Processing , pages =

    REARANK: Reasoning Re-ranking Agent via Reinforcement Learning , author =. Empirical Methods in Natural Language Processing , pages =

  13. [13]

    AAAI Conference on Artificial Intelligence , page=

    TFRank: Think-Free Reasoning Enables Practical Pointwise LLM Ranking , author=. AAAI Conference on Artificial Intelligence , page=

  14. [14]

    arXiv preprint arXiv:2003.07820 , year=

    Overview of the TREC 2019 deep learning track , author=. arXiv preprint arXiv:2003.07820 , year=

    Pith/arXiv arXiv 2019

  15. [15]

    arXiv preprint arXiv:2102.07662 , year=

    Overview of the TREC 2020 deep learning track , author=. arXiv preprint arXiv:2102.07662 , year=

    Pith/arXiv arXiv 2020

  16. [16]

    arXiv preprint arXiv:2505.09388 , year=

    Qwen3 technical report , author=. arXiv preprint arXiv:2505.09388 , year=

    Pith/arXiv arXiv

  17. [17]

    International Conference on Learning Representations , year=

    Hongjin SU and Howard Yen and Mengzhou Xia and Weijia Shi and Niklas Muennighoff and Han-yu Wang and Liu Haisu and Quan Shi and Zachary S Siegel and Michael Tang and Ruoxi Sun and Jinsung Yoon and Sercan O Arik and Danqi Chen and Tao Yu , pages=. International Conference on Learning Representations , year=

  18. [18]

    Neural Information Processing System , year=

    Nandan Thakur and Nils Reimers and Andreas R. Neural Information Processing System , year=

  19. [19]

    International ACM SIGIR Conference , pages=

    Jimmy Lin and Xueguang Ma and Sheng-Chieh Lin and Jheng-Hong Yang and Ronak Pradeep and Rodrigo Frassetto Nogueira , title=. International ACM SIGIR Conference , pages=

  20. [20]

    Weiwei Sun and Lingyong Yan and Xinyu Ma and Shuaiqiang Wang and Pengjie Ren and Zhumin Chen and Dawei Yin and Zhaochun Ren , booktitle=. Is Chat

  21. [21]

    Conference on Language Modeling , year=

    Rank1: Test-Time Compute for Reranking in Information Retrieval , author=. Conference on Language Modeling , year=

  22. [22]

    ACM SIGKDD Conference , pages =

    Zeng, Ziyang and Jing, Heming and Chen, Jindong and Li, Xiangli and Liu, Hongyu and He, Yixuan and Li, Zhengyu and Sun, Yige and Xie, Zheyong and Yang, Yuqing and Cao, Shaosheng and Fan, Jun and Wu, Yi and Hu, Yao , title =. ACM SIGKDD Conference , pages =

  23. [23]

    International Conference on Information and Knowledge Management , pages=

    Rcagent: Cloud Root Cause Analysis by Autonomous Agents with Tool-augmented Large Language Models , author=. International Conference on Information and Knowledge Management , pages=

  24. [24]

    International Conference on Learning Representations , year=

    Beyond Sequential Reranking: Reranker-Guided Search Improves Reasoning Intensive Retrieval , author=. International Conference on Learning Representations , year=

  25. [25]

    Transactions on Machine Learning Research , page=

    Holistic Evaluation of Language Models , author=. Transactions on Machine Learning Research , page=

  26. [26]

    Association for Computational Linguistics , pages =

    ProRank: Prompt Warmup via Reinforcement Learning for Small Language Models Reranking , author =. Association for Computational Linguistics , pages =

  27. [27]

    Findings of the Association for Computational Linguistics , pages =

    Large Language Models are Effective Text Rankers with Pairwise Ranking Prompting , author =. Findings of the Association for Computational Linguistics , pages =

  28. [28]

    2024 , pages =

    Luo, Jian and Chen, Xuanang and He, Ben and Sun, Le , booktitle =. 2024 , pages =

    2024

  29. [29]

    arXiv preprint arXiv:2605.05242 , year=

    Beyond Semantic Similarity: Rethinking Retrieval for Agentic Search via Direct Corpus Interaction , author=. arXiv preprint arXiv:2605.05242 , year=

    Pith/arXiv arXiv

  30. [30]

    2025 , pages =

    Weller, Orion and Chang, Benjamin and MacAvaney, Sean and Lo, Kyle and Cohan, Arman and Van Durme, Benjamin and Lawrie, Dawn and Soldaini, Luca , booktitle=. 2025 , pages =

    2025

  31. [31]

    2024 , pages =

    Mao, Kelong and Deng, Chenlong and Chen, Haonan and Mo, Fengran and Liu, Zheng and Sakai, Tetsuya and Dou, Zhicheng , booktitle =. 2024 , pages =

    2024

  32. [32]

    arXiv preprint arXiv:2508.06412 , year=

    Sample-efficient LLM Optimization with Reset Replay , author=. arXiv preprint arXiv:2508.06412 , year=

    Pith/arXiv arXiv

  33. [33]

    DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning , journal=

    Guo, Daya and Yang, Dejian and Zhang, Haowei and others , year=. DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning , journal=

  34. [34]

    Yue Yu and Wei Ping and Zihan Liu and Boxin Wang and Jiaxuan You and Chao Zhang and Mohammad Shoeybi and Bryan Catanzaro , booktitle=. Rank

  35. [35]

    Neural Information Processing System , page=

    Chain of Thought Prompting Elicits Reasoning in Large Language Models , author=. Neural Information Processing System , page=

  36. [36]

    Transactions on Information Systems , page=

    From Matching to Generation: A Survey on Generative Information Retrieval , author=. Transactions on Information Systems , page=

  37. [37]

    Agentic Reasoning and Tool Integration for

    Joykirat Singh and Yash Pandya and Pranav Vajreshwari and Raghav Magazine and Akshay Nambi , journal=. Agentic Reasoning and Tool Integration for

  38. [38]

    International Conference on Learning Representations , year=

    Rethinking Reasoning in Document Ranking: Why Chain-of-Thought Falls Short , author=. International Conference on Learning Representations , year=

  39. [39]

    FacTool: Factuality Detection in Generative

    Ethan Chern and Steffi Chern and Shiqi Chen and Weizhe Yuan and Kehua Feng and Chunting Zhou and Junxian He and Graham Neubig and Pengfei Liu , page=. FacTool: Factuality Detection in Generative. Conference on Language Modeling , year=

  40. [40]

    International Conference on Learning Representations , year=

    Zhibin Gou and Zhihong Shao and Yeyun Gong and yelong shen and Yujiu Yang and Nan Duan and Weizhu Chen , page=. International Conference on Learning Representations , year=

  41. [41]

    International ACM SIGIR Conference , page=

    A Setwise Approach for Effective and Highly Efficient Zero-shot Ranking with Large Language Models , author=. International ACM SIGIR Conference , page=

  42. [42]

    arXiv preprint arXiv:2508.07050 , year=

    ReasonRank: Empowering Passage Ranking with Strong Reasoning Ability , author=. arXiv preprint arXiv:2508.07050 , year=

    Pith/arXiv arXiv

  43. [43]

    Liu, Wenhan and Ma, Xinyu and Sun, Weiwei and Zhu, Yutao and Li, Yuchen and Yin, Dawei and Dou, Zhicheng , booktitle =

  44. [44]

    Transactions of the Association for Computational Linguistics , page=

    Natural Questions: A Benchmark for Question Answering Research , author=. Transactions of the Association for Computational Linguistics , page=

  45. [45]

    Joshi, Mandar and Choi, Eunsol and Weld, Daniel and Zettlemoyer, Luke , booktitle =

  46. [46]

    Association for Computational Linguistics , pages =

    When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories , author =. Association for Computational Linguistics , pages =

  47. [47]

    , booktitle =

    Yang, Zhilin and Qi, Peng and Zhang, Saizheng and Bengio, Yoshua and Cohen, William and Salakhutdinov, Ruslan and Manning, Christopher D. , booktitle =

  48. [48]

    Constructing A Multi-hop

    Ho, Xanh and Duong Nguyen, Anh-Khoa and Sugawara, Saku and Aizawa, Akiko , booktitle =. Constructing A Multi-hop

  49. [49]

    arXiv preprint arXiv:2108.00573 , year=

    MuSiQue: Multihop Questions via Single-hop Question Composition , author=. arXiv preprint arXiv:2108.00573 , year=

    Pith/arXiv arXiv

  50. [50]

    Findings of the Empirical Methods in Natural Language Processing , pages =

    Measuring and Narrowing the Compositionality Gap in Language Models , author =. Findings of the Empirical Methods in Natural Language Processing , pages =

  51. [51]

    arXiv preprint arXiv:2411.04368 , year=

    Measuring short-form factuality in large language models , author=. arXiv preprint arXiv:2411.04368 , year=

    Pith/arXiv arXiv

  52. [52]

    arXiv preprint arXiv:2512.10791 , year=

    The FACTS Leaderboard: A Comprehensive Benchmark for Large Language Model Factuality , author=. arXiv preprint arXiv:2512.10791 , year=

    arXiv

  53. [53]

    arXiv preprint arXiv:2506.05176 , year=

    Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models , author=. arXiv preprint arXiv:2506.05176 , year=

    Pith/arXiv arXiv

  54. [54]

    arXiv preprint arXiv:2601.04720 , year=

    Qwen3-VL-Embedding and Qwen3-VL-Reranker: A Unified Framework for State-of-the-Art Multimodal Retrieval and Ranking , author=. arXiv preprint arXiv:2601.04720 , year=

    Pith/arXiv arXiv