Pith. sign in

REVIEW 2 major objections 4 minor 1 cited by

A lightweight router that decides per instance whether an LLM should reason before ranking can beat both always-reason and never-reason on accuracy per token.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 08:03 UTC pith:3XSYLB6J

load-bearing objection A genuinely useful routing idea for LLM ranking, but the headline token-savings number is not yet trustworthy because the prefill cost of the checklist probe is excluded. the 2 major comments →

arxiv 2601.18146 v2 pith:3XSYLB6J submitted 2026-01-26 cs.IR

Think When Needed: Model-Aware Reasoning Routing for LLM-based Ranking

Huizhong Guo , Tianjun Wei , Dongxia Wang , Yingpeng Du , Ziyan Wang , Jie Zhang , Zhu Sun This is my paper
classification cs.IR
keywords LLM rankingadaptive reasoningreasoning routingtoken efficiencyPareto frontierlistwise rankinginformation retrievalrecommendation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper claims that the right question for LLM-based ranking is not whether reasoning helps, but when. It proposes a lightweight, model-aware router that, before generation, predicts for each ranking instance whether a reasoning pass will improve the output enough to justify its token cost, and then applies either Think or Non-Think accordingly. Across three public ranking datasets and several open-source LLM scales, the router reports consistently better ranking utility at lower token consumption than always reasoning or never reasoning—for example, +6.3% NDCG@10 with -49.5% tokens on a movie recommendation benchmark. The point is practical: ranking systems that must meet latency budgets can allocate reasoning to the instances that actually benefit, rather than paying for it on every query.

Core claim

On its own terms, the paper establishes that per-instance reasoning routing is both feasible and beneficial for LLM-based ranking. The router is trained to regress a compute-aware advantage score—the expected NDCG gain of Think over Non-Think minus a penalty for extra tokens—using features extracted in a single forward pass: statistical summaries of context and candidate embeddings, plus model-aware difficulty signals obtained from masked yes/no checklist probes that elicit the backbone LLM's own uncertainty. At deployment, a control parameter selects an operating point on the validation Pareto frontier, trading utility against cost. The reported result is a ranking utility at or above the a

What carries the argument

The central object is the model-aware reasoning router head: a lightweight regressor trained to predict the per-instance advantage of Think over Non-Think before generation. It consumes two kinds of pre-generation signals—ranking-aware features such as candidate dispersion and context-candidate alignment, and model-aware difficulty signals from a checklist of paired, direction-balanced yes/no questions whose response probabilities are read from the model's hidden states under a block-diagonal causal mask. A cost-aware routing rule then sends an instance to Think only when predicted advantage exceeds a tunable token penalty, and deployment anchors on the validation Pareto frontier select the

Load-bearing premise

The central efficiency claim rests on counting only generated tokens: if the extra prefill and checklist-probing tokens are included in the token budget, the reported savings may shrink or disappear.

What would settle it

Recompute the end-to-end token cost per instance, including the router's extra forward pass and the checklist suffix, and compare total tokens for the routed policy against always-Non-Think and always-Think; if total routed tokens are not below both, the reduced-token-consumption claim is false.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Latency-sensitive ranking systems can expose a tunable accuracy-efficiency frontier instead of committing to one fixed reasoning mode.
  • Always-Thinking is not a safe default: on a substantial fraction of instances it hurts ranking utility while multiplying token cost, and this pattern persists even for larger LLMs.
  • A router trained for one backbone does not need to encode a single notion of difficulty; model-aware checklist signals let the same framework adapt to different LLMs.
  • Prompt changes shift which instances benefit from reasoning, but the router can re-allocate computation and preserve most of its advantage.
  • When reasoning is clearly harmful for a backbone, the router can approach Non-Think utility rather than forcing Think.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The headline token savings count generated tokens only; the router's extra prefill pass and checklist suffix add input tokens that are not included in the reported -49.5%. Recomputing end-to-end including prefill could materially shrink, or in some settings reverse, the efficiency gain.
  • The paper itself notes that the checklist and feature set may miss nuanced semantic difficulty and require re-tuning across domains or prompt formats, so the routing benefit is conditional on having a validation set and checklist matched to the deployment domain.
  • The same checklist-probing mechanism could transfer to other LLM-as-judge tasks, where model uncertainty about an instance is a general signal for deciding whether to spend more computation.
  • The regression-based advantage formulation leaves room for multi-level routing: rather than binary Think/Non-Think, predicted advantage could later select reasoning depth or style.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper proposes a reasoning routing framework for LLM-based ranking. A lightweight router head decides per instance, before generation, whether to use a reasoning mode (Think) or a direct-output mode (Non-Think). The router is trained to predict the compute-aware advantage of Think over Non-Think (Eq. 3) using ranking-aware features (e.g., candidate dispersion, context�candidate alignment) and model-aware difficulty signals derived from a masked checklist of diagnostic questions appended to the prompt (Section 4.2). At deployment, the router can operate at different points on a validation Pareto frontier, enabling flexible trade-offs between ranking utility and token cost. Experiments on MS-MARCO, MovieLens, and Amazon-VG with Qwen3-4B/8B/14B, Gemma3-12B, and GPT-Oss-20B report consistent ranking utility improvements with reduced token consumption, e.g., +6.3% NDCG@10 with �49.5% tokens on MovieLens with Qwen3-4B.

Significance. If the efficiency accounting is correct, this is a practical and timely contribution. The router is lightweight, does not modify the backbone architecture, and addresses a real deployment concern: reasoning prompts improve ranking but at substantial computational cost. The paper's empirical scope is broad (three datasets, five backbones) and the code is promised in an anonymous repository. The Pareto-frontier policy selection is a useful practical feature. I do not find circularity: the router is trained on observed advantage labels derived from separate Think/Non-Think runs, and the features are pre-generation signals. The main risks are empirical: the token-efficiency metric omits prefill/checklist costs, and the reported averages lack error bars. These are fixable within the manuscript's scope.

major comments (2)
  1. [Section 5.1.1 / Table 1 / Section 4.2] The token-efficiency metric counts only generated tokens. The router path appends a masked checklist suffix to the prompt and extracts ranking-aware features from the backbone's prefill hidden states, so every instance pays that prefill cost. The paper does not report input/prefill token counts, checklist length, or feature-extraction overhead. In the headline example (MovieLens/Qwen3-4B), Ours uses 194 generated tokens vs Think 384; the reported �49.5% saving could shrink or reverse once prefill tokens are included. This is load-bearing for the central accuracy-efficiency claim. Please report total token cost (prefill + generation) and recompute the headline metrics.
  2. [Table 1] Results are reported as averages over five runs without error bars or significance tests. Some claimed improvements are small: e.g., MS-MARCO Qwen3-8B N@10 = 88.31 (Non-Think) vs 88.68 (Ours); Gemma3-12B MovieLens N@10 = 25.96 (Non-Think) vs 26.12 (Ours). Without variance estimates, the claim of consistent improvements across settings is not fully supported. Please include standard deviations or confidence intervals and, where appropriate, significance tests.
minor comments (4)
  1. [Section 4.2 / Figure 4] The paper states that block-diagonal causal masking ensures checklist probing has 'no effect on ranking generation,' but it is not explained how generation positions are prevented from attending to the checklist tokens. Please clarify whether the checklist is removed before decoding or masked out from generation attention.
  2. [Section 4.1 / 4.2] The complete feature set and checklist questions are not included in the paper; they are relegated to the open-source code. Adding an appendix with the actual checklist and feature list would improve reproducibility and allow reviewers to assess completeness.
  3. [Figure 5] The Pareto curves are computed on validation sets. It would strengthen the paper to report test-set performance at the selected operating points (Knee, Utopia, Epsilon, UMax) to demonstrate that the selected policies generalize.
  4. [General] Minor typos and inconsistencies: '<Ouptut>' in Figure 4 should be '<Output>'; 'Features Selection' in Section 4.3 should be 'Feature Selection'; 'Self-select' vs 'Self-Select' are used inconsistently.

Circularity Check

0 steps flagged

No significant circularity: the router is a supervised meta-model trained on observed Think/Non-Think advantage labels, and its pre-generation features do not encode the target.

full rationale

The central derivation is not circular. The training label in Eq. (3), A_p = (U_Think - U_Non) - lambda (T_Think - T_Non), is computed from two separate generation runs with observed ranking utility and token counts. The router features in Eq. (2) and Section 4.2 are extracted from the backbone's hidden states and checklist probabilities before decoding; they do not contain the label. The router is fit to these labels in Eq. (4) and evaluated on a held-out test split (Table 1), so the test results are not the training target reproduced by construction. The final routing rule in Eq. (12) uses the predicted advantage, not the observed label, despite the shared symbol A_p; this is a notation reuse, not a definitional identity. The self-citations ([29], [36], [38]) are used as background for the reasoning template, negative-sampling setup, and checklist inspiration; none is invoked as a uniqueness theorem or as the exclusive justification that routing works. The checklist and feature components are ablated and compared against external datasets and open-source backbones, so the central claim has independent empirical content. The main weakness of the paper is that the reported token metric counts only generation tokens and omits the prefill/checklist cost introduced by the router; that is an efficiency-measurement concern, not circularity, because it does not make the predicted outcome equal to an input by definition. Overall, no load-bearing circular step was identified.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 0 invented entities

The ledger shows the router is a supervised meta-model with two tunable scalar knobs (λ, η) and a set of unverified behavioral assumptions about the checklist probes and embedding statistics. No new physical or conceptual entities are postulated.

free parameters (4)
  • λ (cost multiplier in advantage label)
    Eq 3 defines the training label A_p = (U_Think - U_Non) - λ(T_Think - T_Non). The value of λ controls the accuracy-efficiency tradeoff and is not reported in the paper; the headline result depends on it.
  • η (routing threshold) = selected per deployment policy on the validation Pareto frontier
    Eq 12 a_p(η) = Think if A_p - η(T_Think - T_Non) ≥ 0. η is swept to generate the Pareto frontier and frozen after selection; it determines the operating point reported in Table 1.
  • w_T, w_U (Utopia policy weights)
    Eq 8 weights for the Utopia anchor; not used in the primary UMax results but part of the proposed policy kit.
  • Feature-selection regularization strength (ManiFeSt-based)
    The two-stage feature refinement in Section 4.1 uses sparsity-inducing regularization and cross-setting consistency thresholds; the strengths/thresholds are not specified.
axioms (4)
  • domain assumption An upstream retriever or sampler provides the candidate set; the LLM only ranks the given list.
    Section 3.1 problem setup: focus is on ranking a fixed candidate set, not candidate generation.
  • domain assumption Checklist response probabilities reflect the LLM's internal perception of instance difficulty.
    Section 4.2 introduces masked checklist probing; this premise justifies model-aware difficulty signals but is not independently validated.
  • domain assumption Block-diagonal causal masking ensures probe isolation without affecting ranking generation.
    Section 4.2: this is needed so that the checklist features are non-intrusive; any leakage would distort the router signals.
  • standard math Gradient-boosted regressor with monotonicity constraint can faithfully map features to the advantage.
    Eq 4 assumes f_phi can be optimized with ∂f/∂ΔT ≤ 0; this is an ML architectural choice.

pith-pipeline@v1.3.0-alltime-deepseek · 18502 in / 13477 out tokens · 131384 ms · 2026-08-03T08:03:56.365006+00:00 · methodology

0 comments
read the original abstract

Large language models (LLMs) are increasingly applied to ranking tasks in retrieval and recommendation. Although reasoning prompting can enhance ranking utility, our preliminary exploration reveals that its benefits are inconsistent and come at a substantial computational cost, suggesting that when to reason is as crucial as how to reason. To address this issue, we propose a reasoning routing framework that employs a lightweight, plug-and-play router head to decide whether to use direct inference (Non-Think) or reasoning (Think) for each instance before generation. The router head relies solely on pre-generation signals: i) compact ranking-aware features (e.g., candidate dispersion) and ii) model-aware difficulty signals derived from a diagnostic checklist reflecting the model's estimated need for reasoning. By leveraging these features before generation, the router outputs a controllable token that determines whether to apply the Think mode. Furthermore, the router can adaptively select its operating policy along the validation Pareto frontier during deployment, enabling dynamic allocation of computational resources toward instances most likely to benefit from Think under varying system constraints. Experiments on three public ranking datasets with different scales of open-source LLMs show consistent improvements in ranking utility with reduced token consumption (e.g., +6.3\% NDCG@10 with -49.5\% tokens on MovieLens with Qwen3-4B), demonstrating reasoning routing as a practical solution to the accuracy-efficiency trade-off.

Figures

Figures reproduced from arXiv: 2601.18146 by Dongxia Wang, Huizhong Guo, Jie Zhang, Tianjun Wei, Yingpeng Du, Zhu Sun, Ziyan Wang.

Figure 1
Figure 1. Figure 1: Accuracy–efficiency trade-offs across reasoning [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 3
Figure 3. Figure 3: Prompt template for reasoning routing. each candidate item 𝑖 and returns a top-𝐾 ranked list 𝐿 𝐾 = 𝜏𝐾 ( { 𝑠𝜃 (𝑖 | 𝑃,𝐶, I ) : 𝑖 ∈ I } ) , (1) where 𝜏𝐾 (·) returns the 𝐾 highest-scoring items. For the IR task, the context is a search query 𝑄 (i.e., 𝐶 = 𝑄) and candidates are documents or passages. Given (𝑃, 𝑄, I), the LLM produces a list with items ordered by predicted relevance to 𝑄. For the Rec task, the co… view at source ↗
Figure 4
Figure 4. Figure 4: An overview of our lightweight, model-aware, per-instance router framework. [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Pareto frontier curves on the validation sets of three ranking datasets, showing the trade-off between ranking utility [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Top-5 feature importances of the mode router on the validation sets for three datasets (MovieLens, Amazon-VG, [PITH_FULL_IMAGE:figures/full_fig_p009_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Prompt-sensitive analysis using Qwen3-14B on MS [PITH_FULL_IMAGE:figures/full_fig_p009_7.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. R2-Router: A New Paradigm for LLM Routing with Reasoning

    cs.CL 2026-02 conditional novelty 6.0

    R2-Router jointly selects the LLM and an output-token budget, modeling each model as a quality-cost curve rather than a fixed point, and reports 4-5x cost savings on its new R2-Bench.

Reference graph

Works this paper leans on

47 extracted references · 13 linked inside Pith · cited by 1 Pith paper

  1. [1]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Floren- cia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report.arXiv preprint arXiv:2303.08774 (2023)

    Pith/arXiv arXiv 2023

  2. [2]

    Pranjal Aggarwal and Sean Welleck. 2025. L1: Controlling how long a reasoning model thinks with reinforcement learning.arXiv preprint arXiv:2503.04697(2025)

    Pith/arXiv arXiv 2025

  3. [3]

    Alex Beutel, Jilin Chen, Tulsee Doshi, Hai Qian, Li Wei, Yi Wu, Lukasz Heldt, Zhe Zhao, Lichan Hong, Ed H Chi, et al . 2019. Fairness in recommendation ranking through pairwise comparisons. InProceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2212–2220

    2019

  4. [4]

    Sijia Chen and Baochun Li. 2024. Toward adaptive reasoning in large language models with thought rollback.arXiv preprint arXiv:2412.19707(2024)

    Pith/arXiv arXiv 2024

  5. [5]

    David Cohen, Tal Shnitzer, Yuval Kluger, and Ronen Talmon. 2023. Few-sample feature selection via feature manifold learning. InInternational Conference on Machine Learning. PMLR, 6296–6319

    2023

  6. [6]

    Guanting Dong, Yutao Zhu, Chenghao Zhang, Zechen Wang, Ji-Rong Wen, and Zhicheng Dou. 2025. Understand what LLM needs: Dual preference alignment for retrieval-augmented generation. InProceedings of the ACM on Web Conference

    2025

  7. [7]

    Claus Thorn Ekstrøm, Thomas Alexander Gerds, and Andreas Kryger Jensen

  8. [8]

    Yichao Fu, Siqi Zhu, Runlong Su, Aurick Qiao, Ion Stoica, and Hao Zhang. 2024. Efficient llm scheduling by learning to rank.Advances in Neural Information Processing Systems37 (2024), 59006–59029

    2024

  9. [9]

    Shibo Hao, Sainbayar Sukhbaatar, DiJia Su, Xian Li, Zhiting Hu, Jason Weston, and Yuandong Tian. 2024. Training large language models to reason in a continuous latent space.arXiv preprint arXiv:2412.06769(2024)

    Pith/arXiv arXiv 2024

  10. [10]

    F Maxwell Harper and Joseph A Konstan. 2015. The movielens datasets: History and context.Acm Transactions on Interactive Intelligent Systems (TIIS)5, 4 (2015), 1–19

    2015

  11. [11]

    Minjie Hong, Yan Xia, Zehan Wang, Jieming Zhu, Ye Wang, Sihang Cai, Xiaoda Yang, Quanyu Dai, Zhenhua Dong, Zhimeng Zhang, and Zhou Zhao. [n. d.]. LLM- BS: Enhancing Large Language Models for Recommendation through Exogenous Behavior-Semantics Integration. InThe Web Conference 2025

    2025

  12. [12]

    Bairu Hou, Yang Zhang, Jiabao Ji, Yujian Liu, Kaizhi Qian, Jacob Andreas, and Shiyu Chang. 2025. Thinkprune: Pruning long chain-of-thought of llms via reinforcement learning.arXiv preprint arXiv:2504.01296(2025)

    Pith/arXiv arXiv 2025

  13. [13]

    Kalervo Järvelin and Jaana Kekäläinen. 2002. Cumulated gain-based evaluation of IR techniques.ACM Transactions on Information Systems (TOIS)20, 4 (2002), 422–446

    2002

  14. [14]

    Dongfu Jiang, Xiang Ren, and Bill Yuchen Lin. 2023. LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusion. InPro- ceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 14165–14178

    2023

  15. [15]

    Xiaoxi Li, Jiajie Jin, Yujia Zhou, Yuyao Zhang, Peitian Zhang, Yutao Zhu, and Zhicheng Dou. 2025. From matching to generation: A survey on generative information retrieval.ACM Transactions on Information Systems43, 3 (2025), 1–62

    2025

  16. [16]

    Wenhan Liu, Xinyu Ma, Weiwei Sun, Yutao Zhu, Yuchen Li, Dawei Yin, and Zhicheng Dou. 2025. ReasonRank: Empowering Passage Ranking with Strong Reasoning Ability.arXiv preprint arXiv:2508.07050(2025)

    Pith/arXiv arXiv 2025

  17. [17]

    Zheng Liu, Yujia Zhou, Yutao Zhu, Jianxun Lian, Chaozhuo Li, Zhicheng Dou, Defu Lian, and Jian-Yun Nie. 2024. Information retrieval meets large language models. InCompanion Proceedings of the ACM Web Conference 2024. 1586–1589

    2024

  18. [18]

    Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al

  19. [19]

    Ananth Mohan, Zheng Chen, and Kilian Weinberger. 2011. Web-search ranking with initialized gradient boosted regression trees. InProceedings of the Learning to Rank Challenge. 77–89

    2011

  20. [20]

    Jianmo Ni, Jiacheng Li, and Julian McAuley. 2019. Justifying recommendations using distantly-labeled reviews and fine-grained aspects. InProceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 188–197

    2019

  21. [21]

    OpenAI. 2025. gpt-oss-120b & gpt-oss-20b Model Card. arXiv:2508.10925 [cs.CL] https://arxiv.org/abs/2508.10925

    Pith/arXiv arXiv 2025

  22. [22]

    Andrew Parry, Sean MacAvaney, and Debasis Ganguly. 2024. Top-down parti- tioning for efficient list-wise ranking.arXiv preprint arXiv:2405.14589(2024)

    Pith/arXiv arXiv 2024

  23. [23]

    Jayr Pereira, Andre Assumpcao, and Roberto Lotufo. 2024. Check-Eval: A checklist-based approach for evaluating text quality.arXiv preprint arXiv:2407.14467(2024). Think When Needed: Model-Aware Reasoning Routing for LLM-based Ranking Conference acronym ’XX, June 03–05, 2018, Woodstock, NY

    Pith/arXiv arXiv 2024

  24. [24]

    Jakub Podolak, Leon Perić, Mina Janićijević, and Roxana Petcu. 2025. Beyond reproducibility: Advancing zero-shot llm reranking efficiency with setwise inser- tion. InProceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval. 3205–3213

    2025

  25. [25]

    Ronak Pradeep, Sahel Sharifymoghaddam, and Jimmy Lin. 2023. RankZephyr: Effective and Robust Zero-Shot Listwise Reranking is a Breeze!arXiv preprint arXiv:2312.02724(2023)

    Pith/arXiv arXiv 2023

  26. [26]

    Amar Shah and Zoubin Ghahramani. 2016. Pareto frontier learning with ex- pensive correlated objectives. InInternational Conference on Machine Learning. PMLR, 1919–1927

    2016

  27. [27]

    DiJia Su, Sainbayar Sukhbaatar, Michael Rabbat, Yuandong Tian, and Qinqing Zheng. 2024. Dualformer: Controllable Fast and Slow Thinking by Learning with Randomized Reasoning Traces. InThe Thirteenth International Conference on Learning Representations

    2024

  28. [28]

    Weiwei Sun, Lingyong Yan, Xinyu Ma, Shuaiqiang Wang, Pengjie Ren, Zhumin Chen, Dawei Yin, and Zhaochun Ren. 2023. Is ChatGPT Good at Search? Investi- gating Large Language Models as Re-Ranking Agents. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Houda Bouamor, Juan Pino, and Kalika Bali (Eds.). Association for ...

    doi:10.18653/v1/2023.emnlp-main.923 2023

  29. [29]

    Zhu Sun, Hongyang Liu, Xinghua Qu, Kaidong Feng, Yan Wang, and Yew Soon Ong. 2024. Large language models for intent-driven session recommendations. InProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval. 324–334

    2024

  30. [30]

    Jiakai Tang, Sunhao Dai, Teng Shi, Jun Xu, Xu Chen, Wen Chen, Jian Wu, and Yuning Jiang. 2025. Think before recommend: Unleashing the latent reasoning power for sequential recommendation.arXiv preprint arXiv:2503.22675(2025)

    Pith/arXiv arXiv 2025

  31. [31]

    Gemma Team. 2025. Gemma 3. (2025). https://goo.gle/Gemma3Report

    2025

  32. [32]

    Qwen Team. 2025. Qwen3 Technical Report. arXiv:2505.09388 [cs.CL] https: //arxiv.org/abs/2505.09388

    Pith/arXiv arXiv 2025

  33. [33]

    Xu Wan, Wei Wang, Wenyue Xu, Wotao Yin, Jie Song, and Mingyang Sun. 2025. AdapThink: Adaptive Thinking Preferences for Reasoning Language Model.arXiv preprint arXiv:2506.18237(2025)

    Pith/arXiv arXiv 2025

  34. [34]

    Yu Wang, Lei Sang, Yi Zhang, and Yiwen Zhang. 2025. Intent representation learning with large language model for recommendation. InProceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval. 1870–1879

    2025

  35. [35]

    Yuhan Wang, Qing Xie, Zhifeng Bao, Mengzi Tang, Lin Li, and Yongjian Liu. 2025. Enhancing Transferability and Consistency in Cross-Domain Recommendations via Supervised Disentanglement. InProceedings of the Nineteenth ACM Conference on Recommender Systems. 104–113

    2025

  36. [36]

    Ziyan Wang, Yingpeng Du, Zhu Sun, Haoyan Chua, Kaidong Feng, Wenya Wang, and Jie Zhang. 2025. Re2llm: reflective reinforcement large language model for session-based recommendation. InProceedings of the AAAI Conference on Artificial Intelligence, Vol. 39. 12827–12835

    2025

  37. [37]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models.Advances in Neural Information Processing Systems35 (2022), 24824–24837

    2022

  38. [38]

    Tianjun Wei, Wei Wen, Ruizhi Qiao, Xing Sun, and Jianghong Ma. 2025. RocketE- val: Efficient automated LLM evaluation via grading checklist. InThe Thirteenth International Conference on Learning Representations

    2025

  39. [39]

    Siye Wu, Jian Xie, Yikai Zhang, Aili Chen, Kai Zhang, Yu Su, and Yanghua Xiao

  40. [40]

    Yuki Yano, Yukihiro Tagami, and Akira Tajima. 2016. Quantifying query ambi- guity with topic distributions. InProceedings of the 25th ACM International on Conference on Information and Knowledge Management. 1877–1880

    2016

  41. [41]

    Jun Yin, Zhengxin Zeng, Mingzheng Li, Hao Yan, Chaozhuo Li, Weihao Han, Jianjin Zhang, Ruochen Liu, Hao Sun, Weiwei Deng, et al. 2025. Unleash LLMs Potential for Sequential Recommendation by Coordinating Dual Dynamic Index Mechanism. InProceedings of the ACM on Web Conference 2025. 216–227

    2025

  42. [42]

    Shengzhe Zhang, Liyi Chen, Dazhong Shen, Chao Wang, and Hui Xiong. 2025. Hierarchical Time-Aware Mixture of Experts for Multi-Modal Sequential Recom- mendation. InProceedings of the ACM on Web Conference 2025. 3672–3682

    2025

  43. [43]

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena.Advances in Neural Information Processing Systems36 (2023), 46595–46623

    2023

  44. [44]

    Shengyao Zhuang, Honglei Zhuang, Bevan Koopman, and Guido Zuccon. 2024. A setwise approach for effective and highly efficient zero-shot ranking with large language models. InProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval. 38–47

    2024

  45. [2019]

    Sequential rank agreement methods for comparison of ranked lists.Bio- statistics20, 4 (2019), 582–598

    2019

  46. [2023]

    Self-refine: Iterative refinement with self-feedback.Advances in Neural Information Processing Systems36 (2023), 46534–46594

    2023

  47. [2025]

    ARM: Adaptive Reasoning Model.arXiv preprint arXiv:2505.20258(2025)

    arXiv 2025