Pith. sign in

REVIEW 4 major objections 7 minor 40 references

JointRank: Rank Large Set with Single Pass

T0 review · 4 major / 7 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read JointRank reranks large candidate sets in a single parallel pass by aggregating implicit pairwise comparisons from overlapping blocks, and on the paper's TREC DL-2019 setup it beats full-context listwise reranking.

desk verdict JointRank is a practical single-pass block-rank-aggregation method that mostly delivers on its latency claim; the evaluation is solid but needs significance testing and clearer hyperparameter handling before I'd trust the headline nDCG. read the letter →

arxiv 2506.22262 v1 pith:7H2BISZC submitted 2025-06-27 cs.IR

Evgeny Dedov This is my paper
classification cs.IR
keywords large-setrerankinglistwiserankaggregationblockdesignPartiallyBalancedIncompletePagezero-shotLLMrankingTRECDL-2019
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

JointRank is a model-agnostic reranking method for candidate sets that are too large for an LLM to rank in one helpful context. The paper claims that by splitting the set into overlapping blocks, ranking every block independently in parallel, and then aggregating the implied pairwise comparisons with PageRank or Winrate, a global ranking can be recovered that matches or beats iterative large-set rerankers and clearly beats full-context listwise reranking on large unordered sets. This matters because latency-sensitive applications such as retrieval-augmented generation and interactive search cannot afford the sequential LLM calls that existing large-set rerankers require. On the paper's TREC DL-2019 shuffled-top-1000 setup, JointRank with gpt-4.1-mini achieves an nDCG@10 of 70.88 against 57.68 for full-context listwise reranking, while cutting latency from 21 to 8 seconds.

What carries the argument

The load-bearing object is the block design: a way of grouping the candidate set into equal-size, overlapping subsets so that every item appears in a fixed number of blocks and the resulting comparison graph is connected. From each block's local ranking the method reads implicit pairwise comparisons (every higher-ranked item beats every lower-ranked item in that block), and the union of these edges across blocks forms a tournament graph. PageRank — an algorithm that assigns scores to nodes of a directed graph — is then run on this graph to produce the global ranking; average winrate is the simpler fallback that the paper finds nearly as effective when the design is well balanced. The design's purpose is to make the implied comparisons dense and balanced enough that the aggregation can reconstruct a high-quality global order from partial local orders.

What would settle it

Repeat the paper's shuffled-top-1000 TREC DL-2019 protocol with an oracle block ranker that orders each block by ground-truth relevance before PageRank aggregation, keeping the same block designs. If the aggregated nDCG@10 does not land near the reported oracle upper bound (96.40) and well above the full-context baseline (57.68), then the block-and-aggregate pipeline itself is not responsible for the reported gain and the central claim would need to be revised.

Watch

Extended reading notes

Core claim

On its own terms, the paper establishes that an overlapping partition plus rank aggregation can replace both a single full-context listwise call and multi-call iterative reranking. The method reads each block's local ranking as a set of pairwise comparisons, builds a tournament graph across the whole candidate set, and lets PageRank (or the simpler average winrate) recover a total order. Through synthetic oracle experiments the paper finds that balanced block designs, especially partially balanced incomplete block designs and equi-replicate designs, produce the most reliable global rankings, and that PageRank is the most consistent aggregation rule. The empirical headline is the shuffled-top-1000 TREC DL-2019 result with gpt-4.1-mini: nDCG@10 of 70.88 for JointRank versus 57.68 for full-context listwise reranking, with latency falling from 21 to 8 seconds.

Load-bearing premise

The load-bearing assumption is that the local rankings produced for each block translate into pairwise comparisons that are consistent enough that reconstructing a global ranking from them approximates true relevance; the paper notes that real predicted comparisons may be intransitive and offers no formal robustness guarantee for that case.

Editorial extensions

If this is right

  • End-to-end latency for reranking a set of any size becomes dominated by one parallel batch of block rankings, not by the number of candidates, as long as the compute layer can run blocks concurrently.
  • On the paper's TREC DL-2019 setup, practitioners using a long-context LLM on hundreds of unordered passages get a direct quality-latency win: 70.88 nDCG@10 in 8 seconds instead of 57.68 in 21 seconds for full-context listwise ranking.
  • The design choice is the main quality lever: balanced designs such as PBIBD and equi-replicate designs with PageRank beat random or sliding-window blocking and simpler aggregation rules in the paper's experiments.
  • Block size matters more than block count because each block generates $k(k-1)/2$ pairwise comparisons, so the largest block the model can still rank accurately is the best operating point.
  • JointRank is model-agnostic, so any listwise or setwise reranker can be plugged in without retraining, making the quality-latency trade-off tunable through the hyperparameters $r$, $k$, and block count.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not claim, but it would follow, that weighted aggregation using score differences from score-based listwise rankers could exceed ordinal PageRank; the paper notes distance-based weighting had no effect and leaves score-based weighting as future work.
  • The transitivity caveat in the paper (Section 4.4) means the covering-design guarantee is conditional; real LLM comparisons are often intransitive, so the practical robustness of JointRank rests on treating the graph as a tournament rather than on a proven bound.
  • The paper reports only a few zero-shot models and no supervised rerankers, and its Limitations section says the general design recommendations are not formally proved; extending the benchmarks would show whether the method's advantage transfers beyond the tested settings.
  • For API deployments, the reported latency advantage assumes enough parallel throughput; the paper's own latency tables show some throttling effect, so on rate-limited endpoints the practical gain will depend on the provider's parallelism policies.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper proposes JointRank, a model-agnostic reranking method for candidate sets that exceed an LLM's input limit. It partitions candidates into overlapping blocks, reranks each block independently in a single parallel pass, derives implicit pairwise comparisons from block-local rankings, and aggregates them into a global ranking using PageRank, winrate, or other tournament algorithms. The authors evaluate block designs (random, sliding window, equi-replicate, PBIBD) and aggregators on synthetic oracle data, and on TREC DL-2019 top-100 and shuffled top-1000 with several LLMs, plus BEIR with gpt-4.1-mini. The headline result is that on shuffled top-1000, JointRank with k=100, r=3 achieves nDCG@10 of 70.88 versus 57.68 for full-context listwise reranking with lower wall-clock latency, at the cost of more total inferences and prompt tokens.

Significance. If the empirical results are robust, JointRank offers a practical latency-quality trade-off for zero-shot listwise reranking of large candidate sets, and the block-design/rank-aggregation framing is a useful perspective. Strengths of the paper include a public implementation, synthetic oracle experiments that isolate design and aggregation choices, coverage statistics quantifying graph connectivity, and latency measurements on real APIs. The paper's own limitations section correctly notes the absence of formal proofs and the limited model coverage. The main risk is experimental: real-data tables contain point estimates without variance or significance information, and there is no robustness analysis for noisy or intransitive block-local comparisons, both of which are load-bearing for the central claim.

major comments (4)
  1. [§6.1, Tables 8–9; §6.3, Table 10] All real-data results are reported as single point estimates with no error bars, confidence intervals, or paired significance tests. This is load-bearing because the three JointRank configurations in Table 9 differ by 68.65, 70.88, and 69.25 for nDCG@10, a spread comparable to the difference from some baselines, and the BEIR averages in Table 10 likewise have no variance information. The headline comparison of 70.88 versus 57.68 for FullContextListwise is therefore not shown to be outside run-to-run or API-induced noise.
  2. [§4.4, §5.1, Figs. 3–4] The paper concedes that the covering-design guarantee holds only for transitive comparisons and that predicted comparisons should be treated as a tournament graph, yet the recommended EBD is not covering. The synthetic oracle experiments with v=1000 show that even with perfect block-level rankings, EBD-PageRank does not fully recover the true order (Figs. 3–4). The manuscript provides no error-propagation or robustness analysis for how noisy or intransitive LLM comparisons affect final nDCG, so the reported 70.88 could depend on the particular random EBD draw and API conditions rather than on a robust property of the method.
  3. [§6.1–6.2, Table 9] Hyperparameters r and k are selected on the same TREC DL-2019 test data, and no validation protocol or selection rule is described. The effect of r is non-monotone in Table 9 (68.65, 70.88, 69.25 for r=2,3,4), and BEIR uses different k values with no explicit justification. The choice of r=3 therefore needs to be justified in a way that does not depend on the test set, or the sensitivity to this choice must be quantified.
  4. [§4.5, Table 9] The latency comparison to FullContextListwise conflates parallel speedup with total work: JointRank (r=3, k=100) uses 30 inferences and 284k prompt tokens versus 1 inference and 91k prompt tokens for FullContextListwise, and the paper notes that API throttling may affect latency. For the 'single pass' claim, the authors should report ideal parallel span as well as measured latency, or provide a cost-normalized comparison, so that the 8s versus 21s result is not an artifact of API scheduling.
minor comments (7)
  1. [Abstract and §1] 'exceed a model input limits' should be 'exceed a model's input limit' or 'exceed model input limits'.
  2. [§4.4] 'Egienvalues' is a typo for 'Eigenvalues'.
  3. [References] References [7] and [10] have garbled author lists and missing venue information; these should be corrected.
  4. [Table 8 footnote] The footnote states that the TourRank score comes from the authors' reproduction and acknowledges responsibility for potential setup issues; the paper should clarify whether the same reproduction caveat applies to the other baseline scores.
  5. [§5.1, Fig. 2] The synthetic experiments should state how nDCG@10 is computed when the global order is only partially recovered, including how ties are broken.
  6. [§6.2] The phrase 'well-likely due to throttling on the API side' should be reworded, and reporting measured rate-limit or retry events would be more informative than speculation.
  7. [Table 10] The average latency column should state whether it is an arithmetic mean over datasets and whether per-dataset latencies include queueing or retries.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: JointRank's global ranking is reconstructed by standard rank aggregation from block-level LLM rankings; no target metric or fitted parameter is fed back into the derivation.

full rationale

JointRank's derivation chain is self-contained: the algorithm constructs overlapping blocks (design), obtains independent listwise LLM rankings per block, converts those orderings into pairwise comparisons, and aggregates them via PageRank/Winrate into a global ranking. None of these steps defines the output in terms of the evaluation metric (nDCG@10) or fits a parameter to the target labels; the global ranking is a function of the block rankings, not of the ground truth. The synthetic oracle experiments are an independent sanity check of aggregation methods, not circular evidence, and the real benchmarks are external. The paper contains no load-bearing self-citations: references to prior block-design and rank-aggregation literature are standard external results. The main non-independence is that hyperparameters (r, k, and the final r=3 choice) were selected or reported on the same TREC DL-2019 set; the paper honestly lists all tested configurations and does not rename this tuning as a prediction. The Limitations section explicitly acknowledges the lack of formal proofs and incomplete BEIR comparisons, which are robustness/validity concerns rather than circularity. No equation reduces to its own input.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central method relies on standard rank aggregation machinery and on domain assumptions about LLM ranking behavior. Free parameters are the block size and replication count, chosen by hand or by test-set evaluation. No new entities are introduced.

free parameters (2)
  • block size k = 20 (top-100), 100 (top-1000), 10 or 20 (BEIR)
    Free design parameter; set to fit LLM context limits and tuned via synthetic and real experiments (Section 5, Section 6).
  • replication count r = 4 (Mistral top-100), 2-4 (gpt-4.1-mini top-1000, best 3), 2 (BEIR)
    Chosen by hand based on quality and latency; the best value for the headline TREC result is picked from test-set runs (Section 6.2, Table 9).
assumptions (4)
  • domain assumption The LLM listwise ranker produces a reliable total order within each block.
    JointRank builds pairwise comparisons from block rankings; if block rankings are poor, the global ranking degrades. The paper only validates this with a perfect Oracle in synthetic experiments (Section 5.1).
  • domain assumption Pairwise comparisons derived from local rankings are sufficiently consistent for PageRank/winrate aggregation to recover the global order.
    The paper notes predicted comparisons may be intransitive and that covering-design guarantees assume consistency (Section 4.4), so this is an unproven assumption for real models.
  • domain assumption The block design is connected, so the comparison graph has a single component.
    EBD connectivity is stated as empirically highly probable but not guaranteed (Section 4.4).
  • domain assumption TREC DL-2019 and BEIR relevance judgments are valid ground truth for nDCG@10.
    Standard IR evaluation assumption; not an axiom of the method but of the evaluation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of JointRank: Rank Large Set with Single Pass." pith.science (2026). https://pith.science/paper/7H2BISZC

@misc{pith2026250622262,
  author       = {Pith},
  title        = {Pith review of: JointRank: Rank Large Set with Single Pass},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7H2BISZC}},
  note         = {Machine review of arXiv:2506.22262}
}
read the original abstract

Efficiently ranking relevant items from large candidate pools is a cornerstone of modern information retrieval systems -- such as web search, recommendation, and retrieval-augmented generation. Listwise rerankers, which improve relevance by jointly considering multiple candidates, are often limited in practice: either by model input size constraints, or by degraded quality when processing large sets. We propose a model-agnostic method for fast reranking large sets that exceed a model input limits. The method first partitions candidate items into overlapping blocks, each of which is ranked independently in parallel. Implicit pairwise comparisons are then derived from these local rankings. Finally, these comparisons are aggregated to construct a global ranking using algorithms such as Winrate or PageRank. Experiments on TREC DL-2019 show that our method achieves an nDCG@10 of 70.88 compared to the 57.68 for full-context listwise approach using gpt-4.1-mini as long-context model, while reducing latency from 21 to 8 seconds. The implementation of the algorithm and the experiments is available in the repository: https://github.com/V3RGANz/jointrank

Figures

Figures reproduced from arXiv: 2506.22262 by the authors.

Figure 1
Figure 1. Joint Rank algorithm overview 4.3 Block Construction Designs We use the terminology and notation of experimental design here; the core principle is blocking - separating initial set into multiple subsets. Other basic principles of experimental design are random￾ization (random elements in block as well as their random order within block), replication (each item occurs in several blocks), and balance (constant block … view at source ↗
Figure 2
Figure 2. Blocks Count vs 𝑛𝐷𝐶𝐺@10 for Aggregators (Equi￾Replicate Design 𝑣 = 100) nDCG scores, with the Equi-Replicate design trailing by only one point. Thus, both experiments consistently demonstrate the superior performance of PBIBDs. However, since it may be impossible or challenging to construct a PBIBD for certain parameter combi￾nations due to structural constraints, we recommend using the Equi-Replicate Block Design i… view at source ↗
Figure 3
Figure 3. nDCG@10 for EBD-PageRank𝑣 = 1000, 𝑏 ∈ [50..1000] [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Accuracy@1 for EBD-PageRank 𝑣 = 1000, 𝑏 ∈ [50..1000] • 𝑛𝐷𝐶𝐺@10 ( [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

40 extracted references · 17 canonical work pages

  1. [1]

    Junjie Huang, Jizheng Chen, Jianghao Lin, Jiarui Qin, Ziming Feng, Weinan Zhang, and Yong Yu. 2024. A comprehensive survey on retrieval methods in recommender systems. (2024). https://arxiv.org/abs/2407.21022 arXiv: 2407.21022 [cs.IR]

    arXiv 2024

  2. [2]

    Shashank Rajput et al. 2023. Recommender systems with generative retrieval. (2023). https://arxiv.org/abs/2305.05065 arXiv: 2305.05065 [cs.IR]

    arXiv 2023

  3. [3]

    Yunfan Gao et al. 2024. Retrieval-augmented generation for large language models: a survey. (2024). https://arxiv.org/abs/2312.10997 arXiv: 2312.10997 [cs.CL]

    arXiv 2024

  4. [4]

    Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang

    Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2023. Lost in the middle: how language models use long contexts. (2023). https://arxiv.org/abs/2307.03172 arXiv: 2307.03172 [cs.CL]

    arXiv 2023

  5. [5]

    Tan Yu, Anbang Xu, and Rama Akkiraju. 2024. In defense of rag in the era of long-context language models. (2024). https://arxiv.org/abs/2409.01666 arXiv: 2409.01666 [cs.CL]

    arXiv 2024

  6. [6]

    Mathew Jacob, Erik Lindgren, Matei Zaharia, Michael Carbin, Omar Khattab, and Andrew Drozdov. 2024. Drowning in documents: consequences of scaling reranker inference. (2024). https://arxiv.org/abs/2411.11767 arXiv: 2411.11767 [cs.IR]

    arXiv 2024

  7. [7]

    Weiwei Sun, Lingyong Yan, Xinyu Ma, Shuaiqiang Wang, Zhumin Chen Pengjie Ren, and Zhaochun Ren Dawei Yin. 2023. Is chatgpt good at search? investigat- ing large language models as re-ranking agents. (2023). https://arxiv.org/abs/2 304.09542

    work page 2023

  8. [8]

    Shengyao Zhuang, Honglei Zhuang, Bevan Koopman, and Guido Zuccon. 2024. A setwise approach for effective and highly efficient zero-shot ranking with large language models. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’24). Association for Computing Machinery, Washington DC, USA, ...

    arXiv 2024

Show all 40 references
  1. [9]

    Yiqun Chen et al. 2024. Tourrank: utilizing large language models for documents ranking with a tournament-inspired strategy. (2024). https://arxiv.org/pdf/240 6.11678

    2024

  2. [10]

    Debasis Ganguly Andrew Parry Sean MacAvaney. 2024. Top-down partitioning for efficient list-wise ranking. (2024). https://arxiv.org/abs/2405.14589v1

    2024 arXiv

  3. [11]

    Voorhees

    Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, and Ellen M. Voorhees. 2020. Overview of the trec 2019 deep learning track. (2020). https://a rxiv.org/abs/2003.07820 arXiv: 2003.07820 [cs.IR]

    2020 arXiv

  4. [12]

    Siyi Wang, Qi Deng, Shiwei Feng, Hong Zhang, and Chao Liang. 2024. A survey on rank aggregation. In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, IJCAI-24 . Kate Larson, (Ed.) Survey Track. International Joint Conferences on Artific...

    2024 doi

  5. [13]

    J Ch. 1781. De. borda. mémoire sur les élections au scrutin. Comptes Rendus de l’Académie des Sciences

  6. [14]

    Percy Liang et al. 2023. Holistic evaluation of language models. (2023). https: //arxiv.org/abs/2211.09110 arXiv: 2211.09110 [cs.CL]

    2023 arXiv

  7. [15]

    Honglei Zhuang, Zhen Qin, Kai Hui, Junru Wu, Le Yan, Xuanhui Wang, and Michael Bendersky. 2023. Beyond yes and no: improving zero-shot llm rankers via scoring fine-grained relevance labels. ArXiv, abs/2310.14122. https://api.se manticscholar.org/CorpusID:264426465

    2023 arXiv

  8. [16]

    Zhen Qin et al. 2023. Large language models are effective text rankers with pairwise ranking prompting. (2023). https://arxiv.org/abs/2306.17563 arXiv: 2306.17563 [cs.IR]

    2023 arXiv

  9. [17]

    Xueguang Ma, Xinyu Zhang, Ronak Pradeep, and Jimmy Lin. 2023. Zero-shot listwise document reranking with a large language model. (2023). https://arxiv .org/abs/2305.02156 arXiv: 2305.02156 [cs.IR]

    2023 arXiv

  10. [18]

    Ronak Pradeep, Sahel Sharifymoghaddam, and Jimmy Lin. 2023. Rankzephyr: effective and robust zero-shot listwise reranking is a breeze! (2023). https://arx iv.org/abs/2312.02724 arXiv: 2312.02724 [cs.IR]

    2023 arXiv

  11. [19]

    Ronak Pradeep, Sahel Sharifymoghaddam, and Jimmy Lin. 2023. Rankvicuna: zero-shot listwise document reranking with open-source large language models. (2023). https://arxiv.org/abs/2309.15088 arXiv: 2309.15088 [cs.IR]

    2023 arXiv

  12. [20]

    Revanth Gangi Reddy, JaeHyeok Doo, Yifei Xu, Md Arafat Sultan, Deevya Swain, Avirup Sil, and Heng Ji. 2024. First: faster improved listwise reranking with single token decoding. (2024). https://arxiv.org/abs/2406.15657 arXiv: 2406.15657 [cs.IR]

    2024 arXiv

  13. [21]

    Manveer Singh Tamber, Ronak Pradeep, and Jimmy Lin. 2023. Scaling down, litting up: efficient zero-shot listwise reranking with seq2seq encoder-decoder models. (2023). https://arxiv.org/abs/2312.16098 arXiv: 2312.16098 [cs.IR]

    2023 arXiv

  14. [22]

    Ruiyang Ren, Yuhao Wang, Kun Zhou, Wayne Xin Zhao, Wenjie Wang, Jing Liu, Ji-Rong Wen, and Tat-Seng Chua. 2024. Self-calibrated listwise reranking with large language models. (2024). https://arxiv.org/abs/2411.04602 arXiv: 2411.04602 [cs.IR]

    2024 arXiv

  15. [23]

    Charles Eric Leiserson, Ronald L Rivest, Thomas H Cormen, and Clifford Stein

  16. [24]

    Robertson and Steve Walker

    Stephen E. Robertson and Steve Walker. 1994. Some simple effective approxi- mations to the 2-poisson model for probabilistic weighted retrieval. In Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. https://api.semanticscholar.org/C...

    1994

  17. [25]

    Arpad E. Elo. 1978. The Rating of Chessplayers, Past and Present . Arco Pub., New York. isbn: 0668047216 9780668047210. http://www.amazon.com/Rating-Chess -Players-Past-Present/dp/0668047216

    1978

  18. [26]

    Sergey Brin and Lawrence Page. 1998. The anatomy of a large-scale hypertex- tual web search engine. Computer Networks and ISDN Systems , 30, 107–117. doi: 10.1016/S0169-7552(98)00110-X

    1998 doi

  19. [27]

    Shah and Martin J

    Nihar B. Shah and Martin J. Wainwright. 2018. Simple, robust and optimal ranking from pairwise comparisons. Journal of Machine Learning Research , 18, 199, 1–38. http://jmlr.org/papers/v18/16-206.html. Evgeny Dedov

    2018

  20. [28]

    Sahand Negahban, Sewoong Oh, and Devavrat Shah. 2017. Rank centrality: ranking from pairwise comparisons. Operations Research, 65, 1, 266–287. eprint: https://doi.org/10.1287/opre.2016.1534. doi: 10.1287/opre.2016.1534

    2017

  21. [29]

    Phillip Bonacich. 1987. Power and centrality: a family of measures. American Journal of Sociology, 92, 5, 1170–1182. Retrieved Apr. 24, 2025 from http://ww w.jstor.org/stable/2780000

    1987

  22. [30]

    Bradley and Milton E

    Ralph A. Bradley and Milton E. Terry. 1952. Rank analysis of incomplete block designs: i. the method of paired comparisons. Biometrika, 39, 324–345. doi: 10.2307/2334029

    1952 doi

  23. [31]

    Douglas C Montgomery. 2017. Design and analysis of experiments . John wiley & sons

    2017

  24. [32]

    Ray C Bose and Takatsugu Shimamoto. 1952. Classification and analysis of partially balanced incomplete block designs with two associate classes. Journal of the American statistical association , 47, 258, 151–184

    1952

  25. [33]

    Xingjian Bai and Christian Coester. 2023. Sorting with predictions. In Pro- ceedings of the 37th International Conference on Neural Information Processing Systems (NIPS ’23) Article 1155. Curran Associates Inc., New Orleans, LA, USA, 22 pages

    2023

  26. [34]

    Dmitry Ustalov. 2025. Reliable, Reproducible, and Really Fast Leaderboards with Evalica. In Proceedings of the 31st International Conference on Computational Linguistics: System Demonstrations. Association for Computational Linguistics, Abu Dhabi, UAE, 46–53. https://aclanthol...

    2025 arXiv

  27. [35]

    Fred Glover and Manuel Laguna. 1998. Tabu search. Springer

    1998

  28. [36]

    Mistral AI Team. 2023. Mistral 7b. https://mistral.ai/news/announcing-mistral- 7b. Accessed: 2025-04-21. (2023)

    2023

  29. [37]

    Mistral AI Team. 2025. Mistral small 3. https://mistral.ai/news/mistral-small-3. Accessed: 2025-04-21. (2025)

    2025

  30. [38]

    OpenAI. 2025. Introducing gpt-4.1 in the api. https://openai.com/index/gpt-4-1/. Accessed: 2025-04-21. (2025)

    2025

  31. [39]

    Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, and Iryna Gurevych. 2021. BEIR: a heterogeneous benchmark for zero-shot evaluation of information retrieval models. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Tra...

    2021

  32. [1994]

    Introduction to algorithms. Vol. 3. MIT press Cambridge, MA, USA

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.