Pith. sign in

REVIEW 4 major objections 6 minor 42 references

QUPID: Quantified Understanding for Enhanced Performance, Insights, and Decisions in Korean Search Engines

T0 review · 4 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Two small, differently built language models beat large ones at judging search relevance.

desk verdict Believable engineering win for Korean relevance labeling, but the SLM-vs-LLM headline is confounded by fine-tuning; the internal ablation is the paper's strength. read the letter →

arxiv 2505.07345 v1 pith:274VHBA5 submitted 2025-05-12 cs.CL cs.AIcs.IR

Ohjoon Kwon , Changsu Lee , Jihye Back , Lim Sun Suk , Inho Kang , Donghyeon Jeon This is my paper · ORCID
classification cs.CLcs.AIcs.IR
keywords relevanceassessmentsmalllanguagemodelsheterogeneousensemblequery-documentKoreansearchCohen'skapparankingtoken-probabilityscoring
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

QUPID claims that the best practical judge of whether a search result matches a query is not a large language model but a fine-tuned pair of small models with different architectures: a generative model that scores relevance from token probabilities and an embedding model that scores it from pooled hidden states, combined by weighted averaging. On Korean query-document data across snippets, user-generated content, and general web documents, the ensemble reaches Cohen's kappa 0.646 against human labels, where the strongest zero-shot LLM baseline reaches 0.387, and it does so at 62 milliseconds per judgment versus about 3.3 seconds. The paper also reports that the same relevance scores, plugged into a live search pipeline, improve ranking quality by 1.9% in nDCG@5. A sympathetic reader takes the claim to be that architectural diversity among small models can substitute for raw scale in this task.

What carries the argument

The load-bearing object is the heterogeneous ensemble itself: a weighted average $s_{\text{final}} = w_{\text{gen}} s_{\text{gen}} + w_{\text{emb}} s_{\text{emb}}$ of two fine-tuned SLM scores. The generative score follows the token-probability method: the model is fine-tuned to output exactly one of three label tokens, and the score is $\sum_k p_{i,k} y_k$, the probability-weighted expectation over labels. The embedding score mean-pools the hidden states of the query-document pair and maps the pooled vector through a learned linear layer and softmax. Both models are fine-tuned on roughly one million Korean query-document pairs, including synthetic hard negatives, and the final weights are tuned on a held-out validation set. An operational detail that carries much of the efficiency gain is prompt compression: after fine-tuning, the model needs only a ten-token system prompt plus one special task token and emits a single token, which is why inference takes tens of milliseconds rather than seconds.

What would settle it

Recompute QUPID's Cohen's kappa and AUC after removing from the three test sets every pair that is an exact or near-duplicate of a training pair (using, say, MinHash on normalized text), and compare with the LLM baselines on the same filtered sets; if the gap from 0.646 to 0.387 shrinks substantially, the claim that heterogeneous SLMs beat LLMs on genuinely unseen data fails. A complementary check is to hold out a fresh batch of query-document pairs collected after the training corpus was frozen and re-run the same evaluation.

Watch

Extended reading notes

Core claim

The paper's central discovery is that a heterogeneous ensemble of two fine-tuned small language models, one generative and one embedding-based, can label query-document relevance more accurately than leading zero-shot LLMs while running about 60 times faster. The generative arm, QUPID_GEN, is fine-tuned to emit one of three special label tokens and converts their log probabilities into a softmax distribution, with the relevance score taken as the probability-weighted expectation over label values; the embedding arm, QUPID_EMB, mean-pools the token hidden states of a decoder-only model and passes the pooled vector through a linear layer with softmax. Their scores are combined as a weighted average with weights chosen on a validation set. Across the three test collections the ensemble attains an average kappa of 0.646 and relevant/irrelevant AUC of 0.945/0.871, beating every LLM and every same-architecture ensemble tested, and the paper attributes this to complementary strengths of generative reasoning and similarity-based representation.

Load-bearing premise

The test sets of 3,000 snippet, 20,000 user-generated, and 9,000 web-document pairs are drawn from the same three document sources as the roughly one million training pairs, and the paper reports no deduplication or near-duplicate overlap removal; if those test pairs overlap with training data, the reported kappa and AUC are inflated.

Editorial extensions

If this is right

  • If QUPID's claimed advantage holds, production search engines can afford to run relevance assessment on every query-document pair, not just samples, because per-judgment cost drops to 62 ms.
  • Heterogeneous SLM ensembles, mixing a generative and an embedding-based model, become a viable alternative to zero-shot LLM assessors, especially for languages where large models underperform.
  • The same relevance scores can serve multiple pipeline roles: filtering low-quality pairs at precision above 0.9, auditing query rewriting and auto-completion, flagging misleading snippets, and feeding the ranker directly.
  • Fine-tuning on task data can compress long natural-language prompts into a short task prefix, cutting both latency and serving cost.
  • Homogeneous ensembles of the same architecture do not capture the benefit; Table 2 shows that ensembling three or five copies of QUPID_GEN or QUPID_EMB yields smaller gains than combining the two different architectures.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial: the reported comparison mixes two variables, architecture diversity and fine-tuning on roughly one million labeled pairs, so the paper does not isolate which one drives the kappa gain; a fair test would fine-tune a single SLM of comparable size on the same data and compare.
  • Editorial: if the mechanism is genuinely architectural complementarity, the same generative-plus-embedding recipe should transfer to other languages and to tasks like answer relevance in retrieval-augmented generation; the paper only demonstrates Korean at scale, with one English anecdote.
  • Editorial: because the test sets and training corpus share the same three document sources and no deduplication is reported, the practical gap over LLMs might shrink on genuinely fresh data; a temporal split would settle this.
  • Editorial: the 60x speedup partly reflects the short prompt and single-token output, not just model size; an LLM fine-tuned with the same compressed prompt might narrow the latency gap, which would change the cost-benefit comparison.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes QUPID, a relevance-assessment method that combines two small language models of different architectures: a generative SLM that scores relevance through token probabilities and an embedding-based SLM that scores through a pooled representation with a linear head. Both components are fine-tuned on roughly one million Korean query-document pairs, including synthetic hard negatives, and combined by a weighted average. The authors report that QUPID achieves a Cohen's kappa of 0.646 versus 0.387 for the best zero-shot LLM baseline, reduces inference latency to 62 ms versus seconds, and improves production ranking nDCG@5 by 1.9%. They also present use cases for filtering low-quality pairs, evaluating query refinement, and assessing snippet quality.

Significance. If the empirical claims are supported, the paper would make a useful practical contribution: a fine-tuned heterogeneous SLM ensemble could serve as a cheaper, faster alternative to zero-shot LLM relevance assessors in a deployed Korean search engine. The paper's strengths include its real-world production setting, three document-type test sets, and an internal ablation comparing homogeneous and heterogeneous ensembling. However, the central headline claim is weakened by a confounded experimental design and a lack of uncertainty quantification. The paper does not release code or data, which limits reproducibility, but the methodology is straightforward and the internal ablation is informative.

major comments (4)
  1. [§4.2.1 vs. §3.1] The central comparison is confounded: QUPID is fine-tuned on roughly one million labeled query-document pairs (Section 3.1), while all LLM baselines in Table 1 are evaluated zero-shot with a single prompting strategy (Section 4.2.1). The abstract's claim that 'combining two distinct SLMs with different architectures can outperform LLMs' is not established by this design because the comparison measures the combined effect of fine-tuning and architecture, not architecture alone. A fine-tuned LLM on the same data could plausibly match or exceed the reported 0.646 kappa. Please add fine-tuned LLM baselines, or at least a zero-shot variant of QUPID, and qualify the language in the abstract and contributions accordingly.
  2. [Table 1 and Table 5] No error bars, confidence intervals, or significance tests are reported for any kappa, AUC, or nDCG value. The headline difference (0.646 vs. 0.387) and the production nDCG@5 improvement (+1.9% on 719 queries) cannot be assessed for statistical reliability. Please report bootstrap confidence intervals or paired significance tests (e.g., signed tests on per-query nDCG) for the key comparisons.
  3. [§3.1 vs. §4.1] Training and test data are drawn from the same three document types (snippets, UGC, web documents), but the paper does not describe any deduplication or near-duplicate removal between the roughly one million training pairs and the test sets. If near-duplicate query-document pairs occur in both, kappa and AUC values would be inflated. Please report an overlap analysis (e.g., exact-match and embedding-similarity duplicate rates) and, if leakage is found, evaluate on a disjoint held-out set.
  4. [§3.2 and §3.4] The ensemble weights w_gen and w_emb are tuned on a held-out validation set, and the label weights y_k are manually fixed. This is acceptable in principle, but the paper should report the selected weight values and a sensitivity analysis to show that the reported gains are not an artifact of overfitting the validation set. Also state explicitly whether the validation set is disjoint from the three test sets used in Tables 1 and 2.
minor comments (6)
  1. [Abstract and Table 3] The abstract states '60x faster inference times,' but Table 3 shows 62 ms vs. 3258 ms, which is approximately 52.5x; please recompute or reword.
  2. [Table 1] The column header 'AUC (Relevant / Irrelevant)' is ambiguous; please clarify that these are class-wise AUC values and describe how they are computed.
  3. [§4.2.2] The description of the JudgeBlender baseline should clarify whether its constituent models were fine-tuned on Korean data or used zero-shot, and exactly how majority voting and average ensembling were applied.
  4. [Table 2 caption] The caption says the rows are trained 'with different hyperparameters,' but the specific hyperparameters varied are not listed; please provide this information or refer to the appendix.
  5. [Appendix A.3] Table 4 lists a row 'T: inference temperature' but no temperature value appears in the table; the text mentions temperature 3.0 only later. Please make the table consistent with the prose.
  6. [References] The reference to Aho and Ullman (1972) in Section 3.1.2 does not appear related to hard-negative generation; please verify and replace if mis-cited. Also, de Souza P. Moreira et al. (2024) and (2025) appear to be the same preprint; consolidate.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: QUPID's kappa and AUC claims are evaluated against human labels and external LLM baselines; the ensemble weighting is tuned on validation but does not predetermine test outcomes.

full rationale

The paper's central quantitative claims (Cohen's kappa 0.646 vs. 0.387, AUC values, and 60x latency) are empirical measurements on held-out test sets with human-annotated labels (Section 4.1), compared against externally defined LLM and SLM baselines. No equation in Sections 3.2–3.4 derives the reported accuracy from the model's own outputs as ground truth: Eq. (2) defines a relevance score from token probabilities with hand-assigned label weights, and Eq. (8) is a weighted average whose coefficients are tuned on a held-out validation set; neither makes the test-set kappa equal to a fitted quantity by construction. The HCX-S backbone citation (Yoo et al., 2024) is used as a base model, not as an authority that forces the ensemble result. The only notable validity concern, that QUPID is fine-tuned on roughly one million labeled pairs while LLM baselines are zero-shot, is a comparison confound about experimental fairness rather than circularity, because the baselines' scores are independent of the fitted parameters. The nDCG@5 production result lacks an explicit description of the judgment labels, but the paper does not state that QUPID's own scores served as ground truth; absent such a statement, claiming self-evaluation circularity would be speculation. Therefore, no load-bearing step reduces to its own inputs.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

QUPID is a system name, not a newly postulated entity; no new physical or mathematical objects are introduced. The free parameters are standard tuning choices, and the axioms are the assumptions an independent replication would need to verify.

free parameters (3)
  • Label weights y_k (relevant, somewhat relevant, irrelevant) = 1.0, 0.5, 0
    Hand-set in Eq (2) to convert token probabilities into a scalar relevance score; affects both QUPID_GEN and the ensemble.
  • Ensemble weights w_gen and w_emb = not reported
    Selected on a held-out validation set in Section 3.4; the reported kappa and production gains depend on these values.
  • Inference temperature = 3.0
    Set to 3.0 based on Figure 1 tuning; the paper notes lower temperatures produce overly confident single-token probabilities, so this choice is load-bearing for the generative model's scores.
assumptions (4)
  • domain assumption Human annotations provide correct ground truth labels for the four relevance classes.
    Section 3.1.1 describes a voting scheme but reports no inter-annotator agreement; all kappa and AUC scores are computed against these labels.
  • domain assumption The training corpus and the test sets are independent (no near-duplicate leakage).
    Training (Section 3.1) and test (Section 4.1) draw from the same three document types (snippets, UGC, web), but no deduplication or overlap removal is described.
  • ad hoc to paper The zero-shot LLM baselines represent 'state-of-the-art LLM solutions' for relevance labeling.
    Section 4.2.1 uses four LLMs with one prompting strategy; no fine-tuned LLM baseline is included, so the headline comparison is narrow and favorable to QUPID.
  • domain assumption HCX-S supports Korean and transfers to English for the examples in Table 6.
    The paper feeds English text to QUPID in the use-case tables and asserts the backbone's multilingual capability without a dedicated English evaluation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of QUPID: Quantified Understanding for Enhanced Performance, Insights, and Decisions in Korean Search Engines." pith.science (2026). https://pith.science/paper/274VHBA5

@misc{pith2026250507345,
  author       = {Pith},
  title        = {Pith review of: QUPID: Quantified Understanding for Enhanced Performance, Insights, and Decisions in Korean Search Engines},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/274VHBA5}},
  note         = {Machine review of arXiv:2505.07345}
}
read the original abstract

Large language models (LLMs) have been widely used for relevance assessment in information retrieval. However, our study demonstrates that combining two distinct small language models (SLMs) with different architectures can outperform LLMs in this task. Our approach -- QUPID -- integrates a generative SLM with an embedding-based SLM, achieving higher relevance judgment accuracy while reducing computational costs compared to state-of-the-art LLM solutions. This computational efficiency makes QUPID highly scalable for real-world search systems processing millions of queries daily. In experiments across diverse document types, our method demonstrated consistent performance improvements (Cohen's Kappa of 0.646 versus 0.387 for leading LLMs) while offering 60x faster inference times. Furthermore, when integrated into production search pipelines, QUPID improved nDCG@5 scores by 1.9%. These findings underscore how architectural diversity in model combinations can significantly enhance both search relevance and operational efficiency in information retrieval systems.

Figures

Figures reproduced from arXiv: 2505.07345 by the authors.

Figure 2
Figure 2. PR curve of QUPIDENSEMBLE model on the Web-D dataset. olding that prioritizes high precision, even at the cost of some coverage (recall). This approach al￾lows us to minimize side effects, such as filtering out high-quality documents, while effectively filter￾ing out low-quality documents with high accuracy (above 0.95). A.4.2 Efficiency Compare In contrast to the prompting-based approach, the ex￾periment for our mo… view at source ↗
Figure 1
Figure 1. Effect of Temperature on AUC Scores A.3 Hyper Parameters In this section, we detail the hyperparameters used for training the embedding-based model (EMB) and the generative model (GEN) described in our methodology. We also indicate the proportion of synthetic data used, along with any notable imple￾mentation remarks. Refer to [PITH_FULL_IMAGE:figures/full_fig_p011_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

42 extracted references · 4 canonical work pages

  1. [1]

    Aho and Jeffrey D

    Alfred V. Aho and Jeffrey D. Ullman. 1972. The Theory of Parsing, Translation and Compiling, volume 1. Prentice-Hall, Englewood Cliffs, NJ

    1972

  2. [2]

    Yejin Bang, Samuel Cahyawijaya, Nayeon Lee, Wenliang Dai, Dan Su, Bryan Wilie, Holy Lovenia, Ziwei Ji, Tiezheng Yu et al. 2023. https://arxiv.org/abs/2302.04023 A multitask, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity . Preprint, arXiv:2302.04023

    arXiv 2023

  3. [3]

    Felix Brei, Johannes Frey, and Lars-Peter Meyer. 2024. https://arxiv.org/abs/2405.17076 Leveraging small language models for text2sparql tasks to improve the resilience of ai assistance . Preprint, arXiv:2405.17076

    arXiv 2024

  4. [5]

    Moreira, Radek Osmulski, Mengyao Xu, Ronay Ak, Benedikt Schifferer, and Even Oldridge

    Gabriel de Souza P. Moreira, Radek Osmulski, Mengyao Xu, Ronay Ak, Benedikt Schifferer, and Even Oldridge. 2025. https://arxiv.org/abs/2407.15831 Nv-retriever: Improving text embedding models with effective hard-negative mining . Preprint, arXiv:2407.15831

    arXiv 2025

  5. [6]

    Guglielmo Faggioli, Laura Dietz, Charles L. A. Clarke, Gianluca Demartini, Matthias Hagen, Claudia Hauff, Noriko Kando, Evangelos Kanoulas, Martin Potthast et al. 2023. https://doi.org/10.1145/3578337.3605136 Perspectives on large language models for relevance judgment . In Proceedings of the 2023 ACM SIGIR International Conference on Theory of Informatio...

    arXiv 2023

  6. [7]

    Naghmeh Farzi and Laura Dietz. 2024. https://arxiv.org/abs/2410.14044 Best in tau@llmjudge: Criteria-based relevance evaluation with llama3 . Preprint, arXiv:2410.14044

    arXiv 2024

  7. [8]

    Simon Gog, Giulio Ermanno Pibiri, and Rossano Venturini. 2020. https://doi.org/10.1145/3397271.3401432 Efficient and effective query auto-completion . In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’20, page 2271–2280. ACM

    arXiv 2020

  8. [9]

    Kristen Howell, Gwen Christian, Pavel Fomitchov, Gitit Kehat, Julianne Marzulla, Leanne Rolston, Jadin Tredup, Ilana Zimmerman, Ethan Selfridge, and Joseph Bradley. 2023. https://arxiv.org/abs/2306.07402 The economic trade-offs of large language models: A case study . Preprint, arXiv:2306.07402

    arXiv 2023

Show all 42 references
  1. [10]

    Aaron Jaech and Mari Ostendorf. 2018. https://doi.org/10.18653/v1/P18-2111 Personalized language model for query auto-completion . In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 700--705, Melbourne, Au...

    2018 doi

  2. [11]

    Ravindu Jayakody and Gihan Dias. 2024. https://arxiv.org/abs/2407.21330 Performance of recent large language models for a low-resourced language . Preprint, arXiv:2407.21330

    2024 arXiv

  3. [12]

    Gyuwan Kim. 2019. https://doi.org/10.18653/v1/D19-1507 Subword language model for query auto-completion . In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-...

    2019 doi

  4. [13]

    Chankyu Lee, Rajarshi Roy, Mengyao Xu, Jonathan Raiman, Mohammad Shoeybi, Bryan Catanzaro, and Wei Ping. 2025. https://arxiv.org/abs/2405.17428 Nv-embed: Improved techniques for training llms as generalist embedding models . Preprint, arXiv:2405.17428

    2025 arXiv

  5. [14]

    Cole, Kai Hui, Michael Boratko, Rajvi Kapadia et al

    Jinhyuk Lee, Zhuyun Dai, Xiaoqi Ren, Blair Chen, Daniel Cer, Jeremy R. Cole, Kai Hui, Michael Boratko, Rajvi Kapadia et al. 2024. https://arxiv.org/abs/2403.20327 Gecko: Versatile text embeddings distilled from large language models . Preprint, arXiv:2403.20327

    2024 arXiv

  6. [15]

    Chaofan Li, Zheng Liu, Shitao Xiao, Yingxia Shao, and Defu Lian. 2024 a . https://doi.org/10.18653/v1/2024.acl-long.191 L lama2 V ec: Unsupervised adaptation of large language models for dense retrieval . In Proceedings of the 62nd Annual Meeting of the Association for Computa...

    2024 doi

  7. [16]

    Xiaoxi Li, Jiajie Jin, Yujia Zhou, Yuyao Zhang, Peitian Zhang, Yutao Zhu, and Zhicheng Dou. 2024 b . https://arxiv.org/abs/2404.14851 From matching to generation: A survey on generative information retrieval . Preprint, arXiv:2404.14851

    2024 arXiv

  8. [17]

    Zihao Li, Yucheng Shi, Zirui Liu, Fan Yang, Ali Payani, Ninghao Liu, and Mengnan Du. 2024 c . https://arxiv.org/abs/2404.11553 Language ranker: A metric for quantifying llm performance across high and low-resource languages . Preprint, arXiv:2404.11553

    2024 arXiv

  9. [18]

    Jie Liu and Barzan Mozafari. 2024. https://arxiv.org/abs/2403.09060 Query rewriting via large language models . Preprint, arXiv:2403.09060

    2024

  10. [19]

    Xueguang Ma, Liang Wang, Nan Yang, Furu Wei, and Jimmy Lin. 2023. https://arxiv.org/abs/2310.08319 Fine-tuning llama for multi-stage text retrieval . Preprint, arXiv:2310.08319

    2023 arXiv

  11. [20]

    Xuan-Phi Nguyen, Sharifah Mahani Aljunied, Shafiq Joty, and Lidong Bing. 2024. https://arxiv.org/abs/2306.11372 Democratizing llms for low-resource languages by leveraging their english dominant abilities with linguistically-diverse prompts . Preprint, arXiv:2306.11372

    2024 arXiv

  12. [21]

    Jingwei Ni, Tobias Schimanski, Meihong Lin, Mrinmaya Sachan, Elliott Ash, and Markus Leippold. 2025. https://arxiv.org/abs/2406.14162 Diras: Efficient llm annotation of document relevance in retrieval augmented generation . Preprint, arXiv:2406.14162

    2025 arXiv

  13. [22]

    Rodrigo Nogueira, Wei Yang, Jimmy Lin, and Kyunghyun Cho. 2019. https://arxiv.org/abs/1904.08375 Document expansion by query prediction . Preprint, arXiv:1904.08375

    2019 arXiv

  14. [23]

    Rahmani, Emine Yilmaz, Nick Craswell, and Bhaskar Mitra

    Hossein A. Rahmani, Emine Yilmaz, Nick Craswell, and Bhaskar Mitra. 2024 a . https://arxiv.org/abs/2412.13268 Judgeblender: Ensembling judgments for automatic relevance assessment . Preprint, arXiv:2412.13268

    2024 arXiv

  15. [24]

    Rahmani, Emine Yilmaz, Nick Craswell, Bhaskar Mitra, Paul Thomas, Charles L

    Hossein A. Rahmani, Emine Yilmaz, Nick Craswell, Bhaskar Mitra, Paul Thomas, Charles L. A. Clarke, Mohammad Aliannejadi, Clemencia Siro, and Guglielmo Faggioli. 2024 b . https://arxiv.org/abs/2408.08896 Llmjudge: Llms for relevance judgments . Preprint, arXiv:2408.08896

    2024 arXiv

  16. [25]

    Mortensen, and Graham Neubig

    Nathaniel Robinson, Perez Ogayo, David R. Mortensen, and Graham Neubig. 2023. https://doi.org/10.18653/v1/2023.wmt-1.40 C hat GPT MT : Competitive for high- (but not low-) resource languages . In Proceedings of the Eighth Conference on Machine Translation, pages 392--418, Sing...

    2023 doi

  17. [26]

    Devendra Singh Sachan, Mike Lewis, Mandar Joshi, Armen Aghajanyan, Wen tau Yih, Joelle Pineau, and Luke Zettlemoyer. 2023. https://arxiv.org/abs/2204.07496 Improving passage retrieval with zero-shot question generation . Preprint, arXiv:2204.07496

    2023 arXiv

  18. [27]

    Or Sharir, Barak Peleg, and Yoav Shoham. 2020. https://arxiv.org/abs/2004.08900 The cost of training nlp models: A concise overview . Preprint, arXiv:2004.08900

    2020 arXiv

  19. [28]

    Emma Strubell, Ananya Ganesh, and Andrew McCallum. 2019. https://arxiv.org/abs/1906.02243 Energy and policy considerations for deep learning in nlp . Preprint, arXiv:1906.02243

    2019 arXiv

  20. [29]

    Weiwei Sun, Lingyong Yan, Xinyu Ma, Shuaiqiang Wang, Pengjie Ren, Zhumin Chen, Dawei Yin, and Zhaochun Ren. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.923 Is C hat GPT good at search? investigating large language models as re-ranking agents . In Proceedings of the 2023 ...

    2023 doi

  21. [30]

    Zhaoyan Sun, Xuanhe Zhou, and Guoliang Li. 2024. https://arxiv.org/abs/2412.01661 R-bot: An llm-based query rewrite system . Preprint, arXiv:2412.01661

    2024 arXiv

  22. [31]

    Qiaoyu Tang, Jiawei Chen, Zhuoqun Li, Bowen Yu, Yaojie Lu, Cheng Fu, Haiyang Yu, Hongyu Lin, Fei Huang et al. 2024. https://arxiv.org/abs/2403.00801 Self-retrieval: End-to-end information retrieval with one large language model . Preprint, arXiv:2403.00801

    2024 arXiv

  23. [32]

    Paul Thomas, Seth Spielman, Nick Craswell, and Bhaskar Mitra. 2024. https://arxiv.org/abs/2309.10621 Large language models can accurately predict searcher preferences . Preprint, arXiv:2309.10621

    2024 arXiv

  24. [33]

    Shivani Upadhyay, Ronak Pradeep, Nandan Thakur, Nick Craswell, and Jimmy Lin. 2024. https://arxiv.org/abs/2406.06519 Umbrela: Umbrela is the (open-source reproduction of the) bing relevance assessor . Preprint, arXiv:2406.06519

    2024 arXiv

  25. [34]

    Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei. 2024. https://arxiv.org/abs/2401.00368 Improving text embeddings with large language models . Preprint, arXiv:2401.00368

    2024 arXiv

  26. [35]

    Xueru Wen, Xiaoyang Chen, Xuanang Chen, Ben He, and Le Sun. 2023. https://doi.org/10.1145/3539618.3592028 Offline pseudo relevance feedback for efficient and effective single-pass dense retrieval . In Proceedings of the 46th International ACM SIGIR Conference on Research and D...

    2023

  27. [36]

    Zeqiu Wu, Yi Luan, Hannah Rashkin, David Reitter, Hannaneh Hajishirzi, Mari Ostendorf, and Gaurav Singh Tomar. 2022. https://arxiv.org/abs/2112.08558 Conqrr: Conversational query rewriting for retrieval with reinforcement learning . Preprint, arXiv:2112.08558

    2022 arXiv

  28. [37]

    Yukang Xie, Chengyu Wang, Junbing Yan, Jiyong Zhou, Feiqi Deng, and Jun Huang. 2024. https://doi.org/10.1145/3616855.3635690 Making small language models better multi-task learners with mixture-of-task-adapters . In Proceedings of the 17th ACM International Conference on Web S...

    2024

  29. [38]

    Wujiang Xu, Qitian Wu, Zujie Liang, Jiaojiao Han, Xuying Ning, Yunxiao Shi, Wenfang Lin, and Yongfeng Zhang. 2025. https://arxiv.org/abs/2405.17890 Slmrec: Distilling large language models into small for sequential recommendation . Preprint, arXiv:2405.17890

    2025 arXiv

  30. [39]

    Kang Min Yoo, Jaegeun Han, Sookyo In, Heewon Jeon, Jisu Jeong, Jaewook Kang, Hyunwook Kim, Kyung-Min Kim, Munhyong Kim et al. 2024. https://arxiv.org/abs/2404.01954 Hyperclova x technical report . Preprint, arXiv:2404.01954

    2024 arXiv

  31. [40]

    Yutao Zhu, Huaying Yuan, Shuting Wang, Jiongnan Liu, Wenhan Liu, Chenlong Deng, Haonan Chen, Zheng Liu, Zhicheng Dou, and Ji-Rong Wen. 2024. https://arxiv.org/abs/2308.07107 Large language models for information retrieval: A survey . Preprint, arXiv:2308.07107

    2024

  32. [41]

    Honglei Zhuang, Zhen Qin, Kai Hui, Junru Wu, Le Yan, Xuanhui Wang, and Michael Bendersky. 2024. https://arxiv.org/abs/2310.14122 Beyond yes and no: Improving zero-shot llm rankers via scoring fine-grained relevance labels . Preprint, arXiv:2310.14122

    2024 arXiv

  33. [42]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  34. [43]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.