Pith. sign in

REVIEW 4 major objections 4 minor 61 references

Explainable LLM-driven Multi-dimensional Distillation for E-Commerce Relevance Learning

T0 review · 4 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Distilling an LLM's step-by-step relevance reasoning into small online models improves e-commerce search matching, with the largest gains on long-tail queries.

desk verdict A real, deployed industrial recipe for distilling LLM relevance knowledge into small rankers, but the headline gains are confounded by extra unlabeled data and the 'explainable' claim rests on an unevaluated CoT. read the letter →

arxiv 2411.13045 v2 pith:O6GA4ZFY submitted 2024-11-20 cs.IR cs.AIcs.CL

Gang Zhao , Ximing Zhang , Chenji Lu , Hui Zhao , Tianshu Wu , Pengjie Wang , Jian Xu , Bo Zheng This is my paper · ORCID
classification cs.IRcs.AIcs.CL
keywords e-commercerelevanceknowledgedistillationchain-of-thoughtlargelanguagemodelssemanticmatchinglong-tailgeneralization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a large language model's relevance judgments can be made both explainable and transferable by treating the judgment as chain-of-thought reasoning over fine-grained product aspects. The authors build an explainable teacher LLM (ELLM-rele) that outputs step-by-step attribute matching before its Good/Bad verdict, then distill two kinds of knowledge into small online models: the calibrated probability of the verdict and the token-level reasoning trace. The distilled students keep their original architectures and inference speed while gaining relevance accuracy, especially on long-tail queries. Offline gains are 2.2 to 4.1 ROC-AUC points over four baselines; an online A/B test shows +0.17% CTR and +0.89% Goodrate. If true, this is a practical route to putting LLM-grade semantic understanding into deployable search systems.

What carries the argument

The load-bearing components are (1) ELLM-rele, a 7B LLM fine-tuned on CoT annotations generated by two larger models (72B and 70B) with self-consistency and filtered for label agreement, which produces structured reasoning of the form 'attribute in query; attribute in title; match/mismatch; final judgment'; and (2) the MKD head, which converts the teacher's Good/Bad token probabilities into a normalized relevance score and trains students with a Kullback-Leibler divergence objective, while simultaneously transferring the CoT via sequence tagging (CRF) for interaction models or attention-score regulation ($L_2$ distance on cosine attention) for representation models. The CoT trace is the key mechanism: it turns the teacher's latent matching process into explicit token-level supervision that tells the student which spans are relevant or irrelevant and how they align.

What would settle it

An experiment that swaps or scrambles the relevant/irrelevant span tags in the CoT distillation while keeping the final label fixed would distinguish faithful reasoning from superficial alignment: if the student's accuracy on long-tail queries stays unchanged, the reasoning trace is not the carrier of the distilled knowledge. Similarly, a human-annotation study rating whether the generated CoT steps actually reflect the attributes that determined the label would test the premise directly.

Watch

Extended reading notes

Core claim

The central claim is that decomposing e-commerce relevance into a chain of aspect-level match checks (category, product, brand, gender, and so on) and distilling both the resulting score distribution and the reasoning trace into existing student models improves relevance learning in both interaction-based and representation-based architectures. The teacher ELLM-rele outperforms a plain fine-tuned 7B LLM by 0.8% ROC-AUC, showing the CoT formulation itself adds accuracy, not just interpretability. The student models trained with the multi-dimensional distillation outperform their respective baselines by 2.2–4.1% ROC-AUC and 2.5–4.6% Neg PR-AUC, and the long-tail subset shows a larger gap, indicating improved generalization to low-frequency queries.

Load-bearing premise

The framework assumes that the chain-of-thought rationales generated by the larger LLMs — after discarding those that disagree with the human label — are genuinely the reasoning behind the relevance judgment, and not plausible post-hoc justifications.

Editorial extensions

If this is right

  • Deployed student models gain relevance accuracy without added inference latency or parameter growth, since distillation is applied only during training.
  • Long-tail queries benefit more than head queries, narrowing the frequency-driven performance gap that usually hurts low-data relevance models.
  • Both interaction-based and representation-based architectures can absorb the same teacher knowledge through two different CoT distillation losses.
  • The same CoT formulation can be reused for relevance annotation assistance and error analysis, as the paper notes.
  • The teacher's score probabilities carry fine-grained relevance information beyond hard labels, and distilling them adds accuracy over hard-label distillation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the CoT traces are faithful, attention-score regulation may effectively encode hard aspect constraints (e.g., gender mismatch) into token-level attention, which could generalize to other structured matching tasks.
  • The paper does not measure whether the CoT rationales are causally responsible for the LLM's verdict; if they are post-hoc, the student may learn artifacts of phrasing. A controlled test would be to train students on rationales that are shuffled or generated after seeing the label and compare.
  • The multi-dimensional distillation may be portable to other LLM-as-teacher settings (e.g., question answering or retrieval) where decomposable, aspect-based reasoning is available.
  • Using two different large teacher models for annotation may provide a diversity benefit; a testable extension is whether single-teacher annotation underperforms the ensemble.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes ELLM-MKD, a framework that first builds an explainable LLM teacher (ELLM-rele) for e-commerce query-item relevance by recasting relevance judgment as chain-of-thought (CoT) reasoning over fine-grained aspects, and then distills this teacher into small deployable students (BERT, Sentence-BERT, PolyEncoder, ColBERT) through two complementary objectives: relevance score distribution distillation and token-level CoT knowledge distillation. Offline experiments report consistent ROC-AUC/PR-AUC gains over the baseline students, an ablation study attributes gains to the score and CoT components, and a two-week online A/B test shows CTR and Goodrate improvements, especially on long-tail queries.

Significance. The paper addresses a practically important problem—transferring LLM reasoning knowledge into efficient e-commerce relevance models—and the idea of distilling not just scores but also explicit CoT-derived token-level supervision is interesting and plausibly useful. Credit is due for reporting consistent gains across four student architectures, for the online deployment results, and for a fairly detailed ablation that attempts to separate the contributions of score and CoT distillation. The industrial-scale setting (Taobao search ads) gives the empirical claims real-world grounding. However, the headline comparison is confounded by the use of 30M unlabeled pseudo-labeled pairs only in the MKD students, the core KL distillation objective is not written as a valid mathematical expression, and the 'explainable' claim rests on CoT annotations that are filtered for label agreement but not validated for reasoning faithfulness. These issues are fixable but require additional experiments and corrections before the central claims can be accepted.

major comments (4)
  1. [Section 4.4, Table 2 vs Section 4.6, Table 4] The main comparison in Table 2 is confounded: MKD students are trained with an additional 30 million unlabeled query-item pairs pseudo-labeled by the teacher, while the baseline students are trained only on the human-labeled set. The ablation in Table 4 shows that removing all unlabeled data ('w/o MKD') costs -2.8/-2.2 ROC-AUC for MKDColBERT/MKDBERT, which is larger than the gains attributed to score distillation (+0.9/+0.6) and CoT distillation (+0.5/+0.4). Thus the Table 2 differences over the baselines do not isolate the proposed distillation mechanism; they largely reflect the benefit of the extra training data. To support the claim that the multi-dimensional distillation itself improves students, please report a baseline trained on the same pseudo-labeled 30M pairs with hard labels only (e.g., the 'w/o score&CoT' variant) in Table 2, and report gains relative to that baseline. In addition, since the CoT and score contributions are small (0.4-0.9 ROC-AUC), please provide error bars over multiple runs or significance tests.
  2. [Section 3.3.1, Eq. (5)] The KL divergence objective is not mathematically defined as written: s_T(Q_i,T_i) is a scalar in [0,1] after the projection in Eq. (4), while p_S(y|Q_i,T_i) is a probability distribution over the tokens Good/Bad. KL divergence requires two distributions over the same space. Please rewrite the objective as, e.g., KL([s_T, 1-s_T] || [p_S(Good), p_S(Bad)]) or explain an alternative construction. Additionally, Eq. (4) exponentiates probabilities rather than logits; this is an unusual choice that compresses the probability mass and should be justified or replaced with a more standard softmax over the Good/Bad logits.
  3. [Section 3.2.1 and Section 3.3.2] The CoT annotations are constructed by sampling multiple reasoning paths from larger LLMs and then selecting the path whose final judgment matches the human label. This guarantees label agreement but not the validity of the intermediate reasoning; the selected rationales could be post-hoc justifications that do not reflect the true basis of the relevance decision. Because the paper's 'explainable' claim and the CoT distillation mechanism (sequence tagging and attention regulation) assume these rationales are faithful evidence, the absence of any faithfulness check is load-bearing. The ablation shows CoT distillation adds only +0.4-0.5 ROC-AUC, so without a faithfulness study it is unclear whether the CoT signal transfers reasoning knowledge or merely acts as a regularizer. Please provide at least a human evaluation of CoT quality on a sample, an inter-annotator agreement measure, or a counterfactual/attribution test.
  4. [Section 3.3.2, Eq. (6)-(7)] The two CoT distillation losses are not stated consistently with the surrounding text. Eq. (6) is written as a sum of token-level log-probabilities, which is a cross-entropy objective, yet the text says a CRF layer is used; a CRF would include transition scores and a sequence-level partition function. Eq. (7) contains a leading minus sign and a normalization that do not correspond to a standard L2 loss (the L2 distance should be positive). Please correct these equations or clarify the notation so the reported objectives are reproducible.
minor comments (4)
  1. [Throughout] There are several typos and inconsistent terms, e.g., 'aquire' (Abstract and Section 1), 'Warmart-LLM' vs 'Walmart-LLM' (Section 4.2.1 and Table 2), 'tempreaures' (Section 4.3.2), and 'trasnfers' (Section 2.2). A careful proofread is needed.
  2. [Section 4.2.1] The baseline list contains a numbering error: '3)' appears twice (for Poly-Encoder and ColBERT), so the enumeration is confusing and should be corrected.
  3. [Table 1] The row labeled 'Unlabeled' reports 'Good' and 'Bad' counts that are based on pseudo labels, but the column names are identical to those in the labeled splits; please make explicit that these are pseudo-label counts, and clarify what 'CoT term' counts mean for the unlabeled data.
  4. [Section 4.3.2 and Section 4.7] The hyperparameters λ1, λ2, λ3 and the KL temperature are given without sensitivity analysis, and the online A/B test reports only point estimates (+0.17% CTR, +0.89% Goodrate, +1.96% long-tail) with no confidence intervals or significance levels; please add at least a brief sensitivity discussion and, if available, statistical significance for the online metrics.

Circularity Check

0 steps flagged · score 2.0 of 10

No equation-level circularity; the central claim is tested against held-out human labels and online A/B, and the only self-citation is a non-load-bearing methodological citation.

full rationale

Walking the derivation chain: CoT annotations are generated by Qwen2-72B and Llama3-70B and filtered to those whose parsed judgment matches the human label (Section 3.2.1); ELLM-rele is SFT'd on these annotations (Eq. 3); its Good/Bad token probabilities are normalized into relevance scores (Eq. 4) and distilled to students via KL divergence (Eq. 5), while parsed CoT spans become BIO/attention supervision (Section 3.3.2); students are evaluated on held-out human labels and online CTR/Goodrate. No step defines a predicted quantity as a function of a fitted parameter and then reports it as an independent prediction. The score transform in Eq. (4) is a monotonic normalization of p(Good), not a fitted reparameterization. The label-aligned CoT filtering guarantees label agreement but not that the stated reasoning is the actual cause of the LLM's judgment; this is a faithfulness/correctness limitation, not a circular reduction. The central effectiveness claim is externally falsifiable because the test set and online A/B use human labels not derived from the teacher. The only self-citation is Ref [24] (Deep BoW, sharing corresponding author Bo Zheng), used to justify the Neg PR-AUC metric and as related work; it is not load-bearing. Score 2 reflects that one minor non-load-bearing self-citation; there is no circular derivation.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The framework's novelty rests on combination of known techniques; it introduces no new entities. Its few free parameters are standard deep learning hyperparameters, but the aspect-decomposition and token-level supervision assumptions are domain-specific and unvalidated.

free parameters (5)
  • loss weights λ1, λ2, λ3 = 1.0, 0.1, 0.5 (interaction); 1.0, 0.01, 0.5 (representation)
    Hand-chosen hyperparameters controlling how much score and CoT distillation contribute; no sensitivity analysis is reported, and they affect the magnitude of gains.
  • KL temperature = 2.0
    Set for score distillation without reported tuning; affects the softness of the student target distribution.
  • self-consistency samples per model = 5 (10 total)
    Number of CoT generations per query-title pair from each of Qwen2-72B and Llama3-70B; chosen following the cited self-consistency work.
  • LLM generation TopP / TopK / temperature = 0.7 / 50 / 0.8
    Inference hyperparameters for CoT annotation generation, taken from prior work; no sensitivity analysis.
  • long-tail cutoff = UV > 10
    Defines head vs. long-tail queries in the analysis; the specific threshold is arbitrary and changes the head/long-tail split.
assumptions (5)
  • domain assumption E-commerce relevance can be decomposed into a fixed set of aspects (category, product, brand, gender, etc.) such that the final Good/Bad judgment is determined by the conjunction of aspect matches.
    Introduced in Section 3.1 and Table 5; if the aspect list is not exhaustive or aspects interact, the CoT structure and the derived token-level supervision are inaccurate.
  • domain assumption CoT annotations generated by Qwen2-72B and Llama3-70B and filtered by agreement with the human label constitute valid evidence for relevance decisions.
    Section 3.2.1; the annotation pipeline discards disagreeing rationales, so the quality of the remaining rationales is assumed to be high and faithful.
  • ad hoc to paper The probability of the Good token relative to the Bad token after SFT is a meaningful continuous relevance score.
    Section 3.3.1, Eq (4); no calibration analysis or justification that this transform is better than using logits or the raw probabilities.
  • ad hoc to paper In representation-based models, token-level attention (max cosine similarity) should be higher for tokens marked relevant by the CoT and lower for irrelevant tokens.
    Section 3.3.2, Eq (10); this assumption justifies the attention regulation loss and may not hold for all queries or architectures.
  • standard math KL divergence can be applied between the teacher scalar score and the student distribution as written.
    Section 3.3.1, Eq (5); the paper uses KL(s_T || p_S) where s_T is a scalar, which is not a standard KL between distributions; this is an imprecise use that likely means binary cross-entropy or a KL over a two-point distribution.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Explainable LLM-driven Multi-dimensional Distillation for E-Commerce Relevance Learning." pith.science (2026). https://pith.science/paper/O6GA4ZFY

@misc{pith2026241113045,
  author       = {Pith},
  title        = {Pith review of: Explainable LLM-driven Multi-dimensional Distillation for E-Commerce Relevance Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/O6GA4ZFY}},
  note         = {Machine review of arXiv:2411.13045}
}
read the original abstract

Effective query-item relevance modeling is pivotal for enhancing user experience and safeguarding user satisfaction in e-commerce search systems. Recently, benefiting from the vast inherent knowledge, Large Language Model (LLM) approach demonstrates strong performance and long-tail generalization ability compared with previous neural-based specialized relevance learning methods. Though promising, current LLM-based methods encounter the following inadequacies in practice: First, the massive parameters and computational demands make it difficult to be deployed online. Second, distilling LLM models to online models is a feasible direction, but the LLM relevance modeling is a black box, and its rich intrinsic knowledge is difficult to extract and apply online. To improve the interpretability of LLM and boost the performance of online relevance models via LLM, we propose an Explainable LLM-driven Multi-dimensional Distillation framework for e-commerce relevance learning, which comprises two core components: (1) An Explainable LLM for relevance modeling (ELLM-rele), which decomposes the relevance learning into intermediate steps and models relevance learning as a Chain-of-Thought (CoT) reasoning, thereby enhancing both interpretability and performance of LLM. (2) A Multi-dimensional Knowledge Distillation (MKD) architecture that transfers the knowledge of ELLM-rele to current deployable interaction-based and representation-based student models from both the relevance score distribution and CoT reasoning aspects. Through distilling the probabilistic and CoT reasoning knowledge, MKD improves both the semantic interaction and long-tail generalization abilities of student models. Extensive offline evaluations and online experiments on Taobao search ad scene demonstrate that our proposed framework significantly enhances e-commerce relevance learning performance and user experience.

Figures

Figures reproduced from arXiv: 2411.13045 by the authors.

Figure 1
Figure 1. The key insight of ELLM-rele and MKD. Secondly, a Multi-dimensional Knowledge Distillation (MKD) architecture based on ELLM-rele is designed to further enhance smaller and more efficient online relevance model via maximumly exploitation of such knowledge. Specifically, MKD can distill the knowledge of ELLM-rele into both interaction and representation based models from two dimensions: relevance score distribution an… view at source ↗
Figure 2
Figure 2. The overall architecture of our proposed ELLM-MKD. (1) Generating CoT annotations via Larger LLM combined [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Typical translated relevance learning cases. [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

61 extracted references · 31 canonical work pages

  1. [1]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, et al

    work page

  2. [2]

    Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. 2023. Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection. arXiv preprint arXiv:2310.11511 (2023). https://arxiv.org/abs/2310.11511

    arXiv 2023

  3. [3]

    David Carmel, Elad Haramaty, Arnon Lazerson, Liane Lewin-Eytan, and Yoelle Maarek. 2020. Why do people buy seemingly irrelevant items in voice product search? On the relation between product relevance and customer satisfaction in ecommerce. In Proceedings of the 13th international conference on web search and data mining. 79–87

    2020

  4. [4]

    Liu, Ninghao Liu, Sheng Li, Dajiang Zhu, Hongmin Cai, Lichao Sun, Quanzheng Li, Dinggang Shen, Tianming Liu, and Xiang Li

    Haixing Dai, Zheng Liu, Wenxiong Liao, Xiaoke Huang, Yihan Cao, Zihao Wu, Lin Zhao, Shaochen Xu, W. Liu, Ninghao Liu, Sheng Li, Dajiang Zhu, Hongmin Cai, Lichao Sun, Quanzheng Li, Dinggang Shen, Tianming Liu, and Xiang Li

    work page

  5. [5]

    Hall, and Ming-Wei Chang

    Zhuyun Dai, Vincent Zhao, Ji Ma, Yi Luan, Jianmo Ni, Jing Lu, Anton Bakalov, Kelvin Guu, Keith B. Hall, and Ming-Wei Chang. 2022. Promptagator: Few- shot Dense Retrieval From 8 Examples. ArXiv abs/2209.11755 (2022). https: //api.semanticscholar.org/CorpusID:252519173

    arXiv 2022

  6. [6]

    https://api

    AugGPT: Leveraging ChatGPT for Text Data Augmentation. https://api. semanticscholar.org/CorpusID:257631936

    work page

  7. [7]

    Jacob Devlin. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)

    arXiv 2018

  8. [8]

    Jesse Davis and Mark Goadrich. 2006. The relationship between Precision-Recall and ROC curves. In Proceedings of the 23rd international conference on Machine learning. 233–240

    work page 2006

Show all 61 references
  1. [9]

    Jeffrey L. Elman. 1990. Finding Structure in Time. Cognitive Science 14, 2 (1990), 179–211

    1990

  2. [10]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

    2024 arXiv

  3. [11]

    Xingwei He, Zheng-Wen Lin, Yeyun Gong, Alex Jin, Hang Zhang, Chen Lin, Jian Jiao, Siu Ming Yiu, Nan Duan, and Weizhu Chen. 2023. AnnoLLM: Mak- ing Large Language Models to Be Better Crowdsourced Annotators. In North American Chapter of the Association for Computational Linguis...

    2023

  4. [12]

    Yuxian Gu, Li Dong, Furu Wei, and Minlie Huang. 2024. MiniLLM: Knowledge Distillation of Large Language Models. In The Twelfth International Conference on Learning Representations. https://openreview.net/forum?id=5h0qf7IBZZ

    2024

  5. [13]

    Baotian Hu, Zhengdong Lu, Hang Li, and Qingcai Chen. 2014. Convolutional neural network architectures for matching natural language sentences. Advances in neural information processing systems 27 (2014)

    2014

  6. [14]

    Zexue He, Marco Tulio Ribeiro, and Fereshte Khani. 2023. Targeted Data Gener- ation: Finding and Fixing Model Weaknesses. In Annual Meeting of the Associ- ation for Computational Linguistics . https://api.semanticscholar.org/CorpusID: 258960506

    2023

  7. [15]

    Po-Sen Huang, Xiaodong He, Jianfeng Gao, Li Deng, Alex Acero, and Larry Heck. 2013. Learning deep structured semantic models for web search using clickthrough data. Proceedings of the 22nd ACM international conference on Information & Knowledge Management (2013). https://api.s...

    2013

  8. [16]

    Baotian Hu, Zhengdong Lu, Hang Li, and Qingcai Chen. 2015. Convolutional Neu- ral Network Architectures for Matching Natural Language Sentences. Advances in neural information processing systems 3 (2015)

    2015

  9. [17]

    Samuel Humeau, Kurt Shuster, Marie-Anne Lachaux, and Jason Weston. 2019. Poly-encoders: Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence Scoring. In International Conference on Learning Representations . https://api.semanticscholar.org/CorpusID:210063976

    2019

  10. [18]

    Po-Sen Huang, Xiaodong He, Jianfeng Gao, and pages=2333–2338 year=2013 others, booktitle=Proceedings of the 22nd ACM international conference on In- formation & Knowledge Management. [n. d.]. Learning deep structured semantic models for web search using clickthrough data

    2013

  11. [19]

    Martin Josifoski, Marija Sakota, Maxime Peyrard, and Robert West. 2023. Exploit- ing Asymmetry for Synthetic Training Data Generation: SynthIE and the Case of Information Extraction. arXiv:2303.04132 [cs.CL] https://arxiv.org/abs/2303. 04132

    2023 arXiv

  12. [20]

    Samuel Humeau, Kurt Shuster, Marie-Anne Lachaux, and Jason Weston. 2019. Poly-encoders: Transformer architectures and pre-training strategies for fast and accurate multi-sentence scoring. arXiv preprint arXiv:1905.01969 (2019)

    2019 arXiv

  13. [21]

    Khattab and Matei A

    O. Khattab and Matei A. Zaharia. 2020. ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT. Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval (2020). https://api.semanticsch...

    2020

  14. [22]

    Omar Khattab and Matei Zaharia. 2020. Colbert: Efficient and effective passage search via contextualized late interaction over bert. In Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval. 39–48

    2020

  15. [23]

    Y Lecun and L Bottou. 1998. Gradient-based learning applied to document recognition. Proc. IEEE 86, 11 (1998), 2278–2324

    1998

  16. [24]

    John Lafferty, Andrew McCallum, Fernando Pereira, et al . 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Icml, Vol. 1. Williamstown, MA, 3

    2001

  17. [25]

    Zhe Lin, Jiwei Tan, Dan Ou, Xi Chen, Shaowei Yao, and Bo Zheng. 2024. Deep Bag-of-Words Model: An Efficient and Interpretable Relevance Architecture for Chinese E-Commerce. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (Barcelona, Spai...

    2024

  18. [26]

    Zhe Lin, Jiwei Tan, Dan Ou, Xi Chen, Shaowei Yao, and Bo Zheng. 2024. Deep Bag-of-Words Model: An Efficient and Interpretable Relevance Architecture for Chinese E-Commerce. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining . 5398–5408

    2024

  19. [27]

    Navid Mehrdad, Hrushikesh Mohapatra, Mossaab Bagdouri, Prijith Chandran, Alessandro Magnani, Xunfan Cai, Ajit Puthenputhussery, Sachin Yadav, Tony Lee, ChengXiang Zhai, et al. 2024. Large Language Models for Relevance Judgment in Product Search. arXiv preprint arXiv:2406.00247 (2024)

    2024 arXiv

  20. [28]

    Meng, and James R

    Hongyin Luo, Tianhua Zhang, Yung-Sung Chuang, Yuan Gong, Yoon Kim, Xixin Wu, Helen M. Meng, and James R. Glass. 2023. Search Augmented Instruction Learning. In The 2023 Conference on Empirical Methods in Natural Language Processing. https://openreview.net/forum?id=noIvPGG8P1

    2023

  21. [29]

    Hamid Palangi, Li Deng, Yelong Shen, et al . 2014. Semantic modelling with long-short-term memory for information retrieval. arXiv preprint arXiv:1412.6629 (2014)

    2014 arXiv

  22. [30]

    Long Ouyang, Jeffrey Wu, Xu Jiang, et al . 2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems 35 (2022), 27730–27744

    2022

  23. [31]

    Hamid Palangi, Li Deng, Yelong Shen, Jianfeng Gao, Xiaodong He, Jianshu Chen, Xinying Song, and Rabab Kreidieh Ward. 2015. Deep Sentence Embedding Using Long Short-Term Memory Networks: Analysis and Application to Information Retrieval. IEEE/ACM Transactions on Audio, Speech, ...

    2015

  24. [32]

    Palangi, L

    H. Palangi, L. Deng, Y. Shen, J. Gao, X. He, J. Chen, X. Song, and R. Ward. 2014. Semantic Modelling with Long-Short-Term Memory for Information Retrieval. Computer Science (2014)

    2014

  25. [33]

    Liang Pang, Yanyan Lan, Jiafeng Guo, Jun Xu, Shengxian Wan, and Xueqi Cheng

  26. [34]

    Liang Pang, Yanyan Lan, Jiafeng Guo, Jun Xu, and Xueqi Cheng. 2016. Text Matching as Image Recognition. (2016)

    2016

  27. [35]

    Ankur P Parikh, Oscar Täckström, Dipanjan Das, and Jakob Uszkoreit. 2016. A decomposable attention model for natural language inference. arXiv preprint arXiv:1606.01933 (2016)

    2016 arXiv

  28. [36]

    Baolin Peng, Chunyuan Li, Pengcheng He, Michel Galley, and Jianfeng Gao. 2023. Instruction Tuning with GPT-4. arXiv preprint arXiv:2304.03277 (2023)

    2023 arXiv

  29. [37]

    Ankur Parikh, Oscar Tckstrm, Dipanjan Das, and Jakob Uszkoreit. 2016. A Decomposable Attention Model for Natural Language Inference. (2016)

    2016

  30. [38]

    N Reimers. 2019. Sentence-BERT: Sentence Embeddings using Siamese BERT- Networks. arXiv preprint arXiv:1908.10084 (2019)

    2019 arXiv

  31. [39]

    Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. (2019)

    2019

  32. [40]

    Juan Ramos et al. 2003. Using tf-idf to determine word relevance in document queries. In Proceedings of the first instructional conference on machine learning , Vol. 242. Citeseer, 29–48

    2003

  33. [41]

    Kumar Shridhar, Alessandro Stolfo, and Mrinmaya Sachan. 2022. Distilling Reasoning Capabilities into Smaller Language Models. In Annual Meeting of the Association for Computational Linguistics . https://api.semanticscholar.org/ CorpusID:258762841

    2022

  34. [42]

    Krishna Srinivasan, Karthik Raman, Anupam Samanta, Ling-Yen Liao, Luca Bertelli, and Michael Bendersky. 2022. QUILL: Query Intent with Large Lan- guage Models using Retrieval Augmentation and Multi-stage Distillation. ArXiv abs/2210.15718 (2022). https://api.semanticscholar.or...

    2022 arXiv

  35. [43]

    Stephen Robertson, Hugo Zaragoza, et al . 2009. The probabilistic relevance framework: BM25 and beyond. Foundations and Trends® in Information Retrieval 3, 4 (2009), 333–389

    2009

  36. [44]

    Inar Timiryasov and Jean-Loup Tastet. 2023. Baby Llama: knowledge distillation from an ensemble of teachers trained on a small dataset with no performance penalty. ArXiv abs/2308.02019 (2023). https://api.semanticscholar.org/CorpusID: 260611172

    2023 arXiv

  37. [45]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971 (2023). WWW Compan...

    2023 arXiv

  38. [46]

    Weiwei Sun, Lingyong Yan, Xinyu Ma, Pengjie Ren, Dawei Yin, and Zhaochun Ren. 2023. Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agent. ArXiv abs/2304.09542 (2023). https://api.semanticscholar. org/CorpusID:258212638

    2023 arXiv

  39. [48]

    Shengxian Wan, Yanyan Lan, Jun Xu, Jiafeng Guo, Liang Pang, and Xueqi Cheng

  40. [49]

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention Is All You Need. arXiv (2017)

    2017

  41. [50]

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Huai hsin Chi, and Denny Zhou. 2022. Self-Consistency Improves Chain of Thought Reasoning in Language Models. ArXiv abs/2203.11171 (2022). https://api.semanticscholar.org/ CorpusID:247595263

    2022 arXiv

  42. [51]

    arXiv preprint arXiv:1604.04378 (2016)

    Match-srnn: Modeling the recursive matching structure with spatial rnn. arXiv preprint arXiv:1604.04378 (2016)

    2016 arXiv

  43. [52]

    Yidong Wang, Zhuohao Yu, et al . 2023. PandaLM: An Automatic Evaluation Benchmark for LLM Instruction Tuning Optimization. ArXiv abs/2306.05087 (2023). https://api.semanticscholar.org/CorpusID:259108266

    2023 arXiv

  44. [53]

    arXiv e-prints (2016)

    Match-SRNN: Modeling the Recursive Matching Structure with Spatial RNN. arXiv e-prints (2016)

    2016

  45. [54]

    Guan Wang, Sijie Cheng, Xianyuan Zhan, Xiangang Li, Sen Song, and Yang Liu. 2024. OpenChat: Advancing Open-source Language Models with Mixed- Quality Data. In The Twelfth International Conference on Learning Representations . https://openreview.net/forum?id=AOJyfhWYHf

    2024

  46. [55]

    Shaowei Yao, Jiwei Tan, Xi Chen, Juhao Zhang, Xiaoyi Zeng, and Keping Yang

  47. [56]

    Smith, Daniel Khashabi, and Hannaneh Hajishirzi

    Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. 2022. Self-Instruct: Aligning Language Model with Self Generated Instructions

    2022

  48. [58]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems 35 (2022), 24824–24837

    2022

  49. [59]

    An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Cheng- peng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, et al. 2024. Qwen2 technical report. arXiv preprint arXiv:2407.10671 (2024)

    2024 arXiv

  50. [62]

    Zhen Zhang, Yuhua Zhao, Hang Gao, and Mengting Hu. 2024. LinkNER: Link- ing Local Named Entity Recognition Models to Large Language Models using Uncertainty. arXiv:2402.10573 [cs.CL] https://arxiv.org/abs/2402.10573 A Details of CoT Construction To better illustrate the CoT de...

    2024 arXiv

  51. [2016]

    In Proceedings of the AAAI Conference on Artificial Intelligence, Vol

    Text matching as image recognition. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 30

  52. [2022]

    In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

    ReprBERT: distilling BERT to an efficient representation-based relevance model for e-commerce. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining . 4363–4371

  53. [2023]

    arXiv preprint arXiv:2303.08774 (2023)

    Gpt-4 technical report. arXiv preprint arXiv:2303.08774 (2023)

    2023 arXiv

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.