Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

Beyond instruction-conditioning, MoTE: Mixture of Task Experts for Multi-task Embedding Models

T0 review · 4 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Task-expert blocks beat instruction prompts alone for text embeddings

desk verdict Promising architecture for embedding specialization, but the headline result is undercut by a missing IC+TA-CL baseline and sloppy percentage arithmetic. read the letter →

arxiv 2506.17781 v1 pith:VD6HYH3Y submitted 2025-06-21 cs.LG cs.CL

Miguel Romero , Shuoyang Ding , Corey D. Barret , Georgiana Dinu , George Karypis This is my paper · ORCID
classification cs.LGcs.CL
keywords mixtureofexpertstextembeddingsinstruction-conditioningmulti-tasklearningcontrastiveretrievalMTEBtask-awaretraining
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that instruction-conditioning alone cannot fully specialize a multi-task embedding model, because the specialization signal must travel through shared parameters as a small input change. To get around that, it introduces MoTE, a transformer block whose feed-forward network is replaced by a small set of task-specific experts selected by the instruction, and Task-Aware Contrastive Learning (TA-CL), a curriculum that tunes batching and contrastive temperature per task. On the MTEB benchmark, the MoTE model improves retrieval NDCG@10 by +5.21 versus +3.27 for instruction-conditioning, and improves the average over seen tasks by +3.23 versus +2.35, with the same instructions, training data, latency, and number of active parameters. A sympathetic reader would take this as evidence that giving the model dedicated per-task parameters is a more direct route to embedding specialization than enriching the input text alone.

What carries the argument

The central object is the MoTE transformer block, a Mixture-of-Experts block with one MLP expert per task and instruction-based sequence-level routing instead of learned token-level routing. It carries the argument by decoupling task-specific gradients: each expert trains only on its task's examples, while the rest of the network stays shared. TA-CL is the accompanying training curriculum that supplies each expert with the batching strategy and contrastive temperature that suits its task.

What would settle it

Run the same contrastive training with the same task-aware batching and per-task temperatures on the instruction-conditioning model; if its retrieval NDCG@10 gain rises from +3.27 to near +5.21, the MoTE architecture is not the source of the improvement. Separately, recompute the seen-task gains after removing MTEB evaluation datasets whose training examples appear in the contrastive corpora; if the gains collapse, memorization rather than specialization explains them.

Watch

Extended reading notes

Core claim

MoTE replaces the dense transformer blocks of a pretrained encoder with mixture-of-task-experts blocks: each block contains one expert per task, all initialized from the original MLP weights, and a routing mechanism sends the full sequence to the expert named by its instruction. This gives the model task-specialized parameters without increasing the active parameter count or inference latency, because only one expert is active per forward pass and inactive experts can be offloaded. Trained with TA-CL, which builds task-homogeneous batches for retrieval and adjusts contrastive temperature per task, MoTE produces embeddings that are less entangled across tasks (lower inter-task cosine similarity) and achieve higher downstream performance than instruction-conditioning, both on tasks seen during training and on unseen tasks assigned to the closest expert.

Load-bearing premise

The comparison assumes the instruction-conditioning baseline and MoTE differ only in the added expert architecture, even though MoTE is also trained with the task-aware batching and temperature curriculum while the baseline is not, and an IC+TA-CL control is missing.

Editorial extensions

If this is right

  • On seen tasks, MoTE raises the retrieval NDCG@10 gain from +3.27 to +5.21 and the classification gain from +3.37 to +3.79, with the same active parameter count and latency as instruction-conditioning.
  • On unseen tasks, MoTE also beats instruction-conditioning on average (+1.25 vs +0.67), with the largest gains on STS and summarization, suggesting the task experts generalize when a new task is mapped to a related seen task.
  • Routing the whole sequence through one expert (sequence-level routing) outperforms token-level routing by +1.33 average across all 56 MTEB datasets, indicating that task identity is a stronger routing signal than per-token routing.
  • TA-CL alone adds +0.41 average dataset performance over static training, with a Welch's t-test p-value of 1e-4, and retrieval benefits most from homogeneous batching.
  • Because only one expert is active per task, MoTE keeps memory comparable to dense models by offloading inactive experts to CPU or disk, and expert averaging offers a compression path that retains +0.17 over the instruction-conditioned model.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The reported headline gains combine the MoTE architecture with TA-CL; an instruction-conditioning model trained with the same task-aware curriculum would isolate how much of the +5.21 retrieval gain is architectural.
  • If MTEB evaluation datasets overlap with the contrastive training corpora, part of the 'seen task' improvement could reflect memorization rather than specialization, since the paper does not state whether they overlap.
  • A natural extension is to make routing adaptive: keep the instruction-based assignment for known tasks but add a learned fallback for novel instructions, since the paper shows unseen tasks benefit even from nearest-task assignment.
  • For bulk inference, averaging task experts into a single dense network, as sketched in the paper's expert-averaging experiment, should make MoTE practical where GPU memory is tight while retaining most of the retrieval gain.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes MoTE (Mixture of Task Experts), a transformer block with task-specific expert MLPs and instruction-based sequence-level routing, for multi-task text embedding models. The authors argue that instruction-conditioning (IC) alone limits specialization in low-capacity models and that adding task-specialized parameters improves embedding specialization. They train the MoTE model with Task-Aware Contrastive Learning (TA-CL), which uses per-task batching and contrastive temperatures, and evaluate on MTEB. The central empirical claim is that MoTE outperforms IC by a large margin in performance gains over a non-specialized embedding model (+5.21 vs +3.27 NDCG@10 in retrieval; +3.23 vs +2.35 average gain across seen tasks), while keeping instructions, training data, inference time, and active parameter count unchanged. Additional ablations study TA-CL, sequence-level routing versus token-level routing, and integration of MoTE blocks at every versus every-other transformer layer.

Significance. If the central claim were fully substantiated, MoTE would be a useful architectural recipe for improving task specialization in embedding models without increasing active parameters or inference latency, and the TA-CL curriculum would be a practical training contribution. The paper includes a reasonable set of ablations (TA-CL, routing, block placement) and attempts to isolate the source of gains. The work is reproducible in principle: the base checkpoint, training data source, and hyperparameters are specified. However, the main MoTE-versus-IC comparison is currently confounded because the MoTE candidate trains with TA-CL while the IC baseline appears to use the static curriculum; without an IC+TA-CL condition, the headline gains cannot be attributed to the MoTE architecture. The statistical support for the headline result is also missing. The paper is therefore promising but not yet convincing.

major comments (4)
  1. [Section 4 and Appendix D.2; Tables 2 and 4] The headline comparison conflates architecture and training curriculum. Section 4 states that both IC and MoTE use the same instructions, training data, and training setup, but Appendix D.2 specifies that the MoTE candidate is trained with TA-CL (task-specific batching and contrastive temperatures), while no IC+TA-CL baseline is reported. The TA-CL ablation in Table 4 shows that within the MoTE architecture, switching from static training to TA-CL improves retrieval NDCG@10 by +1.04 and average dataset performance by +0.41. Since these gains are of the same order as parts of the reported MoTE-versus-IC differences, the reader cannot tell whether the IC baseline would narrow or close the gap if it also received TA-CL. Please add an IC+TA-CL condition (and preferably an IC+static condition) so that the architectural contribution of MoTE is separated from the curriculum contribution.
  2. [Section 5.1, Tables 2 and 3] No statistical significance or variance is reported for the central MoTE-versus-IC comparison. The paper reports single runs for the headline numbers and uses Welch's t-tests only for the TA-CL ablation (Section 5.4) and the SLR-versus-TLR routing ablation (Section 6). Since the main contribution is the claim that MoTE outperforms IC, please provide either multiple training runs with standard deviations, or a paired significance test across the MTEB datasets for Tables 2 and 3.
  3. [Section 4 and Tables 1-3] The paper does not state whether the contrastive training datasets overlap with the MTEB evaluation datasets. The training data are described as classification, clustering, and retrieval datasets from the cited contrastors configuration, while evaluation is on MTEB, which includes 15 retrieval, 12 classification, and 11 clustering datasets. Without a statement of which MTEB datasets (or their training splits) were used during contrastive training, the 'seen task' gains in Table 2 could reflect memorization rather than specialization. Please specify the overlap or evaluate on disjoint held-out datasets.
  4. [Abstract, Introduction, and Table 2] The percentage gains reported in the abstract and introduction are internally inconsistent with Table 2. The abstract reports '+3.27 to +5.21' as a 64% gain and '+1.81 to +2.60' as a 43% gain, but Table 2 lists IC average gain +2.35 and MoTE average gain +3.23, which is about a 37% relative improvement. The introduction separately claims '50% higher performance gains in critical tasks such as retrieval and 31% higher gains in other tasks'. Please reconcile these numbers and ensure that all reported percentages are computed from the tables or from clearly defined additional tables.
minor comments (6)
  1. [Abstract and Introduction] There is a typo: 'Task-Aware Contrastive Learnign' should be 'Learning'.
  2. [Section 2.2] The sentence 'Small models use a pre-defined set of instructions with task level information such such as' contains a duplicated 'such'.
  3. [Appendix B and Table 7] The abbreviation 'IEM' is used in Table 7 but is not defined earlier; the main text uses 'IC' for the instruction-conditioned model. Please define 'IEM' or replace it with 'IC' for consistency.
  4. [Section 6 and Table 5] The text defines routing as 'Token-Level Routing (TLR)' but the caption of Table 5 calls it 'Task-Level Routing (TLR)'. Please use one consistent name.
  5. [Table 2 and Table 4] The table footnotes say 'validity measure' for clustering; the standard term is 'V-measure'.
  6. [Section 5.2] The phrase 'we randomly 128 randomly generic Wikipedia articles' is ungrammatical; it should read 'we randomly sample 128 generic Wikipedia articles'.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: MoTE's gains are empirical benchmark results, not derived from fitted constants or self-citation.

full rationale

The central comparison is empirical: MoTE and TA-CL are trained and then evaluated on MTEB, with reported gains (Tables 2-6) as measured differences rather than consequences of the definitions. The routing mechanism R:I→E is task-instruction based, but the claimed +5.21 vs +3.27 retrieval gain is not entailed by that mapping; it is observed on an external benchmark under controlled initialization, instructions, and training data. The per-task batching strategies and temperatures (Appendix A) are training choices, and Table 4 presents their marginal effect as an ablation rather than renaming a fit as a prediction. Citations to prior work, including two references co-authored by the present authors, support background claims (classification, search, MoE, contrastive training) and are not load-bearing in the derivation of the main result. The missing IC+TA-CL baseline is an attribution confound that affects interpreting the contribution split between architecture and curriculum, but it is an experimental-design limitation, not circularity. No equation or argument reduces the paper's predictions to its inputs, so no circular step is identified.

Assumptions & free parameters 4 free parameters · 5 assumptions · 1 invented entities

The central claim rests on several unstated or lightly justified assumptions: task instructions fully specify routing, MTEB is a valid benchmark, training data is adequately disjoint from evaluation, and the hyperparameters (temperature, batching, number of experts) are chosen well. These are typical for empirical ML papers, but none are verified with independent evidence beyond the reported results.

free parameters (4)
  • Contrastive temperature for clustering = 0.06
    Selected based on an ablation in Appendix A.2; clustering uses 0.06 while retrieval and classification use 0.03.
  • Per-task batching strategy = homogeneous for retrieval, heterogeneous for classification and clustering
    Selected based on an ablation in Appendix A.1 and used inside TA-CL.
  • Number of task experts per MoTE block = 4
    Matches the four instruction types; the paper does not ablate other counts.
  • Placement of MoTE blocks = every transformer block (EB)
    Ablation in Table 6 shows marginal differences; EB is chosen as the final configuration.
assumptions (5)
  • domain assumption Small input perturbations such as short instruction tokens produce only bounded changes in output embeddings (Lipschitz continuity), limiting instruction-conditioning capacity.
    Invoked in Section 1 to motivate the method; cited to Tang et al. 2024 and not proven in this paper.
  • domain assumption The MTEB benchmark is a valid and sufficient measure of embedding quality for the claims made.
    Used as the sole evaluation tool; no discussion of MTEB's limitations or alternative benchmarks.
  • domain assumption The four instruction strings ('classification: ', 'clustering: ', 'search query: ', 'search document: ') correctly cover the training tasks and provide a sufficient routing signal.
    Routing and specialization depend on this assignment; Section 5.3 further assumes unseen tasks can be mapped to these categories.
  • domain assumption Initializing all experts with a copy of the dense MLP weights preserves the useful pretrained knowledge.
    Section 3.1.1 relies on upcycling; the paper provides no specific verification beyond final performance.
  • domain assumption The training datasets used for contrastive learning are sufficiently disjoint from the MTEB evaluation datasets that reported gains reflect generalization rather than memorization.
    Section 4 does not state whether evaluation datasets overlap with the training data from nomic-ai/contrastors.
invented entities (1)
  • Task-specialized expert modules within MoTE transformer blocks independent evidence
    purpose: Provide task-specific parameters that produce more disentangled embeddings while keeping active parameters and latency unchanged.
    The architecture's effect is measured on MTEB and is externally checkable, but no code or model weights are released.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Beyond instruction-conditioning, MoTE: Mixture of Task Experts for Multi-task Embedding Models." pith.science (2026). https://pith.science/paper/VD6HYH3Y

@misc{pith2026250617781,
  author       = {Pith},
  title        = {Pith review of: Beyond instruction-conditioning, MoTE: Mixture of Task Experts for Multi-task Embedding Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VD6HYH3Y}},
  note         = {Machine review of arXiv:2506.17781}
}
abstract

Dense embeddings are fundamental to modern machine learning systems, powering Retrieval-Augmented Generation (RAG), information retrieval, and representation learning. While instruction-conditioning has become the dominant approach for embedding specialization, its direct application to low-capacity models imposes fundamental representational constraints that limit the performance gains derived from specialization. In this paper, we analyze these limitations and introduce the Mixture of Task Experts (MoTE) transformer block, which leverages task-specialized parameters trained with Task-Aware Contrastive Learning (\tacl) to enhance the model ability to generate specialized embeddings. Empirical results show that MoTE achieves $64\%$ higher performance gains in retrieval datasets ($+3.27 \rightarrow +5.21$) and $43\%$ higher performance gains across all datasets ($+1.81 \rightarrow +2.60$). Critically, these gains are achieved without altering instructions, training data, inference time, or number of active parameters.

Figures

Figures reproduced from arXiv: 2506.17781 by the authors.

Figure 1
Figure 1. Average inter-task cosine similarity between embedding representations for the same sequence. Results [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. a) Overview of the MoTE block, which replaces the standard transformer block in MoTE. Each MoTE [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Inter-task similarity of text representation when using IC and MoTE. We observe that in most cases MoTE [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (2 more)
Figure 5
Figure 5. Figure 5: Task-Specific Performance: Batching Strategy [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]
Figure 7
Figure 7. Figure 7: Optimal contrastive temperature per down [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Position: Text Embeddings Should Capture Implicit Semantics, Not Just Surface Meaning

    cs.CL 2025-06 conditional novelty 5.0 of 10

    State-of-the-art text embeddings lag far behind on tasks requiring pragmatic inference, stance detection, and social meaning, relative to their strong performance on surface semantic benchmarks.

Reference graph

Works this paper leans on

37 extracted references · 6 canonical work pages · cited by 1 Pith paper

  1. [1]

    Eneko Agirre, Daniel Cer, Mona Diab, and Aitor Gonzalez-Agirre. 2012. https://aclanthology.org/S12-1051 S em E val-2012 task 6: A pilot on semantic textual similarity . In * SEM 2012: The First Joint Conference on Lexical and Computational Semantics -- Volume 1: Proceedings of the main conference and the shared task, and Volume 2: Proceedings of the Sixth...

    2012

  2. [2]

    Jimmy Lei Ba. 2016. Layer normalization. arXiv preprint arXiv:1607.06450

    arXiv 2016

  3. [3]

    Chris Burges, Tal Shaked, Erin Renshaw, Ari Lazier, Matt Deeds, Nicole Hamilton, and Greg Hullender. 2005. https://doi.org/10.1145/1102351.1102363 Learning to rank using gradient descent . In Proceedings of the 22nd International Conference on Machine Learning, ICML '05, page 89–96, New York, NY, USA. Association for Computing Machinery

    arXiv 2005

  4. [4]

    Dhivya Chandrasekaran and Vijay Mago. 2021. Evolution of semantic similarity—a survey. ACM Computing Surveys (CSUR), 54(2):1--37

    work page 2021

  5. [5]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805

    arXiv 2018

  6. [6]

    William Fedus, Barret Zoph, and Noam Shazeer. 2022. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. Journal of Machine Learning Research, 23(120):1--39

    2022

  7. [7]

    Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2021. Simcse: Simple contrastive learning of sentence embeddings. arXiv preprint arXiv:2104.08821

    arXiv 2021

  8. [8]

    Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, and Haofen Wang. 2023. Retrieval-augmented generation for large language models: A survey. arXiv preprint arXiv:2312.10997

    arXiv 2023

Show all 37 references
  1. [9]

    Ze-Feng Gao, Peiyu Liu, Wayne Xin Zhao, Zhong-Yi Lu, and Ji-Rong Wen. 2022. Parameter-efficient mixture-of-experts architecture for pre-trained language models. arXiv preprint arXiv:2203.01104

    2022 arXiv

  2. [10]

    Jiawei Han, Micheline Kamber, and Jian Pei. 2012. https://doi.org/10.1016/B978-0-12-381479-1.00002-2 2 - getting to know your data . In Jiawei Han, Micheline Kamber, and Jian Pei, editors, Data Mining (Third Edition), third edition edition, The Morgan Kaufmann Series in Data M...

    2012 doi

  3. [11]

    Ethan He, Abhinav Khattar, Ryan Prenger, Vijay Korthikanti, Zijie Yan, Tong Liu, Shiqing Fan, Ashwath Aithal, Mohammad Shoeybi, and Bryan Catanzaro. 2024. Upcycling large language models into mixture of experts. arXiv preprint arXiv:2410.07524

    2024 arXiv

  4. [12]

    Albert Q Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, et al. 2024. Mixtral of experts. arXiv preprint arXiv:2401.04088

    2024 arXiv

  5. [13]

    Aran Komatsuzaki, Joan Puigcerver, James Lee-Thorp, Carlos Riquelme Ruiz, Basil Mustafa, Joshua Ainslie, Yi Tay, Mostafa Dehghani, and Neil Houlsby. 2022. Sparse upcycling: Training mixture-of-experts from dense checkpoints. arXiv preprint arXiv:2212.05055

    2022 arXiv

  6. [14]

    Dmitry Lepikhin, HyoukJoong Lee, Yuanzhong Xu, Dehao Chen, Orhan Firat, Yanping Huang, Maxim Krikun, Noam Shazeer, and Zhifeng Chen. 2020. Gshard: Scaling giant models with conditional computation and automatic sharding. arXiv preprint arXiv:2006.16668

    2020 arXiv

  7. [15]

    u ttler, Mike Lewis, Wen-tau Yih, Tim Rockt \

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich K \"u ttler, Mike Lewis, Wen-tau Yih, Tim Rockt \"a schel, et al. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Proc...

    2020

  8. [16]

    Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692

    2019 arXiv

  9. [17]

    Niklas Muennighoff, Hongjin Su, Liang Wang, Nan Yang, Furu Wei, Tao Yu, Amanpreet Singh, and Douwe Kiela. 2024. Generative representational instruction tuning. arXiv preprint arXiv:2402.09906

    2024 arXiv

  10. [18]

    Niklas Muennighoff, Nouamane Tazi, Lo \" c Magne, and Nils Reimers. 2022. Mteb: Massive text embedding benchmark. arXiv preprint arXiv:2210.07316

    2022 arXiv

  11. [19]

    Arvind Neelakantan, Tao Xu, Raul Puri, Alec Radford, Jesse Michael Han, Jerry Tworek, Qiming Yuan, Nikolas Tezak, Jong Wook Kim, Chris Hallacy, et al. 2022. Text and code embeddings by contrastive pre-training. arXiv preprint arXiv:2201.10005

    2022 arXiv

  12. [20]

    Zach Nussbaum, John X Morris, Brandon Duderstadt, and Andriy Mulyar. 2024. Nomic embed: Training a reproducible long context text embedder. arXiv preprint arXiv:2402.01613

    2024 arXiv

  13. [21]

    James O ' Neill, Polina Rozenshtein, Ryuichi Kiryo, Motoko Kubota, and Danushka Bollegala. 2021. https://doi.org/10.18653/v1/2021.emnlp-main.568 I wish I would have loved this one, but I didn ' t -- a multilingual dataset for counterfactual detection in product review . In Pro...

    2021 doi

  14. [22]

    Aaron van den Oord, Yazhe Li, and Oriol Vinyals. 2018. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748

    2018 arXiv

  15. [23]

    Sachin Ravi, Sebastian Musslick, Maia Hamin, Theodore L Willke, and Jonathan D Cohen. 2020. Navigating the trade-off between multi-task learning and learning to multitask in deep neural networks. arXiv preprint arXiv:2007.10527

    2020 arXiv

  16. [24]

    Andrew Rosenberg and Julia Hirschberg. 2007. https://aclanthology.org/D07-1043 V -measure: A conditional entropy-based external cluster evaluation measure . In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natura...

    2007

  17. [25]

    Hongjin Su, Weijia Shi, Jungo Kasai, Yizhong Wang, Yushi Hu, Mari Ostendorf, Wen-tau Yih, Noah A Smith, Luke Zettlemoyer, and Tao Yu. 2022. One embedder, any task: Instruction-finetuned text embeddings. arXiv preprint arXiv:2212.09741

    2022 arXiv

  18. [26]

    Eric Tang, Bangding Yang, and Xingyou Song. 2024. Understanding llm embeddings for regression. arXiv preprint arXiv:2411.14708

    2024 arXiv

  19. [27]

    Naftali Tishby, Fernando C Pereira, and William Bialek. 2000. The information bottleneck method. arXiv preprint physics/0004057

    2000 arXiv

  20. [28]

    Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furu Wei. 2022. Text embeddings by weakly-supervised contrastive pre-training. arXiv preprint arXiv:2212.03533

    2022 arXiv

  21. [29]

    Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei. 2023. Improving text embeddings with large language models. arXiv preprint arXiv:2401.00368

    2023 arXiv

  22. [30]

    Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei. 2024. Multilingual e5 text embeddings: A technical report. arXiv preprint arXiv:2402.05672

    2024 arXiv

  23. [31]

    Yining Wang, Liwei Wang, Yuanzhi Li, Di He, and Tie-Yan Liu. 2013. A theoretical analysis of ndcg type ranking measures. In Conference on learning theory, pages 25--54. PMLR

    2013

  24. [32]

    Zichen Wang, Steven A Combs, Ryan Brand, Miguel Romero Calvo, Panpan Xu, George Price, Nataliya Golovach, Emmanuel O Salawu, Colby J Wise, Sri Priya Ponnapalli, et al. 2021. Lm-gvp: A generalizable deep learning framework for protein property prediction from sequence and struc...

    2021

  25. [33]

    Colby Wise, Vassilis N Ioannidis, Miguel Romero Calvo, Xiang Song, George Price, Ninad Kulkarni, Ryan Brand, Parminder Bhatia, and George Karypis. 2020. Covid-19 knowledge graph: accelerating information retrieval and discovery for scientific literature. arXiv preprint arXiv:2...

    2020 arXiv

  26. [34]

    Jiaming Xu, Peng Wang, Guanhua Tian, Bo Xu, Jun Zhao, Fangyuan Wang, and Hongwei Hao. 2015. Short text clustering via convolutional neural networks. In Proceedings of the 1st Workshop on Vector Space Modeling for Natural Language Processing, pages 62--69

    2015

  27. [35]

    Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman, and Chelsea Finn. 2020. Gradient surgery for multi-task learning. Advances in Neural Information Processing Systems, 33:5824--5836

    2020

  28. [36]

    Seniha Esen Yuksel, Joseph N Wilson, and Paul D Gader. 2012. Twenty years of mixture of experts. IEEE transactions on neural networks and learning systems, 23(8):1177--1193

    2012

  29. [37]

    Shengyu Zhang, Linfeng Dong, Xiaoya Li, Sen Zhang, Xiaofei Sun, Shuhe Wang, Jiwei Li, Runyi Hu, Tianwei Zhang, Fei Wu, et al. 2023. Instruction tuning for large language models: A survey. arXiv preprint arXiv:2308.10792

    2023

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.