University of Indonesia at SemEval-2025 Task 11: Evaluating State-of-the-Art Encoders for Multi-Label Emotion Detection

Alfan Farizki Wicaksono; Eduardus Tjitrahardja; Eryawan Presma Yulianrifat; Ikhlasul Akmal Hanif; Jaycent Gunawan Ongris; Muhammad Falensi Azmi; Rahmat Bryan Naufal

REVIEW 3 major objections 5 minor 17 references

University of Indonesia at SemEval-2025 Task 11: Evaluating State-of-the-Art Encoders for Multi-Label Emotion Detection

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Frozen prompt-based embedding encoders with a tree classifier beat fully fine-tuned multilingual transformers on a 28-language emotion detection task.

desk verdict A solid shared-task recipe paper whose practical result (BGE + per-emotion prompts + CatBoost) is credible, but whose 'classifier-only beats fine-tuning' claim is confounded with encoder choice and needs re-framing. read the letter →

arxiv 2505.16460 v1 pith:4NMV2G5K submitted 2025-05-22 cs.CL cs.AI

Ikhlasul Akmal Hanif , Eryawan Presma Yulianrifat , Jaycent Gunawan Ongris , Eduardus Tjitrahardja , Muhammad Falensi Azmi , Rahmat Bryan Naufal , Alfan Farizki Wicaksono This is my paper

classification cs.CLcs.AI

keywords multi-labelemotionclassificationmultilingualdetectionprompt-basedembeddingsBGECatBoostfrozenencodersSemEval-2025Task11F1-macro

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper is a system description for SemEval 2025 Task 11 Track A, multi-label emotion classification in 28 languages. It tries to establish that a surprisingly simple pipeline—frozen, prompt-based embedding encoders with a shallow tree classifier—outperforms the standard practice of fully fine-tuning multilingual transformer models. On the development set, the best BGE-with-CatBoost configuration reaches 55.40 F1-macro, compared with 47.10 for the best fully fine-tuned model, and the gap is statistically significant. On the official test set, the team's ensemble of four BGE models with weighted voting reaches 56.58 average F1-macro, which the paper reports as significantly surpassing the strongest decoder-based baseline from earlier work. If this holds, it means high-quality pretrained embeddings plus a cheap classifier can be a better use of resources than fine-tuning for multilingual emotion detection.

What carries the argument

The load-bearing mechanism is the combination of a frozen, prompt-conditioned embedding encoder and a shallow tree classifier. BGE (bge-multilingual-gemma2) and mE5 are prompt-based multilingual encoders: each input is prefixed with an instruction, and in the best configuration the model is prompted separately for each emotion, using a query like 'Represent this text for identifying the presence of the emotion {{EMOTION}}' to produce one embedding per emotion. CatBoost, a gradient-boosted tree model, is trained on these frozen embeddings with class weights to handle label imbalance. For the final submission, four BGE-plus-CatBoost models are combined by weighted voting, where each model's vote is signed and weighted by development-set F1-macro. The paper's argument is that these encoders already encode multilingual semantics well enough that fine-tuning the transformer is unnecessary, and that targeted prompts sharpen the representation for each emotion.

What would settle it

Re-run the mBERT and XLM-R fine-tuning baselines on the same BRIGHTER development splits with a small hyperparameter search, for example learning rates from 1e-5 to 5e-5, 3 to 6 epochs, and multiple seeds, and check whether any configuration reaches or exceeds the 55.40 development F1-macro of BGEV2-CB-ALL; if one does, the paper's central superiority claim over fine-tuning would not survive.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that classifier-only training on frozen embedding encoders, especially BGE with one prompt per emotion, beats end-to-end fine-tuning for multilingual multi-label emotion classification. The authors report a Mann-Whitney U test with U = 456 and p < 0.001 separating prompt-based encoder models from the rest, while the best single BGE configuration (BGEV2-CB-ALL) scores 55.40 on development and the best fine-tuned model (mBERT-MO-ALL-AL) scores 47.10. The submitted ensemble, combining BGE models trained on all languages or per language with CatBoost and weighted voting, achieves 56.58 average F1-macro on the official test set and outperforms the Qwen2.5-72B decoder baseline from the BRIGHTER paper with W = 285.0 and p < 0.001. The paper also finds that emotion-specific prompts improve over a single general prompt and that training on all languages does not significantly beat language-specific training.

Load-bearing premise

The conclusion that frozen prompt-based encoders are better than fine-tuning rests on the assumption that the fine-tuning baselines were given a fair comparison, since they were run with one fixed hyperparameter setting and no search.

Editorial extensions

If this is right

Fully fine-tuning multilingual transformers is not required to do well on multi-label emotion detection across many languages; frozen prompt-based embeddings plus CatBoost produce the best scores in this paper.
Emotion-specific prompting is a cheap and effective intervention: BGEV2 with one prompt per emotion outperforms BGEV1 with a single general prompt on development.
Multilingual joint training yields no significant benefit over language-specific training, so per-language models can be preferred when compute is limited.
Ensembling several embedding-classifier models with weighted voting improves test performance over the best single model in 25 of 28 languages.
Because the winning pipeline uses only frozen encoders and a tree classifier, strong results are achievable without large-scale fine-tuning compute.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A reader might infer from this result that, for low-resource multilingual emotion tasks, the marginal compute is better spent on stronger embedding models and prompt design than on fine-tuning hyperparameters.
The comparison would be more persuasive if the fine-tuning baselines received the same hyperparameter budget; a modest search over learning rate, epochs, and seeds could close part of the six-point development gap.
The same frozen-embedding-plus-tree-classifier recipe may transfer to other multilabel text tasks, such as hate-speech or stance detection, where label imbalance and low-resource languages are common.
The paper's own stated limitation is the lack of qualitative analysis, so an error-analysis pass on the weakest languages, such as VMW and YOR, could show whether the measured gap reflects true model quality or artifacts of the evaluation.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Read this for the recipe, not for the broad claim. The paper is a solid SemEval system report: it tests many encoder-classifier combinations on BRIGHTER, reports per-language dev scores, and finds that frozen bge-multilingual-gemma2 embeddings with per-emotion prompts plus a CatBoost head give the best F1-macro (55.40 dev, 56.58 test ensemble), beating their fine-tuned mBERT/XLMR baselines and the Qwen2.5-72B reference from the BRIGHTER paper. The per-emotion prompt ablation (BGE V2) is a nice, simple result, and the ensemble weighting is sensible.

The honest parts: prompts and hyperparameters are in the appendix, the tables are extensive, and the external Qwen comparison on the same test set is a reasonable datapoint. This is above-average transparency for a shared-task paper.

The soft spots are real and concentrated in the interpretation. The headline 'classifier-only significantly outperforms fully fine-tuned transformers' is not controlled: the frozen models are BGE/mE5/Jina, while the fine-tuned models are mBERT/XLMR. Different scale, different pretraining. The paper's own held-encoder comparison goes the other way: frozen XLMR-CB-ALL is 38.48, fine-tuned XLMR-BR-LANG is 45.61. So the claim should be 'these particular embeddings, used as feature extractors, work very well here,' not 'classifier-only beats fine-tuning.' Second, the fine-tuning baselines are untuned (one lr, 4 epochs, seed 42), and several runs collapse to near zero. A tuned fine-tuned model could close part of the 8-point dev gap, though the margin is large enough that the practical recipe likely survives. Third, there are small accuracy issues: the abstract's 56.58 does not obviously match Table 1's average, the prompt-comparison line mentions 'two samples,' and the Mann-Whitney test treats all configurations as independent, which is loose. No code, no error bars.

My take: the data and the math are fine, the citations are appropriate, and the practical result is credible. What needs fixing is the framing and a missing controlled ablation. For a shared-task report, that is a minor-to-moderate revision, not a rejection. Send it to review, ask for the broader claim to be narrowed or backed by a fine-tuned BGE / frozen mBERT condition, and for corrected numbers and code release.

Referee Report

3 major / 5 minor

Summary. This paper reports the University of Indonesia's system for SemEval 2025 Task 11 Track A, multilingual multi-label emotion detection across 28 languages. The authors compare two regimes: classifier-only training on frozen pretrained encoders (mE5, BGE, Jina, XLM-R) with classical classifiers, and end-to-end fine-tuning of mBERT and XLM-R with Focal or Asymmetric loss. On the development set, BGE with emotion-specific prompts and CatBoost achieves the best average F1-macro (55.40), and the submitted ensemble achieves 56.58 average F1-macro on the official test set, which the paper reports as significantly surpassing the Qwen2.5-72B decoder baseline of the BRIGHTER paper. The central conclusion is that classifier-only training with prompt-based encoders significantly outperforms full fine-tuning.

Significance. If the central claim is accepted, the paper would provide a practically useful result for low-resource multilingual emotion detection: frozen modern embedding encoders plus tree-based classifiers can beat fine-tuned multilingual transformers at lower training cost, and emotion-specific prompts add measurable gains. The paper's strengths are its systematic coverage of many encoder/classifier/prompt/loss combinations, the inclusion of per-language development tables, and the use of nonparametric statistical tests. However, the main methodological comparison is confounded and the fine-tuning baselines are not tuned, so the paper's headline claim is stronger than the evidence supports. The result is still a credible system description, but the general conclusion needs either additional controlled experiments or a substantially more careful formulation.

major comments (3)

[§5.1, §7, Abstract] The central claim that classifier-only training on prompt-based encoders 'significantly' outperforms full fine-tuning is not supported by a controlled comparison. The classifier-only group uses BGE, mE5, Jina, and XLM-R, while the fine-tuning group uses only mBERT and XLM-R, so the comparison varies both method and encoder. The only encoder that appears in both regimes is XLM-R, and there fine-tuning helps: XLMR-CB-ALL scores 38.48 (Table 12), while fine-tuned XLMR-BR-LANG scores 45.61 (Table 6). The Mann-Whitney U test in §5.1 therefore compares model families rather than training regimes. Please either add a frozen mBERT condition and/or a fine-tuned BGE/mE5 condition, or restrict the conclusion to the models actually compared, e.g., 'in our configuration, BGE embeddings with CatBoost outperformed untuned mBERT and XLM-R fine-tuning.'
[§4, Table 3, Table 6] The fine-tuning baselines are not tuned and may not be representative of full fine-tuning. Section 4 states that no hyperparameter tuning was performed, fixing learning rate 3e-5, four epochs, seed 42, and batch size 32. Several fine-tuning runs collapse on many languages (e.g., XLMR-MO-LANG-AL 27.85, XLMR-MO-LANG-FL 25.61, XLMR-SEP-LANG-FL 21.25, with several languages at 0.00 in Table 12). The best fine-tuned model (mBERT-MO-ALL-AL 47.10) trails BGE (55.40), but the gap may be partly due to undertrained or unlucky runs rather than to a fundamental advantage of classifier-only training. Please add seed averaging or a modest hyperparameter search, or explicitly label these results as untuned baselines and soften the corresponding conclusion.
[§5.2, Table 1] The test-set claim that the ensemble 'significantly surpasses' the best decoder-based model from prior work should report the exact comparison conditions. The Qwen2.5-72B scores are taken from the BRIGHTER paper, but Table 1 has missing entries for seven languages; the Wilcoxon test W=285, p<0.001 should state the number of paired languages used and whether the missing languages were excluded. It would also help to state whether the external baseline was produced with the same evaluation script and test split as the submitted predictions. Without these details, the comparison is not fully auditable.

minor comments (5)

[Table 5] The caption says 'Performance scores of the classifier-only training model on the test set', but Section 5.1 and the surrounding text describe development-set results; please change the caption to 'development set'.
[§5.2] Model V2 is described as 'an ensemble of four models: BGEV2-CB-ALL, BGE-CB-LANG, BGE-CB-LANG, BGE-CB-ALL'; the two middle entries appear to be duplicates and the model names do not match the naming scheme used elsewhere (e.g., BGEV1-CB-LANG vs. BGEV2-CB-LANG). Please correct this list.
[§5.1] The prompt-ablation sentence says 'this experiment was conducted only on CB models with two samples' and reports increases 'from 5.3% to 5.5% and from 54.0% to 55.0%'. The intended comparison is presumably BGEV1-CB-ALL (53.52) to BGEV2-CB-ALL (55.40) and BGEV1-CB-LANG (54.19) to BGEV2-CB-LANG (55.39); the text should say so and should not refer to 'two samples'.
[§1] The phrase 'where we freeze the classifier' in the introduction is misleading; the frozen component is the encoder, and the classifier is trained. Please rephrase.
[§5.1] The sentence 'Different prompt lead to different results' has a subject-verb agreement error and should be rephrased.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the paper is an empirical shared-task comparison with independently trained baselines and standard dev-set model selection.

full rationale

This is an empirical system description, not a derivation. The central claim that frozen prompt-based encoders (BGE, mE5) plus a tree classifier outperform fully fine-tuned mBERT/XLM-R rests on direct experiments on a shared dev set, with each baseline trained and evaluated independently. No equation defines one result in terms of another, and no fitted parameter is renamed as a prediction. The ensemble weights are selected using development performance and then applied to the official test set, which is standard model selection rather than circular fitting. The comparison with Qwen2.5-72B uses an external baseline from the BRIGHTER paper, and the MMTEB reference is external evidence, not a self-citation. The main weakness is experimental validity: the headline comparison varies both the regime and the encoder architecture, and the fine-tuning hyperparameters are fixed without tuning, so the six-point gap may partly reflect model capacity and recency rather than the classifier-only versus fine-tuning regime. That is a legitimate correctness concern, but it is not a circularity in the derivation chain because the paper does not define success in terms of its inputs. No load-bearing step reduces to a self-citation, an ansatz smuggled in by citation, or an equation that is true by construction.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The paper's central claims rest on the benchmark being valid, on frozen embeddings retaining emotion semantics, and on the fine-tuning baselines being representative. No invented entities are introduced. The class-weight formula and loss hyperparameters are taken from prior work, and the ensemble weight mapping is unspecified.

free parameters (3)

Focal loss alpha and gamma = alpha from class weights (Eq. 1), gamma=2
Hand-chosen in Section 3.2 following Lin et al. 2017; used only in fine-tuning baselines, but affects the fairness of the comparison with frozen-embedding models.
Asymmetric loss gamma+, gamma-, margin m = gamma+=0, gamma-=4, m=0.05
Taken from Ridnik et al. 2021 in Section 3.2; these hyperparameters influence the fine-tuning baseline scores that the central comparison relies on.
Ensemble weight mapping = unspecified
Section 5.2 says weights are 'based on development set performance and handling zero weights', but the exact mapping from dev score to weight is not given, so the ensemble procedure is under-specified and not exactly reproducible.

assumptions (5)

domain assumption BRIGHTER dataset annotations and official F1-macro metric are a valid benchmark.
The paper adopts the shared task's data, splits, and evaluation without questioning annotation quality; cited as Muhammad et al. 2025a,b, Sections 2 and 4.
domain assumption Frozen encoder embeddings (BGE, mE5) retain emotion-relevant semantics across all 28 languages.
Section 3.1 freezes pretrained encoders; Section 5.1 interprets their superior dev scores as reflecting MMTEB-validated embedding quality.
standard math Non-parametric significance tests are appropriate for comparing per-language F1 scores.
Section 5.1 applies Wilcoxon signed-rank and Mann-Whitney U tests to per-language F1 values, implicitly assuming paired observations and exchangeability.
domain assumption The fixed hyperparameter schedule (Table 3) fairly represents the fine-tuning approach.
Section 4 states no hyperparameter tuning was performed, and the paper's central comparison depends on these untuned fine-tuned runs being representative.
domain assumption The Qwen2.5-72B results reported in the BRIGHTER paper are directly comparable to the authors' test-set runs.
Section 5.2 compares against Qwen2.5-72B numbers from Muhammad et al. 2025a, but the paper does not state whether the official test set or identical preprocessing was used.

how reviews work

0 comments

Cite this review

Pith. "Pith review of University of Indonesia at SemEval-2025 Task 11: Evaluating State-of-the-Art Encoders for Multi-Label Emotion Detection." pith.science (2026). https://pith.science/paper/4NMV2G5K

@misc{pith2026250516460,
  author       = {Pith},
  title        = {Pith review of: University of Indonesia at SemEval-2025 Task 11: Evaluating State-of-the-Art Encoders for Multi-Label Emotion Detection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4NMV2G5K}},
  note         = {Machine review of arXiv:2505.16460}
}

read the original abstract

This paper presents our approach for SemEval 2025 Task 11 Track A, focusing on multilabel emotion classification across 28 languages. We explore two main strategies: fully fine-tuning transformer models and classifier-only training, evaluating different settings such as fine-tuning strategies, model architectures, loss functions, encoders, and classifiers. Our findings suggest that training a classifier on top of prompt-based encoders such as mE5 and BGE yields significantly better results than fully fine-tuning XLMR and mBERT. Our best-performing model on the final leaderboard is an ensemble combining multiple BGE models, where CatBoost serves as the classifier, with different configurations. This ensemble achieves an average F1-macro score of 56.58 across all languages.

Figures

Figures reproduced from arXiv: 2505.16460 by the authors.

**Figure 1.** Our system overview and a specific emotion. This configuration is referred to as SEP. Architecture Used. In this experiment, we employ multilingual BERT (mBERT) and XLMR as the underlying architectures for fine-tuning. These models serve as the backbone for our emotion classification framework, leveraging their multilingual pretraining to enhance contextual understanding across diverse languages. Loss Function. Du… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

17 extracted references · 6 canonical work pages

[3]

Preprint, arXiv:1911.02116

Unsuper- vised cross-lingual representation learning at scale. Preprint, arXiv:1911.02116. DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bocha...

arXiv 1911
[6]

Preprint, arXiv:2502.13595

Mmteb: Massive multilingual text embedding benchmark. Preprint, arXiv:2502.13595. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al- Dahle, Aiesha Letman, Akhil Mathur, Alan Schel- ten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mi- tra, Archie Sravankumar, Artem Koren...

arXiv
[7]

Preprint, arXiv:2407.21783

The llama 3 herd of models. Preprint, arXiv:2407.21783. M.A. Hearst, S.T. Dumais, E. Osuna, J. Platt, and B. Scholkopf

arXiv
[8]

In Proceedings of the 18th International Workshop on Semantic Evalua- tion (SemEval-2024), pages 1405–1411, Mexico City, Mexico

PEAR at SemEval-2024 task 1: Pair encoding with augmented re-sampling for semantic textual relatedness. In Proceedings of the 18th International Workshop on Semantic Evalua- tion (SemEval-2024), pages 1405–1411, Mexico City, Mexico. Association for Computational Linguistics. Yova Kementchedjhieva and Ilias Chalkidis

work page 2024
[10]

In Findings of the Asso- ciation for Computational Linguistics: EACL 2023, pages 1825–1836, Dubrovnik, Croatia

Best practices in the creation and use of emotion lexicons. In Findings of the Asso- ciation for Computational Linguistics: EACL 2023, pages 1825–1836, Dubrovnik, Croatia. Association for Computational Linguistics. Saif Mohammad and Svetlana Kiritchenko

work page 2023
[12]

Preprint, arXiv:2210.07316

Mteb: Massive text embedding benchmark. Preprint, arXiv:2210.07316. Shamsuddeen Hassan Muhammad, Nedjma Ousid- houm, Idris Abdulmumin, Jan Philip Wahle, Terry Ruas, Meriem Beloucif, Christine de Kock, Nir- mal Surange, Daniela Teodorescu, Ibrahim Said Ahmad, David Ifeoluwa Adelani, Alham Fikri Aji, Felermino D. M. A. Ali, Ilseyar Alimova, Vladimir Araujo,...

arXiv
[13]

Preprint, arXiv:2303.08774

Gpt-4 technical report. Preprint, arXiv:2303.08774. Liudmila Prokhorenkova, Gleb Gusev, Aleksandr V orobev, Anna Veronika Dorogush, and Andrey Gulin

arXiv
[14]

Preprint, arXiv:2409.10173

jina-embeddings- v3: Multilingual embeddings with task lora. Preprint, arXiv:2409.10173. Adane Nega Tarekegn, Mario Giacobini, and Krzysztof Michalak

arXiv

Show all 17 references

[15]

Preprint, arXiv:2402.05672

Multilingual e5 text embeddings: A technical report. Preprint, arXiv:2402.05672. Janyce Wiebe, Theresa Wilson, and Claire Cardie

arXiv
[16]

Preprint, arXiv:2407.10671

Qwen2 techni- cal report. Preprint, arXiv:2407.10671. Ximing Zhang, Qian-Wen Zhang, Zhao Yan, Ruifang Liu, and Yunbo Cao

arXiv
[2018]

In Proceed- ings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan

Un- derstanding emotions: A dataset of tweets to study interactions between affect categories. In Proceed- ings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan. European Language Resources As- sociation (ELRA). Saif M....

2018
[2019]

Preprint, arXiv:1810.04805

Bert: Pre-training of deep bidirectional transformers for language understand- ing. Preprint, arXiv:1810.04805. Kenneth Enevoldsen, Isaac Chung, Imene Kerboua, Márton Kardos, Ashwin Mathur, David Stap, Jay Gala, Wissam Siblini, Dominik Krzemi ´nski, Genta Indra Winata, Saba St...

arXiv
[2020]

Preprint, arXiv:2005.14165

Language models are few-shot learn- ers. Preprint, arXiv:2005.14165. Jianlyu Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu

2005 arXiv
[2021]

In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 1190–1200, Online

Enhancing label corre- lation feedback in multi-label text classification via multi-task learning. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 1190–1200, Online. Association for Computa- tional Linguistics. Liu Zhuang, Lin Wayne, Shi Ya...

2021
[2023]

In Findings of the Association for Computational Linguistics: ACL 2023, pages 5828–5843, Toronto, Canada

An exploration of encoder-decoder approaches to multi- label classification for legal and biomedical text. In Findings of the Association for Computational Linguistics: ACL 2023, pages 5828–5843, Toronto, Canada. Association for Computational Linguistics. Tsung-Yi Lin, Priya G...

2023
[2024]

In Findings of the Associa- tion for Computational Linguistics ACL 2024, pages 2318–2335

M3- embedding: Multi-linguality, multi-functionality, multi-granularity text embeddings through self- knowledge distillation. In Findings of the Associa- tion for Computational Linguistics ACL 2024, pages 2318–2335. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Cha...

2024
[2025]

Preprint, arXiv:2501.12948

Deepseek-r1: Incen- tivizing reasoning capability in llms via reinforce- ment learning. Preprint, arXiv:2501.12948. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova

arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [3]

Preprint, arXiv:1911.02116

Unsuper- vised cross-lingual representation learning at scale. Preprint, arXiv:1911.02116. DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bocha...

arXiv 1911

[2] [6]

Preprint, arXiv:2502.13595

Mmteb: Massive multilingual text embedding benchmark. Preprint, arXiv:2502.13595. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al- Dahle, Aiesha Letman, Akhil Mathur, Alan Schel- ten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mi- tra, Archie Sravankumar, Artem Koren...

arXiv

[3] [7]

Preprint, arXiv:2407.21783

The llama 3 herd of models. Preprint, arXiv:2407.21783. M.A. Hearst, S.T. Dumais, E. Osuna, J. Platt, and B. Scholkopf

arXiv

[4] [8]

In Proceedings of the 18th International Workshop on Semantic Evalua- tion (SemEval-2024), pages 1405–1411, Mexico City, Mexico

PEAR at SemEval-2024 task 1: Pair encoding with augmented re-sampling for semantic textual relatedness. In Proceedings of the 18th International Workshop on Semantic Evalua- tion (SemEval-2024), pages 1405–1411, Mexico City, Mexico. Association for Computational Linguistics. Yova Kementchedjhieva and Ilias Chalkidis

work page 2024

[5] [10]

In Findings of the Asso- ciation for Computational Linguistics: EACL 2023, pages 1825–1836, Dubrovnik, Croatia

Best practices in the creation and use of emotion lexicons. In Findings of the Asso- ciation for Computational Linguistics: EACL 2023, pages 1825–1836, Dubrovnik, Croatia. Association for Computational Linguistics. Saif Mohammad and Svetlana Kiritchenko

work page 2023

[6] [12]

Preprint, arXiv:2210.07316

Mteb: Massive text embedding benchmark. Preprint, arXiv:2210.07316. Shamsuddeen Hassan Muhammad, Nedjma Ousid- houm, Idris Abdulmumin, Jan Philip Wahle, Terry Ruas, Meriem Beloucif, Christine de Kock, Nir- mal Surange, Daniela Teodorescu, Ibrahim Said Ahmad, David Ifeoluwa Adelani, Alham Fikri Aji, Felermino D. M. A. Ali, Ilseyar Alimova, Vladimir Araujo,...

arXiv

[7] [13]

Preprint, arXiv:2303.08774

Gpt-4 technical report. Preprint, arXiv:2303.08774. Liudmila Prokhorenkova, Gleb Gusev, Aleksandr V orobev, Anna Veronika Dorogush, and Andrey Gulin

arXiv

[8] [14]

Preprint, arXiv:2409.10173

jina-embeddings- v3: Multilingual embeddings with task lora. Preprint, arXiv:2409.10173. Adane Nega Tarekegn, Mario Giacobini, and Krzysztof Michalak

arXiv