A diagnostic benchmark of text and code transformations finds embedding similarity metrics often conflate opposition with equivalence; LLM judges discriminate better, and Euclidean distance improves code embeddings.
MT6: Multilingual Pretrained Text-to-Text Transformer with Translation Pairs
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
Multilingual T5 (mT5) pretrains a sequence-to-sequence model on massive monolingual texts, which has shown promising results on many cross-lingual tasks. In this paper, we improve multilingual text-to-text transfer Transformer with translation pairs (mT6). Specifically, we explore three cross-lingual text-to-text pre-training tasks, namely, machine translation, translation pair span corruption, and translation span corruption. In addition, we propose a partially non-autoregressive objective for text-to-text pre-training. We evaluate the methods on eight multilingual benchmark datasets, including sentence classification, named entity recognition, question answering, and abstractive summarization. Experimental results show that the proposed mT6 improves cross-lingual transferability over mT5.
citation-role summary
background 1
citation-polarity summary
fields
cs.CL 1years
2025 1verdicts
REJECT 1roles
background 1polarities
unclear 1representative citing papers
citing papers explorer
-
How Small Transformation Expose the Weakness of Semantic Similarity Measures
A diagnostic benchmark of text and code transformations finds embedding similarity metrics often conflate opposition with equivalence; LLM judges discriminate better, and Euclidean distance improves code embeddings.