REVIEW 8 cited by
mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrieval
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
We present systematic efforts in building long-context multilingual text representation model (TRM) and reranker from scratch for text retrieval. We first introduce a text encoder (base size) enhanced with RoPE and unpadding, pre-trained in a native 8192-token context (longer than 512 of previous multilingual encoders). Then we construct a hybrid TRM and a cross-encoder reranker by contrastive learning. Evaluations show that our text encoder outperforms the same-sized previous state-of-the-art XLM-R. Meanwhile, our TRM and reranker match the performance of large-sized state-of-the-art BGE-M3 models and achieve better results on long-context retrieval benchmarks. Further analysis demonstrate that our proposed models exhibit higher efficiency during both training and inference. We believe their efficiency and effectiveness could benefit various researches and industrial applications.
Forward citations
Cited by 8 Pith papers
-
Bekko Embedding: Parameter-Efficient Multilingual Retrieval with Ultra-Compact Encoders
Bekko a8m, with 7.7M active parameters, scores 56.2 on MMTEB Multilingual v2 Retrieval, beating mE5 models and BGE-M3, while a25m reaches 57.5, on par with gte-multilingual-base.
-
IssueExec: A Test-Driven Approach for Localizing Software Engineering Issues
IssueExec localizes issue-relevant code by retrieving related tests and following their execution traces, beating prior localization methods on SWE-bench Lite.
-
ImportSnare: Directed "Code Manual" Hijacking in Retrieval-Augmented Code Generation
Documentation poisoning with hidden ranking and suggestion sequences can make RAG-based code generators confidently recommend malicious dependencies, even at 0.01% poisoning ratios.
-
Language Models Improve When Pretraining Data Matches Target Tasks
Ranking pretraining documents by similarity to benchmark training examples (BETR) yields consistent benchmark gains and a 2.1x compute multiplier over DCLM-Baseline.
-
Towards Applying Large Language Models to Complement Single-Cell Foundation Models
A fusion model called scMPT, combining scGPT with an LLM text encoder, improves single-cell cell type classification on most tested datasets, and the paper shows the LLM relies on marker genes and simple expression patterns.
-
Text-to-LoRA: Instant Transformer Adaption
A hypernetwork can generate task-specific LoRA adapters from a text description, and when trained with supervised fine-tuning it zero-shot outperforms a multi-task LoRA baseline on ten benchmarks.
-
Granite Embedding R2 Models
Granite Embedding R2 is an Apache-2.0 family of ModernBERT-based retrieval and reranking models that posts high average scores on several benchmarks but falls short of top code-retrieval and reranking baselines.
-
From Domain Documents to Requirements: Retrieval-Augmented Generation in the Space Industry
A RAG-based pipeline with neural label classification generates draft space requirements from mission documents, shown in a single qualitative case study.
Discussion (0). Sign in to comment.