REVIEW 6 cited by
mmE5: Improving Multimodal Multilingual Embeddings via High-quality Synthetic Data
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
mmE5: Improving Multimodal Multilingual Embeddings via High-quality Synthetic Data
read the original abstract
Multimodal embedding models have gained significant attention for their ability to map data from different modalities, such as text and images, into a unified representation space. However, the limited labeled multimodal data often hinders embedding performance. Recent approaches have leveraged data synthesis to address this problem, yet the quality of synthetic data remains a critical bottleneck. In this work, we identify three criteria for high-quality synthetic multimodal data. First, broad scope ensures that the generated data covers diverse tasks and modalities, making it applicable to various downstream scenarios. Second, robust cross-modal alignment makes different modalities semantically consistent. Third, high fidelity ensures that the synthetic data maintains realistic details to enhance its reliability. Guided by these principles, we synthesize datasets that: (1) cover a wide range of tasks, modality combinations, and languages, (2) are generated via a deep thinking process within a single pass of a multimodal large language model, and (3) incorporate real-world images with accurate and relevant texts, ensuring fidelity through self-evaluation and refinement. Leveraging these high-quality synthetic and labeled datasets, we train a multimodal multilingual E5 model mmE5. Extensive experiments demonstrate that mmE5 achieves state-of-the-art performance on the MMEB Benchmark and superior multilingual performance on the XTD benchmark. Our codes, datasets and models are released in https://github.com/haon-chen/mmE5.
Forward citations
Cited by 6 Pith papers
-
Douyin Multimodal Embedding Model Technical Report
Latent typed reasoning plus cross-conditional reconstruction during training improves multimodal retrieval accuracy while keeping inference a standard dense bi-encoder, yielding 74.8 (2B) and 78.4 (9B) on MMEB-v2.
-
FreeRet: MLLMs as Training-Free Retrievers
FreeRet enables pretrained MLLMs to act as training-free retrievers via semantically grounded embeddings and reasoning-based reranking, outperforming models trained on millions of pairs on MMEB benchmarks.
-
FreeRet: MLLMs as Training-Free Retrievers
A prompt-and-layer tweak lets pretrained multimodal LLMs serve as competitive retrieval systems without any additional training, with reranking framed as multiple-choice questions to reduce label bias.
-
MetaEmbed: Scaling Multimodal Retrieval at Test-Time with Flexible Late Interaction
MetaEmbed trains fixed learnable Meta Tokens to produce granularity-organized multi-vector embeddings that support test-time scaling in multimodal retrieval.
-
Reconstructing Content with Collaborative Attention for Universal Multimodal Representation Learning
CoCoA forces an MLLM to reconstruct masked text through a single EOS token, improving multimodal embedding quality on MMEB-V1 and matching MoCa at 3B with far less pretraining data.
-
Magic-MM-Embedding: Towards Visual-Token-Efficient Universal Multimodal Embedding with MLLMs
Visual token compression (4x fewer tokens) plus a three-stage generative/contrastive/judge-curated training pipeline yields state-of-the-art MLLM-based retrieval accuracy at lower inference cost.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.