SemBridge: Language Transfer in Sparse Encoders via Multilingual Semantic Bridges

Heuiseok Lim; Hyeonseok Moon; Jia-Heui Ju; Seongtae Hong; Youngjoon Jang

arxiv: 2605.26002 · v1 · pith:ACU7XKWFnew · submitted 2026-05-25 · 💻 cs.IR

SemBridge: Language Transfer in Sparse Encoders via Multilingual Semantic Bridges

Seongtae Hong , Youngjoon Jang , Jia-Heui Ju , Hyeonseok Moon , Heuiseok Lim This is my paper

Pith reviewed 2026-06-29 20:12 UTC · model grok-4.3

classification 💻 cs.IR

keywords sparse encoderscross-lingual transfermultilingual embeddingsinformation retrievalzero-shot retrievallanguage adaptationsemantic alignmenttoken initialization

0 comments

The pith

SemBridge initializes each non-English sparse token as a linear combination of a few source-language synonyms chosen through multilingual dense embeddings.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Sparse encoders deliver high-precision retrieval by assigning importance weights inside a fixed vocabulary, yet their English-centric vocabularies block direct use in other languages. SemBridge overcomes the barrier by treating dense multilingual models as semantic bridges that align source and target vocabularies. For every target token the method identifies a small set of semantically close source tokens and initializes the target embedding as their weighted sum, discarding unrelated source material. The resulting starting point reduces semantic noise and lets subsequent fine-tuning converge faster. Experiments across five languages and four sparse architectures report better zero-shot retrieval and stronger post-fine-tuning scores than prior initialization baselines.

Core claim

SemBridge establishes semantic alignments between source and target vocabularies using multilingual dense embeddings as a bridge. Rather than directly relying on all source tokens, SemBridge selects a small set of semantically related source-language tokens and uses them to initialize each target-language token, effectively filtering out semantic noise and reconstructing target tokens as precise linear combinations of core synonyms. This accelerates convergence during fine-tuning and improves training efficiency.

What carries the argument

SemBridge, the initialization procedure that selects a small set of semantically related source-language tokens via multilingual dense embeddings and reconstructs each target token as their linear combination.

If this is right

Zero-shot retrieval performance improves across five languages and four sparse architectures relative to prior baselines.
Fine-tuning reaches higher final retrieval scores than the same architectures initialized by existing methods.
Training converges in fewer steps, raising overall training efficiency.
High-precision sparse retrieval becomes deployable in non-English settings without full vocabulary redesign.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The same bridge-based selection could be tested on dense encoders to see whether it also reduces cross-lingual fine-tuning cost.
If the number of selected source tokens can be made even smaller while preserving performance, memory and compute savings would increase.
The method implicitly treats dense multilingual models as fixed teachers; relaxing that teacher to an updated or domain-specific dense model might further improve alignment quality.

Load-bearing premise

Multilingual dense embeddings supply sufficiently accurate semantic alignments that allow reliable selection of a small set of source tokens whose linear combination reconstructs each target token without introducing semantic noise.

What would settle it

Running the same zero-shot and fine-tuning experiments on the five languages and four architectures and observing no consistent gains in retrieval metrics over the existing baselines would falsify the central claim.

Figures

Figures reproduced from arXiv: 2605.26002 by Heuiseok Lim, Hyeonseok Moon, Jia-Heui Ju, Seongtae Hong, Youngjoon Jang.

**Figure 2.** Figure 2: Overview of the SemBridge embedding ini [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** Figure 3: Training loss trajectories for Splade-v3 on Chinese, Korean, and Russian using Baseline, OFA, FOCUS, [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

**Figure 4.** Figure 4: Zero-shot retrieval performance (nDCG@10) on (a) WebFAQ and (b) MIRACL. For SemBridge, the [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗

**Figure 5.** Figure 5: Efficiency vs. performance trade-off on the [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗

read the original abstract

Sparse encoders offer high-precision retrieval by representing term importance within a vocabulary space, yet their English-centric structures pose a critical impediment to language transfer for non-English languages. To overcome this structural limitation, we propose SemBridge, a novel embedding initialization method designed for cross-lingual adaptation in sparse encoders by leveraging multilingual bridge models. SemBridge establishes semantic alignments between source and target vocabularies using multilingual dense embeddings as a bridge. Rather than directly relying on all source tokens, SemBridge selects a small set of semantically related source-language tokens and uses them to initialize each target-language token, effectively filtering out semantic noise and reconstructing target tokens as precise linear combinations of core synonyms. This accelerates convergence during fine-tuning and improves training efficiency. Extensive experiments across five languages and four sparse architectures demonstrate that SemBridge achieves superior zero-shot retrieval performance and consistently improves retrieval performance after fine-tuning compared to existing baselines. These results validate SemBridge as a practical solution for deploying high-performance sparse retrieval systems in diverse linguistic environments.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

SemBridge gives a practical initialization method for cross-lingual sparse encoders but needs stronger evidence on the size of gains.

read the letter

The key takeaway is that SemBridge introduces a way to initialize sparse encoder tokens for new languages by using multilingual dense embeddings to find and combine a small number of related source tokens. This is presented as better than direct transfer or other baselines for zero-shot performance and fine-tuning speed across five languages and four architectures.

The paper does a reasonable job laying out the method and running it across multiple languages and model types. The idea of filtering noise by selecting synonyms via the bridge is straightforward and could be practical if it holds.

Where it falls short is the lack of concrete experimental details. We don't see the actual performance deltas, the baselines used, or any analysis of when the bridge fails. The central assumption that the dense alignments are clean enough for linear reconstruction needs more testing, like checking for introduced noise on specific terms or comparing to simpler mapping methods.

This kind of work is for IR folks dealing with multilingual sparse retrieval systems. Someone looking for engineering tips on adapting these models would find it relevant. The idea is solid enough on its face that it merits peer review, though the authors will likely need to add more evidence and controls in revision.

I'd say send it to referees with a request for the full results tables and ablations.

Referee Report

1 major / 0 minor

Summary. The paper proposes SemBridge, an embedding initialization technique for sparse encoders that uses multilingual dense embeddings as bridges to select a small set of semantically related source-language tokens and initialize each target-language token as their linear combination. The central claim is that this yields superior zero-shot retrieval performance and consistent gains after fine-tuning relative to existing baselines, demonstrated across five languages and four sparse architectures.

Significance. If the reported gains are robust, the work would offer a practical route to language transfer for sparse retrieval models, which currently suffer from English-centric vocabularies. The idea of using dense bridges to filter noise while preserving term-importance structure is a targeted contribution to multilingual IR.

major comments (1)

[Abstract] Abstract: the claim of 'superior zero-shot retrieval performance and consistently improves retrieval performance after fine-tuning' across five languages and four architectures is presented without any metrics, baselines, statistical tests, or even the names of the languages and architectures. This information is load-bearing for the central empirical claim and is absent from the provided manuscript text.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the detailed feedback. We address the major comment on the abstract below and will revise the manuscript to improve clarity.

read point-by-point responses

Referee: [Abstract] Abstract: the claim of 'superior zero-shot retrieval performance and consistently improves retrieval performance after fine-tuning' across five languages and four architectures is presented without any metrics, baselines, statistical tests, or even the names of the languages and architectures. This information is load-bearing for the central empirical claim and is absent from the provided manuscript text.

Authors: We agree the abstract would be strengthened by greater specificity. In the revised version, we will name the five languages and four sparse architectures explicitly. Due to typical abstract length limits, we will not add full metrics or statistical tests there, but will reference the key baselines and the nature of the gains. All quantitative results, including metrics, baselines, and significance tests, appear in the experiments section of the full manuscript. revision: yes

Circularity Check

0 steps flagged

No significant circularity detected

full rationale

The paper describes an empirical initialization method that leverages external multilingual dense embeddings to select and linearly combine source tokens for target vocabulary initialization in sparse encoders. No equations, fitted parameters, or self-citations are presented in the provided text that reduce any claimed prediction or result to an input by construction. The central claims concern measured retrieval performance gains across languages and architectures; these are externally falsifiable via experiments and do not rely on self-definitional loops, renamed known results, or load-bearing self-citations for their validity. The derivation chain is therefore self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

Review is based solely on the abstract; therefore the ledger records only the domain assumptions explicitly invoked by the described method. No free parameters or invented entities are named in the abstract.

axioms (1)

domain assumption Multilingual dense embeddings provide accurate semantic alignments between source and target vocabularies that can be used to select core synonyms.
Invoked to justify the bridge-based selection of source tokens for each target token.

pith-pipeline@v0.9.1-grok · 5713 in / 1170 out tokens · 24795 ms · 2026-06-29T20:12:33.758051+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

2 extracted references · 2 canonical work pages

[1]

arXiv preprint arXiv:2402.14714 , year=

Efficient and effective vocabulary expan- sion towards multilingual large language models. Preprint, arXiv:2402.14714. Weize Kong, Jeffrey M Dudek, Cheng Li, Mingyang Zhang, and Michael Bendersky. 2023. Sparseembed: Learning sparse lexical representations with contex- tual embeddings for retrieval. InProceedings of the 46th International ACM SIGIR confere...

work page arXiv 2023
[2]

Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval

Cpt: a pre-trained unbalanced transformer for both chinese language understanding and generation. Science China Information Sciences, 67(5):152102. Lee Xiong, Chenyan Xiong, Ye Li, Kwok-Fung Tang, Jialin Liu, Paul Bennett, Junaid Ahmed, and Arnold Overwijk. 2020. Approximate nearest neighbor neg- ative contrastive learning for dense text retrieval. arXiv ...

work page Pith review arXiv 2020

[1] [1]

arXiv preprint arXiv:2402.14714 , year=

Efficient and effective vocabulary expan- sion towards multilingual large language models. Preprint, arXiv:2402.14714. Weize Kong, Jeffrey M Dudek, Cheng Li, Mingyang Zhang, and Michael Bendersky. 2023. Sparseembed: Learning sparse lexical representations with contex- tual embeddings for retrieval. InProceedings of the 46th International ACM SIGIR confere...

work page arXiv 2023

[2] [2]

Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval

Cpt: a pre-trained unbalanced transformer for both chinese language understanding and generation. Science China Information Sciences, 67(5):152102. Lee Xiong, Chenyan Xiong, Ye Li, Kwok-Fung Tang, Jialin Liu, Paul Bennett, Junaid Ahmed, and Arnold Overwijk. 2020. Approximate nearest neighbor neg- ative contrastive learning for dense text retrieval. arXiv ...

work page Pith review arXiv 2020