Tokenizing Crosslingual Homographs

Rotem Brillant; Yuval Pinter

REVIEW 4 major objections 4 minor 28 references

Tokenizing Crosslingual Homographs

T0 review · 4 major / 4 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Swapping the first character of a shared word for a language-specific cue before tokenizer training lets multilingual models keep 'bank' (English) apart from 'bank' (German), yielding small but repeated gains in English-to-X translation.

desk verdict Careful, honest empirical study of a cheap tokenizer intervention; the abstract overstates the intrinsic claim, and the missing random-character control is a real gap, but the paper deserves a serious referee. read the letter →

arxiv 2607.17689 v1 pith:SEXO3V6W submitted 2026-07-20 cs.CL

Rotem Brillant , Yuval Pinter This is my paper

classification cs.CL

keywords cross-lingualhomographsfalsefriendsmultilingualtokenizationsubwordvocabularieslanguagecuesbyte-pairencoding(BPE)UnigramLMmachinetranslation

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Multilingual tokenizers build one shared vocabulary for many languages, so a word that looks identical in two languages—'bank' in English and German, say—typically receives a single subword unit and a single embedding, even when its meanings differ. This paper proposes marking such cross-lingual homographs during tokenizer training by replacing their first character with a language-specific Unicode cue, a one-to-one, recoverable transformation that costs nothing at inference. The paper first establishes that BPE and UnigramLM treat homographs in a largely language-agnostic way, while the context-sensitive SaGe tokenizer diverges more strongly, and that the cues close this gap. In English-to-X machine translation the intervention yields small but repeated BLEU improvements under BPE—significant on FLORES+ for English-Italian—while its effect under UnigramLM is inconsistent and sometimes negative. The paper concludes that a lightweight language signal at the tokenization level is a promising direction, while acknowledging the gains are modest and the test languages are all related Latin-script ones.

What carries the argument

The central object is the language cue: for each language pair, a set of 26 Unicode Extended Latin characters, each mapped one-to-one to a lowercase English letter. During tokenizer training, whenever a word from a prebuilt cross-lingual homograph set appears, its first character is replaced by the cue for that language and letter. The one-to-one mapping keeps the transformation recoverable and lets the tokenizer still share the word's remainder (such as the substring 'ank' across a cued English and German 'bank'), so the intervention creates language-specific initial subword units without fully separating the vocabularies. Supporting machinery includes the tokenization-pattern categories (s

What would settle it

Train BPE tokenizers on the same English–German corpus under three control conditions: cues replaced by randomly chosen rare Extended Latin letters, cues placed at the word's final character instead of the first, and identical words cued in only a random half of their occurrences. If the random-letter or final-position conditions reproduce the paper's same_splits reduction and BLEU gains, the claimed mechanism—a systematic initial language marker—is not what drives the effect.

Watch

Extended reading notes

Core claim

The paper's central claim is that standard subword tokenizers erase language identity from identical surface forms: trained on concatenated multilingual text, BPE and UnigramLM mostly assign cross-lingual homographs (and false friends) the same segmentation a monolingual tokenizer would, while the context-sensitive SaGe tokenizer splits them differently across languages more often. Replacing the first character of each homograph in the training corpus with a language-unique Unicode cue reverses that: cued BPE and ULM tokenizers shift from the pervasive 'same_splits' pattern toward 'different_splits', closing the gap with SaGe. In downstream English-to-X machine translation with a shared 8,00

Load-bearing premise

The claim collapses if the effects come from the particular hand-picked cue characters rather than from language information itself, since the 26 cues per language were chosen from the Extended Latin block and verified only for rarity, with no ablation over alternative cue sets or positions.

Editorial extensions

If this is right

Language identity can be supplied at the input stage rather than only through model-side language embeddings: because the cue substitution is one-to-one and invertible, the intervention is nearly free to deploy and leaves the rest of the pipeline untouched.
Cued BPE models beat their baseline in most of the 24 evaluated setups, with bootstrap-significant gains on FLORES+ for English-Italian (+0.71/+0.75 BLEU) and a positive aggregate bootstrap mean (+0.17).
Marking homographs moves BPE and UnigramLM tokenization behavior toward the language-divergent pattern of context-sensitive tokenizers like SaGe, reducing the 'same_splits' cases that dominate standard training.
The benefit is specific to the tokenizer algorithm: UnigramLM's top-down selection responds erratically to the rare cued items, occasionally losing BLEU (English-Swedish on FLORES+, −2.28), so tokenizer choice matters for whether cueing helps.
Cueing changes vocabulary structure—slightly shorter tokens, higher Rényi efficiency, finer segmentation—while word-level language-model perplexity is generally slightly worse, so downstream gains are not explained by better language modeling per se.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the mechanism is language information itself, the cue protocol should generalize across cue sets and positions: a natural next experiment abandons the hand-picked Extended Latin letters for randomly sampled rare characters, and moves the cue to the word-final position, to test whether the effects come from the initial-position language marker or from any surface perturbation.
The intervention is orthogonal to model-side language embeddings, so a plausible extension is to stack both signals—corpus-level cues at tokenization and language embeddings at training—which could compound the small gains the paper reports.
The same trick could adapt to multi-script ambiguity, where tokenizers already split by Unicode script: marking shared or confusable forms at script boundaries with cues may address the cross-script homograph cases the paper leaves to future work.
The BPE-versus-UnigramLM contrast suggests a design heuristic: when a multilingual vocabulary is built with a frequency-driven bottom-up algorithm, low-frequency orthographic markers propagate consistently; with likelihood-based selection they do not, so choosing the tokenizer may be as important as choosing the signal.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

This is a careful, honestly hedged empirical study of a simple idea: replace the first character of cross-lingual homographs with language-specific Unicode cues during tokenizer training. What is actually new is the cue mechanism itself and the comparative analysis of how BPE, UnigramLM, and SaGe tokenize homographs. The authors show clearly that standard BPE and ULM treat homographs in a largely language-agnostic way, while SaGe diverges more, and they report their downstream MT results with bootstrap confidence intervals and a limitations section that does not hide the mixed picture. That is real work, and the paper advances the discussion of homograph tokenization.

The soft spots are real but not fatal. First, the abstract claims the intervention "removes this gap," but no table shows the cued tokenizer's homograph divergence. Tables 1–2 only compare baseline and SaGe. The intrinsic section reports vocabulary length, Rényi efficiency, fertility, and perplexity, but not the direct measure that would support the headline claim. That is a genuine mismatch between the abstract and the evidence. Second, the stress-test note is right: there is no control using equally rare Unicode characters with a permuted or language-agnostic assignment. The BPE gains are small (+0.17 BLEU aggregate, mostly non-significant), and adding 52 rare characters could plausibly change merge order and vocabulary composition without carrying language-specific information. Without that control, the claim that language cues, rather than rare-character perturbation, cause the gains is not fully established. Third, no code or data are released, which makes the already-small effects harder to evaluate. The word-level perplexity being generally worse is acknowledged and handled well.

That said, the paper's own conclusions are appropriately modest. It does not claim a breakthrough; it claims preliminary evidence. The missing random-character control is a fair demand, but it does not invalidate the paper's descriptive findings about tokenizer behavior, nor does it make the intervention uninteresting as a proof of concept.

Who is this for? Anyone working on multilingual tokenizer design or on the interaction between segmentation and cross-lingual transfer. It deserves a serious referee. I would send it to review with a request for a direct cued-vs-baseline intrinsic table, a random-character control, and a code/data release. The central idea is worth engaging with; it just needs to be backed by the evidence the abstract promises.

Referee Report

4 major / 4 minor

Summary. The paper studies cross-lingual homographs in multilingual subword tokenization. It first shows that BPE and UnigramLM tend to segment homographs identically in monolingual and multilingual settings, whereas the context-sensitive SaGe tokenizer produces more divergent segmentations. It then proposes a tokenizer-level intervention: during tokenizer training, replace the first character of each selected homograph with a language-specific Unicode cue, leaving the rest of the surface form intact. The intrinsic analysis reports modest changes in vocabulary statistics, and the downstream English-to-X machine translation experiments show small BLEU gains under BPE in several settings but a mixed, often negative picture under UnigramLM, with bootstrap confidence intervals excluding zero in only a few cases. The paper concludes that lightweight language cues at the tokenizer level are a promising but not yet robust direction.

Significance. If the central claim is established, the paper would make a useful contribution by showing that injecting minimal language-identity information at the tokenization stage can, in some settings, improve multilingual MT with almost no architectural change. The study is transparent about many limitations and includes bootstrap intervals, multiple languages, two tokenizer families, and multiple evaluation sets, which are strengths. However, the evidence as presented is preliminary: the headline intrinsic claim is not actually tabulated, and the downstream MT gains are mostly statistically indistinguishable from zero. The absence of a control condition for the specific cue characters is a substantial threat to the causal interpretation. The paper would be strengthened materially by adding such controls and by reporting the missing cued-tokenizer intrinsic results.

major comments (4)

[§3.2, Appendix A, Tables 6–7, Appendix F] The central causal claim is that language-specific cue information, not merely a rare-character perturbation, drives the BPE improvements. No control is reported: the experiments compare baseline against the single hand-selected 26-character-per-language mapping in Appendix A, with no permuted assignment or equally rare non-linguistic character control. Since the aggregate BPE bootstrap difference is only +0.1724 BLEU, and only 2 of 24 BPE confidence intervals exclude zero, a generic change to the alphabet could plausibly produce the same pattern. The ULM aggregate is negative (-0.2495), reinforcing sensitivity to tokenizer details. The Limitations section acknowledges scope but does not address this confound.
[§2.1, Tables 1–2, Abstract] The abstract and introduction state that the proposed intervention 'removes this gap' between standard tokenizers and SaGe, but Tables 1–2 compare baseline BPE/ULM with their SaGe variants only; no cued-tokenizer column is shown. Thus the intrinsic gap-removal claim is not supported by any tabulated result in the paper. The authors should add cued BPE/UnigramLM distributions to Tables 1–2 or an equivalent table, and preferably test whether the cued distribution moves toward the SaGe pattern rather than merely away from the baseline.
[Appendix A] The method's reproducibility depends on the exact cue-character mapping, but the mapping table referenced as 'Table 8' is absent from the manuscript. The text says the table presents the full mapping from English letters to language-specific Unicode replacements, yet no such table appears. This omission also prevents any independent assessment of whether the chosen characters are indeed rare and appropriate. Please include the actual mapping.
[§4.2, Statistical Robustness, Appendix F] The downstream evidence is weaker than the prose sometimes suggests. Only 2 of 24 BPE bootstrap intervals exclude zero, both for EN–IT on FLORES+, while three UnigramLM intervals significantly favor the baseline, notably EN–SV. The aggregate BPE mean is dominated by the EN–IT result, and removing the two Swedish FLORES+ datasets reduces the ULM aggregate from -0.2495 to +0.0699. The phrase 'repeated appearance of small improvements' overstates the pattern; the authors should either temper this conclusion or provide additional evidence, such as a proper control condition or a paired analysis across all cells, that the positive point estimates are not idiosyncratic.

minor comments (4)

[Figure 7 caption] The caption lists 'EN–SE' for Swedish, while the text and all other tables use 'SV'. Please correct for consistency.
[Table 12, EN-SV FLORES+ HG row] The row appears to contain a formatting/typo ('30.0811 30.86'), with an apparently truncated cued BLEU value. Please fix.
[Table 2 header] The column headers 'EN L2 EN All All EMD' are dense and hard to parse; consider aligning them explicitly with the five pattern categories listed in §2.1.
[Appendix F] The bootstrap tables report mean differences and intervals, but not the number of sentences per evaluation set or the homograph-hit rate. Reporting those would help readers assess the strength of the evidence for the homograph-focused subsets.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the cue intervention is evaluated on external MT benchmarks; intrinsic effects are direct mechanisms, not fitted predictions.

full rationale

The paper contains no derived prediction that is equivalent to its inputs. The method is an operational data transformation: first characters of selected homographs are replaced by language-specific cue characters (§3.2). The intrinsic observations that cued tokenizers are slightly shorter, more efficient, and more finely segmented are descriptive statistics of the resulting tokenizers; the paper explicitly labels the length effect as 'expected' and reports word-level perplexity when token-level perplexity is segmentation-dependent (§4.1). The downstream claim is not circular: BLEU scores on OPUS-100 and FLORES+ are external benchmarks (Tables 6–7), bootstrap estimates are reported (Appendix F), and no parameter is fitted to those benchmarks and then reported as a prediction. The cue alphabet in Appendix A was selected for rarity, not optimized on BLEU; absence of a random-character control is a confounding concern, not circularity. The same-group SaGe citation (Yehezkel and Pinter, 2023) is not load-bearing: SaGe tokenizations are directly computed and reported in Tables 1–2 rather than assumed. There is no imported uniqueness theorem and no ansatz smuggled via self-citation. The abstract's phrase 'our intervention removes this gap' is under-supported in the body (no cued EMD table is shown), but an unsupported claim is a support gap, not a reduction to the paper's own inputs. Overall, the central result is an empirical, externally benchmarked finding; no circular step is exhibited.

Assumptions & free parameters 3 free parameters · 4 assumptions · 1 invented entities

The central claim rests on a hand-chosen cue character set, frequency filters, and the assumption that tokenizer-level divergence (as exemplified by SaGe) is beneficial. These are not fitted to data in a statistical sense, but they are design choices that shape the results.

free parameters (3)

Cue character mapping (26 Unicode chars per language) = hand-selected; mapping in Appendix A not fully reproduced
The choice of which Unicode characters serve as cues is arbitrary; only rarity was checked, no ablation across cue sets. The method's effect may depend on this specific choice.
Homograph frequency filter threshold = 5
Words appearing fewer than 5 times in either language corpus are excluded; this hand-chosen threshold shapes the homograph set and could affect results.
Minimum homograph length = 3
Words shorter than 3 characters removed to avoid irregular forms like Roman numerals; hand-chosen preprocessing choice.

assumptions (4)

domain assumption A shared subword unit receives a single embedding that must serve all languages; therefore shared homographs cause representational ambiguity.
Motivates the entire problem statement (§1); if models could fully disambiguate via context, the intervention would be unnecessary.
domain assumption BPE/UnigramLM trained on concatenated data assign one subword unit to identical surface forms; language-agnostic treatment is the default.
Used to frame the intrinsic analysis (§2.1, Table 2).
domain assumption SaGe's divergence from monolingual tokenization is a desirable property to emulate.
Abstract and §2.1 treat SaGe's larger 'different_splits' as the target; the paper does not demonstrate that this divergence is causally linked to better downstream performance.
ad hoc to paper Replacing only the first character of a homograph preserves enough surface overlap to keep the tokenizer compact while signaling language.
Core design choice in §3.2; no ablation over cue position or cue type.

invented entities (1)

Language-specific Unicode cue characters
purpose: Replace first character of homographs during tokenizer training so the tokenizer can learn language-distinct subword units.
Artificial symbol set introduced by the paper; its only support is the paper's own tokenizer/LM/MT experiments, with no external falsifiable handle.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Tokenizing Crosslingual Homographs." pith.science (2026). https://pith.science/paper/SEXO3V6W

@misc{pith2026260717689,
  author       = {Pith},
  title        = {Pith review of: Tokenizing Crosslingual Homographs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SEXO3V6W}},
  note         = {Machine review of arXiv:2607.17689}
}

read the original abstract

Multilingual language models rely on shared subword vocabularies to represent multiple languages within a limited number of token units. While such sharing is often useful, it can also create cases in which identical surface forms are treated too uniformly across languages, even when their meanings or usage differ. We investigate this limitation through cross-lingual homographs and false friends, and examine whether introducing language information earlier in the tokenization process can improve their treatment. We propose a simple tokenizer-level intervention based on language cues: language-specific characters replacing initial characters of shared-vocabulary words, reducing common identity during vocabulary construction. In intrinsic analysis, we find through tokenizer-level statistics that BPE and UnigramLM often treat cross-lingual homographs in a largely language agnostic way, whereas the context-sensitive SaGe tokenizer diverges more strongly; our intervention removes this gap. In downstream English-to-X machine translation, our cues yield modest improvements in several settings, especially under BPE, although the effect is not consistent across all languages and evaluation sets. Overall, the findings suggest that adding lightweight language information at the tokenizer level is a promising direction for further exploration.

Figures

Figures reproduced from arXiv: 2607.17689 by the authors.

**Figure 2.** Overview of multilingual language model pipeline: standard flow vs. language-cue injection. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** Token fertility comparison between baseline [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Figures from the paper (4 more)

**Figure 4.** Figure 4: Token fertility comparison between baseline [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

**Figure 6.** Figure 6: Token fertility comparison between baseline [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]

**Figure 5.** Figure 5: Token fertility comparison between baseline [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]

**Figure 7.** Figure 7: Token fertility comparison between baseline [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

28 extracted references · 3 canonical work pages

[1]

aari1995 . 2026. False friends english--german. https://huggingface.co/datasets/aari1995/false_friends_en_de. Hugging Face dataset, accessed March 11, 2026

2026
[2]

Roee Aharoni, Melvin Johnson, and Orhan Firat. 2019. https://doi.org/10.18653/v1/N19-1388 Massively multilingual neural machine translation . In Proceedings of the 2019 Conference of the North A merican Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) , pages 3874--3884, Minneapolis, M...

doi:10.18653/v1/n19-1388 2019
[3]

Eymen Efe Altun. 2026. all-words-in-all-languages. https://github.com/eymenefealtun/all-words-in-all-languages. GitHub repository, accessed March 11, 2026

2026
[4]

Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. http://arxiv.org/abs/1911.02116 Unsupervised cross-lingual representation learning at scale

arXiv 2020
[5]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. https://doi.org/10.18653/v1/N19-1423 BERT : Pre-training of deep bidirectional transformers for language understanding . In Proceedings of the 2019 Conference of the North A merican Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long a...

doi:10.18653/v1/n19-1423 2019
[6]

Dirk Goldhahn, Thomas Eckart, and Uwe Quasthoff. 2012. https://aclanthology.org/L12-1154/ Building large monolingual dictionaries at the L eipzig corpora collection: From 100 to 200 languages . In Proceedings of the Eighth International Conference on Language Resources and Evaluation ( LREC '12) , pages 759--765, Istanbul, Turkey. European Language Resour...

2012
[7]

Julie Kallini, Dan Jurafsky, Christopher Potts, and Martijn Bartelds. 2025. http://arxiv.org/abs/2509.18750 False friends are not foes: Investigating vocabulary overlap in multilingual language models

arXiv 2025
[8]

Yuka Kitamura, Jiahao Huang, and Akiko Aizawa. 2025. https://doi.org/10.18653/v1/2025.ijcnlp-long.96 Doppelganger- JC : Benchmarking the LLM s' understanding of cross-lingual homographs between J apanese and C hinese . In Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia-Pacific Chapte...

work page doi:10.18653/v1/2025.ijcnlp-long.96 2025

Show all 28 references

[9]

Taku Kudo. 2018. https://doi.org/10.18653/v1/P18-1007 Subword regularization: Improving neural network translation models with multiple subword candidates . In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), page...

2018 doi
[10]

Taku Kudo and John Richardson. 2018. http://arxiv.org/abs/1808.06226 Sentencepiece: A simple and language independent subword tokenizer and detokenizer for neural text processing

2018 arXiv
[11]

Guillaume Lample and Alexis Conneau. 2019. http://arxiv.org/abs/1901.07291 Cross-lingual language model pretraining

2019 arXiv
[12]

Junyoung Lee, Marco Cognetta, Sangwhan Moon, and Naoaki Okazaki. 2025. https://doi.org/10.18653/v1/2025.loresmt-1.8 Jamo-level subword tokenization in low-resource K orean machine translation . In Proceedings of the Eighth Workshop on Technologies for Machine Translation of Lo...

2025 doi
[13]

Leipzig Corpora Collection . 2021. https://wortschatz.uni-leipzig.de/en/download/ Leipzig corpora used in this study . This work used the following Leipzig corpora for tokenizer training: [English Wikipedia 2016 1M, German Wikipedia 2021 1M, French Wikipedia 2021 1M, Spanish W...

2021
[14]

Davis Liang, Hila Gonen, Yuning Mao, Rui Hou, Naman Goyal, Marjan Ghazvininejad, Luke Zettlemoyer, and Madian Khabsa. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.813 XLM - V : Overcoming the vocabulary bottleneck in multilingual masked language models . In Proceedings of...

2023 doi
[15]

Tomasz Limisiewicz, Ji r \'i Balhar, and David Mare c ek. 2023. https://doi.org/10.18653/v1/2023.findings-acl.350 Tokenization impacts multilingual language modeling: Assessing vocabulary allocation and overlap across languages . In Findings of the Association for Computationa...

2023 doi
[16]

NLLB Team , Marta R. Costa-juss \`a , James Cross, Onur C elebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Sun, Skyler Wang, Guillaume Wenzek, Al Youngblood, Bapi Akula, Loic Barrault, Gabriel Mejia Gonzalez,...

2024
[17]

Open Language Data Initiative . 2024. Flores+: A multilingual dataset for evaluation. https://huggingface.co/datasets/openlanguagedata/flores_plus. Hugging Face dataset card, accessed April 11, 2026

2024
[18]

Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli. 2019. fairseq: A fast, extensible toolkit for sequence modeling. In Proceedings of NAACL-HLT 2019: Demonstrations

2019
[19]

Telmo Pires, Eva Schlinger, and Dan Garrette. 2019. https://doi.org/10.18653/v1/P19-1493 How multilingual is multilingual BERT ? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4996--5001, Florence, Italy. Association for Compu...

2019 doi
[20]

Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. 2018. Improving language understanding by generative pre-training

2018
[21]

Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. https://doi.org/10.18653/v1/P16-1162 Neural machine translation of rare words with subword units . In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages ...

2016 doi
[22]

Eshaan Tanwar, Gayatri Oke, and Tanmoy Chakraborty. 2026. http://arxiv.org/abs/2501.09127 Multilingual llms struggle to link orthography and semantics in bilingual word processing

2026
[23]

J \"o rg Tiedemann. 2012. http://www.lrec-conf.org/proceedings/lrec2012/pdf/463_Paper.pdf Parallel data, tools and interfaces in OPUS . In Proceedings of the Eighth International Conference on Language Resources and Evaluation ( LREC '12) , pages 2214--2218, Istanbul, Turkey. ...

2012
[24]

Giorgos Vernikos and Andrei Popescu-Belis. 2021. https://doi.org/10.18653/v1/2021.findings-emnlp.224 Subword mapping and anchoring across languages . In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 2633--2647, Punta Cana, Dominican Republic. Ass...

2021 doi
[25]

Wiktionary contributors . 2026. Appendix: Glossary of false friends. https://en.wiktionary.org/wiki/Appendix:Glossary_of_false_friends. Wiktionary, accessed March 11, 2026

2026
[26]

Shaked Yehezkel and Yuval Pinter. 2023. https://doi.org/10.18653/v1/2023.eacl-main.45 Incorporating context into subword vocabularies . In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 623--635, Dubrovnik, Cr...

2023 doi
[27]

Biao Zhang, Philip Williams, Ivan Titov, and Rico Sennrich. 2020. https://doi.org/10.18653/v1/2020.acl-main.148 Improving massively multilingual neural machine translation and zero-shot translation . In Proceedings of the 58th Annual Meeting of the Association for Computationa...

2020 doi
[28]

Vil \'e m Zouhar, Clara Meister, Juan Gastaldi, Li Du, Mrinmaya Sachan, and Ryan Cotterell. 2023. https://doi.org/10.18653/v1/2023.acl-long.284 Tokenization and the noiseless channel . In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (...

2023 doi

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.

[1] [1]

aari1995 . 2026. False friends english--german. https://huggingface.co/datasets/aari1995/false_friends_en_de. Hugging Face dataset, accessed March 11, 2026

2026

[2] [2]

Roee Aharoni, Melvin Johnson, and Orhan Firat. 2019. https://doi.org/10.18653/v1/N19-1388 Massively multilingual neural machine translation . In Proceedings of the 2019 Conference of the North A merican Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) , pages 3874--3884, Minneapolis, M...

doi:10.18653/v1/n19-1388 2019

[3] [3]

Eymen Efe Altun. 2026. all-words-in-all-languages. https://github.com/eymenefealtun/all-words-in-all-languages. GitHub repository, accessed March 11, 2026

2026

[4] [4]

Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. http://arxiv.org/abs/1911.02116 Unsupervised cross-lingual representation learning at scale

arXiv 2020

[5] [5]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. https://doi.org/10.18653/v1/N19-1423 BERT : Pre-training of deep bidirectional transformers for language understanding . In Proceedings of the 2019 Conference of the North A merican Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long a...

doi:10.18653/v1/n19-1423 2019

[6] [6]

Dirk Goldhahn, Thomas Eckart, and Uwe Quasthoff. 2012. https://aclanthology.org/L12-1154/ Building large monolingual dictionaries at the L eipzig corpora collection: From 100 to 200 languages . In Proceedings of the Eighth International Conference on Language Resources and Evaluation ( LREC '12) , pages 759--765, Istanbul, Turkey. European Language Resour...

2012

[7] [7]

Julie Kallini, Dan Jurafsky, Christopher Potts, and Martijn Bartelds. 2025. http://arxiv.org/abs/2509.18750 False friends are not foes: Investigating vocabulary overlap in multilingual language models

arXiv 2025

[8] [8]

Yuka Kitamura, Jiahao Huang, and Akiko Aizawa. 2025. https://doi.org/10.18653/v1/2025.ijcnlp-long.96 Doppelganger- JC : Benchmarking the LLM s' understanding of cross-lingual homographs between J apanese and C hinese . In Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia-Pacific Chapte...

work page doi:10.18653/v1/2025.ijcnlp-long.96 2025