Pith. sign in

REVIEW 4 major objections 6 minor 45 references

Dictionaries to the Rescue: Cross-Lingual Vocabulary Transfer for Low-Resource Languages Using Bilingual Dictionaries

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that bilingual dictionaries alone can transfer a pretrained model's vocabulary to a low-resource language, outperforming corpus-based baselines on named-entity recognition and perplexity.

desk verdict A genuinely new dictionary-only vocabulary transfer method with strong low-resource NER results, but the iterative-removal story is undercut by the authors' own ablation and the perplexity evaluation is not trustworthy as presented. read the letter →

arxiv 2506.01535 v1 pith:PKBOEBAS submitted 2025-06-02 cs.CL cs.AI

Haruki Sakajo , Yusuke Ide , Justin Vasselli , Yusuke Sakai , Yingtao Tian , Hidetaka Kamigaito , Taro Watanabe This is my paper
classification cs.CLcs.AI
keywords cross-lingualvocabularytransferlow-resourcelanguagesbilingualdictionariesBPEtokenizerembeddinginitializationlanguage-adaptivepre-trainingperplexityevaluation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that bilingual dictionaries alone, with no monolingual or parallel corpora, are enough to transfer a pretrained language model's vocabulary to a low-resource language. The proposed method exploits a property of BPE tokenizers: when a subword is removed from the vocabulary, tokenization falls back to shorter subwords, so iteratively deleting mapped subwords exposes ever-smaller pieces for alignment. Target subword embeddings are then initialized as count-weighted averages of English subword embeddings obtained from dictionary entry-definition pairs treated as a tiny parallel corpus. If the paper is right, adapting language models to endangered and under-resourced languages becomes practical wherever a dictionary exists, even with less than ten percent of the data that corpus-based methods need. The experiments report that the dictionary approach outperforms the FOCUS baseline on named-entity recognition and word-normalized perplexity for Uyghur, Khmer, Sanskrit, and Manchu.

What carries the argument

The central object is the BPE tokenizer's fallback behavior under subword deletion: removing a subword from the vocabulary forces the tokenizer to decompose the same word into shorter subwords. The method turns this property into an iterative mapping loop, tokenizing dictionary entries and definitions, aligning subword tokens with a statistical IBM Model 2 aligner, creating type-level target-to-source mappings, deleting mapped subwords, and repeating until no new subwords are mapped. This loop is what lets a tiny dictionary cover a large fraction of the target vocabulary, and the initialized embedding for each target subword is the count-weighted average of the English subword embeddings it mapped to.

What would settle it

Run the identical target tokenizer and language-adaptive pre-training pipeline with randomly initialized target embeddings instead of dictionary-aligned ones; if the random-initialized model matches the dictionary-initialized model on named-entity recognition F1 and word-normalized perplexity, the dictionary alignment step is not what causes the reported gains.

Watch

Extended reading notes

Core claim

The central claim is that a dictionary-only vocabulary transfer can beat corpus-based embedding initialization for low-resource languages. The authors train a byte-level BPE tokenizer on dictionary entries alone, align target and source subwords from entry-definition pairs with a token-level statistical aligner, and map each target subword to one or more source subwords. The key move is to delete newly mapped subwords from the target tokenizer and re-tokenize, so BPE falls back to shorter subwords that were previously hidden inside longer merges; repeating until no new mappings appear assigns English embeddings to most target subwords, with unmapped ones receiving the source UNK embedding. Target embeddings are initialized as count-weighted averages of their aligned source embeddings. In experiments, this procedure outperforms FOCUS on named-entity recognition F1 for Uyghur, Khmer, Sanskrit, and Manchu and, after language-adaptive pre-training, gives the best word-length-normalized perplexity for causal language models across all seven tested languages, while FOCUS retains an edge for high-resource German and Japanese.

Load-bearing premise

The causal-language-model results assume that word-length-normalized perplexity fairly compares models with different tokenizers; if that normalization favors the dictionary-trained tokenizer, the large perplexity improvements in Table 4 could be a tokenization artifact rather than evidence that the embeddings helped.

Editorial extensions

If this is right

  • Dictionary-based transfer outperforms the FOCUS baseline on low-resource languages while using under 10% of the initialization data, according to the word counts in Table 5.
  • Language-adaptive pre-training after dictionary initialization yields the best word-normalized perplexity for causal language models in all seven languages tested, as reported in Table 4.
  • The iterative removal step is crucial for masked-language-model performance: skipping it drops named-entity recognition F1 sharply for most low-resource languages after language-adaptive pre-training, as shown in Table 9.
  • Higher mapped-subword coverage correlates with better causal-language-model performance, and unmapped tokens act as noise, according to the coverage analysis in Table 6.
  • The language-specific tokenizer reduces token over-fragmentation compared with the multilingual Llama 3.1 tokenizer, although FOCUS compresses even further, as reported in Table 7.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension the paper does not run is to add inflected or conjugated word forms to the dictionary; the paper's morphology analysis predicts this should close the gap for fusional languages like German and Old English.
  • A downstream causal-language-model task in a language where one exists would check whether the word-normalized perplexity ranking survives a metric that does not depend on tokenizer normalization choices.
  • The iterative fallback mechanism suggests a general recipe: any tokenizer with deterministic fallback could host the same removal-and-realign loop, not only BPE.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes a dictionary-based method for cross-lingual vocabulary transfer. It trains a byte-level BPE tokenizer on target-language dictionary entries, aligns target subwords to source subwords by running fast_align on entry/definition pairs, and initializes target embeddings as count-weighted averages of source embeddings. The key algorithmic novelty is an iterative removal step: after mapping subwords, those subwords are deleted from the target tokenizer so that shorter subwords can be mapped in subsequent iterations. The method is evaluated on NER with RoBERTa and XLM-R and on word-length-normalized perplexity with Llama 3.1 and Gemma 2, across seven languages including four low-resource languages (Uyghur, Sanskrit, Khmer, Manchu). The paper reports improvements over the FOCUS baseline in NER for the low-resource languages and large perplexity reductions after language-adaptive pre-training.

Significance. If the NER results hold, the paper contributes a practical and refreshingly simple resource: a dictionary-only transfer method that needs no parallel corpus and works for languages with different scripts than English, including the extremely low-resource Manchu case. The code is released, and the use of descriptive-linguist dictionaries is well motivated. However, the CLM perplexity evidence is currently not convincing because the comparison metric is not validated for the proposed tokenizer and because the paper's own ablation shows the removal step often hurts CLM perplexity. The central claim, as stated in the abstract, therefore needs additional controlled evaluation before it can be accepted.

major comments (4)
  1. [Section 4 (Evaluation Metrics), Tables 4 and 10] The claim that the proposed method 'outperforms existing methods' for CLMs rests on word-length-normalized perplexity, but the paper's own ablation (Table 10) shows that the iterative removal step, which is the core novelty, increases perplexity in most languages relative to the first-iteration mapping. For example, Llama 3.1 German with LAPT goes from 88.61 with removal to 25.00 without removal, and Gemma 2 German goes from 595.14 to 18.98. Because the normalized perplexity is computed across different tokenizers and the normalization is not validated for the proposed tokenizer (a concern the Limitations section explicitly concedes), the large improvements in Table 4 over FOCUS may be a segmentation artifact rather than evidence that the estimated embeddings helped. Please provide a controlled comparison, for example by evaluating all methods with the same tokenizer or by reporting a tokenizer-independent metric, and either remove or substantially qualify the CLM perplexity claim.
  2. [Table 3 (FOCUS baseline, Sanskrit and Khmer rows)] The FOCUS(XLM-R)+LAPT baseline reports exactly the same micro-F1 score, 12.33, for both Sanskrit and Khmer. This pattern strongly suggests a collapsed baseline or an evaluation bug rather than a genuine result. If the baseline has failed for these languages, the claim that the proposed method outperforms existing methods on Sanskrit and Khmer is unsubstantiated. Please re-run FOCUS for these languages, report standard deviations, and sanity-check the baseline against trivial predictors such as the most frequent class.
  3. [Section 4 (Datasets, Training) and Table 7] The perplexity evaluation does not describe how the test data are constructed or how they are separated from the LAPT training data. Table 7 reports only the number of test subwords per tokenizer, and the training section says LAPT uses up to 3,000 samples from the same Wikipedia source. If the test documents overlap with LAPT training documents, the perplexity gains in Table 4 are inflated. In addition, all runs use a single seed (Table 12), so no variance is reported for any of the headline numbers. Please disclose the exact document-level train/test split and add multiple seeds or bootstrap confidence intervals for the main comparisons.
  4. [Section 3.3 and Table 6] For target subwords that cannot be mapped to any source subword, the method falls back to the UNK embedding, or to random initialization for Llama 3.1 (footnote 2). Table 6 shows that in several languages more than 10% of subwords are unmapped (e.g., only 77.35% mapped for Manchu), so a substantial fraction of the embedding table is initialized without any lexical signal. The paper does not ablate this default, making it unclear whether the reported NER gains come from the learned mappings or from the way unmapped tokens happen to be initialized. Please add an ablation that initializes unmapped subwords with a neutral vector such as the mean source embedding or zero and report the effect on Table 3.
minor comments (6)
  1. [Figure 5 caption] The caption contains typos: 'slho' should be 'solho' and 'lern' should be 'learn'.
  2. [Table 4] Several perplexity values are formatted as 'inf', '2.30×10 19', or '2.83×10 11', which makes the table hard to read; consider using log-scale or consistent scientific notation with a spacing fix.
  3. [Section 3.3] The notation is inconsistent: the text defines A_t as the set of source subwords for target subword t, but the formula uses M_t; please align the notation throughout.
  4. [Algorithm 1] The pseudo-code uses 'M new' with a space and later 'M_new'; also 'MapSubwords(A)' is not defined explicitly in the text, so it is unclear whether it consumes only the alignment counts or also the existing mapping.
  5. [References] The reference 'Wikimedia Foundation. Wikimedia downloads' is incomplete; please add the URL and access date.
  6. [Abstract and Conclusion] The abstract and conclusion say the method 'outperforms existing methods' without restricting the claim to NER, while Section 5.7 shows that the removal step hurts CLM perplexity in most languages; please qualify the summary claims to match the evidence.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: mapping is fitted on dictionary data, evaluations are external; minor non-load-bearing self-citation.

full rationale

The proposed method (Algorithm 1) fits target-source subword mappings using dictionary entry-definition pairs via fast_align, then initializes embeddings as a weighted average (Section 3.3) of source embeddings. The reported results are NER F1 on WikiANN/ManNER and word-length-normalized perplexity on held-out text; neither quantity is defined by the fitted mapping. The removal step is an algorithmic procedure, not a fitted parameter renamed as a prediction. The Limitations section itself flags the cross-tokenizer perplexity comparison as potentially unfair, but this is a metric-validity concern, not circularity. The only self-citation (Vasselli et al., 2025, motivating that dictionaries help language modeling) is not load-bearing: the method's contribution is the iterative BPE-removal mapping, and its evaluation is external to that citation. Hence no circular step reduces the paper's predictions to its inputs.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The method introduces no new physical or mathematical entities. It relies on four domain assumptions and three fitted components: the fast_align alignment parameters, the dictionary-trained BPE tokenizer, and the LAPT fine-tuning hyperparameters. These are standard tools, but the central claim depends on them.

free parameters (3)
  • fast_align IBM Model 2 translation probabilities = not reported (estimated on dictionary pairs)
    The target-to-source subword alignment counts c(s,t) that define the embedding weights come from fast_align trained on dictionary entry-definition pairs (Algorithm 1, line 3). The method's performance is sensitive to this fitted alignment.
  • Target BPE tokenizer merge operations = vocabulary sizes in Table 6 (1,634 to 50,265)
    The byte-level BPE tokenizer is trained on dictionary entries (Section 3.1); the resulting segmentations determine which subwords can be mapped and how the test data is tokenized for evaluation.
  • LAPT training budget and hyperparameters = up to 3,000 samples; lr 1e-4; batch 8/16; epochs 2/50 (Tables 12-13)
    Continuous pre-training after initialization strongly affects the reported perplexity and NER scores; these choices are made by the authors, not derived.
assumptions (5)
  • domain assumption Bilingual dictionaries with target-word-to-source-definition pairs are available for low-resource languages, including those with scarce online text.
    The entire method and the experimental setup (Section 4, Table 1) rely on such dictionaries; if they are unavailable for the target language, the method cannot be applied. The paper notes MUSE dictionary quality issues in the Limitations.
  • domain assumption Removing a subword from a BPE tokenizer forces fallback to shorter subwords, so iterative removal reveals shorter mappable units.
    This BPE property is the central mechanism of Section 3.2 (Algorithm 1, step 4). It is a standard property of BPE, but it is an unproved-in-paper assumption that the resulting shorter segments are semantically mappable to source subwords.
  • domain assumption fast_align, running IBM Model 2 on dictionary-size 'corpora' (e.g., 1,131 entries for Uyghur), produces subword alignments accurate enough for embedding initialization.
    Section 3.2 uses fast_align on extremely small parallel data. The paper does not validate alignment quality, only downstream performance.
  • domain assumption Word-length-normalized perplexity is a fair cross-tokenizer comparison metric.
    Section 4 Evaluation Metrics cites Roh et al. 2020 for this normalization, and the Limitations acknowledge it 'might raise questions about fair comparison'. The CLM conclusions in Table 4 rest on this assumption.
  • domain assumption English source embeddings carry enough cross-lingual signal for weighted averages to produce usable target subword embeddings.
    Section 3.3 initializes target embeddings as weighted averages of English subword embeddings; the method assumes the source-language space is a valid anchor for target subwords, including languages with different scripts.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Dictionaries to the Rescue: Cross-Lingual Vocabulary Transfer for Low-Resource Languages Using Bilingual Dictionaries." pith.science (2026). https://pith.science/paper/PKBOEBAS

@misc{pith2026250601535,
  author       = {Pith},
  title        = {Pith review of: Dictionaries to the Rescue: Cross-Lingual Vocabulary Transfer for Low-Resource Languages Using Bilingual Dictionaries},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PKBOEBAS}},
  note         = {Machine review of arXiv:2506.01535}
}
read the original abstract

Cross-lingual vocabulary transfer plays a promising role in adapting pre-trained language models to new languages, including low-resource languages. Existing approaches that utilize monolingual or parallel corpora face challenges when applied to languages with limited resources. In this work, we propose a simple yet effective vocabulary transfer method that utilizes bilingual dictionaries, which are available for many languages, thanks to descriptive linguists. Our proposed method leverages a property of BPE tokenizers where removing a subword from the vocabulary causes a fallback to shorter subwords. The embeddings of target subwords are estimated iteratively by progressively removing them from the tokenizer. The experimental results show that our approach outperforms existing methods for low-resource languages, demonstrating the effectiveness of a dictionary-based approach for cross-lingual vocabulary transfer.

Figures

Figures reproduced from arXiv: 2506.01535 by the authors.

Figure 1
Figure 1. Conceptual illustration of our proposed method. Embeddings of target subwords are initial￾ized using those of corresponding source subwords. languages, where parallel corpora are often unavail￾able (Artetxe et al., 2017; Fang and Cohn, 2017), or when there is little lexical overlap between source and target languages that use different scripts. Dictionaries offer a potential solution to this is￾sue. When reading in … view at source ↗
Figure 2
Figure 2. An illustration of the subword mapping part in the proposed method. We (1) tokenize definitions and [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. The perplexity distribution and box plot of [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (3 more)
Figure 5
Figure 5. Figure 5: The subword embedding visualization with t [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]
Figure 6
Figure 6. Figure 6: The perplexity distribution and box plot of Llama 3.1-based our proposed method. [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]
Figure 7
Figure 7. Figure 7: The perplexity distribution and box plot of Llama 3.1-based our proposed method without the removal. [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

45 extracted references · 17 canonical work pages

  1. [1]

    Oliver Adams, Adam Makarucha, Graham Neubig, Steven Bird, and Trevor Cohn. 2017. https://aclanthology.org/E17-1088/ Cross-lingual word embeddings for low-resource language modeling . In Proceedings of the 15th Conference of the E uropean Chapter of the Association for Computational Linguistics: Volume 1, Long Papers , pages 937--947, Valencia, Spain. Asso...

    work page 2017

  2. [2]

    Orevaoghene Ahia, Sachin Kumar, Hila Gonen, Jungo Kasai, David Mortensen, Noah Smith, and Yulia Tsvetkov. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.614 Do all languages cost the same? tokenization in the era of commercial language models . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 9904--9923, ...

    doi:10.18653/v1/2023.emnlp-main.614 2023

  3. [3]

    Mikel Artetxe, Gorka Labaka, and Eneko Agirre. 2017. https://doi.org/10.18653/v1/P17-1042 Learning bilingual word embeddings with (almost) no bilingual data . In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 451--462, Vancouver, Canada. Association for Computational Linguistics

    work page doi:10.18653/v1/p17-1042 2017

  4. [4]

    Ehsaneddin Asgari, Yassine El Kheir, and Mohammad Ali Sadraei Javaheri. 2025. https://arxiv.org/abs/2502.00894 Morphbpe: A morpho-aware tokenizer bridging linguistic complexity for efficient llm training across morphologies . Preprint, arXiv:2502.00894

    arXiv 2025

  5. [5]

    Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017. https://doi.org/10.1162/tacl_a_00051 Enriching word vectors with subword information . Transactions of the Association for Computational Linguistics, 5:135--146

    doi:10.1162/tacl_a_00051 2017

  6. [6]

    Brown, Stephen A

    Peter F. Brown, Stephen A. Della Pietra, Vincent J. Della Pietra, and Robert L. Mercer. 1993. https://aclanthology.org/J93-2003/ The mathematics of statistical machine translation: Parameter estimation . Computational Linguistics, 19(2):263--311

    work page 1993

  7. [7]

    Chau, Lucy H

    Ethan C. Chau, Lucy H. Lin, and Noah A. Smith. 2020. https://doi.org/10.18653/v1/2020.findings-emnlp.118 Parsing with multilingual BERT , a small corpus, and a small treebank . In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1324--1334, Online. Association for Computational Linguistics

    doi:10.18653/v1/2020.findings-emnlp.118 2020

  8. [8]

    Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzm \'a n, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. https://doi.org/10.18653/v1/2020.acl-main.747 Unsupervised cross-lingual representation learning at scale . In Proceedings of the 58th Annual Meeting of the Association for Comp...

    doi:10.18653/v1/2020.acl-main.747 2020

Show all 45 references
  1. [9]

    Konstantin Dobler and Gerard de Melo. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.829 FOCUS : Effective embedding initialization for monolingual specialization of multilingual models . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Process...

    2023 doi

  2. [10]

    Chris Dyer, Victor Chahuneau, and Noah A. Smith. 2013. https://aclanthology.org/N13-1073/ A simple, fast, and effective reparameterization of IBM model 2 . In Proceedings of the 2013 Conference of the North A merican Chapter of the Association for Computational Linguistics: Hu...

    2013

  3. [11]

    Meng Fang and Trevor Cohn. 2017. https://doi.org/10.18653/v1/P17-2093 Model transfer for tagging low-resource languages using a bilingual dictionary . In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 587...

    2017 doi

  4. [12]

    https://dumps.wikimedia.org Wikimedia downloads

    Wikimedia Foundation. https://dumps.wikimedia.org Wikimedia downloads

  5. [13]

    Fabian Gloeckle, Badr Youbi Idrissi, Baptiste Roziere, David Lopez-Paz, and Gabriel Synnaeve. 2024. https://proceedings.mlr.press/v235/gloeckle24a.html Better & faster large language models via multi-token prediction . In Proceedings of the 41st International Conference on Mac...

    2024

  6. [14]

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Art...

    2024 arXiv

  7. [15]

    HyoJung Han, Akiko Eriguchi, Haoran Xu, Hieu Hoang, Marine Carpuat, and Huda Khayrallah. 2025. https://openreview.net/forum?id=KxQRHOre9D Adapters for altering LLM vocabularies: What languages benefit the most? In The Thirteenth International Conference on Learning Representations

    2025

  8. [16]

    Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. https://openreview.net/forum?id=nZeVKeeFYf9 Lo RA : Low-rank adaptation of large language models . In International Conference on Learning Representations

    2022

  9. [17]

    Yova Kementchedjhieva, Mareike Hartmann, and Anders S gaard. 2019. https://doi.org/10.18653/v1/D19-1328 Lost in evaluation: Misleading benchmarks for bilingual dictionary induction . In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and ...

    2019 doi

  10. [18]

    Taku Kudo and John Richardson. 2018. https://doi.org/10.18653/v1/D18-2012 S entence P iece: A simple and language independent subword tokenizer and detokenizer for neural text processing . In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processin...

    2018 doi

  11. [19]

    Taku Kudo, Kaoru Yamamoto, and Yuji Matsumoto. 2004. https://aclanthology.org/W04-3230/ Applying conditional random fields to J apanese morphological analysis . In Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing, pages 230--237, Barcelona...

    2004

  12. [20]

    Guillaume Lample, Alexis Conneau, Ludovic Denoyer, and Marc'Aurelio Ranzato. 2017. https://arxiv.org/abs/1711.00043 Unsupervised machine translation using monolingual corpora only . Preprint, arXiv:1711.00043

    2017 arXiv

  13. [21]

    Sangah Lee, Sungjoo Byun, Jean Seo, and Minha Kang. 2024. https://aclanthology.org/2024.lrec-main.961/ M an NER & M an POS : Pioneering NLP for endangered M anchu language . In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resour...

    2024

  14. [22]

    Jind r ich Libovick \'y and Jind r ich Helcl. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.421 Lexically grounded subword segmentation . In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 7403--7420, Miami, Florida, USA. Assoc...

    2024 doi

  15. [23]

    Yihong Liu, Peiqin Lin, Mingyang Wang, and Hinrich Schuetze. 2024. https://doi.org/10.18653/v1/2024.findings-naacl.68 OFA : A framework of initializing unseen subword embeddings for efficient large-scale multilingual continued pretraining . In Findings of the Association for C...

    2024 doi

  16. [24]

    Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. https://arxiv.org/abs/1907.11692 Ro BERT a: A robustly optimized bert pretraining approach . Preprint, arXiv:1907.11692

    2019 arXiv

  17. [25]

    Benjamin Minixhofer, Fabian Paischer, and Navid Rekabsaz. 2022. https://doi.org/10.18653/v1/2022.naacl-main.293 WECHSEL : Effective initialization of subword embeddings for cross-lingual transfer of monolingual language models . In Proceedings of the 2022 Conference of the Nor...

    2022 doi

  18. [26]

    Ponti, and Ivan Vuli\' c

    Benjamin Minixhofer, Edoardo M. Ponti, and Ivan Vuli\' c . 2024. https://proceedings.neurips.cc/paper_files/paper/2024/file/532ce4fcf853023c4cf2ac38cbc5d002-Paper-Conference.pdf Zero-shot tokenizer transfer . In Advances in Neural Information Processing Systems, volume 37, pag...

    2024

  19. [27]

    Luca Moroni, Giovanni Puccetti, Pere-Llu \'i s Huguet Cabot, Andrei Stefan Bejgu, Alessio Miaschi, Edoardo Barba, Felice Dell ' Orletta, Andrea Esuli, and Roberto Navigli. 2025. https://aclanthology.org/2025.findings-naacl.371/ Optimizing LLM s for I talian: Reducing token fer...

    2025

  20. [28]

    Jerry Norman. 2013. A comprehensive Manchu-English dictionary . Harvard-Yenching Institute Monograph Series. Harvard University, Asia Center, Cambridge, MA

    2013

  21. [29]

    Xiaoman Pan, Boliang Zhang, Jonathan May, Joel Nothman, Kevin Knight, and Heng Ji. 2017. https://doi.org/10.18653/v1/P17-1178 Cross-lingual name tagging and linking for 282 languages . In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (...

    2017 doi

  22. [30]

    Aleksandar Petrov, Emanuele La Malfa, Philip Torr, and Adel Bibi. 2023. https://openreview.net/forum?id=78yDLKi95p Language model tokenizers introduce unfairness between languages . In Thirty-seventh Conference on Neural Information Processing Systems

    2023

  23. [31]

    Trinh Pham, Khoi Le, and Anh Tuan Luu. 2024. https://doi.org/10.18653/v1/2024.acl-long.174 U ni B ridge: A unified approach to cross-lingual transfer learning for low-resource languages . In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistic...

    2024 doi

  24. [32]

    Afshin Rahimi, Yuan Li, and Trevor Cohn. 2019. https://doi.org/10.18653/v1/P19-1015 Massively multilingual transfer for NER . In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 151--164, Florence, Italy. Association for Computatio...

    2019 doi

  25. [33]

    Fran c ois Remy, Pieter Delobelle, Hayastan Avetisyan, Alfiya Khabibullina, Miryam de Lhoneux, and Thomas Demeester. 2024. https://openreview.net/forum?id=sBxvoDhvao Trans-tokenization and cross-lingual vocabulary transfers: Language adaptation of LLM s for low-resource NLP . ...

    2024

  26. [34]

    Jihyeon Roh, Sang-Hoon Oh, and Soo-Young Lee. 2020. https://arxiv.org/abs/2011.13220 Unigram-normalized perplexity as a language model performance measure with different vocabulary sizes . Preprint, arXiv:2011.13220

    2020 arXiv

  27. [35]

    Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. https://doi.org/10.18653/v1/P16-1162 Neural machine translation of rare words with subword units . In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages ...

    2016 doi

  28. [36]

    Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, Johan Ferret, Peter Liu, Pouya Tafti, Abe Friesen, Michelle Casbon, Sabela Ramos, Ravin Kumar, Charline Le Lan...

    2024 arXiv

  29. [37]

    Laurens van der Maaten and Geoffrey Hinton. 2008. http://jmlr.org/papers/v9/vandermaaten08a.html Visualizing data using t-sne . Journal of Machine Learning Research, 9(86):2579--2605

    2008

  30. [38]

    Justin Vasselli, Haruki Sakajo, Arturo Mart \'i nez Peguero, Frederikus Hudi, and Taro Watanabe. 2025. https://aclanthology.org/2025.americasnlp-1.13/ Leveraging dictionaries and grammar rules for the creation of educational materials for indigenous languages . In Proceedings ...

    2025

  31. [39]

    Zihan Wang, Karthikeyan K, Stephen Mayhew, and Dan Roth. 2020. https://doi.org/10.18653/v1/2020.findings-emnlp.240 Extending multilingual BERT to low-resource languages . In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 2649--2656, Online. Associ...

    2020 doi

  32. [40]

    L.J. Whaley. 1997. https://books.google.co.jp/books?id=S7oRYzV5SJgC Introduction to Typology: The Unity and Diversity of Language . SAGE Publications

    1997

  33. [41]

    Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, and ...

    2020 doi

  34. [42]

    Atsuki Yamaguchi, Aline Villavicencio, and Nikolaos Aletras. 2024 a . https://doi.org/10.18653/v1/2024.findings-emnlp.396 An empirical study on cross-lingual vocabulary adaptation for efficient language model inference . In Findings of the Association for Computational Linguis...

    2024 doi

  35. [43]

    Atsuki Yamaguchi, Aline Villavicencio, and Nikolaos Aletras. 2024 b . https://arxiv.org/abs/2406.11477 How can we effectively expand the vocabulary of llms with 0.01gb of target language text? Preprint, arXiv:2406.11477

    2024

  36. [44]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  37. [45]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.