Pith. sign in

REVIEW 4 major objections 5 minor 42 references

A language model's next-token entropy doubles as a language vector: low surprise means close kin, high surprise means distance.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Entropy2Vec turns the cross-lingual surprise of monolingual language models into dense language embeddings that resemble typological families and match curated vectors in downstream tasks.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection A novel and clearly-presented unsupervised language vector method, but the script/resource confound the authors themselves concede keeps the typology claim from being convincing as-is; it deserves a careful review rather than a pass. the 4 major comments →

arxiv 2509.05060 v1 pith:KFLM3EIC submitted 2025-09-05 cs.CL

Entropy2Vec: Crosslingual Language Modeling Entropy as End-to-End Learnable Language Representations

Patrick Amadeus Irawan , Ryandito Diandaru , Belati Jagad Bintang Syuhada , Randy Zakya Suchrady , Alham Fikri Aji , Genta Indra Winata , Fajri Koto , Samuel Cahyawijaya This is my paper
classification cs.CL MSC 68T50
keywords cross-lingual language representationlanguage modeling entropylinguistic typologylanguage vectorsperplexityunsupervised embeddingsmultilingual NLPlow-resource languages
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that the entropy of a monolingual language model's next-token predictions, measured on many other languages, is itself a dense, unsupervised language vector: a model trained on one language is more surprised by structurally distant languages than by close ones. On 33 languages from Glot500, the authors build a matrix of cross-entropy scores, use each row as a language embedding, induce typological trees with DBSCAN, and compare those trees to Glottolog and to URIEL/URIEL+ vectors. They also plug the vectors into the LinguAlchemy regularizer and report accuracy competitive with curated typological vectors on SIB-200 and MASSIVE, including low-resource and unseen languages. If the hypothesis holds, typology can be derived on demand from text alone, without expert-curated static inventories.

Core claim

Entropy2Vec's central claim is that cross-lingual entropy is a typological fingerprint. For each language Li, a small GPT-2 model is trained on monolingual text, and the model's average cross-entropy L(θ_i,D_j) is computed on held-out corpora of every language. The vector Z_Li = [L(θ_i,D_1),...,L(θ_i,D_n)] is the language embedding: low entries mark languages the model finds easy (structurally similar), high entries mark difficult ones. The authors claim these rows align with established families—Malayic languages cluster together, Philippine languages group near Malayic—and match or beat URIEL/URIEL+ in tree reconstruction and downstream LinguAlchemy regularization.

What carries the argument

The central object is the cross-entropy row Z_Li = [L(θ_i,D_1),...,L(θ_i,D_n)]: the average token-level cross-entropy of a language model θ_i trained only on language Li when evaluated on each corpus D_j. It turns the already-learned generalization behavior of autoregressive LMs into a geometric language representation; DBSCAN clustering on these rows induces the typological trees, and the rows themselves serve as dense language vectors in downstream regularization.

Load-bearing premise

The vector rows are assumed comparable across languages even though each language has its own character tokenizer, script, and corpus size; if cross-entropy differences are driven by those artifacts rather than by structural similarity, the embeddings measure encoding and data abundance, not typology.

What would settle it

Take two typologically close languages written in different scripts (e.g., Thai and Lao) and two unrelated languages sharing a script; if the induced tree consistently separates the close pair while uniting the same-script pair, the entropy signal is dominated by encoding rather than structure. A direct test is to re-train the models on transliterated corpora and check whether the tree realigns with Glottolog.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Any language with enough monolingual text can receive a vector without expert annotation, avoiding the missing-feature and sparse-coverage problems of WALS-style inventories.
  • Because each vector is computed by running a model on corpora, it can be refreshed or continually updated, making typology dynamic rather than a static snapshot.
  • Entropy2Vec alone is competitive with URIEL/URIEL+ for LinguAlchemy regularization on XLM-R, and concatenating with baseline vectors helps mBERT, so it can substitute for or augment curated vectors depending on the backbone.
  • Induced trees reproduce broad Glottolog groupings and known families (Malayic, Philippine), indicating the entropy matrix carries a real cross-lingual structural signal.
  • Since DBSCAN tree construction does not require a pre-specified cluster count, the method scales to arbitrary language sets without manual priors on the number of families.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper's own Limitations section concedes a script/encoding effect: Thai and Lao are split early despite shared ancestry, so the current vectors likely mix orthography with structure; a transliteration control would quantify how much of the signal is typological.
  • Cross-entropy is sensitive to tokenizer vocabulary and corpus size (here capped at 1M sentences, but far smaller for some languages), so part of the vectors may encode resource abundance rather than linguistic similarity; equal-size, script-normalized corpora would test this.
  • The 33-language prototype suggests an obvious scaling path to hundreds or thousands of languages using existing web corpora, but the cost is one full LM training per language; an approximate or shared-encoder variant could make that practical.
  • If the hypothesis survives controls, entropy rows over a shared tokenizer could serve as a living typology, with periodic re-inference on new corpora tracking language change over time.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes Entropy2Vec, a method for constructing dense language representations by training a character-level GPT-2 model on each of 33 languages and using the average cross-entropy of model θ_i on corpus D_j as the (i,j)-th entry of a language-vector matrix (§3.2). The vectors are then used to induce typological trees via DBSCAN and compared to Glottolog using RF/LCA distances (§4), and are also plugged into the LinguAlchemy regularizer for XLM-R/mBERT fine-tuning on SIB-200 and MASSIVE (§5). The central claim is that cross-entropy of a monolingual LM on another language's text reflects structural similarity: low entropy implies high similarity, high entropy implies divergence.

Significance. If the core hypothesis is validated, Entropy2Vec would offer a scalable, unsupervised, and dynamically updatable alternative to static typological inventories, with clear benefits for low-resource and under-documented languages. The paper provides a concrete pipeline, corpus choices, and comparisons against URIEL/URIEL+, and the downstream integration with LinguAlchemy is a sensible testbed. However, the main scientific claim is not yet isolated from major confounds—per-language tokenizers, unequal corpus sizes, and training trajectories—several of which are acknowledged in the paper's own Limitations section. The significance of the contribution therefore hinges on whether these confounds can be controlled for or shown not to dominate.

major comments (4)
  1. [§3.2, Appendix B, Table 6] The definition Z_Li = [L(θ_i,D_1),...,L(θ_i,D_n)] assumes that cross-entropy values are comparable across rows. But each θ_i is trained with a language-specific character-level tokenizer, on corpora ranging from ~7k (tdt) to ~1M sentences, with early stopping up to 150 epochs. Thus each entry mixes tokenizer/script coverage, model convergence, and resource availability with structural similarity. The Limitations section explicitly concedes that the representation is influenced by encoding and that Thai/Lao split despite shared ancestry. To support the abstract's hypothesis, the authors need a controlled experiment: e.g., a shared tokenizer across languages, subsampling to a common corpus size, and an ablation comparing Entropy2Vec against a script/character-overlap baseline. Without this, the Glottolog alignment in Fig. 2c and Table 1 cannot be attributed to typology.
  2. [§4.1] The tree construction is underspecified. DBSCAN is a flat clustering algorithm, yet the paper says it is used to generate 'hierarchical language clusters' and that the resulting clusters are 'transformed into tree structures'; no details are given for how the hierarchy is obtained, how cluster labels such as 'Unsplit_L1_*' are assigned, or how the hyperparameters min_samples=0.3 and epsilon=0.1 were selected. Since the central typology evaluation (Table 1) depends entirely on this transformation, the procedure must be precisely defined and ideally validated against standard hierarchical clustering methods with appropriate hyperparameter choices.
  3. [§4.2, Table 1] The quantitative evidence for the claim that Entropy2Vec 'aligns' with Glottolog is weak. The reported differences are small (LCA MAE 8.60 vs. 9.35 for URIEL Syntax; RF 17 vs. 18), with no confidence intervals, no significance tests, and no repeated runs. Given the small number of languages (33) and the loose clustering procedure, these differences are within likely noise. The paper overstates the conclusion that Entropy2Vec 'captures key linguistic relationships' without statistical support.
  4. [§5.2, Tables 3 and 4] The downstream comparisons also lack error bars or significance tests. For XLM-R, Entropy2Vec's best average (81.6 on SIB-200 unseen) is slightly below URIEL+ Syntax (81.8); for mBERT, the differences from URIEL+ variants are within roughly one accuracy point. With 13 training languages and no reported variance across seeds, these differences cannot support the claim of 'competitive or even superior' performance. Reporting standard deviations across multiple seeds and paired significance tests is necessary.
minor comments (5)
  1. [§3.1] Typo: 'where and θ represents' should read 'where θ represents'.
  2. [§3.2] Typo: 'ans closely similar languages' should be 'and closely similar languages'.
  3. [§4.2] Typo: 'similarity to Glottog' and 'the the' constructions; please proofread.
  4. [Figure 2] The caption says 'Tree comparison across methods'; please clarify which subtrees are shown and how they were pruned, since Figure 2c appears to contain unsplit/cluster labels that are not defined in the text.
  5. [§4.1] The phrase 'the its concatenated variants' is ungrammatical; should read 'the concatenated variants'.

Circularity Check

0 steps flagged

No significant circularity: Entropy2Vec vectors are direct cross-entropy measurements, not fitted to the typological or downstream targets they are evaluated against.

full rationale

The derivation chain is self-contained. The language vector Z_Li is defined in Section 3.2 as [L(θ_i,D_1),...,L(θ_i,D_n)], i.e., raw cross-entropies of a monolingual LM on each corpus; no parameter is fitted to Glottolog, URIEL, SIB-200, or MASSIVE. The typological evaluation compares DBSCAN trees (with fixed min_samples=0.3 and epsilon=0.1) to Glottolog via RF/LCA metrics; these external comparisons are independent support. Downstream LinguAlchemy evaluation uses Entropy2Vec as a regularizer against URIEL/URIEL+ baselines, so the claim is not defined in terms of its target. The self-citations in Section 3.2 motivate the hypothesis that LM generalization depends on language closeness, but the hypothesis itself is tested empirically against external benchmarks and the vectors are not constructed from those citations. The Limitations admission that encoding (script/tokenizer) splits Thai/Lao — 'the representation is influenced by the encoding, causing similar languages to split due to differing encodings' — is an acknowledged confound and a correctness risk, not a definitional reduction; it weakens the typological interpretation but does not make the derivation circular. Therefore no circular step can be exhibited and the score is 0.

Axiom & Free-Parameter Ledger

2 free parameters · 4 axioms · 0 invented entities

The central claim rests on the assumption that LM cross-entropy is a monotone proxy for structural similarity across languages, plus comparability across tokenizers, scripts, and corpora, and the validity of Glottolog as ground truth. No new physical or theoretical entities are introduced. The only hand-set numeric parameters are the DBSCAN hyperparameters.

free parameters (2)
  • DBSCAN epsilon = 0.1
    Chosen by hand and fixed across all vector variants; no sensitivity analysis is reported. It directly affects tree structure and all typology comparisons.
  • DBSCAN min_samples = 0.3
    Chosen by hand and fixed across all experiments; no sensitivity analysis is reported. It controls minimum cluster size and therefore tree shape.
axioms (4)
  • domain assumption Cross-entropy of a language model trained on language A, evaluated on language B, monotonically reflects structural similarity between A and B.
    Central hypothesis stated in Section 3.2 and the abstract; never derived or controlled against tokenizer, script, and model-quality confounds.
  • domain assumption Entropy values are comparable across languages despite per-language character tokenizers, scripts, and unequal corpus sizes.
    Invoked by defining Z_Li in Section 3.2 without normalization; the Limitations section concedes encoding influences the representation.
  • domain assumption Glottolog tree is a valid gold standard for typological distance.
    Used as ground truth in Section 4.1; the paper notes other trees exist but does not test sensitivity to the choice.
  • domain assumption DBSCAN with post-processing yields a meaningful hierarchical representation of language relationships.
    Section 4.1 and Appendix C justify DBSCAN qualitatively, but the conversion from flat clusters to hierarchical trees is not specified.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Entropy2Vec: Crosslingual Language Modeling Entropy as End-to-End Learnable Language Representations." pith.science (2026). https://pith.science/paper/KFLM3EIC

@misc{pith2026250905060,
  author       = {Pith},
  title        = {Pith review of: Entropy2Vec: Crosslingual Language Modeling Entropy as End-to-End Learnable Language Representations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KFLM3EIC}},
  note         = {Machine review of arXiv:2509.05060}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

We introduce Entropy2Vec, a novel framework for deriving cross-lingual language representations by leveraging the entropy of monolingual language models. Unlike traditional typological inventories that suffer from feature sparsity and static snapshots, Entropy2Vec uses the inherent uncertainty in language models to capture typological relationships between languages. By training a language model on a single language, we hypothesize that the entropy of its predictions reflects its structural similarity to other languages: Low entropy indicates high similarity, while high entropy suggests greater divergence. This approach yields dense, non-sparse language embeddings that are adaptable to different timeframes and free from missing values. Empirical evaluations demonstrate that Entropy2Vec embeddings align with established typological categories and achieved competitive performance in downstream multilingual NLP tasks, such as those addressed by the LinguAlchemy framework.

Figures

Figures reproduced from arXiv: 2509.05060 by Alham Fikri Aji, Belati Jagad Bintang Syuhada, Fajri Koto, Genta Indra Winata, Patrick Amadeus Irawan, Randy Zakya Suchrady, Ryandito Diandaru, Samuel Cahyawijaya.

Figure 1
Figure 1. Figure 1: ENTROPY2VEC framework. Monolingual LMs are trained per language, and cross-lingual perplex￾ity is used as an unsupervised signal to derive language vectors and induce typological trees, aligning well with expert-curated taxonomies. These embeddings capture semantic and syntac￾tic properties, facilitating cross-lingual transfer in various NLP tasks. Nonetheless, existing methods primarily focus on monolingu… view at source ↗
Figure 2
Figure 2. Figure 2: Tree comparison across methods: (a) Glottolog gold tree, (b) tree derived from URIEL [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

42 extracted references · 20 canonical work pages · 1 internal anchor

  1. [1]

    Alabi, Yanke Mao, Haonan Gao, and En-Shiun Annie Lee

    David Ifeoluwa Adelani, Hannah Liu, Xiaoyu Shen, Nikita Vassilyev, Jesujoba O. Alabi, Yanke Mao, Haonan Gao, and En-Shiun Annie Lee. 2024. https://doi.org/10.18653/v1/2024.eacl-long.14 SIB -200: A simple, inclusive, and big evaluation dataset for topic classification in 200+ languages and dialects . In Proceedings of the 18th Conference of the European Ch...

    doi:10.18653/v1/2024.eacl-long.14 2024

  2. [2]

    Muhammad Farid Adilazuarda, Samuel Cahyawijaya, Alham Fikri Aji, Genta Indra Winata, and Ayu Purwarianti. 2024. https://arxiv.org/abs/2401.06034 Lingualchemy: Fusing typological and geographical elements for unseen language generalization . Preprint, arXiv:2401.06034

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  3. [3]

    A. V. Aho, J. E. Hopcroft, and J. D. Ullman. 1973. https://doi.org/10.1145/800125.804056 On finding lowest common ancestors in trees . In Proceedings of the Fifth Annual ACM Symposium on Theory of Computing, STOC '73, page 253–265, New York, NY, USA. Association for Computing Machinery

    work page arXiv 1973

  4. [4]

    Do, Yan Xu, and Pascale Fung

    Yejin Bang, Samuel Cahyawijaya, Nayeon Lee, Wenliang Dai, Dan Su, Bryan Wilie, Holy Lovenia, Ziwei Ji, Tiezheng Yu, Willy Chung, Quyet V. Do, Yan Xu, and Pascale Fung. 2023. https://doi.org/10.18653/v1/2023.ijcnlp-main.45 A multitask, multilingual, multimodal evaluation of C hat GPT on reasoning, hallucination, and interactivity . In Proceedings of the 13...

    doi:10.18653/v1/2023.ijcnlp-main.45 2023

  5. [5]

    Hale, and Hannah Rose Kirk

    Andrew Michael Bean, Simeon Hellsten, Harry Mayne, Jabez Magomere, Ethan A Chi, Ryan Andrew Chi, Scott A. Hale, and Hannah Rose Kirk. 2024. https://openreview.net/forum?id=cLga8GStdk LINGOLY : A benchmark of olympiad-level linguistic reasoning puzzles in low resource and extinct languages . In The Thirty-eight Conference on Neural Information Processing S...

    work page 2024

  6. [6]

    Balthasar Bickel and Johanna Nichols. 2002. Autotypologizing databases and their use in fieldwork. In Proceedings of the international LREC workshop on resources and tools in field linguistics, Las Palmas, volume 2627. ISLE and DOBES Nijmegen

    work page 2002

  7. [7]

    Balthasar Bickel, Johanna Nichols, Taras Zakharko, Alena Witzlack-Makarevich, Kristine Hildebrandt, Michael Rießler, Lennart Bierkandt, Fernando Zúñiga, and John B Lowe. 2023. https://doi.org/10.5281/zenodo.7976754 The autotyp database (v1.1.1)

    work page doi:10.5281/zenodo.7976754 2023

  8. [8]

    David Bradley. 2009. Burma, thailand, cambodia, laos and vietnam. The Routledge handbook of sociolinguistics around the world, pages 98--107

    work page 2009

  9. [9]

    David Bradley. 2023. Sociolinguistics in mainland southeast asia. In The Routledge Handbook of Sociolinguistics Around the World, pages 227--237. Routledge

    work page 2023

  10. [10]

    Thomas Brochhagen, Gemma Boleda, Eleonora Gualdoni, and Yang Xu. 2023. From language development to language evolution: A unified view of human lexical creativity. Science, 381(6656):431--436

    work page 2023

  11. [11]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, and 12 others. 2020. https://arxiv.org/abs/2005.14165 Lan...

    Pith/arXiv arXiv 2020

  12. [12]

    Samuel Cahyawijaya. 2024. https://arxiv.org/abs/2409.13897 Llm for everyone: Representing the underrepresented in large language models . Preprint, arXiv:2409.13897

    Pith/arXiv arXiv 2024

  13. [13]

    Samuel Cahyawijaya, Holy Lovenia, Alham Fikri Aji, Genta Winata, Bryan Wilie, Fajri Koto, Rahmad Mahendra, Christian Wibisono, Ade Romadhony, Karissa Vincentio, Jennifer Santoso, David Moeljadi, Cahya Wirawan, Frederikus Hudi, Muhammad Satrio Wicaksono, Ivan Parmonangan, Ika Alfina, Ilham Firdausi Putra, Samsul Rahmadani, and 29 others. 2023 a . https://d...

    doi:10.18653/v1/2023.findings-acl.868 2023

  14. [14]

    Samuel Cahyawijaya, Holy Lovenia, Fajri Koto, Dea Adhista, Emmanuel Dave, Sarah Oktavianti, Salsabil Akbar, Jhonson Lee, Nuur Shadieq, Tjeng Wawan Cenggoro, Hanung Linuwih, Bryan Wilie, Galih Muridan, Genta Winata, David Moeljadi, Alham Fikri Aji, Ayu Purwarianti, and Pascale Fung. 2023 b . https://doi.org/10.18653/v1/2023.ijcnlp-main.60 N usa W rites: Co...

    doi:10.18653/v1/2023.ijcnlp-main.60 2023

  15. [15]

    Samuel Cahyawijaya, Holy Lovenia, Fajri Koto, Rifki Putri, Wawan Cenggoro, Jhonson Lee, Salsabil Akbar, Emmanuel Dave, Nuurshadieq Nuurshadieq, Muhammad Mahendra, Rr Putri, Bryan Wilie, Genta Winata, Alham Aji, Ayu Purwarianti, and Pascale Fung. 2024. https://doi.org/10.18653/v1/2024.acl-long.796 Cendol: Open instruction-tuned generative large language mo...

    doi:10.18653/v1/2024.acl-long.796 2024

  16. [16]

    Samuel Cahyawijaya, Holy Lovenia, Tiezheng Yu, Willy Chung, and Pascale Fung. 2023 c . https://doi.org/10.18653/v1/2023.sealp-1.5 I nstruct A lign: High-and-low resource language alignment via continual crosslingual instruction tuning . In Proceedings of the First Workshop in South East Asian Language Processing, pages 55--78, Nusa Dua, Bali, Indonesia. A...

    doi:10.18653/v1/2023.sealp-1.5 2023

  17. [17]

    Samuel Cahyawijaya, Genta Indra Winata, Bryan Wilie, Karissa Vincentio, Xiaohong Li, Adhiguna Kuncoro, Sebastian Ruder, Zhi Yuan Lim, Syafri Bahar, Masayu Khodra, Ayu Purwarianti, and Pascale Fung. 2021. https://doi.org/10.18653/v1/2021.emnlp-main.699 I ndo NLG : Benchmark and resources for evaluating I ndonesian natural language generation . In Proceedin...

    work page doi:10.18653/v1/2021.emnlp-main.699 2021

  18. [18]

    Lyle Campbell and Ver \'o nica Grondona. 2008. Ethnologue: Languages of the world. Language, 84(3):636--641

    2008

  19. [19]

    Morten H Christiansen and Simon Kirby. 2003. Language evolution: Consensus and controversies. Trends in cognitive sciences, 7(7):300--307

    work page 2003

  20. [20]

    Team Cohere, :, Aakanksha, Arash Ahmadian, Marwan Ahmed, Jay Alammar, Milad Alizadeh, Yazeed Alnumay, Sophia Althammer, Arkady Arkhangorodsky, Viraat Aryabumi, Dennis Aumiller, Raphaël Avalos, Zahara Aviv, Sammie Bae, Saurabh Baji, Alexandre Barbet, Max Bartolo, Björn Bebensee, and 211 others. 2025. https://arxiv.org/abs/2504.00698 Command a: An enterpris...

    Pith/arXiv arXiv 2025

  21. [21]

    Michael C Corballis. 2017. Language evolution: a changing perspective. Trends in cognitive sciences, 21(4):229--236

    work page 2017

  22. [22]

    Dryer and Martin Haspelmath, editors

    Matthew S. Dryer and Martin Haspelmath, editors. 2013. https://doi.org/10.5281/zenodo.7385533 WALS Online (v2020.3) . Zenodo

    work page doi:10.5281/zenodo.7385533 2013

  23. [23]

    Norman Fairclough. 2009. Language and globalization

    work page 2009

  24. [24]

    Jack FitzGerald, Christopher Hench, Charith Peris, Scott Mackie, Kay Rottmann, Ana Sanchez, Aaron Nash, Liam Urbach, Vishesh Kakarala, Richa Singh, Swetha Ranganath, Laurie Crist, Misha Britan, Wouter Leeuwis, Gokhan Tur, and Prem Natarajan. 2023. https://doi.org/10.18653/v1/2023.acl-long.235 MASSIVE : A 1 M -example multilingual natural language understa...

    doi:10.18653/v1/2023.acl-long.235 2023

  25. [25]

    Lenore A Grenoble. 2021. Language shift. In Oxford research encyclopedia of linguistics

    work page 2021

  26. [26]

    Martin Haspelmath. 2005. The world atlas of language structures. Oxford University Press

    work page 2005

  27. [27]

    Alfred B Hudson. 1970. A note on selako: Malayic dayak and land dayak languages in western borneo. Sarawak Museum Journal, 18(36-37):301--318

    work page 1970

  28. [28]

    Ayyoob Imani, Peiqin Lin, Amir Hossein Kargaran, Silvia Severini, Masoud Jalili Sabet, Nora Kassner, Chunlan Ma, Helmut Schmid, Andr \'e Martins, Fran c ois Yvon, and Hinrich Sch \"u tze. 2023. https://doi.org/10.18653/v1/2023.acl-long.61 Glot500: Scaling multilingual corpora and language models to 500 languages . In Proceedings of the 61st Annual Meeting...

    doi:10.18653/v1/2023.acl-long.61 2023

  29. [29]

    Hoang, Eric Khiu, A

    Aditya Khan, Mason Shipton, David Anugraha, Kaiyao Duan, Phuong H. Hoang, Eric Khiu, A. Seza Do g ru \"o z, and En-Shiun Annie Lee. 2025. https://aclanthology.org/2025.coling-main.463/ URIEL +: Enhancing linguistic inclusion and usability in a typological and multilingual knowledge base . In Proceedings of the 31st International Conference on Computationa...

    work page 2025

  30. [30]

    Mortensen, Ke Lin, Katherine Kairis, Carlisle Turner, and Lori Levin

    Patrick Littell, David R. Mortensen, Ke Lin, Katherine Kairis, Carlisle Turner, and Lori Levin. 2017. https://aclanthology.org/E17-2002/ URIEL and lang2vec: Representing languages as typological, geographical, and phylogenetic vectors . In Proceedings of the 15th Conference of the E uropean Chapter of the Association for Computational Linguistics: Volume ...

    2017

  31. [31]

    Kampman, Joel Ruben Antony Moniz, Muhammad Ravi Shulthan Habibi, Frederikus Hudi, Railey Montalan, Ryan Ignatius Hadiwijaya, Joanito Agili Lopo, William Nixon, B \"o rje F

    Holy Lovenia, Rahmad Mahendra, Salsabil Maulana Akbar, Lester James Validad Miranda, Jennifer Santoso, Elyanah Aco, Akhdan Fadhilah, Jonibek Mansurov, Joseph Marvin Imperial, Onno P. Kampman, Joel Ruben Antony Moniz, Muhammad Ravi Shulthan Habibi, Frederikus Hudi, Railey Montalan, Ryan Ignatius Hadiwijaya, Joanito Agili Lopo, William Nixon, B \"o rje F. K...

    doi:10.18653/v1/2024.emnlp-main.296 2024

  32. [32]

    Sebastian Nordhoff and Harald Hammarstr \"o m. 2011. Glottolog/langdoc: Defining dialects, languages, and language families as collections of resources. In First International Workshop on Linked Science 2011-In conjunction with the International Semantic Web Conference (ISWC 2011)

    work page 2011

  33. [33]

    Guilherme Penedo, Hynek Kydlíček, Vinko Sabolčec, Bettina Messmer, Negar Foroutan, Amir Hossein Kargaran, Colin Raffel, Martin Jaggi, Leandro Von Werra, and Thomas Wolf. 2025. https://arxiv.org/abs/2506.20920 Fineweb2: One pipeline to scale them all -- adapting pre-training data processing to every language . Preprint, arXiv:2506.20920

    Pith/arXiv arXiv 2025

  34. [34]

    Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners

    2019

  35. [35]

    Lawrence A Reid and Hsiu-chuan Liao. 2004. A brief syntactic typology of philippine languages

    work page 2004

  36. [36]

    Robinson and L.R

    D.F. Robinson and L.R. Foulds. 1981. https://doi.org/10.1016/0025-5564(81)90043-2 Comparison of phylogenetic trees . Mathematical Biosciences, 53(1):131--147

    doi:10.1016/0025-5564(81)90043-2 1981

  37. [37]

    Stefanie Siebenh \"u tter. 2019. Sociocultural influences on linguistic geography: religion and language in southeast asia. In Handbook of the changing world language map, pages 2825--2843. Springer

    work page 2019

  38. [38]

    Genta Indra Winata, Alham Fikri Aji, Samuel Cahyawijaya, Rahmad Mahendra, Fajri Koto, Ade Romadhony, Kemal Kurniawan, David Moeljadi, Radityo Eko Prasojo, Pascale Fung, Timothy Baldwin, Jey Han Lau, Rico Sennrich, and Sebastian Ruder. 2023. https://doi.org/10.18653/v1/2023.eacl-main.57 N usa X : Multilingual parallel sentiment dataset for 10 I ndonesian l...

    doi:10.18653/v1/2023.eacl-main.57 2023

  39. [39]

    Rush, Stella Biderman, Albert Webson, Pawan Sasanka Ammanamanchi, Thomas Wang, Benoît Sagot, and 375 others

    BigScience Workshop, :, Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ilić, Daniel Hesslow, Roman Castagné, Alexandra Sasha Luccioni, François Yvon, Matthias Gallé, Jonathan Tow, Alexander M. Rush, Stella Biderman, Albert Webson, Pawan Sasanka Ammanamanchi, Thomas Wang, Benoît Sagot, and 375 others. 2023. https://arxiv.org/abs/2211.0...

    Pith/arXiv arXiv 2023

  40. [40]

    Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. 2021. https://doi.org/10.18653/v1/2021.naacl-main.41 m T 5: A massively multilingual pre-trained text-to-text transformer . In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics...

    doi:10.18653/v1/2021.naacl-main.41 2021

  41. [41]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  42. [42]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.