Edeflip: Supervised Word Translation between English and Yoruba

Ikeoluwa Abioye; Jiani Ge

REVIEW 3 major objections 5 minor 10 references

Edeflip: Supervised Word Translation between English and Yoruba

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Normalizing embeddings nearly triples top-1 English-Yoruba word translation precision.

desk verdict An honest small-scale study whose headline interaction effect is not credible until train/eval overlap is ruled out. read the letter →

arxiv 2506.13020 v1 pith:BEWMUTPK submitted 2025-06-16 cs.CL cs.AIcs.LG

Ikeoluwa Abioye , Jiani Ge This is my paper

classification cs.CLcs.AIcs.LG

keywords embeddingalignmentProcrusteswordtranslationprecisionYorubalow-resourcelanguagefastTextembeddingsnormalizationEnglish-Yoruba

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether supervised embedding alignment, a method that maps two monolingual word-vector spaces onto each other without parallel corpora, can work for a low-resource language pair. It tests English-to-Yoruba translation with four alignments that vary the Yoruba embedding source (noisy Wikipedia versus a curated, diacritic-corrected corpus) and whether embeddings are normalized before Procrustes alignment. The result is that both factors matter, and they interact: normalization lifts curated precision from 6.88% to 19.38% at k=1, while its effect on the Wikipedia embedding is far smaller. The paper concludes that the state-of-the-art method transfers to Yoruba only when the monolingual embeddings are high quality and put on the same scale, and that low-resource language work must therefore attend to embedding curation, not just the alignment algorithm.

What carries the argument

The load-bearing object is the orthogonal mapping matrix $W$ found by Procrustes alignment: $W^* = \arg\max_{W\in O_d(\mathbb{R})} \|WX-Y\|_F = UV^T$, where $U\Sigma V^T = \mathrm{SVD}(YX^T)$. Given a set of anchor word pairs, the method rotates the source embedding space $X$ onto the target space $Y$ so that nearest-neighbor retrieval in the target space acts as translation. The second piece of machinery is the pre-alignment centering and normalization applied to half the conditions, which puts the two vector distributions on a common scale and unlocks the curated embedding's advantage.

What would settle it

Re-run the four alignments after deleting from the 161-pair evaluation set every word pair that also appears in the 3,693-pair training set; if curated-normalized top-1 precision drops well below 19.38% while the unnormalized conditions do not, the claimed interaction effect is an artifact of dictionary overlap.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that supervised Procrustes alignment produces low but usable word translation precision for English-Yoruba only under the right conditions. Normalizing the embedding vectors before alignment increases precision for both embedding sources, and the gain is much larger for the curated Yoruba embedding (from 6.88% to 19.38% top-1) than for the Wikipedia embedding (from 10.56% to 12.42%). The authors interpret this interaction as reflecting the initial similarity of the two spaces: the Wikipedia embeddings are both trained with the same widely used embedding method and thus closer in shape, while the curated Yoruba embedding is trained on a distinct, higher-quality corpus, so it needs normalization to map onto English. Across all conditions, the top-1 precision stays below the published figures for high-resource language pairs, which the authors attribute to embedding size, data quality, and morphological distance.

Load-bearing premise

The reported precision figures assume the 161-pair evaluation dictionary and the 3,693-pair training dictionary are effectively disjoint; if the same English-Yoruba pairs appear in both, the numbers are inflated by memorization of anchor points instead of genuine transfer.

Editorial extensions

If this is right

For low-resource pairs, supplying larger and diacritically correct monolingual embeddings raises nearest-neighbor translation precision without any parallel training data.
Normalizing before alignment is not optional; it yields a large gain for embeddings trained on different corpora and a smaller but positive gain for same-method embeddings.
The interaction between embedding source and normalization means that these two factors cannot be tuned independently when building a low-resource translation system.
The best English-Yoruba condition still translates only about a fifth of source words correctly at k=1, so at current resource levels this method is a baseline, not a deployment-ready translator.
Supervised embedding alignment with a human-created dictionary is a workable starting point for English-Yoruba, with top-1 precision of 19.38% in the curated-normalized condition.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The reported precision likely depends on the separation between the training and evaluation dictionaries; if overlapping English-Yoruba pairs were not removed, a strict holdout replication could find lower, though still positive, precision.
A direct extension would test whether subword-aware or contextualized Yoruba embeddings trained on the same curated corpus beat word-level vectors, and whether normalization still matters for them.
The factorial design (embedding source × normalization) could be rerun on other low-resource pairs to see whether the interaction generalizes or is an artifact of the particular Wikipedia-versus-curated contrast studied here.
If the interaction is general, then the practical bottleneck for low-resource embedding alignment is monolingual data curation, not the alignment algorithm itself.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: this is a small, honest experiment applying supervised Procrustes alignment to English–Yoruba, comparing two embedding sources and normalization. The value is in the concrete low-resource setting and the clear reporting of weak precision numbers. The soft spot is the one that matters: the training and evaluation dictionaries are both derived from Alabi et al. (2020), and the paper never states that overlapping pairs were removed. That makes the reported interaction effect—normalization helping the curated embedding much more than the wiki embedding—hard to credit. ‘Sea’ is a perfectly common word; if any of the 161 eval pairs appear in the 3,693 training pairs, those results are inflated by memorization, and the curated conditions are precisely the ones that would benefit most.

What the paper does well: it takes a genuinely low-resource language pair, uses two publicly available embeddings, and honestly reports that even the best condition gets only 19.4% top-1 precision. The qualitative example with ‘sea’ is helpful, though not systematic. The discussion of why normalization might matter more for the curated embedding is plausible.

The flaws are in proportion. There are no error bars, each condition ran once, and no significance test. That alone would make the interaction claim premature. But the bigger problem is the dictionary overlap. If the authors can show the evaluation set is disjoint from training, or recompute on the disjoint subset, then the paper becomes a useful descriptive study. As it stands, the numbers should not be taken at face value. There is also no code or data, which makes the single-run result impossible to check.

The paper is worth a serious referee because the topic matters and the experiment is easy to fix. My recommendation: send it to review, but with a clear request to address the overlap, run multiple seeds, and share code/data. If the authors cannot rule out overlap, they should drop the interaction claim and present this as a case study of low-resource alignment challenges.

For a reading group, it might be useful as a cautionary example of how easy it is to accidentally leak test data into training in low-resource settings.

Referee Report

3 major / 5 minor

Summary. The paper reports a supervised embedding-alignment study for English-to-Yoruba word translation. The authors align English fastText embeddings with two Yoruba embedding sources (a Wikipedia fastText embedding and a larger curated embedding from Alabi et al. 2020) under two preprocessing conditions (with or without centering and normalization), yielding four conditions. They evaluate top-1/5/10 translation precision on a 161-pair English-Yoruba evaluation set derived from WordSim353 and report that normalization improves precision, with a larger gain for the curated embeddings than for the Wikipedia embeddings; they interpret this as an interaction effect between embedding quality and normalization.

Significance. If the reported results were statistically robust and free of data leakage, the paper would provide a useful, if preliminary, demonstration that for low-resource languages the quality of monolingual embeddings and the preprocessing of embedding spaces matter more than the alignment algorithm itself, challenging a direct transfer of high-resource results. The study is fully empirical, uses publicly available data, and reports concrete numbers that could be reproduced. However, the central claims are currently undermined by the likely overlap between the training and evaluation dictionaries and by the absence of any uncertainty quantification or significance testing; these issues must be resolved before the interaction effect can be taken as established.

major comments (3)

[Section 2.1] The training dictionary (3,693 pairs scraped from Alabi et al.'s online dictionary) and the evaluation dictionary (161 pairs from Alabi et al.'s Yoruba translation of WordSim353) are both derived from Alabi et al. (2020), and the paper does not state that the two sets are disjoint. If common English-Yoruba pairs, or even English source words, appear in both resources, the reported top-k precision values, especially the curated-normalized 19.38% at k=1, could reflect memorization during supervised Procrustes alignment rather than genuine transfer. The authors should either verify and report that no overlapping pairs (or overlapping English source words) exist between the two resources, or remove all overlaps from the evaluation set and recompute all four conditions on the disjoint subset; without such a check the main quantitative claim is not trustworthy.
[Section 3, Table 1] Each condition is evaluated on only 161 word pairs with a single run, and no error bars, confidence intervals, or significance tests are reported. The claimed interaction effect rests entirely on point estimates; for example, the key contrast between curated-normalized (19.38%, roughly 31/161) and wiki-normalized (12.42%, roughly 20/161) is a difference of about 11 correct translations, and a two-proportion z-test would not reject equality at the 5% level. The authors should provide at least bootstrap confidence intervals or a permutation test for the precision differences, and ideally multiple training/evaluation splits, to establish that the interaction is not an artifact of small-sample variation.
[Section 2.2 and Section 4.1] The paper describes the method as following Conneau et al. (2017) but does not state whether the evaluation used the CSLS (cross-domain similarity local scaling) retrieval metric that the MUSE system employs; it appears to use plain nearest-neighbor retrieval. Since Section 4.1 directly compares the obtained absolute precision values to MUSE's published results for Spanish, German, and Chinese (77.4, 68.4, and 40.6), the difference in retrieval metric could account for part of the gap. The authors should specify the retrieval procedure precisely and, if CSLS was not used, avoid direct numerical comparison with MUSE results without noting this protocol difference.

minor comments (5)

[Section 2.2, Eq. (1)] The Procrustes problem is written as an arg max over W of ||WX - Y||_F, but the goal is to minimize that Frobenius norm; the displayed optimization should be arg min, and the stated SVD solution is for the minimizing orthogonal matrix.
[Section 3] The sentence "Overall, the wiki-unnormalized condition consistently yields the lowest precision, whereas the curated-normalized condition the highest" is missing a verb; it should read "and the curated-normalized condition yields the highest precision."
[Section 3] "The top retrievals of the curated-unnormalized model, however, bares no visible semblance to the meaning of 'sea'" contains subject-verb agreement and word-choice errors; it should be "bear no visible resemblance."
[Section 2.1] The authors write that they "scrape the online English-Yoruba dictionary Alabi et al. (2020)"; Alabi et al. (2020) is a research paper, not the dictionary itself, so please identify the actual web resource being scraped and provide its URL or a data availability statement.
[Section 2.2] The normalization step is described only as "center and normalize the embeddings using the tensor operations from the PyTorch package"; please specify whether "normalize" means L2 unit-length normalization and whether centering is mean subtraction across the vocabulary for each dimension, as these choices affect reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper is an empirical evaluation of supervised embedding alignment, not a derivation that reduces to its inputs.

full rationale

The paper's central claims are empirical measurements rather than derivations. It fits a supervised Procrustes alignment using a scraped ground-truth dictionary and then measures English-to-Yoruba word translation precision on an evaluation set derived from WordSim353; the fitted mapping W and the reported precision are not equivalent by construction. The comparison across four conditions is an experimental result, not an identity obtained from definitions, and the interaction between embedding quality and normalization is a data-level finding rather than a consequence of how the variables were defined. No load-bearing self-citations or imported uniqueness theorems appear: the supervised alignment method is cited from Conneau et al. (2017) as external prior work, and the Yoruba resources are cited from Alabi et al. (2020). The only caveat worth noting is that Section 2.1 does not explicitly state that the 161 evaluation pairs are disjoint from the 3,693 training dictionary pairs, both sourced from Alabi et al. (2020); if overlap existed, absolute precisions would be inflated by memorization. However, that would be a benchmark soundness concern, not a demonstrated circularity of the paper's derivation, and the relative comparisons across conditions would not necessarily collapse.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The paper introduces no fitted parameters, no new entities, and no ad hoc mathematical constructs. It borrows pretrained embeddings, a supervised dictionary, and the standard Procrustes method. The main unverified assumptions are the quality differential of the curated embedding and the disjunction between training and evaluation dictionaries.

assumptions (3)

domain assumption The Alabi et al. (2020) curated Yoruba embedding is of higher quality than the fastText Wikipedia Yoruba embedding.
The paper's interpretation of the interaction effect depends on this assumed quality difference, but no independent quality metric is given. Section 2.1 and Section 4.1 rely on the curation described in Alabi et al. (2020).
standard math Procrustes alignment preserves semantic neighborhoods after the linear mapping.
This is the standard assumption of the MUSE method (Conneau et al., 2017), which the paper adopts without proof. The evaluation metric, nearest-neighbor precision, depends on this property.
domain assumption Nearest-neighbor retrieval in the aligned embedding space is a valid measure of word translation quality.
The paper defines translation precision as whether the correct Yoruba word appears among the top-k nearest neighbors. This assumes that translations are nearest neighbors in the aligned space, which is known to be imperfect even for high-resource languages.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Edeflip: Supervised Word Translation between English and Yoruba." pith.science (2026). https://pith.science/paper/BEWMUTPK

@misc{pith2026250613020,
  author       = {Pith},
  title        = {Pith review of: Edeflip: Supervised Word Translation between English and Yoruba},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BEWMUTPK}},
  note         = {Machine review of arXiv:2506.13020}
}

read the original abstract

In recent years, embedding alignment has become the state-of-the-art machine translation approach, as it can yield high-quality translation without training on parallel corpora. However, existing research and application of embedding alignment mostly focus on high-resource languages with high-quality monolingual embeddings. It is unclear if and how low-resource languages may be similarly benefited. In this study, we implement an established supervised embedding alignment method for word translation from English to Yoruba, the latter a low-resource language. We found that higher embedding quality and normalizing embeddings increase word translation precision, with, additionally, an interaction effect between the two. Our results demonstrate the limitations of the state-of-the-art supervised embedding alignment when it comes to low-resource languages, for which there are additional factors that need to be taken into consideration, such as the importance of curating high-quality monolingual embeddings. We hope our work will be a starting point for further machine translation research that takes into account the challenges that low-resource languages face.

Figures

Figures reproduced from arXiv: 2506.13020 by the authors.

**Figure 2.** t-SNE plot for wiki-normalized The interaction effect between embedding qual- [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** t-SNE plot for curated-unnormalized [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: t-SNE plot for curated-normalized ity and normalization is not predicted, though in hindsight understandable. We reason that it is because the English and Yoruba embeddings trained on Wikipedia have a more similar shape from the start, as they are both trained using f…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

10 extracted references · 4 canonical work pages

[1]

Massive vs. Curated Word Embeddings for Low-Resourced Languages. The Case of Yor\`ub\'a and Twi

Jesujoba O. Alabi, Kwabena Amponsah-Kaakyire, David I. Adelani, and Cristina España-Bonet. 2020. http://arxiv.org/abs/1912.02481 Massive vs. curated word embeddings for low-resourced languages. the case of yor\`ub\'a and twi

work page Pith review arXiv 2020
[2]

Mikel Artetxe, Gorka Labaka, and Eneko Agirre. 2019. https://doi.org/10.18653/v1/P19-1494 Bilingual lexicon induction through unsupervised machine translation . In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5002--5007, Florence, Italy. Association for Computational Linguistics

doi:10.18653/v1/p19-1494 2019
[3]

Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017. http://arxiv.org/abs/1607.04606 Enriching word vectors with subword information

arXiv 2017
[4]

Alexis Conneau, Guillaume Lample, Marc'Aurelio Ranzato, Ludovic Denoyer, and Herv \'e J \'e gou. 2017. Word translation without parallel data. arXiv preprint arXiv:1710.04087

arXiv 2017
[5]

Stefan Jansen. 2018. http://arxiv.org/abs/1705.03127 Word and phrase translation with word2vec

work page Pith review arXiv 2018
[6]

Pratik Joshi, Sebastin Santy, Amar Budhiraja, Kalika Bali, and Monojit Choudhury. 2020. https://doi.org/10.18653/v1/2020.acl-main.560 The state and fate of linguistic diversity and inclusion in the NLP world . In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6282--6293, Online. Association for Computational...

doi:10.18653/v1/2020.acl-main.560 2020
[7]

Tomas Mikolov, Edouard Grave, Piotr Bojanowski, Christian Puhrsch, and Armand Joulin. 2017. http://arxiv.org/abs/1712.09405 Advances in pre-training distributed word representations

work page Pith review arXiv 2017
[8]

Tomas Mikolov, Edouard Grave, Piotr Bojanowski, Christian Puhrsch, and Armand Joulin. 2018. Advances in pre-training distributed word representations. In Proceedings of the International Conference on Language Resources and Evaluation (LREC 2018)

work page 2018

Show all 10 references

[9]

URL: " 'urlintro :=

ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before...
[10]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Massive vs. Curated Word Embeddings for Low-Resourced Languages. The Case of Yor\`ub\'a and Twi

Jesujoba O. Alabi, Kwabena Amponsah-Kaakyire, David I. Adelani, and Cristina España-Bonet. 2020. http://arxiv.org/abs/1912.02481 Massive vs. curated word embeddings for low-resourced languages. the case of yor\`ub\'a and twi

work page Pith review arXiv 2020

[2] [2]

Mikel Artetxe, Gorka Labaka, and Eneko Agirre. 2019. https://doi.org/10.18653/v1/P19-1494 Bilingual lexicon induction through unsupervised machine translation . In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5002--5007, Florence, Italy. Association for Computational Linguistics

doi:10.18653/v1/p19-1494 2019

[3] [3]

Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017. http://arxiv.org/abs/1607.04606 Enriching word vectors with subword information

arXiv 2017

[4] [4]

Alexis Conneau, Guillaume Lample, Marc'Aurelio Ranzato, Ludovic Denoyer, and Herv \'e J \'e gou. 2017. Word translation without parallel data. arXiv preprint arXiv:1710.04087

arXiv 2017

[5] [5]

Stefan Jansen. 2018. http://arxiv.org/abs/1705.03127 Word and phrase translation with word2vec

work page Pith review arXiv 2018

[6] [6]

Pratik Joshi, Sebastin Santy, Amar Budhiraja, Kalika Bali, and Monojit Choudhury. 2020. https://doi.org/10.18653/v1/2020.acl-main.560 The state and fate of linguistic diversity and inclusion in the NLP world . In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6282--6293, Online. Association for Computational...

doi:10.18653/v1/2020.acl-main.560 2020

[7] [7]

Tomas Mikolov, Edouard Grave, Piotr Bojanowski, Christian Puhrsch, and Armand Joulin. 2017. http://arxiv.org/abs/1712.09405 Advances in pre-training distributed word representations

work page Pith review arXiv 2017

[8] [8]

Tomas Mikolov, Edouard Grave, Piotr Bojanowski, Christian Puhrsch, and Armand Joulin. 2018. Advances in pre-training distributed word representations. In Proceedings of the International Conference on Language Resources and Evaluation (LREC 2018)

work page 2018