{"id":"c29f770b-8b79-489a-802b-0c8acc501ec9","arxiv_id":"1908.06625","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A semi-supervised bilingual lexicon induction method relaxes the isometric mapping assumption and improves word translation accuracy, especially for distant language pairs, using a GAN, a seed lexicon, and a weak orthogonality loss.","lead":"This paper proposes BLISS, a semi-supervised method for matching words across languages that combines adversarial training, a small seed dictionary, and a soft orthogonality penalty. It also introduces a Gromov-Hausdorff distance-based check of how isometric two embedding spaces are, and reports competitive results on standard benchmarks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Central SOTA claim is internally inconsistent: abstract says 15/18 MUSE pairs, intro says 10/16, and Table 3 shows best on 6/10.","rationale":"I reviewed the strongest claim and the paper's reported results. The claim is empirical: SOTA on 15/18 MUSE pairs. The paper itself reports three different counts: 15/18, 10/16, and 6/10 from Table 3. This is a direct internal contradiction that can be checked by counting the bold entries. It is load-bearing because the headline contribution is the empirical performance. If the 15/18 count is a typo, the central claim is much weaker (best on 6/10 MUSE pairs, and none on VecMap). This warrants a conditional acceptance with a required correction and full results. The orthogonality concern is real but secondary; even if the loss landscape is stable, the paper must still support its stated numbers. I found no other issue that outweighs this internal inconsistency. The review is in good faith: the method may be sound, but the reported results are not trustworthy as written.","tokens_in":13917,"tokens_out":3832,"duration_ms":34690,"concrete_test":"Manually count the bold entries in Table 3: for each of the 10 language pairs, determine the method with the highest accuracy across all groups (the bold number). Compute the number of pairs for which BLISS(R) is bold. Do the same for Table 4. Then compare these counts to 15/18 and 10/16. Also search the appendix and any extra tables for results on all 18 or 16 language pairs. If the counts do not match, ask the authors to provide the full results and correct the abstract.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim, stated in the abstract, is that BLISS 'obtains state of the art results on 15 of 18 language pairs on the MUSE dataset.' The introduction (Section 1) states 'Our best model outperforms the state-of-the-art on 10 of 16 language pairs on the MUSE datasets.' Section 4.3 reports that 'BLISS(R) gives the best accuracy across all baseline methods for 6 out of 10 language pairs' (Table 3). Table 3 contains 10 MUSE language-pair directions; Table 4 contains 2 VecMap pairs, where GeoMM and VecMap(U)++ outperform BLISS. No table or appendix reports 18 or 16 language pairs. The abstract number cannot be reconciled with the presented results, and the two other counts disagree with each other. This is a factual inconsistency in the headline result, not a matter of interpretation. Before the SOTA claim can be accepted, the authors must report the full set of language pairs and specify which count is correct. The orthogonality-instability concern raised by the reader is a secondary issue; the load-bearing problem is that the paper's own numbers do not support its primary empirical conclusion.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies bilingual lexicon induction (BLI) and makes two main contributions. First, it proposes a quantitative diagnostic based on the Gromov–Hausdorff (GH) distance to measure how close two embedding spaces are to being isometric, and it reports correlations between this distance and BLI accuracy. Second, it proposes BLISS, a semi-supervised framework that combines an adversarial distribution-matching loss, a supervised loss on a small seed lexicon, and a weak orthogonality (cyclic consistency) loss, together with iterative Procrustes refinement and a hubness filtering technique. Experiments on MUSE and VecMap datasets compare two instantiations of BLISS against supervised, unsupervised, and semi-supervised baselines. The abstract claims state-of-the-art results on 15 of 18 MUSE language pairs and improved stability from supervision, especially for etymologically distant language pairs.","tokens_in":14157,"tokens_out":10615,"duration_ms":92009,"significance":"If the empirical claims are substantiated, the paper would make a useful contribution: it offers a pre-hoc isometry diagnostic that correlates with BLI performance, and a semi-supervised objective that appears to outperform pure supervised and pure unsupervised baselines on several language pairs, particularly when supervision is scarce. The release of code and the inclusion of a stability analysis are also strengths that support reproducibility. However, the headline state-of-the-art claim is currently supported by inconsistent counts across the abstract, introduction, and results tables, and the relationship between the weak orthogonality loss and the paper's critique of the orthogonality assumption is not directly demonstrated. These issues block acceptance in the current form, but they are addressable in revision.","major_comments":[{"comment":"The central empirical claim is not supported by the reported experiments. The abstract states 'state of the art results on 15 of 18 language pairs on the MUSE dataset'; Section 1 states 'Our best model outperforms the state-of-the-art on 10 of 16 language pairs on the MUSE datasets'; Section 4.3 reports that BLISS(R) gives the best accuracy across all baselines for 6 of 10 language pairs (Table 3), and Table 4 reports 2 VecMap pairs. No table or appendix reports 18 or 16 language pairs; Table 3 contains 10 MUSE direction pairs. Since the SOTA claim in the abstract is the paper's headline result, the authors must either supply the full pair-level results for 18 (or 16) pairs or revise the claim to match the 10 pairs actually presented. The distinction matters because Table 3 also shows GeoMM outperforming BLISS(R) on de-en and en-zh, so the choice of pair set and counting convention changes the verdict.","section":"Abstract; Section 1; Section 4.3; Tables 3 and 4"},{"comment":"The supporting comparison with RCSLS is also inconsistently counted. Section 4.4 says 'BLISS(R) substantially outperforms RCSLS on 6 of 9 language pairs' and points to Table 2. Reading the BLISS(R) and RCSLS rows in Table 2 for the 9 language pairs where RCSLS has non-star values gives BLISS(R) higher on 7 pairs, lower on 1 (en-ru), and tied on 1 (es-fr). Please reconcile this count and make explicit whether ties are counted as outperformance. As it stands, the evidence for the 'particularly well when non-isometric' claim depends on a count that does not match the table.","section":"Section 4.4; Table 2"},{"comment":"The weak orthogonality constraint in Eq. (5) encourages W^T W approximately equal to the identity and is therefore a softened version of the hard orthogonality assumption that motivates the paper. The claim that the supervised and adversarial losses outweigh LW|O exactly when the spaces are non-isometric is an empirical assumption, not demonstrated in the paper. Table 7 shows that the autoencoding loss alone is sensitive to its scaling in the unsupervised setting, but the paper does not report the analogous analysis for BLISS, nor does it report ||I - W^T W|| for the learned BLISS mappings. Without this evidence, the reader cannot tell whether BLISS is actually learning non-orthogonal mappings for high-GH pairs or simply reverting to an orthogonal map with a small regularizer. Please add a sensitivity analysis and, if possible, a direct measurement of the learned W's deviation from orthogonality across language pairs.","section":"Section 3.2, Eqs. (5)-(6); Table 7"}],"minor_comments":[{"comment":"The name is spelled 'Gromov-Hausdroff' in the abstract and 'Gromov Hausdorff' elsewhere; use 'Gromov-Hausdorff' consistently.","section":"Abstract and Section 2"},{"comment":"The second sum is written over xi in Y but uses the target variable; this should be written as a sum over y_i in Y (or equivalent) to avoid a type error.","section":"Eq. (3)"},{"comment":"The column layout mixes per-language-pair values with correlation columns, and the meaning of '*' entries is not defined in the caption. Please add a caption that explains the row and column structure.","section":"Table 2"},{"comment":"Several accuracy advantages are below one point (e.g., en-es 84.3 vs. 84.1); the paper should state whether these differences are stable across runs and ideally report variance or significance.","section":"Table 3"},{"comment":"The phrase 'orthogonal gains' is ambiguous and should be rephrased, for example as 'further gains' or 'additional gains'.","section":"Conclusion"}],"recommendation":"major_revision","confidential_remarks":"The paper's code release and reproducible setup are strengths, but the headline SOTA count must be reconciled before the paper can be accepted. It would be helpful to request the raw per-pair results as part of the revision, since the abstract, introduction, and results sections report different numbers of language pairs."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe thing to know: this paper has a genuinely useful core, but its headline claim is internally inconsistent. The abstract says state of the art on 15 of 18 MUSE pairs; the introduction says 10 of 16; Section 4.3 reports best on 6 of 10, and Table 3 shows 10 pairs. No table lists 18 or 16 pairs. The authors need to say which count is right and show the full set. This is a load-bearing factual issue, not a style quibble.\n\nWhat's new and good: the Gromov-Hausdorff distance is, as far as I know, new to BLI as a diagnostic for isometry. The paper shows GH distance tracks BLI accuracy and correlates with Søgaard et al.'s eigenvector similarity, with a lower bound via bottleneck distance. That is a real contribution even if approximate. The BLISS objective—adversarial distribution matching plus supervised seed loss plus a soft cyclic orthogonality penalty—is a sensible combination, and the stability results (Figure 2, Table 5) demonstrate semi-supervision rescues cases like en-zh where unsupervised training collapses. Code and hyperparameters are public. Credit where due.\n\nSoft spots, in order. The inconsistent SOTA count is the biggest; fixable, but the abstract must match the tables. The weak orthogonality constraint (Eq. 5) softens exactly the assumption the paper critiques; that's okay as a data-driven trade-off, but the paper gives no analysis of when the cyclic loss dominates. This is secondary, since the empirical comparison remains informative. The GH numbers are lower bounds without error bars—treat as suggestive. The hubness filter is simple and mildly useful, not a major innovation.\n\nWho is this for: researchers working on cross-lingual embedding alignment, especially distant language pairs or low-resource settings. It deserves a serious referee; I would not desk-reject. But the authors should be required to reconcile the counts and report all language pairs before the SOTA claim is accepted. Send it to review, with that note.","headline":"Solid semi-supervised BLI method with a useful GH-distance diagnostic, undermined by an internally inconsistent SOTA claim that must be fixed.","tokens_in":14717,"tokens_out":2976,"would_cite":true,"duration_ms":28304,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Relaxing the isometry assumption between embedding spaces yields state-of-the-art bilingual lexicon induction, especially for etymologically distant languages.","keywords":["bilingual lexicon induction","semi-supervised learning","word embeddings","orthogonality assumption","Gromov-Hausdorff distance","adversarial training","hubness filtering","cross-lingual mapping"],"falsifier":"On the same language pairs, train BLISS(R) with the weak orthogonality term removed and with a hard orthogonal projection in its place; if the soft-loss version does not clearly beat the hard-projection baseline on high-GH-distance pairs (e.g., en-ko, en-hi, en-zh), the paper's explanation for its gains collapses. A second check: compute GH-distance estimates on a held-out set of language pairs and test whether their rank order predicts the size of BLISS's advantage over RCSLS; if not, the diagnostic is not doing the predictive work the paper claims.","tokens_in":13723,"feed_emoji":"🔤","tokens_out":9476,"duration_ms":81671,"temperature":0.7,"pith_summary":"The paper sets out to test a hidden premise in bilingual lexicon induction (BLI): that the two word-embedding spaces can be aligned by an orthogonal rotation, i.e., that they are isometric. It argues, with a Gromov-Hausdorff distance estimate, that this isometry assumption weakens as languages become etymologically distant, and that standard orthogonal-mapping methods therefore underperform precisely there. To fix this, the paper proposes BLISS, a semi-supervised framework that jointly optimizes an adversarial distribution-matching loss over all unlabeled embeddings, a supervised alignment loss on a small seed dictionary, and a weak orthogonality constraint that encourages, but does not force, the mapping to be orthogonal. On the MUSE benchmark the paper reports state-of-the-art accuracy on 15 of 18 language pairs, with the clearest gains on non-isometric pairs, and shows that even 50 seed word pairs stabilize training that purely unsupervised methods fail to complete. If correct, this means minimal human-provided word pairs plus a flexible mapping can make bilingual lexicon induction work across typologically diverse languages, not just close relatives.","feed_headline":"Relaxing isometry yields state-of-the-art bilingual lexicons","feed_subtitle":"A small seed dictionary plus a soft orthogonality penalty stabilizes training, winning 15 of 18 MUSE language pairs.","key_machinery":"The load-bearing machinery is the three-term joint loss $L = L_{W|D} + L_{W|S} + L_{W|O}$. $L_{W|D}$ is an adversarial distribution-matching loss (a GAN) that uses all unlabeled word embeddings from both languages; $L_{W|S}$ is a supervised similarity loss on a small seed dictionary, instantiated either as cosine similarity (BLISS(M)) or as the CSLS retrieval criterion (BLISS(R)); $L_{W|O}$ is a weak orthogonality loss that maximizes the similarity between $x$ and $W^T W x$, encouraging $W^T W \\approx I$ without hard projection. The second key object is the Gromov-Hausdorff distance estimate, computed through the bottleneck distance between Vietoris-Rips persistence diagrams of the two embedding spaces, which the paper uses as an a-priori diagnostic of how isometric the spaces are. The claim is that jointly optimizing these three losses lets the model bend the mapping where the spaces are non-isometric and stay close to orthogonal where they are.","core_discovery":"The paper's central claim is that the orthogonality assumption used in most BLI methods—that one embedding space can be rotated rigidly onto the other—does not hold uniformly, and that relaxing it improves translation quality. The supporting evidence is the Gromov-Hausdorff distance between embedding spaces, which the authors approximate with bottleneck distance and show to be low for etymologically close pairs (ru-uk, en-fr) and high for distant pairs (en-ko, en-hi), while correlating strongly with BLI accuracy. BLISS then operationalizes the relaxation by adding a soft autoencoding loss, $L_{W|O} = -\\frac{1}{|X|} \\sum_{x_i} f_a(x_i, W^T W x_i)$, to the supervised and adversarial losses, so $W$ is nudged toward orthogonality only when the data supports it. The claimed result is state-of-the-art performance on 15 of 18 MUSE language pairs and on the VecMap dataset, with the largest margins on high-GH-distance pairs, plus the empirical finding that a small amount of supervision dramatically stabilizes the GAN-based training. This is a claim about both method efficacy and about the geometry of embedding spaces: the failure of isometry is real, measurable, and exploitable.","pith_inferences":["If the GH-distance diagnostic is as predictive as reported, it could be used to decide a priori how much seed supervision a language pair needs, and to extrapolate the method's likely success to resource-poor languages where no seed dictionary exists.","The weak orthogonality constraint resembles a regularizer rather than a geometric prior; a natural extension would be to learn the trade-off between $L_{W|O}$ and the other losses per language pair rather than fixing the coefficient, which the paper does not explore.","The paper's own error analysis (polysemy and antonymy) suggests that the remaining failures are not geometric but semantic; a testable extension is to combine BLISS with sense-aware or contextualized embeddings to see whether the non-isometry error shrinks further.","Because the method works with 50–500 seed pairs, it could serve as a practical warm-start for low-resource machine translation systems, where such lexicons are often available from WordNet-style resources or Wikipedia links.","If the GH-distance diagnostic is as predictive as reported, it could be used to decide a priori how much seed supervision a language pair needs, and to extrapolate the method's likely success to resource-poor languages where no seed dictionary exists."],"forward_implications":["A small seed dictionary (tens to hundreds of word pairs) is enough to convert a failing unsupervised method into a working bilingual lexicon induction system, including for distant language pairs such as English–Chinese and English–Korean.","Purely unsupervised methods are at a systematic disadvantage for fine-grained word choice: they can align semantic clusters but miss distinctions within clusters (e.g., aunt vs. grandmother); adding supervision inside those clusters corrects this.","The Gromov-Hausdorff distance between embedding spaces can serve as an a-priori predictor of how hard a language pair will be for orthogonal-mapping methods, since it correlates strongly with method accuracy and with the empirical non-orthogonality of the learned mapping.","Near-orthogonal rather than strictly orthogonal mappings are sufficient for state-of-the-art BLI; a hard orthogonality projection is not necessary and is sensitive to hyperparameters.","Providing even minimal supervision stabilizes the adversarial training procedure, avoiding the failure-to-converge cases seen in unsupervised GAN-based methods, particularly when embeddings are low-quality or the language pair is distant.","The paper's own error analysis (polysemy and antonymy) suggests that the remaining failures are not geometric but semantic; a testable extension is to combine BLISS with sense-aware or contextualized embeddings to see whether the non-isometry error shrinks further.","Because the method works with 50–500 seed pairs, it could serve as a practical warm-start for low-resource machine translation systems, where such lexicons are often available from WordNet-style resources or Wikipedia links."],"supporting_citations":[{"why":"Supplies the adversarial distribution-matching loss, the CSLS retrieval distance, and the MUSE supervised/unsupervised baselines that BLISS extends and compares against.","marker":"Lample et al. (2018)"},{"why":"Established the orthogonal mapping constraint for BLI, which the paper identifies as the isometry assumption and sets out to relax.","marker":"Xing et al. (2015)"},{"why":"Introduced iterative Procrustes refinement / dictionary expansion, which BLISS uses as post-processing and also appears as a baseline.","marker":"Artetxe et al. (2017)"},{"why":"Argued that orthogonality is necessary for self-consistency of bilingual mappings, motivating the paper's weak orthogonality term.","marker":"Smith et al. (2017)"},{"why":"Provided evidence that unsupervised BLI fails for distant languages and proposed the eigenvector-similarity metric that the GH-distance diagnostic is compared against.","marker":"Søgaard et al. (2018)"},{"why":"Proposed RCSLS, the supervised retrieval-criterion baseline that one BLISS instantiation (BLISS(R)) adopts as its supervised loss and outperforms.","marker":"Joulin et al. (2018)"},{"why":"Showed the bottleneck distance between persistence diagrams lower-bounds the Gromov-Hausdorff distance, the approximation the paper uses to quantify isometry.","marker":"Chazal et al. (2009)"},{"why":"Defined hubness in high-dimensional nearest-neighbor spaces, the basis for the paper's hubness filtering in iterative dictionary expansion.","marker":"Radovanović et al. (2010)"}],"fun_headline_variants":["Semi-supervision beats rigid rotation for bilingual lexicons","Relaxing isometry assumption improves bilingual lexicon induction","Small seeds and soft orthogonality win 15 of 18 MUSE language pairs","Isometry fails for distant languages: measure it, exploit it","Bilingual lexicons without isometry: semi-supervised success"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the three losses in equation (6) can be jointly optimized so that the soft orthogonality penalty is outweighed by the supervised and adversarial losses exactly when the spaces are non-isometric, without causing training instability or harmful local optima; the paper provides no theoretical guarantee of this balance, only empirical results.","fun_headline_variants_meta":{"raw":{"variants":["Semi-supervision beats rigid rotation for bilingual lexicons","Relaxing isometry assumption improves bilingual lexicon induction","Small seeds and soft orthogonality win 15 of 18 MUSE language pairs","Isometry fails for distant languages: measure it, exploit it","Bilingual lexicons without isometry: semi-supervised success"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000479,"raw_usage":{"total_tokens":2392,"prompt_tokens":985,"completion_tokens":1407,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":601,"completion_tokens_details":{"reasoning_tokens":1318}},"tokens_in":601,"tokens_out":1407,"duration_ms":11422,"temperature":1.0,"reasoning_tokens":1318,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T12:38:10.804353+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On the same language pairs, train BLISS(R) with the weak orthogonality term removed and with a hard orthogonal projection in its place; if the soft-loss version does not clearly beat the hard-projection baseline on high-GH-distance pairs (e.g., en-ko, en-hi, en-zh), the paper's explanation for its gains collapses. A second check: compute GH-distance estimates on a held-out set of language pairs and test whether their rank order predicts the size of BLISS's advantage over RCSLS; if not, the diagnostic is not doing the predictive work the paper claims.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the adversarial distribution-matching loss, the CSLS retrieval distance, and the MUSE supervised/unsupervised baselines that BLISS extends and compares against."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduced iterative Procrustes refinement / dictionary expansion, which BLISS uses as post-processing and also appears as a baseline."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Argued that orthogonality is necessary for self-consistency of bilingual mappings, motivating the paper's weak orthogonality term."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Showed the bottleneck distance between persistence diagrams lower-bounds the Gromov-Hausdorff distance, the approximation the paper uses to quantify isometry."}],"review_version":1}