Understanding Undesirable Word Embedding Associations

David Duvenaud; Graeme Hirst; Kawin Ethayarajh

REVIEW 4 major objections 5 minor 16 references

Understanding Undesirable Word Embedding Associations

T0 review · 4 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read For matrix-factorization embeddings, subspace-projection debiasing provably matches unbiased training, and WEAT overestimates bias.

desk verdict A solid, important paper whose WEAT critique and RIPA measure are genuinely useful, but whose central debiasing theorem rests on an unproved imported lemma and an overclaimed equivalence. read the letter →

arxiv 1908.06361 v1 pith:GNYZJ4E2 submitted 2019-08-18 cs.CL

Kawin Ethayarajh , David Duvenaud , Graeme Hirst This is my paper · ORCID

classification cs.CL

keywords wordembeddingsgenderbiasdebiasingWEATRIPAmatrixfactorizationSGNSGloVe

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper aims to establish that the standard post hoc debiasing of word embeddings can be placed on a theoretical footing. It claims that for any embedding model that implicitly factorizes a word-context matrix, subtracting the projection onto the bias subspace is, under stated conditions, equivalent to training on an unbiased corpus. It also claims that WEAT, the most common association test for embeddings, systematically overestimates bias because of a hidden equal-frequency requirement and a maximal effect size. The paper then derives a new association measure, RIPA, and uses it to show that skip-gram with negative sampling leaves most words as gendered as they are in the corpus but amplifies genderedness for stereotyped and gender-specific words. A reader should care because this clarifies when debiasing is justified and offers a more reliable way to measure associations.

What carries the argument

The central object is the bias subspace $B = \operatorname{span}(\{\vec x - \vec y \mid (x,y)\in S\})$ and the operation of replacing a word vector $\vec w$ by $\vec w - \operatorname{proj}_B \vec w$. The load-bearing identity is that under perfect reconstruction the context matrix is a scalar multiple of the word matrix, $C = \lambda W$; this turns orthogonality in word space into unbiasedness of the reconstructed co-occurrence matrix. RIPA is the inner product $\beta(\vec w;\vec b)=\langle \vec w,\vec b\rangle$, where $\vec b$ is the first principal component of the difference vectors, and for SGNS that inner product reduces to a constant multiple of $\log[p(w\mid x)/p(w\mid y)]$.

What would settle it

On a model with near-zero reconstruction error, compute the least-squares scalar $\lambda$ minimizing $\|C - \lambda W\|_F$; if the relative residual $\|C - \lambda W\|_F/\|C\|_F$ is not small while reconstruction error is small, then the scalar-multiple assumption behind the Debiasing Theorem and RIPA's information-theoretic form is violated and the central claims would not apply to that model.

Watch

Extended reading notes

Core claim

For any embedding model that implicitly factorizes a word-context matrix, the paper claims that subtracting from each word vector its projection onto the bias subspace $B = \operatorname{span}(\{\vec x - \vec y \mid (x,y)\in S\})$ yields a word matrix whose reconstructed co-occurrence matrix is unbiased with respect to the word pairs in $S$. The paper also claims that WEAT, the cosine-similarity test most used for word-embedding associations, systematically overestimates bias: its neutrality condition implicitly requires equal corpus frequencies of the attribute words, and its effect size is always maximal for two target words. It introduces RIPA, $\beta(\vec w;\vec b) = \langle \vec w,\vec b\rangle$ with $\vec b$ the first principal component of the defining difference vectors, which for SGNS equals a constant times $\log[p(w\mid x)/p(w\mid y)]$ under perfect reconstruction. Experiments show that SGNS leaves the genderedness of most neutral words unchanged but amplifies the gender association of stereotyped and gender-specific words, and that an unsupervised rule for choosing which words to debias preserves gender-appropriate analogies better than the previously used identification list.

Load-bearing premise

The central proofs assume that, under perfect reconstruction, the context matrix is a scalar multiple of the word matrix ($C = \lambda W$); if that proportionality fails, the equalities that carry the debiasing guarantee and RIPA's interpretation do not follow.

Editorial extensions

If this is right

For SGNS and GloVe, the subspace-projection debiasing method carries a theoretical guarantee, provided vectors are not normalized first and the bias subspace is the span of difference vectors rather than a principal-component subspace.
WEAT results should be re-interpreted: unequal corpus frequencies of attribute words can make neutral words appear biased, and with two target words the effect size is always at its maximum in one direction.
RIPA provides an interpretable and bounded association score; for SGNS its value is proportional to $\log[p(w\mid x)/p(w\mid y)]$, so it can be compared across relation vectors.
For SGNS, most words are not made more gendered by the embedding model, but gender-stereotyped and gender-specific words are amplified relative to the corpus.
An unsupervised criterion—debias a word only when its gender-defining RIPA is smaller than its bias-defining RIPA—preserves almost all gender-appropriate analogies and blocks most gender-biased ones.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Read strictly, the paper proves unbiasedness of the reconstructed matrix, not that debiased vectors equal vectors trained directly on an unbiased corpus; the two differ unless additional assumptions hold.
The same bias-subspace and RIPA constructions apply to any ordered word-pair association, so the method can be carried over to other protected attributes by changing only the defining pairs; the paper states this extension for associations generally but does not test it.
Because normalization is shown to break the guarantee, bias measurements that normalize embeddings before comparing them may systematically miss the association the model encodes; future tests could use unnormalized inner products and corpus frequency information.
RIPA's dependence on co-occurrence counts suggests that a proper significance test for association would need corpus counts, not just embedding vectors, to separate spurious from real associations.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Read this paper. It does two things well: it dismantles WEAT and it proposes RIPA, a more interpretable association measure. The WEAT critique is the strongest part. Proposition 2--that with two target words the effect size is always maxed out at 2--is a clean proof that the test's significance numbers are meaningless. Proposition 1, the frequency dependence, is also a genuine flaw that hasn't been articulated this clearly. RIPA is a useful practical tool, and the information-theoretic decomposition for SGNS is elegant.

The weak spot is the debiasing theorem. It depends on Lemma 2 from the authors' 2018 paper, which says C=λW under perfect reconstruction. That lemma is not proved here, and the justification offered--a symmetric map whose 'relative geometry' forces a single eigenvalue--is a sketch, not a proof. If C is not a scalar multiple of W, Eq. 2 fails and the theorem collapses. That's load-bearing, so the conditions need to be stated and the lemma proved in this paper.

The abstract's 'equivalent to training on an unbiased corpus' also goes beyond what the proof shows. The theorem establishes that the reconstructed matrix is unbiased, not that the debiased vectors are identical to ones you'd get from training on an unbiased corpus. That's a meaningful overstatement.

Two more minor points. The claim that WEAT systematically overestimates bias is mostly supported for the two-target-word case; the paper should be explicit that the general case is an extrapolation. And the experiments set λ and α by hand (λ←1, α←−1) rather than deriving them, which makes the quantitative claims in Table 2 more fragile than the p-values suggest.

Overall, the paper is worth engaging seriously. The WEAT critique alone earns it a reading, and RIPA will be cited. It needs revision on the lemma and the equivalence claim before the central theorem is taken as established. I'd send it out.

Referee Report

4 major / 5 minor

Summary. The paper addresses the theoretical foundations of measuring and removing gender bias in word embeddings. It argues that for embedding models that implicitly factorize a word-context matrix (e.g., SGNS, GloVe), post-hoc debiasing by subspace projection is, under certain conditions, equivalent to training on an unbiased corpus. It then proves two properties of the WEAT association test: that an unbiased word can appear biased when attribute words have unequal corpus frequencies (Proposition 1), and that the effect size is maximal when each target set contains a single word (Proposition 2). Based on the debiasing theory, the authors introduce a new association measure, RIPA, and use it to compare corpus-level and embedding-space gender associations. Experiments on SGNS show that neutral words are not, on average, made more gendered, while gender-stereotyped and gender-specific words are amplified. The paper also proposes an unsupervised heuristic for selecting gender-appropriate words to leave out of debiasing.

Significance. If the central theorems hold, the paper would provide the first theoretical justification for the widely used Bolukbasi et al. (2016) subspace-projection debiasing method, and it would supply a principled alternative to WEAT in RIPA. The empirical finding that SGNS amplifies gendered associations only for already-gendered words is also of practical interest. The paper is commendably explicit about definitions and attempts formal proofs, and the Table 1 demonstration of WEAT's manipulability is striking. However, the main theoretical results depend on an imported lemma from prior work whose conditions are not stated or proved in this manuscript, and the abstract's 'equivalent to training on an unbiased corpus' formulation is stronger than the theorem actually establishes.

major comments (4)

[Section 3, proof of Debiasing Theorem] The proof of the Debiasing Theorem is load-bearing and hinges on Lemma 2 of Ethayarajh et al. (2018), which asserts that under perfect reconstruction there exists λ∈R with C=λW. This lemma is not proved in the manuscript, and the in-text justification (a symmetric map A from W to C whose 'relative geometry' forces a single eigenvalue) is a heuristic rather than a proof. If C is not a scalar multiple of W, Eq. (2) fails, and the conclusion that orthogonality of w_d to x−y implies unbiasedness of the reconstructed matrix does not follow. The authors should either prove the lemma in an appendix, state its precise hypotheses, or restrict the theorem to the cases where the lemma is known to hold.
[Abstract and Section 3] The claim that post-hoc debiasing is 'equivalent to training on an unbiased corpus' goes beyond what is proved. The theorem shows that the reconstructed matrix W_d C^T has equal entries for (w,x) and (w,y) for every w not in S, i.e., that this matrix is unbiased in the sense of Definition 1. It does not show that the debiased word vectors W_d equal the word matrix obtained by factorizing an unbiased corpus, nor that the reconstruction error is the same. The equivalence claim in the abstract and in the text should be weakened to something like 'produces a reconstructed word-context matrix that is unbiased with respect to S'.
[Section 5.1, Eq. (5), and Section 6.2, Table 2] The information-theoretic form of RIPA in Eq. (5) relies on the same C=λW lemma and on the assumption that the squared word-embedding norm is linear in log frequency with model-specific constants α and λ. In Section 6.2, these constants are set by hand (λ←1, α←−1) with only a brief expectation or a minimization heuristic; no sensitivity analysis is reported. The significance tests in Table 2 (p<0.001 vs. p=0.84) compare corpus and embedding genderedness using these hand-set constants, so the quantitative conclusions are conditional on unvalidated parameter choices. The authors should provide a sensitivity analysis or justify the constants from the model itself.
[Section 4, Proposition 2, and Abstract] Proposition 2 shows that when each target set contains exactly one word and each attribute set contains exactly one word, the effect size is exactly 2 in one direction and −2 in the other, because the denominator of the effect size is the standard deviation of two values. This is a correct observation, but the abstract's global conclusion that 'WEAT systematically overestimates bias' is not established by this proposition. The paper does not show that the effect size is inflated for realistic target sets of eight or more words, nor that the permutation-test p-values are anti-conservative in that regime. The claim should be scoped to the special case analyzed, with the manipulation in Table 1 presented as an illustrative construction rather than a general property.

minor comments (5)

[Table 1 caption] The caption states that 'In both cases, the associations are highly statistically significant,' but the rows labeled 'inconclusive' have p=0.5 and are not significant; the caption should refer only to the significant rows or explain the role of the inconclusive rows.
[Section 6.2, Eq. (7)] The notation for g and \hat{g} is inconsistent between the text and the equation: the text says g denotes the RIPA in embedding space and \hat{g} the perfect-reconstruction (corpus) value, but Eq. (7) defines g as the embedding-space quantity and \hat{g} as the corpus quantity, and Table 2's column headings further conflate the two. Please make the notation uniform.
[Section 4, Proposition 1] The derivation of Eq. (4) implicitly assumes that the word norms \|x\| and \|y\| are positive and that the linear relationship \|w\|^2 = α1 log p(w) + α2 holds with the same constants for all words; these assumptions should be stated explicitly.
[Section 5.1, Eq. (5)] The algebraic steps from the first line of Eq. (5) to the second are not shown; in particular, the appearance of the factor 1/sqrt(λ) is not derived, making the expression difficult to verify. A brief derivation or an intermediate line would help.
[Section 3] The sentence 'For a detailed explanation, we refer the reader to the proof of that lemma' is not adequate for a self-contained paper when the lemma is load-bearing; the lemma should at least be restated precisely, and ideally proved in an appendix.

Circularity Check

1 steps flagged · score 4.0 of 10

The Debiasing Theorem and RIPA both rest on the self-cited, unproved C=λW lemma; if that lemma fails, Eq. (2) and Eq. (5) collapse.

uniqueness imported from authors [Section 3 (Debiasing Theorem proof); Section 5.1 (Eq. 5)]
"From Lemma 2 of Ethayarajh et al. (2018), we also know that under perfect reconstruction, ∃ λ∈R, C = λW. For a detailed explanation, we refer the reader to the proof of that lemma. In short, if a linear word analogy holds over S (i.e., the word pairs have the same difference vector), then there exists a real symmetric matrixA that maps W to C. A’s eigenvectors form a basis for the word space but A can only have non-distinct eigenvalues if the relative geometry of the word space is to be preserved."

The proof's key step is an imported uniqueness result from the authors' own prior work, with no proof or hypotheses stated here. The 'in short' explanation is a heuristic symmetry argument, not a derivation. If C is not a scalar multiple of W, Eq. (2) fails, so orthogonality of the debiased vector no longer implies unbiasedness of the reconstructed matrix W_dC^T, and the claimed equivalence to training on an unbiased corpus does not follow. The same lemma is the sole bridge in Section 5.1 that converts RIPA into a corpus-side information-theoretic quantity (Eq. 5). Thus the paper's central theoretical claims reduce to a self-cited lemma whose conditions are unspecified, making the load-bearing step a self-citation rather than an independently established result.

full rationale

The main theoretical result of the paper is not a restatement of its own definitions, but its proof is not self-contained: the Debiasing Theorem and the RIPA interpretation both rely on Lemma 2 of Ethayarajh et al. (2018), a prior paper by the same authors, which asserts that under perfect reconstruction the context matrix is a scalar multiple of the word matrix. The present text gives only a brief symmetry heuristic for this lemma and does not state its assumptions or proof; if the lemma fails, the identity ⟨w, x_c−y_c⟩=λ⟨w, x−y⟩ fails, and the theorem's conclusion is unsupported. This is a genuine self-citation that is load-bearing, so the score is elevated. However, the paper also contains independent content: Proposition 2's criticism of WEAT effect sizes follows directly from the definition of the effect size, and the empirical RIPA comparison is a calibrated measurement rather than a prediction forced by construction. The hand-chosen constants λ←1 and α←−1 are acknowledged estimates, and they affect the quantitative magnitudes in Table 2, but they are not fitted to the word categories whose amplification is claimed, so they do not constitute a fitted-input-called-prediction circularity under the hard rules. The abstract's phrase 'equivalent to training on an unbiased corpus' overstates the theorem, which actually proves unbiasedness of the reconstructed matrix, but that is a precision gap rather than a circular step. Overall, the central derivation is substantially dependent on an unproved self-cited uniqueness lemma, warranting a moderate circularity score of 4.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The central claims rest on lemmas and constants imported from the authors' prior work (Ethayarajh et al. 2018) and on empirical regularities such as the norm-log probability relation. The headline empirical result uses two hand-set or fitted constants, λ and α. No new physical or metaphysical entities are introduced.

free parameters (2)

lambda (λ) = 1
Set to 1 in Section 6.2 with 'we expect λ≈1 in practice'. It scales the corpus RIPA values ĝ, so the reported amplification of gender associations depends on this choice.
alpha (α) = -1
Section 6.2: α is set to -1 because it minimizes the difference between ||x−y|| and its information-theoretic interpretation over the gender-defining pairs. This is a fitted constant and directly controls the corpus genderedness scale, hence the 'SGNS amplifies' conclusion.

assumptions (5)

domain assumption Under perfect reconstruction, the word matrix W and context matrix C satisfy C = λW for some real λ (Lemma 2 of Ethayarajh et al. 2018).
Invoked in Section 3 proof of Debiasing Theorem and Section 5.1 RIPA derivation. Without it, equation (2) does not follow.
domain assumption The squared norm of an SGNS vector is linear in the log probability of the word: ||w||^2 = α1 log p(w) + α2.
Used in Proposition 1 (Section 4) to conclude that equal association under WEAT requires p(x)=p(y). This is an empirical regularity from prior work, not an exact theorem.
domain assumption The word pairs used to define the bias subspace form a linear word analogy, i.e. their difference vectors are approximately equal, and the same csPMI constant applies.
Assumed when interpreting RIPA as log[p(w|x)/p(w|y)] and when taking the first principal component as the relation vector (Section 5.1).
domain assumption No reconstruction error for the theoretical equivalence; empirical results treat reconstruction error as zero-centered noise.
The Debiasing Theorem and RIPA derivation are stated under perfect reconstruction (Sections 3 and 5.1). Real embeddings only approximate this.
domain assumption Reconstruction error for a word pair follows a zero-centered normal with variance decreasing in co-occurrence frequency (from Ethayarajh et al. 2018).
Used in Section 5.2 to argue WEAT's statistical significance is 'disingenuous'.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Understanding Undesirable Word Embedding Associations." pith.science (2026). https://pith.science/paper/GNYZJ4E2

@misc{pith2026190806361,
  author       = {Pith},
  title        = {Pith review of: Understanding Undesirable Word Embedding Associations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GNYZJ4E2}},
  note         = {Machine review of arXiv:1908.06361}
}

abstract

Word embeddings are often criticized for capturing undesirable word associations such as gender stereotypes. However, methods for measuring and removing such biases remain poorly understood. We show that for any embedding model that implicitly does matrix factorization, debiasing vectors post hoc using subspace projection (Bolukbasi et al., 2016) is, under certain conditions, equivalent to training on an unbiased corpus. We also prove that WEAT, the most common association test for word embeddings, systematically overestimates bias. Given that the subspace projection method is provably effective, we use it to derive a new measure of association called the $\textit{relational inner product association}$ (RIPA). Experiments with RIPA reveal that, on average, skipgram with negative sampling (SGNS) does not make most words any more gendered than they are in the training corpus. However, for gender-stereotyped words, SGNS actually amplifies the gender association in the corpus.

Figures

Figures reproduced from arXiv: 1908.06361 by the authors.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

16 extracted references · 6 canonical work pages

[1]

URL: " 'urlintro :=

ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...
[2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
[3]

Sanjeev Arora, Yuanzhi Li, Yingyu Liang, Tengyu Ma, and Andrej Risteski. 2016. A latent variable model approach to PMI -based word embeddings. Transactions of the Association for Computational Linguistics, 4:385--399

work page 2016
[4]

Yoshua Bengio, R \'e jean Ducharme, Pascal Vincent, and Christian Jauvin. 2003. A neural probabilistic language model. Journal of Machine Learning Research, 3(Feb):1137--1155

2003
[5]

Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In Advances in Neural Information Processing Systems, pages 4349--4357

2016
[6]

Aylin Caliskan, Joanna J Bryson, and Arvind Narayanan. 2017. Semantics derived automatically from language corpora contain human-like biases. Science, 356(6334):183--186

2017
[7]

Ronan Collobert and Jason Weston. 2008. A unified architecture for natural language processing: Deep neural networks with multitask learning. In Proceedings of the 25th International Conference on Machine Learning, pages 160--167. ACM

work page 2008
[8]

Kawin Ethayarajh, David Duvenaud, and Graeme Hirst. 2018. Towards understanding linear word analogies. arXiv preprint arXiv:1810.04882

arXiv 2018

Show all 16 references

[9]

Hila Gonen and Yoav Goldberg. 2019. Lipstick on a pig: Debiasing methods cover up systematic gender biases in word embeddings but do not remove them. arXiv preprint arXiv:1903.03862

2019 arXiv
[10]

Thomas K Landauer and Susan T Dumais. 1997. A solution to Plato's problem: The latent semantic analysis theory of acquisition, induction, and representation of knowledge. Psychological review, 104(2):211

1997
[11]

Omer Levy and Yoav Goldberg. 2014. Neural word embedding as implicit matrix factorization. In Advances in Neural Information Processing Systems, pages 2177--2185

2014
[12]

Omer Levy, Yoav Goldberg, and Ido Dagan. 2015. Improving distributional similarity with lessons learned from word embeddings. Transactions of the Association for Computational Linguistics, 3:211--225

2015
[13]

Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems, pages 3111--3119

2013
[14]

David Mimno and Laure Thompson. 2017. The strange geometry of skip-gram with negative sampling. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2873--2878

2017
[15]

Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. GloVe : Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1532--1543

2014
[16]

Jieyu Zhao, Yichao Zhou, Zeyu Li, Wei Wang, and Kai-Wei Chang. 2018. Learning gender-neutral word embeddings. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4847--4853

2018

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

URL: " 'urlintro :=

ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...

[2] [2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

[3] [3]

Sanjeev Arora, Yuanzhi Li, Yingyu Liang, Tengyu Ma, and Andrej Risteski. 2016. A latent variable model approach to PMI -based word embeddings. Transactions of the Association for Computational Linguistics, 4:385--399

work page 2016

[4] [4]

Yoshua Bengio, R \'e jean Ducharme, Pascal Vincent, and Christian Jauvin. 2003. A neural probabilistic language model. Journal of Machine Learning Research, 3(Feb):1137--1155

2003

[5] [5]

Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In Advances in Neural Information Processing Systems, pages 4349--4357

2016

[6] [6]

Aylin Caliskan, Joanna J Bryson, and Arvind Narayanan. 2017. Semantics derived automatically from language corpora contain human-like biases. Science, 356(6334):183--186

2017

[7] [7]

Ronan Collobert and Jason Weston. 2008. A unified architecture for natural language processing: Deep neural networks with multitask learning. In Proceedings of the 25th International Conference on Machine Learning, pages 160--167. ACM

work page 2008

[8] [8]

Kawin Ethayarajh, David Duvenaud, and Graeme Hirst. 2018. Towards understanding linear word analogies. arXiv preprint arXiv:1810.04882

arXiv 2018