Interpretable Syntactic Representations Enable Hierarchical Word Vectors

Biraj Silwal

REVIEW 5 major objections 5 minor 18 references

Interpretable Syntactic Representations Enable Hierarchical Word Vectors

T0 review · 5 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that dense word vectors can be projected onto an 8-dimensional syntactic subspace whose coordinates correspond to the eight parts of speech, producing compact, human-readable vectors that, when combined with the…

desk verdict Original post-processing idea with a circular WordNet-based interpretability test and overclaimed performance gains; worth a look but needs major revision. read the letter →

arxiv 2411.08384 v1 pith:PIAWOF44 submitted 2024-11-13 cs.CL cs.LG

Biraj Silwal This is my paper · ORCID

classification cs.CLcs.LG

keywords syntacticrepresentationwordembeddingsinterpretabilityhierarchicalvectorspart-of-speechNetpost-processingincrementallearning

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that word embeddings, which are normally dense and opaque, can be rewritten as short 8-number vectors where each number measures how strongly a word behaves like one of the eight parts of speech. It derives these numbers by averaging all WordNet word vectors per part of speech and projecting every word onto that subspace. It then concatenates or weights the original embedding with this compact representation to form hierarchical vectors. The paper claims these hierarchical vectors beat the plain pretrained vectors on average across several downstream tasks, while the 8-dimensional representations themselves align with human judgments of a word's part of speech. The point is that interpretability does not have to come at the cost of task performance.

What carries the argument

The load-bearing object is the syntactic subspace defined by the transition matrix C, whose eight rows are the mean pretrained vectors of all WordNet words belonging to each part of speech. The projection S_v = (C^T)^+ x_v, using the Moore–Penrose inverse, maps a full word vector to an 8-dimensional coordinate vector whose relative values are claimed to encode the word's part of speech. The hierarchical vectors are composite representations: overcomplete ones formed by the Kronecker product of the syntactic and original vectors, and weighted ones formed by an element-wise weighted average. The subspace construction, the projection step, and the weighting rule together carry the argument.

What would settle it

Take a random half of the WordNet words per part of speech, build the eight average vectors from them, project the held-out half, and compare the largest coordinate's label against WordNet's label; if accuracy for held-out words drops far below the in-sample accuracy reported in Table 4, the subspace is overfit to the particular word list rather than capturing general syntactic regularities. A weaker but also decisive test is to run the interpretable representation on a balanced corpus labeled by a statistical tagger and measure agreement on words that are absent from WordNet.

Watch

Extended reading notes

Core claim

The central claim is that the syntactic regularities hidden inside ordinary dense word vectors can be isolated as an 8-dimensional subspace whose axes are the eight parts of speech, and that these reduced coordinates are both meaningful to humans and useful for learning. The paper constructs a transition matrix C whose rows are the average word vectors of nouns, verbs, adjectives, adverbs, pronouns, prepositions, conjunctions, and interjections, taken from WordNet words that also appear in the pretrained vocabulary. Any word vector is then projected onto this subspace using the Moore–Penrose inverse, yielding an 8-dimensional syntactic representation. Three normalizations of this representation are defined: absolute, interpretable (scaled to 0.5–1 with 1 marking the dominant part of speech), and L2-normalized. These representations are combined with the original vector either by Kronecker product, giving an overcomplete hierarchical vector, or by weighting the original vector's coordinates with the syntactic scores, giving a weighted hierarchical vector of the same dimension as the input. On news classification, question classification, noun-phrase bracketing, sentiment analysis, and attribute-discrimination tasks, the hierarchical vectors match or exceed the base Word2Vec and GloVe vectors on average, although statistical significance is demonstrated for only a subset of tasks.

Load-bearing premise

The method assumes that averaging all WordNet word vectors for each part of speech yields a subspace in which any word's projection has coordinates whose relative sizes reliably indicate its part of speech, and it never checks this premise against words outside WordNet or against an independent POS oracle.

Editorial extensions

If this is right

If the claim holds, any pretrained static embedding can be post-processed into an 8-dimensional syntactic description without retraining on raw text, making large vocabularies cheap to inspect and compare.
The hierarchical vectors give a direct way to inject syntactic information into downstream classifiers: a classifier can start from a small, interpretable feature set and then enlarge it to the full vector, matching human-like incremental learning.
The interpretable representations can be used to label a word's dominant part of speech by reading off the coordinate closest to 1, yielding a lightweight POS tagger for vocabulary not covered by an external tagger.
The weighted hierarchical vectors preserve the original dimension, so they can be dropped into existing models that expect pretrained embeddings without changing architecture or training pipelines.
The overcomplete variant, despite its very large condition number, still matches base performance on several tasks, suggesting that the added syntactic channels carry signal rather than pure noise.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The same subspace idea could be tested on contextual embeddings from masked language models, where the projection might reveal whether POS information is as linearly recoverable as it appears in static vectors; the paper does not attempt this.
Because the subspace is built from WordNet averages, a natural stress test is to replace WordNet with a POS-tagged corpus and see whether the eight axes change; this would separate the method from the particular lexical resource.
The reported loss on word-similarity benchmarks, combined with gains on classification tasks, suggests the syntactic projection removes semantic nuance; a practical extension would be to blend the syntactic and original vectors with a tunable weight rather than fix the composition rule.
The extremely high condition number of the overcomplete vectors hints that the Kronecker product amplifies rare-axis noise; applying a spectral truncation before concatenation might recover the similarity scores without hurting classification.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The core idea is genuinely fresh: project pretrained word vectors onto eight WordNet-derived part-of-speech centroids, then combine those 8-dimensional projections with the original vectors via Kronecker products or weighted averages. It is cheap, simple, and the qualitative top-word lists show recognizable syntactic groupings (the pronoun and interjection dimensions are particularly telling). The author also deserves credit for an honest limitations section, including the polysemy issue and the degraded word-similarity scores, and for attempting statistical significance tests on the downstream results.

But the central claims are not established. The interpretability validation is circular: the POS subspace is built by averaging WordNet words per POS, and then the same WordNet words are projected onto that subspace and 'classified' by the maximal coordinate. High accuracy is expected by construction, even if the pretrained embeddings carry no generalizable POS signal. The independent eight-class experiment in Appendix G gives 58.78%, which is closer to what you'd expect from an honest held-out evaluation, but there is no indication that those words were excluded from the subspace construction.

Beyond circularity, the math is sloppy in load-bearing places. Equations 2-3 have mismatched dimensions: C is defined as (V∩W)×8, so C^T·S cannot equal a row of X, and the projection formula as written cannot produce a 1×8 syntactic representation. Equations 5-6 normalize to [0.5, 1.5], yet the text says the range is [0.5, 1] and the classification rule relies on the value 1 marking the class. These are not minor typos; they affect how the method is supposed to work. Also, Table 1 shows only WOA (and GOL by 0.03 points) beating the base on average, so the abstract's 'hierarchical vectors outperform' is a clear overstatement. No code or data is provided, which makes it hard to separate presentation error from substantive confusion.

Who is this for? A researcher working on interpretable embedding post-processing might find a seed of an idea, but they would need to redo the evaluation and fix the math. The paper is not ready as is, but it deserves serious referee time rather than a desk reject because the idea is original and the flaws are fixable. I would advise a major-revision outcome: use a held-out POS tagger or non-WordNet labels for the interpretability test, correct the equations and normalization, release code, and temper the performance claims.

Referee Report

5 major / 5 minor

Summary. The paper proposes a post-processing method that projects pretrained word embeddings (Word2Vec and GloVe) onto an 8-dimensional 'syntactic subspace' whose axes are derived from average vectors of WordNet words for each part of speech. The resulting Syntactic Representations are then combined with the original vectors, either by Kronecker product (overcomplete) or by weighting, to produce Hierarchical Vectors. The paper evaluates these vectors on seven downstream tasks and word-similarity benchmarks, reports accuracy for POS classification using WordNet labels, gives qualitative top-word examples, and argues that the representations are interpretable and that the hierarchical vectors perform at least as well as, and sometimes better than, the base embeddings.

Significance. A reliable method for converting dense embeddings into low-dimensional, interpretable syntactic axes, with no loss on downstream tasks, would be a useful contribution. The paper includes a broad benchmark suite, a statistical significance test, and qualitative word lists that are suggestive. However, the significance is substantially undercut by three issues: the primary interpretability experiment is circular, the abstract's blanket performance claim is not supported by the per-variant results in Table 1, and the core equations contain notation and normalization errors. These problems affect the central claims of the paper, not just its presentation.

major comments (5)

[Section 7.1 and Section 3] The word-classification experiment is circular as a validation of the interpretability claim. The syntactic subspace is built in Section 3 from averages of WordNet words for each POS, and the oracle for the classification experiment in Section 7.1 is the same WordNet resource. A test word's projection is computed from directions that include that word's own POS class average, so high accuracy on this construction is expected even if the embeddings contain no POS signal beyond the class centroids themselves. The paper needs an evaluation on held-out words (for example, a split of WordNet synsets or leave-one-out) or, preferably, on POS labels from an independent annotation source such as a tagged corpus. The Google Text Normalization experiment in Appendix G is not a substitute: it reports only a confusion matrix and an accuracy of 58.78%, with no description of how the 36 tags were conflated, no baseline, and no held-out split.
[Abstract and Section 6.2, Table 1] The abstract's claim that 'subsequent hierarchical vectors outperform the original vectors in benchmark tests' is contradicted by Table 1. Only WOA (81.79 vs 80.49 for Word2Vec) and GOL (83.37 vs 83.34 for GloVe) have higher averages than their base vectors; the other ten variants all have lower averages. Even WOA and GOL do not improve uniformly across tasks: WOA loses on NP bracketing and discriminative attributes, and GOL's average margin is only 0.03 percentage points. The statistical significance table (Table 2) does not state which hierarchical variant enters the comparison, and for NP bracketing and sentiment the reported p-values (0.75 and 0.42/0.97) give no support. The paper should replace the blanket claim with a per-variant claim restricted to the tasks and variants for which the improvement is reliable, and it should address the substantial drops on word similarity in Table 7 (for example, WOA on SimLex-999 falls from 44.20 to 36.89, and GOA from 40.83 to 29.81) rather than dismissing them via Faruqui et al.
[Appendix A.2, Eqs. (5)-(6)] The stated normalization does not produce values in [0.5, 1]. With I_intermediate = (S_i - min S)/(max S_i - min S), the maximum of I_intermediate is 1, so Eq. (5) gives I_i = 1.5 at that coordinate. This contradicts both the description 'scales them to a range between 0.5 and 1' and the claim in Section 7.1 that the predicted label is the dimension with value 1. The formula should presumably be I_i = 0.5 * I_intermediate + 0.5, or the prediction rule should be stated as selecting the argmax coordinate. This needs to be corrected because the Interpretable Syntactic Representation is the basis for the hierarchical variants and for the classification experiment.
[Section 3, Eqs. (1)-(3)] The mathematical construction of the syntactic subspace is ambiguous. With C in R^{(V∩W)×8}, Eq. (2) treats C^T as an 8×(V∩W) matrix acting on S_{v,:}, but Eq. (3) then uses S_{v,:} as an 8-vector. The subscript in Eq. (1) sums over V while the set of nouns is N, and the notation C_{1,:} is used for what appears to be a coordinate of a single POS. The transition matrix construction and the projection in Eqs. (2)-(3) must be rewritten with consistent dimensions so that the method is reproducible.
[Appendix B.2, Eq. (9)] The definition of the Weighted Hierarchical Vector is incomplete. The index j in 'j ∈ [a, b]' is undefined, and the expression V^s_{i:} × V^r_{j:} is not a standard product with the stated dimensions. Since these vectors are evaluated in Table 1, the operation must be specified precisely. As written, Eq. (9) does not define a reproducible construction.

minor comments (5)

[Section 2] There are several citation and punctuation errors in the Related Work section, including 'Faruqui et al. (2015)Subramanian et al. (2018)Panigrahi et al. (2019)' and '(Mathew et al., 2020)(Engler et al., 2023)', which should be corrected.
[Table 7] In the Weighted Word2Vec block of Table 7, the row labels repeat WOA/WOI/WOL instead of WWA/WWI/WWL, so the table as printed cannot be interpreted reliably.
[Table 3 and Section 6.5] The condition number is reported as '>100000' without the corresponding effective smallest singular value; please state the exact value and the threshold used to judge it problematic.
[Appendix E and Figure 4] The word-list-size experiments are presented without error bars or statistical tests, so the conclusion that the change is 'not consistent' is not quantified.
[General] The paper does not state whether code or data will be released; a reproducibility statement would be helpful.

Circularity Check

2 steps flagged · score 6.0 of 10

The headline interpretability result is a self-match: the syntactic subspace is fit on WordNet POS averages and then "predicted" on the same WordNet POS labels, so the word-classification accuracy in Table 4 is expected by construction; the downstream benchmark gains are external and are not circular.

fitted input called prediction [Section 3, Eqs. (1)-(3); Section 7.1, Table 4; Appendix A.2, Eqs. (5)-(6).]
"To build the syntactic subspace, we leverage linguistic knowledge obtained from a pre-constructed linguistic resource, WordNet... We gather words belonging to each part of speech from WordNet... For this experiment, we consider WordNet to be the oracle. We extract all words present in WordNet to generate respective Interpretable Syntactic Representations... The dimension of Interpretable Syntactic Representation with value "1" is selected as the predicted label."

The coordinate directions of the syntactic subspace C are per-POS averages of WordNet word vectors (Eq. 1 for nouns, analogous for the other seven classes). The Interpretable Syntactic Representation (Eqs. 5-6) scales each word's projection so that the largest coordinate becomes 1, and Sec. 7.1 predicts that word's part of speech by selecting the coordinate equal to 1, using WordNet as the oracle. Thus the projection used for prediction is fit to the very WordNet POS labels being predicted: each word is scored by closeness to its own class centroid, so high classification accuracy on the construction vocabulary is expected by construction and does not demonstrate that the coordinates recover POS for unseen words or independent annotation sources.
self definitional [Section 7.2, Table 5.]
"As shown in table 5, we find that the representations are able to produce excellent syntactic groupings and the produced groups are coherent with the built dimension."

Each 'built dimension' is literally the average of WordNet words belonging to that part of speech (Sec. 3, Eq. 1 and its analogues). Ranking words by their projection on the noun dimension therefore surfaces words with high alignment to the average noun vector; observing that these words are mostly nouns restates how the dimension was constructed rather than providing independent evidence of interpretability. This qualitative check is an internal consistency result, not an external validation.

full rationale

Score is 6, not higher, because the circularity is confined to the interpretability validation. The syntactic subspace is constructed from WordNet POS averages and then evaluated by predicting those same WordNet POS labels, so the word-classification accuracy in Table 4 is a by-construction self-match. The Appendix G Google Text Normalization classification is a partial external check, but the paper's headline interpretability numbers come from the WordNet oracle and no held-out/non-WordNet validation is provided for Table 4. The other central component, hierarchical vectors outperforming base vectors on downstream benchmarks, is evaluated on external task labels and is not circular. There is no load-bearing self-citation or uniqueness-theorem chain. The qualitative top-word table is also self-referential, as the dimensions are defined by the same POS averages used to interpret them.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new physical or ontological entities; hierarchical vectors and the syntactic subspace are data structures and mathematical constructs. The main implicit additions are the hand-chosen eight POS coordinates and the assumption that POS information is linearly recoverable from pretrained embeddings, neither of which is independently validated.

free parameters (2)

number and identity of syntactic coordinates (eight parts of speech) = 8
The coordinate set is fixed by hand to the traditional eight POS classes; no derivation or ablation justifies this choice for the embedding geometry.
normalization offset for interpretable representations = 0.5
The 0.5 lower bound in Eq 5 is a display choice; it does not affect downstream vectors but affects the stated interpretability rule and is internally inconsistent with the value-1 selection rule.

assumptions (4)

domain assumption WordNet POS membership is a reliable oracle for a word's syntactic category.
Used in Section 3 to build the subspace and in Section 7.1 as the ground truth for interpretability.
domain assumption Pretrained word vectors encode part-of-speech distinctions linearly, so averages per POS define a meaningful subspace.
Central to Eq 1-3; if false, the projection coordinates carry no syntactic signal.
standard math The Moore-Penrose inverse of C^T gives the correct least-squares projection.
Invoked in Eq 3; the math is standard, but the notation in Eq 2-3 is dimensionally inconsistent as written.
domain assumption The coordinate with the largest (or normalized-value 1) entry identifies the word's POS.
Used in Section 7.1 for classification; the paper is internally inconsistent about whether the rule is argmax or value equal to 1.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Interpretable Syntactic Representations Enable Hierarchical Word Vectors." pith.science (2026). https://pith.science/paper/PIAWOF44

@misc{pith2026241108384,
  author       = {Pith},
  title        = {Pith review of: Interpretable Syntactic Representations Enable Hierarchical Word Vectors},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PIAWOF44}},
  note         = {Machine review of arXiv:2411.08384}
}

read the original abstract

The distributed representations currently used are dense and uninterpretable, leading to interpretations that themselves are relative, overcomplete, and hard to interpret. We propose a method that transforms these word vectors into reduced syntactic representations. The resulting representations are compact and interpretable allowing better visualization and comparison of the word vectors and we successively demonstrate that the drawn interpretations are in line with human judgment. The syntactic representations are then used to create hierarchical word vectors using an incremental learning approach similar to the hierarchical aspect of human learning. As these representations are drawn from pre-trained vectors, the generation process and learning approach are computationally efficient. Most importantly, we find out that syntactic representations provide a plausible interpretation of the vectors and subsequent hierarchical vectors outperform the original vectors in benchmark tests.

Figures

Figures reproduced from arXiv: 2411.08384 by the authors.

**Figure 1.** Examples of Syntactic Representations These Syntactic Representations are continuous and highly reduced transformations of the original representations. These representations isolate the syntactic regularities existing within the large original word vectors and when used together in a incremental learning approach as Hierarchical Vectors, outperforms the original vectors in benchmark tests. Hierarchical Vectors util… view at source ↗

**Figure 2.** Sport Classification Accuracy per size of word list. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

**Figure 3.** Effect of transformation on the embedding space [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Figures from the paper (5 more)

**Figure 4.** Figure 4: Word Classification Accuracy per size of word list. [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

**Figure 5.** Figure 5: Religion Classification Accuracy per size of word list. [PITH_FULL_IMAGE:figures/full_fig_p015_5.png]

**Figure 6.** Figure 6: Computer Classification Accuracy per size of word list. [PITH_FULL_IMAGE:figures/full_fig_p016_6.png]

**Figure 7.** Figure 7: Result of the Word Classification experiment. Accuracy represents the ratio of number of [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]

**Figure 8.** Figure 8: Result of the Word Classification experiment. Accuracy represents the ratio of number of [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

18 extracted references · 11 canonical work pages

[7]

doi: https://doi.org/10.1016/B978-0-12-394435-1.00015-6

ISBN 978-0-12- 394435-1. doi: https://doi.org/10.1016/B978-0-12-394435-1.00015-6. URL https://www. sciencedirect.com/science/article/pii/B9780123944351000156. Jiang Guo, Wanxiang Che, Haifeng Wang, and Ting Liu. Revisiting embedding features for simple semi-supervised learning. In Proceedings of the 2014 conference on empirical methods in natural language...

work page doi:10.1016/b978-0-12-394435-1.00015-6
[8]

Semeval-2018 task 10: Capturing discrimi- native attributes

Alicia Krebs, Alessandro Lenci, and Denis Paperno. Semeval-2018 task 10: Capturing discrimi- native attributes. In Proceedings of the 12th international workshop on semantic evaluation , pp. 732–740,

work page 2018
[9]

Fish transporters and miracle homes: How compositional distributional semantics can help np parsing

10 Angeliki Lazaridou, Eva Maria Vecchi, and Marco Baroni. Fish transporters and miracle homes: How compositional distributional semantics can help np parsing. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pp. 1908–1913,

work page 2013
[14]

Recursive deep models for semantic compositionality over a sentiment treebank

Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empirical methods in natural language pro- cessing, pp. 1631–1642,

2013
[15]

Wizmap: Scalable interactive visualization for exploring large machine learning embeddings

Zijie J Wang, Fred Hohman, and Duen Horng Chau. Wizmap: Scalable interactive visualization for exploring large machine learning embeddings. arXiv preprint arXiv:2306.09328,

arXiv
[16]

Unlike conventional normalization, which scales data to a specific range, we inten- tionally preserve these representations in their unnormalized form

A T YPES OF SYNTACTIC REPRESENTATIONS A.1 A BSOLUTE SYNTACTIC REPRESENTATIONS These Syntactic Representations are the actual outcomes obtained through post-processing of the original vectors. Unlike conventional normalization, which scales data to a specific range, we inten- tionally preserve these representations in their unnormalized form. The absolute ...

work page 2015
[17]

which consists of a training/dev/test split of 6920/872/1821 sentences. We generate the feature vector by averaging over all constituent words of the sentence, only on the data instances with non-neutral labels, and report the test accuracy obtained by the model with the highest validation accuracy. C.6 W ORD SIMILARITY The Word Similarity test intends to...

work page 2015
[18]

Each pair of data in all of these datasets are annotated a human generated similarity score

and MT-771 (Halawi et al., 2012). Each pair of data in all of these datasets are annotated a human generated similarity score. We compute the cosine similarity of all of the pairs of words in each of the above dataset and report the Spearman’s rank correlation coefficient ρ between the model generated similarity and the human annotated similarity. Only th...

work page 2012

Show all 18 references

[1993]

The polar framework: Polar opposites enable interpretability of pre-trained word embeddings

Binny Mathew, Sandipan Sikdar, Florian Lemmerich, and Markus Strohmaier. The polar framework: Polar opposites enable interpretability of pre-trained word embeddings. In Proceedings of The Web Conference 2020, pp. 1548–1558,

2020
[1995]

All-but-the-top: Simple and effective postprocessing for word representations

Jiaqi Mu, Suma Bhat, and Pramod Viswanath. All-but-the-top: Simple and effective postprocessing for word representations. arXiv preprint arXiv:1702.01417,

arXiv
[2003]

Visually analyzing contextualized embeddings

Matthew Berger. Visually analyzing contextualized embeddings. In 2020 IEEE Visualization Con- ference (VIS), pp. 276–280. IEEE,

2020
[2013]

Learning question classifiers

Xin Li and Dan Roth. Learning question classifiers. In COLING 2002: The 19th International Conference on Computational Linguistics,

2002
[2014]

Bert: Pre-training of deep bidirectional transformers for language understanding

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805,

arXiv
[2015]

Problems with evaluation of word embeddings using word similarity tasks

Manaal Faruqui, Yulia Tsvetkov, Pushpendre Rastogi, and Chris Dyer. Problems with evaluation of word embeddings using word similarity tasks. arXiv preprint arXiv:1605.02276,

arXiv
[2018]

The secret is in the spec- tra: Predicting cross-lingual task performance with spectral similarity measures

Haim Dubossarsky, Ivan Vuli ´c, Roi Reichart, and Anna Korhonen. The secret is in the spec- tra: Predicting cross-lingual task performance with spectral similarity measures. arXiv preprint arXiv:2001.11136,

2001 arXiv
[2019]

Glove: Global vectors for word representation

Jeffrey Pennington, Richard Socher, and Christopher D Manning. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp. 1532–1543,

2014
[2020]

Sensepolar: Word sense aware interpretability for pre-trained contextual word embeddings

Jan Engler, Sandipan Sikdar, Marlene Lutz, and Markus Strohmaier. Sensepolar: Word sense aware interpretability for pre-trained contextual word embeddings. arXiv preprint arXiv:2301.04704 ,

arXiv
[2023]

Sparse overcomplete word vector representations

Manaal Faruqui, Yulia Tsvetkov, Dani Yogatama, Chris Dyer, and Noah Smith. Sparse overcomplete word vector representations. arXiv preprint arXiv:1506.02004,

arXiv

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.

[1] [7]

doi: https://doi.org/10.1016/B978-0-12-394435-1.00015-6

ISBN 978-0-12- 394435-1. doi: https://doi.org/10.1016/B978-0-12-394435-1.00015-6. URL https://www. sciencedirect.com/science/article/pii/B9780123944351000156. Jiang Guo, Wanxiang Che, Haifeng Wang, and Ting Liu. Revisiting embedding features for simple semi-supervised learning. In Proceedings of the 2014 conference on empirical methods in natural language...

work page doi:10.1016/b978-0-12-394435-1.00015-6

[2] [8]

Semeval-2018 task 10: Capturing discrimi- native attributes

Alicia Krebs, Alessandro Lenci, and Denis Paperno. Semeval-2018 task 10: Capturing discrimi- native attributes. In Proceedings of the 12th international workshop on semantic evaluation , pp. 732–740,

work page 2018

[3] [9]

Fish transporters and miracle homes: How compositional distributional semantics can help np parsing

10 Angeliki Lazaridou, Eva Maria Vecchi, and Marco Baroni. Fish transporters and miracle homes: How compositional distributional semantics can help np parsing. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pp. 1908–1913,

work page 2013

[4] [14]

Recursive deep models for semantic compositionality over a sentiment treebank

Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empirical methods in natural language pro- cessing, pp. 1631–1642,

2013

[5] [15]

Wizmap: Scalable interactive visualization for exploring large machine learning embeddings

Zijie J Wang, Fred Hohman, and Duen Horng Chau. Wizmap: Scalable interactive visualization for exploring large machine learning embeddings. arXiv preprint arXiv:2306.09328,

arXiv

[6] [16]

Unlike conventional normalization, which scales data to a specific range, we inten- tionally preserve these representations in their unnormalized form

A T YPES OF SYNTACTIC REPRESENTATIONS A.1 A BSOLUTE SYNTACTIC REPRESENTATIONS These Syntactic Representations are the actual outcomes obtained through post-processing of the original vectors. Unlike conventional normalization, which scales data to a specific range, we inten- tionally preserve these representations in their unnormalized form. The absolute ...

work page 2015

[7] [17]

which consists of a training/dev/test split of 6920/872/1821 sentences. We generate the feature vector by averaging over all constituent words of the sentence, only on the data instances with non-neutral labels, and report the test accuracy obtained by the model with the highest validation accuracy. C.6 W ORD SIMILARITY The Word Similarity test intends to...

work page 2015

[8] [18]

Each pair of data in all of these datasets are annotated a human generated similarity score

and MT-771 (Halawi et al., 2012). Each pair of data in all of these datasets are annotated a human generated similarity score. We compute the cosine similarity of all of the pairs of words in each of the above dataset and report the Spearman’s rank correlation coefficient ρ between the model generated similarity and the human annotated similarity. Only th...

work page 2012