Pith. sign in

REVIEW 4 major objections 6 minor 47 references

A polar coordinate system represents syntax in large language models

T0 review · 4 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read In the activations of large language models, dependency syntax is laid out as a polar coordinate system: distance marks whether a relation exists, while relative angle marks its type and direction.

desk verdict A clean, useful extension of the Structural Probe showing that dependency type and direction are linearly decodable from angle, but the 'spontaneous' claim needs a label-permutation control. read the letter →

arxiv 2412.05571 v1 pith:XAEXHRN5 submitted 2024-12-07 cs.CL

Pablo Diego-Simón , Stéphane D'Ascoli , Emmanuel Chemla , Yair Lakretz , Jean-Rémi King This is my paper · ORCID
classification cs.CL
keywords polarcoordinatesystemsyntactictreesdependencygrammarprobinglargelanguagemodelsUniversalDependencieslinearreadoutneuralrepresentations
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that large language models do not merely place syntactically related words near one another: they organize dependency trees as a polar coordinate system in a low-dimensional subspace of their activations. In that subspace, the distance between two word embeddings marks whether a dependency relation exists, while the relative angle between their difference vector and a prototype vector marks the relation's type, and the sign of that angle marks which word is the head. The authors introduce a single linear transformation, the Polar Probe, trained to satisfy both distance and angular constraints, and they report that it reads the type and direction of syntactic relations roughly twice as accurately as the earlier distance-only probe. If the claim is right, the main symbolic structure of syntax is not an external overlay on neural vectors but a readable geometry the models spontaneously acquire.

What carries the argument

The central object is the probed edge vector $B_P s_{ij}$, the image under a linear map of the difference between two contextualized word embeddings; this is what the paper calls the polar coordinate system. Its squared norm encodes whether two words are syntactically linked, and its direction encodes the relation's type and head. The map $B_P$ is the machinery: a single linear transformation trained on the sum of a distance objective and a contrastive angular objective, so that edges of the same dependency type are pulled into the same direction and edges of different types are pushed toward orthogonality. Readout then reduces to comparing each probed edge with type prototype vectors by cosine similarity, making the whole tree linearly decodable.

What would settle it

A decisive control would train the Polar Probe on the same model activations and corpus but with the dependency labels, or their head directions, randomly permuted during training; if the probe still identifies the true labels on a held-out test set, the claimed geometry would not be specifically about syntax, whereas chance-level performance would confirm that it is.

Watch

Extended reading notes

Core claim

The central claim is that a single linear readout of contextualized word embeddings can recover the full labeled and directed dependency tree, not just the unlabeled graph. Writing the edge embedding between words $i$ and $j$ as $s_{ij}=h_i-h_j$, the Polar Probe learns one linear map $B_P$ such that the predicted existence of a relation is $\hat d=\|B_P s_{ij}\|^2$ (Eq. 8), the predicted type is $\hat t=\arg\max_c |\cos(B_P s_{ij}, V_c)|$ where $V_c$ is the average probed vector for type $c$ (Eq. 9), and the head is $h_i$ when that cosine is nonnegative and $h_j$ otherwise (Eq. 10). The probe is trained jointly on a tree-distance loss and a contrastive angle loss, and the result is that same-type edges become collinear while different-type edges become orthogonal in the probed space. On the English Web Treebank, the probe reaches about 95% AUC for dependency-type classification versus 74% for the structural probe subspace, preserves the structural score for whether a dependency exists, and peaks at the same intermediate layer across three language models. The same angular code is recovered when a main clause is embedded in nested relative clauses, which the paper takes as evidence that the geometry is systematic across tree levels.

Load-bearing premise

The load-bearing premise is that the manual grammar annotations used as ground truth are the correct specification of syntactic relations, their types, and their head directions, because the polar coordinate system is only measured against those labels.

Editorial extensions

If this is right

  • Dependency type and head direction are linearly readable from activation differences, completing the distance-only picture and removing the need for a separate labeled probe.
  • The full labeled tree fits in about 128 dimensions, far smaller than the layer width, suggesting that syntax occupies a compact geometric subspace rather than being spread across all units.
  • The code appears at the same intermediate layer across several families of models and sharpens with model scale and recency, so it is a reproducible property of trained representations rather than a quirk of one architecture.
  • The same relation type is coded by the same direction whether it appears in a short sentence or inside a long-nested relative clause, supporting a coordinate system that is invariant to surface position.
  • Because a probe trained on a randomly initialized model performs at chance, the polar geometry is inherited from what the model learned, not manufactured by the probe's objective.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not test causality; a natural extension would be to rotate the angular coordinate of a probed edge and check whether the model's syntactic behavior shifts accordingly, which would show the code is used rather than merely correlational.
  • The same distance-plus-angle scheme could be adapted to other structured domains the paper does not probe, such as semantic role relations, knowledge-graph edges, or object-feature binding in vision.
  • Because the probe is supervised by one annotation scheme, an unsupervised variant would be needed to decide whether the polar geometry would be discovered from the activations alone or is partly imposed by the chosen labels.
  • The explicit direction sign suggests a cross-linguistic test: if the same relation types point the same way in head-initial and head-final languages under the same annotation framework, the code would be a candidate universal syntactic primitive.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. This paper introduces the Polar Probe, a linear transformation trained on LLM contextual embeddings with a joint objective that combines the Structural Probe distance loss (Eq. 3) and an angular contrastive loss (Eq. 4). The resulting probe represents each dependency edge as a vector B_P(h_i - h_j); the squared norm of this vector is claimed to encode the existence of a syntactic relation (Eq. 8), the absolute cosine to type-specific prototypes is claimed to encode the relation type (Eq. 9), and the sign of that cosine is claimed to encode head direction (Eq. 10). The authors evaluate the probe on the English Web Treebank across several LLMs (Llama-2, Mistral, BERT-large, GPT-2, Pythia), report that it outperforms the Structural Probe on labeled and directed attachment, analyze layer- and dimensionality-dependent performance, and present a small controlled nested-sentence dataset. They conclude that LLMs spontaneously learn a polar-coordinate geometry that explicitly represents labeled directed dependency trees.

Significance. If the geometry is genuinely intrinsic to LLM activations, the paper would provide a simple linear readout for labeled directed dependency trees, contributing to the long-standing debate about symbolic versus connectionist representations. The manuscript has several concrete strengths: evaluation on a held-out test set, a range of model families and sizes, the Random-LLM and No-Probe controls, and a dimensionality analysis that goes beyond the original Structural Probe. However, the load-bearing interpretation that the polar geometry is 'spontaneous' and 'explicitly represents' UD syntax is not yet fully supported: the probe is trained with UD labels, and the angular objective imposes the collinearity and orthogonality structure that is later measured. The quantitative comparison with the Structural Probe is not like-for-like, and the headline numerical claims are reported without confidence intervals or significance tests. With additional controls and clarifications, the central results could be made solid, but as presented the evidence is incomplete.

major comments (4)
  1. [§2.4, Eq. (11); §3 'Comparison with baselines'; §4 'Limitations'] The 'spontaneous' and 'explicitly represents' claims require a label-permutation control. The angular objective (Eq. 4) explicitly pushes same-type edges toward collinearity and different-type edges toward orthogonality, so measuring that structure in the probe output partly checks the training objective. The Random-LLM and No-Probe baselines do not establish that the true UD labels are privileged: a real LLM's activations contain abundant lexical, part-of-speech, and positional regularities (as the No-Probe 80% AUC shows), and a supervised linear probe could plausibly fit permuted labels to a similar degree. I request a control in which the same LLM activations are trained with randomly permuted relation labels (or permuted head/direction assignments), preserving label frequencies and edge distributions; without it, the evidence supports linear decodability of UD labels, not a spontaneous syntax-specific polar geometry.
  2. [§3 'Comparison with baselines'; Fig. 2.B; Fig. 3] The comparison with the Structural Probe on label accuracy is apples-to-oranges. The Structural Probe is trained only to minimize the distance objective (Eq. 3) and is not designed to encode relation type or direction, so lower label accuracy is expected and does not demonstrate that the Polar Probe's geometry is superior. To support the claimed 'nearly two folds' improvement, the authors should compare against a supervised linear readout trained on top of the Structural Probe's output embeddings, or against a linear classifier applied to raw activations; otherwise the baseline is not a fair control for label and direction accuracy.
  3. [§3, Fig. 2.B, Fig. 3, Fig. 6] The paper reports no confidence intervals, error bars, or significance tests for the headline AUC, balanced accuracy, or LAS scores. Given that the central quantitative claims include 'substantially outperforms' and 'becomes increasingly precise in the latest frontier models,' the authors should report variance across test splits, random seeds, or bootstrap resamples, and where appropriate perform significance tests for the differences between probes.
  4. [§2.1, Eq. (1); §2.3, Eqs. (4)-(7); §2.4, Eqs. (9)-(10)] The definition of the edge embedding s_i,j = h_i - h_j does not specify whether the ordered pair (w_i, w_j) is always (head, dependent), always (dependent, head), or in sentence order. This matters because Eq. (4) pushes all same-type edges toward positive collinearity and Eq. (7)/(10) reads head direction from the sign of the cosine relative to the prototype. If the ordering is not consistently head-to-dependent, the angular objective conflicts with direction encoding. The paper should clarify the ordering, and include a validation that the predicted direction correlates with UD gold heads independently of sentence position (e.g., by checking that swapping the order of the pair flips the predicted head).
minor comments (6)
  1. [§2.4, Eq. (11)] The loss notation is inconsistent: Eq. (4) defines L_A with the Angular Probe matrix B_A, but Eq. (11) uses L_A within the Polar Probe objective for B_P; the authors should state that B_A is replaced by B_P during joint training or introduce a separate notation.
  2. [Fig. 1 caption] The caption contains a typo: 'finds a a linear transform' should read 'finds a linear transform.'
  3. [§3 'Comparison with baselines'] The sentence 'The same relative results across probes are conserved for the Balanced Accuracy score' does not report the actual Balanced Accuracy numbers; please provide them in the text or table.
  4. [§2.5 'Controlled dataset'] The text says 'as shown in Fig: 5' but the reference should be to Figure 5 without the colon; also the dataset description would benefit from a table of the 100 sentences and their level counts for reproducibility.
  5. [Fig. 6] The y-axis label 'Structural and Label Score' is vague, the 'SOTA' point is not explained in the caption or text, and there is no legend for the dashed versus solid markers; please clarify.
  6. [§3 'Comparison with baselines'] The phase 'nearly two folds' is awkward; I suggest 'nearly two-fold' or 'roughly twice as accurate.'

Circularity Check

0 steps flagged · score 0.0 of 10

No load-bearing circularity: the Polar Probe is supervised but evaluated on held-out data; the 'spontaneous' framing is an interpretive overreach, not a derivation-level circularity.

full rationale

The derivation chain is not circular. The Polar Probe is trained on the UD-labeled training split through the joint objective (Eqs. 4 and 11) and then evaluated on held-out sentences (Section 2.7, Section 3). Its high test-set accuracy for dependency type and direction is therefore genuine generalization, not a restatement of the training objective. The polar structure visualized in Fig. 2C (same-type collinearity, cross-type orthogonality) does coincide with the Angular Probe's objective in Eq. 4, but because it is measured on the test set, it is evidence that the learned geometry transfers to unseen activations rather than a tautology. The paper itself acknowledges the supervised nature of the probe in the Limitations: 'the Polar Probe is based on a supervised task: we optimize a linear transformation that maximally retrieves a known syntactic structure from the neural activations.' This tempers the abstract's 'spontaneously' wording, but it is a framing concern, not a circular derivation. Self-citations (e.g., King and Dehaene 2014 for linear readouts; Lakretz et al. 2021b for the controlled dataset) are incidental and not load-bearing. The absence of a label-permutation control weakens the stronger 'spontaneous syntax-specific geometry' interpretation, especially since the No-Probe baseline already reaches AUC=80% on type labels, but that is an external validity and control-selection gap, not an internal circularity. Overall, the central empirical claim — that a linear polar readout can recover labeled, directed dependency trees on held-out data — rests on a train/test separation and does not reduce to its inputs by construction.

Assumptions & free parameters 4 free parameters · 4 assumptions · 1 invented entities

The central claim rests on the fitted probe matrix, two hand-set hyperparameters (lambda and dimensionality), and prototype vectors computed from training data. The ground-truth dependency annotations and the linear-readout definition of representation are domain assumptions, while the polar coordinate system is an invented structure defined by the probe itself.

free parameters (4)
  • Probe matrix B_P (and angular/structural variants) = learned linear map, not released
    The central claim is that such a matrix exists and maps activations to a polar geometry; the matrix is optimized on the training set (Section 2.6), so the representational finding is conditional on this fitted transform.
  • lambda (loss weight) = 10.0
    Set by hand to balance structural and angular objectives (Section 2.6); affects the geometry found.
  • Probe dimensionality k'' = 128
    Chosen as the peak of validation performance (Section 3, Fig 4); the claim of low-dimensional subspace depends on this selection.
  • Prototype vectors V_c = centroids of 10,000 training edges per type
    Used for classification; computed from training set (Section 2.7).
assumptions (4)
  • domain assumption Universal Dependencies annotations (EWT) are the correct ground truth for syntactic trees, including head direction and relation labels.
    The probe is trained and evaluated against these annotations (Section 2.5); if they are wrong, the polar geometry is an artifact of the annotation scheme.
  • domain assumption A representation is defined as linearly readable information; a linear probe is the appropriate measurement.
    Stated in Section 2.1 following the linear readout definition; excludes nonlinear codes.
  • domain assumption Euclidean geometry and the difference embedding h_i - h_j capture relational information.
    The probe operates on edge embeddings s_ij = h_i - h_j (Eq 1); hyperbolic alternatives are mentioned as a limitation in Section 4.
  • standard math The Structural Probe's distance-based representation is valid and provides the existence signal.
    The Polar Probe builds on Hewitt and Manning (2019); the structural objective is inherited.
invented entities (1)
  • Polar coordinate system for syntax in activation space
    purpose: Postulated geometric code where distance marks link existence and angle marks relation type and direction.
    The system is defined by the fitted probe; there is no independent measurement outside the probe that confirms the geometry. It is a representational model imposed by training, not a discovered entity with independent falsifiable predictions beyond probe accuracy.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A polar coordinate system represents syntax in large language models." pith.science (2026). https://pith.science/paper/XAEXHRN5

@misc{pith2026241205571,
  author       = {Pith},
  title        = {Pith review of: A polar coordinate system represents syntax in large language models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XAEXHRN5}},
  note         = {Machine review of arXiv:2412.05571}
}
read the original abstract

Originally formalized with symbolic representations, syntactic trees may also be effectively represented in the activations of large language models (LLMs). Indeed, a 'Structural Probe' can find a subspace of neural activations, where syntactically related words are relatively close to one-another. However, this syntactic code remains incomplete: the distance between the Structural Probe word embeddings can represent the existence but not the type and direction of syntactic relations. Here, we hypothesize that syntactic relations are, in fact, coded by the relative direction between nearby embeddings. To test this hypothesis, we introduce a 'Polar Probe' trained to read syntactic relations from both the distance and the direction between word embeddings. Our approach reveals three main findings. First, our Polar Probe successfully recovers the type and direction of syntactic relations, and substantially outperforms the Structural Probe by nearly two folds. Second, we confirm that this polar coordinate system exists in a low-dimensional subspace of the intermediate layers of many LLMs and becomes increasingly precise in the latest frontier models. Third, we demonstrate with a new benchmark that similar syntactic relations are coded similarly across the nested levels of syntactic trees. Overall, this work shows that LLMs spontaneously learn a geometry of neural activations that explicitly represents the main symbolic structures of linguistic theory.

Figures

Figures reproduced from arXiv: 2412.05571 by the authors.

Figure 1
Figure 1. Dependency trees hypothesized in linguistics and in neural networks. A. According to the dependency grammar framework, the sentences can be described as linear sequences of words connected by an acyclic graph. B. More precisely, such acyclic graph is both labeled and directed, where each edge has a direction, representing the hierarchy of the syntactic relation, and a label, representing the type of syntactic relati… view at source ↗
Figure 2
Figure 2. The Polar Probe reliably identifies dependency types. A. PCA visualization of edges linearly read by the Polar Probe. The color of each edge corresponds to one of three different dependency types (‘nsubj’, ‘obj’, ‘det’): the linear readouts point in systematic directions. B. AUC and Balanced Accuracy metrics obtained for dependency type classification. C. Pairwise cosine similarity (0=orthogonal, 1=collinear) matric… view at source ↗
Figure 2
Figure 2. A shows a Principal Component Analysis (PCA) projection of the dependency relation [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figures from the paper (6 more)
Figure 3
Figure 3. Figure 3: The Polar Probe outperforms the Structural Probe at identifying labeled and directed dependen￾cies. A. For dependency existence, the Polar Probe matches the UUAS performance of the Structural probe, peaking at layer 16. B. For dependency type, the Polar Probe outperfor…
Figure 4
Figure 4. Figure 4: The optimal dimensionality for the Polar Probe is an order of magnitude small than model’s layer size. Polar Probe performance as a function of dimensionality, measured by A. UUAS, B. Dependency Type Accuracy and C. LAS for Llama-2-7b-hf as a function of k ′′, the dime…
Figure 5
Figure 5. Figure 5: Visualization of the dependency tree uncovered by the Polar Probe on a set of sentences with increasingly complex hierarchical structures. A. We display a PCA visualization of the distributions of word embeddings (once linearly read out by the Polar Probe), for the dif…
Figure 6
Figure 6. Figure 6: Polar Probe performance on the EN-EWT dataset for Language Models with different families and [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]
Figure 7
Figure 7. Figure 7: Comparative analysis of Polar Probe performance on the EN-EWT dataset as a function of sentence [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]
Figure 8
Figure 8. Figure 8: Polar Probe performance across different sentence structures and dependency types in a controlled [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

47 extracted references · 40 canonical work pages

  1. [1]

    Caucheteux, C., Gramfort, A., and King, J.-R. (2021). Disentangling syntax and semantics in the brain with deep networks. In International conference on machine learning , pages 1336--1348. PMLR

    work page 2021

  2. [2]

    and King, J.-R

    Caucheteux, C. and King, J.-R. (2022). Brains and algorithms partially converge in natural language processing. Communications biology , 5(1):134

    work page 2022

  3. [3]

    Chen, B., Fu, Y., Xu, G., Xie, P., Tan, C., Chen, M., and Jing, L. (2021). Probing bert in hyperbolic spaces. arXiv preprint arXiv:2104.03869

    arXiv 2021

  4. [4]

    A., Hewitt, J., and Manning, C

    Chi, E. A., Hewitt, J., and Manning, C. D. (2020). Finding universal grammatical relations in multilingual BERT . In Jurafsky, D., Chai, J., Schluter, N., and Tetreault, J., editors, Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics , pages 5564--5577, Online. Association for Computational Linguistics

    work page 2020

  5. [5]

    Chomsky, N. (1957). Syntactic Structures . De Gruyter

    work page 1957

  6. [6]

    Chomsky, N. (2014). The minimalist program . MIT press

    work page 2014

  7. [7]

    and Rizzi, L

    Cinque, G. and Rizzi, L. (2009). The cartography of syntactic structures

    work page 2009

  8. [8]

    R., Cross, J., C elebi, O., Elbayad, M., Heafield, K., Heffernan, K., Kalbassi, E., Lam, J., Licht, D., Maillard, J., et al

    Costa-juss \`a , M. R., Cross, J., C elebi, O., Elbayad, M., Heafield, K., Heffernan, K., Kalbassi, E., Lam, J., Licht, D., Maillard, J., et al. (2022). No language left behind: Scaling human-centered machine translation. arXiv preprint arXiv:2207.04672

    arXiv 2022

Show all 47 references
  1. [9]

    Desai, K., Nickel, M., Rajpurohit, T., Johnson, J., and Vedantam, S. R. (2023). Hyperbolic image-text representations. In Krause, A., Brunskill, E., Cho, K., Engelhardt, B., Sabato, S., and Scarlett, J., editors, Proceedings of the 40th International Conference on Machine Lear...

    2023

  2. [10]

    Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2019). Bert: Pre-training of deep bidirectional transformers for language understanding. pages 4171--4186. Association for Computational Linguistics

    2019

  3. [11]

    Dhingra, B., Shallue, C., Norouzi, M., Dai, A., and Dahl, G. (2018). Embedding text in hyperbolic spaces

    2018

  4. [12]

    DiCarlo, J. J. and Cox, D. D. (2007). Untangling invariant object recognition. Trends in cognitive sciences , 11(8):333--341

    2007

  5. [13]

    Eisape, T., Gangireddy, V., Levy, R., and Kim, Y. (2022). Probing for incremental parse states in autoregressive language models. In Goldberg, Y., Kozareva, Z., and Zhang, Y., editors, Findings of the Association for Computational Linguistics: EMNLP 2022 , pages 2801--2813, Ab...

    2022

  6. [14]

    Evanson, L., Lakretz, Y., and King, J.-R. (2023). Language acquisition: do children and language models follow similar learning stages? arXiv preprint arXiv:2306.03586

    2023 arXiv

  7. [15]

    T., Campanelli, L., Li, J., Bhattasali, S., Pallier, C., and Brennan, J

    Hale, J. T., Campanelli, L., Li, J., Bhattasali, S., Pallier, C., and Brennan, J. R. (2022). Neurocomputational models of language processing. Annual Review of Linguistics , 8:427--446

    2022

  8. [16]

    and Manning, C

    Hewitt, J. and Manning, C. D. (2019). A structural probe for finding syntax in word representations. pages 4129--4138. Association for Computational Linguistics

    2019

  9. [17]

    Huang, Q., Smolensky, P., He, X., Deng, L., and Wu, D. (2017). Tensor product generation networks for deep nlp modeling. arXiv preprint arXiv:1709.09118

    2017 arXiv

  10. [18]

    and Huth, A

    Jain, S. and Huth, A. (2018). Incorporating context into language encoding models for fmri. Advances in neural information processing systems , 31

    2018

  11. [19]

    Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D

    Jiang, A. Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D. S., de las Casas, D., Bressand, F., Lengyel, G., Lample, G., Saulnier, L., Lavaud, L. R., Lachaux, M.-A., Stock, P., Scao, T. L., Lavril, T., Wang, T., Lacroix, T., and Sayed, W. E. (2023). Mistral 7b

    2023

  12. [20]

    Joshi, A. K. and Schabes, Y. (1997). Tree-adjoining grammars. In Handbook of Formal Languages: Volume 3 Beyond Words , pages 69--123. Springer

    1997

  13. [21]

    and Dehaene, S

    King, J.-R. and Dehaene, S. (2014). Characterizing the dynamics of mental representations: the temporal generalization method. Trends in cognitive sciences , 18(4):203--210

    2014

  14. [22]

    Kingma, D. P. and Ba, J. (2014). Adam: A method for stochastic optimization

    2014

  15. [23]

    and Bandettini, P

    Kriegeskorte, N. and Bandettini, P. (2007). Analyzing for information, not activation, to exploit high-resolution fmri. Neuroimage , 38(4):649--662

    2007

  16. [24]

    Lakretz, Y., Dehaene, S., and King, J.-R. (2020). What limits our capacity to process nested long-range dependencies in sentence comprehension? Entropy , 22(4):446

    2020

  17. [25]

    Lakretz, Y., Desbordes, T., King, J.-R., Crabb \'e , B., Oquab, M., and Dehaene, S. (2021a). Can rnns learn recursive nested subject-verb agreements? arXiv preprint arXiv:2101.02258

    2021 arXiv

  18. [26]

    Lakretz, Y., Hupkes, D., Vergallito, A., Marelli, M., Baroni, M., and Dehaene, S. (2021b). Mechanisms for handling nested dependencies in neural-network language models and humans. Cognition , 213:104699

    2021

  19. [27]

    Lakretz, Y., Kruszewski, G., Desbordes, T., Hupkes, D., Dehaene, S., and Baroni, M. (2019). The emergence of number and syntax units in. pages 11--20. Association for Computational Linguistics

    2019

  20. [28]

    and Mare c ek, D

    Limisiewicz, T. and Mare c ek, D. (2021). Introducing orthogonal constraint in structural probes. In Zong, C., Xia, F., Li, W., and Navigli, R., editors, Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Co...

    2021

  21. [29]

    Linzen, T., Dupoux, E., and Goldberg, Y. (2016). Assessing the ability of lstms to learn syntax-sensitive dependencies. Transactions of the Association for Computational Linguistics , 4:521--535

    2016

  22. [30]

    M \"u ller-Eberstein, M., van der Goot, R., and Plank, B. (2022). Probing for labeled dependency trees. In Muresan, S., Nakov, P., and Villavicencio, A., editors, Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , ...

    2022

  23. [31]

    and Kiela, D

    Nickel, M. and Kiela, D. (2017). Poincar\' e embeddings for learning hierarchical representations. In Guyon, I., Luxburg, U. V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnett, R., editors, Advances in Neural Information Processing Systems , volume 30. Curr...

    2017

  24. [32]

    Nivre, J., Zeman, D., Ginter, F., and Tyers, F. (2017). Universal Dependencies . ACL Anthology

    2017

  25. [33]

    Palangi, H., Huang, Q., Smolensky, P., He, X., and Deng, L. (2017). Grammatically-interpretable learned representations in deep nlp models. In Advances in Neural Information Processing Systems Workshop

    2017

  26. [34]

    Pallier, C., Devauchelle, A.-D., and Dehaene, S. (2011). Cortical representation of the constituent structure of sentences. Proceedings of the National Academy of Sciences , 108:2522--2527

    2011

  27. [35]

    Pasquiou, A., Lakretz, Y., Hale, J., Thirion, B., and Pallier, C. (2022). Neural language models are not born equal to fit brain data, but training helps. In ICML 2022-39th International Conference on Machine Learning , page 18

    2022

  28. [36]

    Pasquiou, A., Lakretz, Y., Thirion, B., and Pallier, C. (2023). Information-restricted neural language models reveal different brain regions’ sensitivity to semantics, syntax, and context. Neurobiology of Language , 4(4):611--636

    2023

  29. [37]

    Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., and Duchesnay, E. (2011). Scikit-learn: Machine learning in P ython. Jou...

    2011

  30. [38]

    Reddy, A. J. and Wehbe, L. (2021). Can fmri reveal the representation of syntactic structure in the brain? Advances in Neural Information Processing Systems , 34:9843--9856

    2021

  31. [39]

    Robins, R. H. (2013). A Short History of Linguistics . Routledge

    2013

  32. [40]

    Silveira, N., Dozat, T., de Marneffe, M.-C., Bowman, S., Connor, M., Bauer, J., and Manning, C. D. (2014). A gold standard dependency corpus for E nglish. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC-2014)

    2014

  33. [41]

    Smolensky, P. (1987). Connectionist ai, symbolic ai, and the brain. Artificial Intelligence Review , 1:95--109

    1987

  34. [42]

    Smolensky, P. (1990). Tensor product variable binding and the representation of symbolic structures in connectionist systems. Artificial Intelligence , 46:159--216

    1990

  35. [43]

    T., Fernandez, R., Goldrick, M., and Gao, J

    Smolensky, P., McCoy, R. T., Fernandez, R., Goldrick, M., and Gao, J. (2022). Neurocompositional computing: From the central paradox of cognition to a new generation of ai systems. AI Magazine , 43(3):308–322

    2022

  36. [44]

    Soulos, P., McCoy, T., Linzen, T., and Smolensky, P. (2019). Discovering the compositional structure of vector representations with role learning networks. arXiv preprint arXiv:1910.09113

    2019 arXiv

  37. [45]

    Tesnière, L. (1953). Esquisse d'une syntaxe structurale . Klincksieck

    1953

  38. [46]

    Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., Bikel, D., Blecher, L., Ferrer, C. C., Chen, M., Cucurull, G., Esiobu, D., Fernandes, J., Fu, J., Fu, W., Fuller, B., Gao, C., Goswami, V., Goyal, N....

    2023

  39. [47]

    C., Pimentel, T., Saphra, N., and Cotterell, R

    White, J. C., Pimentel, T., Saphra, N., and Cotterell, R. (2021). A non-linear structural probe. In Toutanova, K., Rumshisky, A., Zettlemoyer, L., Hakkani-Tur, D., Beltagy, I., Bethard, S., Cotterell, R., Chakraborty, T., and Zhou, Y., editors, Proceedings of the 2021 Conferen...

    2021

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.