{"id":"e68a2117-3f15-48c2-b59b-31d96be74c84","arxiv_id":"1908.06629","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Artificial sentences generated from standard grammar classes have average dependency distances below the random baseline, indicating memory constraints are embedded in grammatical formalisms.","lead":"This paper tested whether formal grammar rules, which linguists treat as memory-free, secretly encode human memory limits. It found that several classes of dependency grammars produce sentences with shorter-than-chance word-to-word distances, suggesting cognitive constraints are built into grammar itself.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Dependency-distance proxy for memory is untested, and order-sensitive class constraints could produce the reported reduction without implicating memory; the headline claim is thereby underdetermined.","rationale":"The reader identified the same weakest assumption: dependency distance as a proxy for memory constraints is imported from prior literature rather than tested. I agree that this is the most load-bearing point, because the entire interpretive leap from 'artificial grammar classes have below-chance mean dependency distances' to 'memory limitations are hidden in grammar' depends on that premise. The empirical core of the paper is credible: exhaustive enumeration for n <= 10, large random samples for n > 10, and a transparent comparison against the (n+1)/3 baseline. My concern is not about the computation but about what the computation means. Secondary weaknesses exist, including the lack of error bars on sampled estimates for small effective sample sizes (e.g., 30-69 structures for the rarest class-length combinations) and the unresolved citeYadav2019a placeholder in the discussion; these are real but do not change the primary concern. The proposed reading-time test would provide an independent check of whether dependency distance is a valid memory proxy, which is exactly the step needed to convert the reported statistical finding into a claim about memory. Because the paper's quantitative contribution survives even if the proxy issue is unresolved, I keep the reader's CONDITIONAL verdict rather than moving to REJECT or ACCEPT.","tokens_in":13399,"tokens_out":16225,"duration_ms":186242,"concrete_test":"Validate the proxy on existing reading-time data: fit word-by-word mixed-effects regressions in the Dundee corpus, or an equivalent public eye-tracking corpus, with dependency distance of the incoming dependency as a fixed effect, controlling for word length, log frequency, and sentence position, and with participant and item random intercepts. If dependency distance shows a robust positive relationship with reading time after controls, the proxy is empirically supported and the memory interpretation is mitigated; if the effect is absent or reversed, below-chance distances in artificial grammar classes do not license the memory-limitation conclusion.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's conclusion that memory limitations have permeated grammatical descriptions rests on interpreting a below-baseline average dependency distance <d>_AS as evidence of memory constraints. The premise that dependency distance is a valid proxy for memory is asserted on p. 5 with a citation, not derived or tested here. This matters because the grammar classes studied (projective, planar, WG1, MH4, MH5, 1EC) are defined by order-sensitive restrictions: planarity forbids crossings, projectivity forbids crossings plus covered roots, WG1 bounds gap degree, MHk bounds multi-headedness, and 1EC bounds endpoint-crossing structure. Each such restriction couples tree structure to the linear string and, by itself, biases the distribution of distances; for instance, forbidding an arc between two positions removes a long-distance configuration from the class's support. The observed <d>_AS < (n+1)/3 is therefore a statement about the combinatorial consequences of these formal constraints. The additional claim that the bias is specifically due to memory is not identified by the data. The limitation paragraph on p. 8 addresses the objection that future classes might not show the effect, but it never addresses the possibility that the effect is a generic property of order-sensitive grammar classes. Without independent support for the distance-memory mapping, the central claim is underdetermined.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper tests the cognitive-independence assumption of formal grammars by computing the average dependency distance <d>_AS of uniformly sampled dependency structures in several grammar classes (projective, planar, WG1, MH4, MH5, and 1EC) for sentence lengths n = 3 to 25. For n ≤ 10 the authors exhaustively enumerate all labelled directed trees; for n > 10 they draw 10^9 uniform random Prüfer-code samples. They report that for all classes considered, <d>_AS falls below the random-linear-arrangement baseline <d>_rla = (n+1)/3 for sufficiently large n, and they interpret this as evidence that memory limitations are inherent to grammatical descriptions. An analysis of attested treebanks (UD, Stanford, Prague) provides a reference curve. The paper concludes that it may be impossible to build a parsimonious theory of human linguistic productivity independent of non-linguistic cognitive constraints.","tokens_in":13630,"tokens_out":5810,"duration_ms":64511,"significance":"If the interpretive step is accepted, the finding would challenge a foundational assumption in linguistics and would have implications for language acquisition, language evolution, and NLP. The study has clear strengths: the exhaustive enumeration for n ≤ 10 is exact, the Prüfer-code sampling is uniform and well described, the data and code are made available, and the 'all structures' class reproduces the known baseline, which is a useful sanity check. The main risk is that the central inference—from below-baseline average dependency distance to memory constraints—is underdetermined. The observed reduction may be a combinatorial consequence of order-sensitive class definitions rather than a specific signature of memory. With additional control analyses or a more carefully scoped conclusion, the contribution would be solid and valuable.","major_comments":[{"comment":"The paper's central inference from below-baseline <d>_AS to memory constraints depends on the premise that dependency distance is a valid proxy for memory limitations. That premise is asserted with citations (22, 23) but is not derived or validated in this manuscript. Because every class studied is defined by an order-sensitive restriction (planarity forbids crossings, projectivity adds a constraint on the root, WG1 bounds gap degree, MHk bounds multi-headedness, and 1EC bounds endpoint-crossing), the reduction in mean distance could simply be a combinatorial by-product of excluding certain long-distance or crossing configurations from the support. The limitation paragraph on p. 8 addresses the possibility that future classes may not show the effect, but it does not address the possibility that any order-sensitive class with similar restrictions would show it. To make the headline claim load-bearing, the authors should either (i) provide independent evidence for the distance-memory mapping, (ii) include control classes defined by order-sensitive restrictions with no known link to memory and show that they do not show the same drop, or (iii) explicitly rephrase the conclusion as a claim about the combinatorial structure of grammar classes rather than about memory. Without one of these, the conclusion that 'memory limitations have permeated grammatical descriptions' is underdetermined by the data presented.","section":"p. 5 and Supplementary Text"},{"comment":"The sampled estimates for n > 10 are presented without confidence intervals or standard errors, and some displayed points rest on very few structures (for example, 69 trees for MH5 at n = 25). The statement that <d>_AS is below chance 'for sufficiently large n in all classes' is an asymptotic claim whose tail is supported by these low-count estimates. Please report sampling error (e.g., bootstrap or standard error of the mean), state the effective sample size for each point, and verify that the below-baseline ordering is statistically significant at the displayed points. This is especially important because the main figure suppresses points with fewer than 30 structures, while the text's qualitative claim extends into the regime where counts are small.","section":"Fig. 2B / Fig. 3 / Materials and Methods"}],"minor_comments":[{"comment":"The phrase 'we cannot not exclude' appears to be a typo for 'we cannot exclude'.","section":"p. 8"},{"comment":"The inline reference '—citeYadav2019a' appears as a broken LaTeX citation and should be replaced with a proper reference.","section":"p. 8"},{"comment":"The summary says 'Analyses of billions of sentences,' but the artificial structures are not sentences; 'billions of syntactic structures' would be more accurate.","section":"One Sentence Summary"},{"comment":"The 30-structure cutoff is justified only by a generic rule of thumb (ref. 46); please provide a more specific justification or a sensitivity analysis for the displayed curves.","section":"Materials and Methods"},{"comment":"The caption states that points with fewer than 30 structures are not shown for n > n*, but a reader cannot tell from the figure which classes have sparse points at each n; adding per-point sample sizes or a supplementary table would improve transparency.","section":"Fig. 2B caption"}],"recommendation":"major_revision","confidential_remarks":"The paper's main observation is likely correct as a combinatorial fact about the grammar classes studied, but the title and abstract overstate the cognitive interpretation. The gap between the formal result and the memory-based conclusion is the main obstacle; I would not recommend acceptance without either a control analysis or a substantially weakened conclusion. The manuscript may also be a better fit for a specialized quantitative linguistics venue than for a general journal, but that is an editorial decision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Hi [Colleague],\n\nQuick take: the paper has a solid, well-documented computational result that deserves serious refereeing, but the headline claim—that memory limitations are therefore hidden in grammar—bets more on the proxy than the data can carry.\n\nWhat's actually new: they uniformly sample dependency trees from several grammar classes (projective, planar, WG1, MH4, MH5, 1EC) with sentence length controlled, and show that mean dependency distance stays below the (n+1)/3 random baseline for sufficiently large n. For n<=10 they enumerate exhaustively; beyond that they sample 10^9 trees per n and report points with at least 30 structures. That's a clean improvement over earlier work on projective and 2-component classes, which had length-control caveats. The data and code are available, and the methodology is transparent.\n\nCredit where due: the baseline is carefully justified, the distinction between exact enumeration and sampled estimates is explicit, and the undersampling discussion is honest. The observation that all these linguistically motivated classes land below chance is a genuine result.\n\nSoft spots: First, the leap from below-chance dependency distances to 'memory limitations have permeated grammatical descriptions' rests on the premise that dependency distance is a proxy for memory. The paper cites prior work for this, which is fair, but it doesn't test the alternative: that the formal constraints themselves—planarity, projectivity, bounded gap degree—simply bias the distribution of distances as a combinatorial side effect. The stress-test note makes this point, and on reading the paper it doesn't hold up fully, because the classes are exactly constraints on linear order; the interpretation is underdetermined. Second, the sampled estimates have no error bars. 69 trees for MH5 at n=25 is thin, and even the 30-threshold rule of thumb doesn't substitute for confidence intervals. Third, the text has an unresolved 'citeYadav2019a' placeholder and a typo ('cannot not exclude'), which suggests the manuscript isn't final.\n\nI'd send this to peer review. The empirical contribution is solid and the interpretive debate is worth having. The authors should add error bars, soften the causal language, and fix the placeholders, but the core finding deserves a proper referee.","headline":"A solid computational result about below-chance dependency distances in grammar classes, but the memory-limitation interpretation needs more support than the paper gives it.","tokens_in":14187,"tokens_out":2827,"would_cite":true,"duration_ms":30899,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Memory limitations are hidden in the formal grammars themselves: uniformly sampled sentences from every tested dependency grammar class have below-chance average dependency distance.","keywords":["dependency grammar","dependency distance","memory constraints","projective trees","mildly non-projective trees","random linear arrangement","cognitive independence","uniform sampling"],"falsifier":"A controlled processing experiment could settle the proxy question: if readers show no measurable increase in processing effort, such as reading time, as average dependency distance rises while other factors are held fixed, the proxy fails and the inference from grammar classes to memory constraints collapses. Alternatively, exhibiting a practically motivated mildly non-projective class whose uniform random sample has average dependency distance at or above (n+1)/3 for large n would refute the universality of the effect.","tokens_in":13180,"feed_emoji":"🧠","tokens_out":5805,"duration_ms":55739,"temperature":0.7,"pith_summary":"This paper tests the long-standing assumption that formal grammars of human language capture an idealized competence, free of the memory limitations of the human brain. The authors sample billions of sentences uniformly from the spaces defined by several state-of-the-art dependency grammar classes and measure average dependency distance, a standard proxy for memory load. They claim that in every class considered, projective, planar, and three mildly non-projective families, the average dependency distance of artificial structures falls below the random baseline (n+1)/3 for sufficiently large sentence length n. If true, this shows that memory constraints are encoded inside the grammar formalisms themselves, undermining the parsimonious idea that linguistic productivity can be described independently of non-linguistic cognition.","feed_headline":"Uniformly sampled grammar structures keep word distances below chance","feed_subtitle":"Billions of dependency trees from standard grammar classes show memory limits are baked into syntax.","key_machinery":"The central object is the average dependency distance <d>, defined as the mean over dependencies of the linear distance, in number of intermediate words plus one, between a head and its dependent in a dependency tree. The argument compares <d>_AS, the average over uniformly sampled artificial syntactic structures from a given grammar class, with the random linear arrangement baseline <d>_rla = (n+1)/3. The machinery is the uniform sampler over labelled directed trees, built via Prüfer codes and rooting, that produces artificial ensembles for n up to 25, making the comparison free of any direct memory constraint.","core_discovery":"The central discovery is that classes of dependency grammars constrain average dependency distance below chance even when no direct memory constraint is imposed. For a sentence of n words, a random linear arrangement of words yields an expected average dependency distance of (n+1)/3; the paper shows that uniformly sampling valid dependency trees from projective, planar, WG1, MH4, MH5, and 1EC classes gives average distances systematically below this baseline for sufficiently large n, with projective and planar classes showing the strongest reduction. The paper interprets this as evidence that memory limitations have permeated grammatical descriptions, making a competence grammar independent of cognitive constraints unlikely.","pith_inferences":["A next step would be to test whether the below-chance effect persists when sampling is weighted by real sentence-length distributions or by attested structures, which would reveal how strong the hidden constraint is relative to actual usage biases.","The same uniform-sampling test could be applied to non-dependency formalisms by defining a distance measure on phrase-structure trees, to see whether memory constraints are universal across grammatical frameworks or specific to dependency-based ones.","The ordering of distance reduction across classes, projective strongest, then planar, then mildly non-projective, suggests a formal hierarchy of implicit memory cost that could be used to compare parsing algorithms by the load they impose.","These results predict that even artificial languages engineered with such grammars, for example in laboratory learning experiments, should show easier processing than surface statistics alone would predict, a directly testable consequence."],"forward_implications":["If correct, the cognitive independence assumption underlying much of generative linguistics loses support: formal grammars are not neutral characterizations of sentencehood but encode memory-related restrictions.","The close derivational ties between dependency grammar and categorial, phrase-structure, and minimalist grammars suggest those frameworks likely embed similar hidden constraints, so the finding is not an artifact of one formalism.","Language acquisition may be explainable by general cognitive constraints such as working memory rather than language-specific innate grammatical knowledge, because the class of possible structures is already memory-shaped.","In language engineering, systems designed to interact with humans may perform better if deliberately constrained to human-like memory limits instead of being given unbounded computational capacity.","The result strengthens the view that the scarcity of crossing dependencies in natural language reflects dependency distance minimization, a processing pressure, rather than an arbitrary grammatical rule."],"supporting_citations":[{"why":"Supplies the random baseline formula (n+1)/3 and earlier statistical evidence that attested dependency distances are below it.","marker":"(25)"},{"why":"Justifies using dependency distance as a proxy for memory constraints, grounding the paper's interpretation.","marker":"(22,23)"},{"why":"Earlier observation of below-baseline distances in projective trees without controlling for sentence length, which the paper revisits with length control and artificial trees.","marker":"(26)"},{"why":"Length-controlled comparison using attested trees and a marginal class; the paper improves on it by using artificial trees free of memory constraints.","marker":"(27)"},{"why":"Provides the formal definitions and coverage statistics of the mildly non-projective classes (WG1, MH4, MH5, 1EC) that the paper samples.","marker":"(33)"},{"why":"Introduces the method for generating random structures from an undirected tree plus a linear arrangement, generalized here to uniform sampling over dependency structures.","marker":"(47)"},{"why":"Prüfer codes are the basis for generating uniformly random labelled trees in the sampling procedure.","marker":"(48)"}],"fun_headline_variants":["Grammar hides memory limits in dependency distance","Dependency grammar keeps word distance below chance","Memory constraints are baked into grammar itself","Why grammar reveals human memory limits","Grammar's hidden tie to memory limitations"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire argument rests on treating average dependency distance as a valid proxy for memory limitations; if dependency distance does not actually track working-memory load during sentence processing, then the below-chance distances in artificial grammar classes would not establish that memory constraints are hidden in grammar.","fun_headline_variants_meta":{"raw":{"variants":["Grammar hides memory limits in dependency distance","Dependency grammar keeps word distance below chance","Memory constraints are baked into grammar itself","Why grammar reveals human memory limits","Grammar's hidden tie to memory limitations"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00037,"raw_usage":{"total_tokens":1897,"prompt_tokens":775,"completion_tokens":1122,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":391,"completion_tokens_details":{"reasoning_tokens":1061}},"tokens_in":391,"tokens_out":1122,"duration_ms":8914,"temperature":1.0,"reasoning_tokens":1061,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T12:38:41.888188+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A controlled processing experiment could settle the proxy question: if readers show no measurable increase in processing effort, such as reading time, as average dependency distance rises while other factors are held fixed, the proxy fails and the inference from grammar classes to memory constraints collapses. Alternatively, exhibiting a practically motivated mildly non-projective class whose uniform random sample has average dependency distance at or above (n+1)/3 for large n would refute the universality of the effect.","supporting_citations":[],"review_version":1}