{"id":"2f052d6e-aca5-49a4-8941-d6855aea20fc","arxiv_id":"2608.05028","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"LMs trained on corpora that never show multi-modifier noun phrases still prefer English's scope-homomorphic modifier order, and noun-modifier association strength does not explain this preference.","lead":"A controlled corpus experiment shows that language models trained without any multi-modifier noun phrases still prefer the scope-homomorphic order of English modifiers. The result bears on whether linguistic biases like word-order preferences are learned from experience or require innate constraints.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Parser-filter recall is the load-bearing premise: the central claim requires that no multi-modifier NPs survived in training, but the Stanza-based filter's recall is never validated and the paper's own 'that one' exception shows the mechanism is fallible.","rationale":"I read the paper in good faith. The experiment is well-designed: a clean FiCT setup, three model sizes, multiple seeds, and an evaluation that plausibly reflects a learned preference. The central result—positive HC deltas across model sizes—is internally consistent and the paper honestly reports its limitations, including the divergence from human Dem-Adj patterns. The reader's weakest assumption is exactly the right point: the claim that models 'never observing multi-modifier NPs during training' is the load-bearing premise, and it depends on an unvalidated parser filter. Appendix A's explicit admission that Stanza misparses clause-initial 'that one' makes the fallibility concrete; the authors patch one pattern but do not establish that no other patterns leak. Because the entire poverty-of-stimulus interpretation collapses if direct evidence remains, this is the single most load-bearing concern. My proposed audit is a direct, feasible test. I do not see a reason to move the reader's verdict: CONDITIONAL remains appropriate until the recall of the filter is measured and, if necessary, the experiment is rerun on a verified-clean corpus.","tokens_in":11007,"tokens_out":6361,"duration_ms":89978,"concrete_test":"Run a recall audit on the filter: sample 1,000 sentences from the 5,884,372 sentences classified as simple-NP, independently parse them with a second parser (e.g., spaCy transformer or UDPipe), and manually adjudicate all disagreements, counting any NP with a shared noun head modified by a demonstrative, numeral, or adjective (including coordinated and appositive cases). If any missed multi-modifier NP is found, estimate recall with a confidence interval; then retrain the models on a corpus with the missed sentences additionally filtered and check whether the HC preferences persist. If preferences disappear, the central claim is unsupported; if they persist, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The strongest claim in the Discussion is that models 'never observ[e] noun phrases containing multiple modifiers during training.' That premise rests entirely on the Stanza dependency-parsing pipeline described in §3.1 and Appendix A. The paper reports that 148,367 of 6,032,739 sentences contained complex NPs and were filtered, but it never measures the filter's recall against a gold standard. The appendix itself shows the parser systematically misparses at least one construction ('that one' as SCONJ instead of demonstrative), which the authors patch with an explicit pattern filter rather than validating the whole extraction. Other constructions are plausible miss candidates: coordinated adjectives ('red and blue'), appositive modifiers, complex numeral expressions, and clause-initial demonstrative+numeral strings. If even a small fraction of multi-modifier NPs survive—say 1% of the 148,367 detected—more than a thousand multi-modifier NPs would remain in the 100M-token corpus; after 40 epochs the model would see each many times. The observed Dem-Num preferences could then reflect direct exposure to 'those two dogs' rather than generalization from impoverished input. This is not an internal inconsistency, but it is a correctness risk on the paper's central empirical claim. A recall audit would settle it; without one, the 'poverty of the stimulus' framing is conditional.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper asks whether Transformer language models acquire the scope-homomorphic noun-phrase ordering bias under poverty-of-the-stimulus conditions. The authors filter a 100M-token Wikipedia sample by removing or perturbing sentences containing multi-modifier NPs, train OPT-style models of 52M, 110M, and 350M parameters with five seeds each, and evaluate them on minimal pairs of the form Dem-Num, Dem-Adj, and Num-Adj. They report HC_delta and HC% values above chance across all sizes, with Dem-Num strongest and Dem-Adj weakest, and then test whether noun-modifier PMI predicts item-level preferences, finding no reliable relationship. The central claim is that models develop a human-like ordering preference despite never observing multi-modifier NPs during training.","tokens_in":11231,"tokens_out":6694,"duration_ms":94979,"significance":"If the central premise is sound, this is a meaningful contribution: it extends Filtered Corpus Training to a well-known ALL bias, uses multi-seed and multi-size training with item-level statistics, and reports a clean null result for PMI. The paper also gives a controlled framework for asking which distributional cues are sufficient to induce scope-homomorphic preferences. However, the headline finding is conditional on the corpus filter's recall, which is not validated against any gold standard; the strength of the 'poverty of the stimulus' interpretation therefore depends entirely on a single, fallible parsing pipeline.","major_comments":[{"comment":"The Discussion's claim that the models 'never observ[e] noun phrases containing multiple modifiers during training' rests entirely on the Stanza-based filter. The manuscript reports that 148,367 of 6,032,739 sentences were flagged, but it never measures the filter's recall against a gold standard. The appendix itself documents a systematic parser failure ('that one' as SCONJ), patched by an explicit pattern filter, which demonstrates that the detection mechanism is imperfect; coordinated modifiers, appositive modifiers, and complex numeral/demonstrative strings are plausible additional miss classes. A false-negative rate of even 1% would leave more than a thousand multi-modifier NPs in the 100M-token corpus, and after 40 epochs the model would see each many times. Please provide a recall audit on a gold-standard sample of Wikipedia sentences, or explicitly restrict the claim to 'no detected multi-modifier NPs' and discuss the consequences of residual direct evidence for the main result. The Limitations section's concession that the corpus 'may contain indirect evidence about modifier relationships' should also be reconciled with the stronger formulation in the abstract and Discussion.","section":"§3.1, Appendix A"},{"comment":"The evaluation corpus mixes 25 train-sourced and 25 test-sourced items per modifier pair, and the text says that test-sourced items may contain individual modifiers not seen during training. If these items contain lexical items that were not in the 100M-token training sample, they are not directly comparable to train-sourced items: the model could be making decisions about unseen word forms rather than about modifier ordering. This is especially relevant for the weak Dem-Adj and Num-Adj effects and for the mixed-effects model in Table 2. Please report train-sourced and test-sourced items separately, clarify how the test split was sampled relative to the training corpus, and state whether any modifier or noun tokens were out-of-vocabulary or unseen in training.","section":"§3.4, §4.3"}],"minor_comments":[{"comment":"The text says 148,367 complex-NP sentences were 'filtered out' but then says they were 'perturbed' to create 664,272 rewritten variants that were included in the corpus; this wording is confusing and should be clarified so that readers can tell whether the original multi-modifier sentences were fully replaced by their single-modifier variants.","section":"Appendix A"},{"comment":"The caption contains a typo: 'Homomorphic consistency elta' should read 'Homomorphic consistency delta'.","section":"Figure 4"},{"comment":"The item-level PMI analysis reports Pearson correlations and agreement percentages but no confidence intervals; given that the correlations are small and the agreement is near chance, reporting uncertainty around these estimates would make the null result more interpretable.","section":"§5.2.2"}],"recommendation":"major_revision","confidential_remarks":"The paper is not ready for acceptance in its current form because the central empirical claim depends on the unvalidated recall of the Stanza-based filter. If the authors can provide a credible recall audit or suitably weaken the 'never observe' claim, the contribution would be publishable. I saw no citation or scope concerns; the paper fits a computational linguistics audience."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a useful paper with a real empirical result, and the authors are honest about its limits. The one thing to watch is the load-bearing premise—'never observing multi-modifier NPs'—which depends on a parser filter whose recall is never checked.\n\nWhat's new: they extend FiCT to scope-homomorphic ordering, train three model sizes on a 100M-token Wikipedia corpus stripped of multi-modifier NPs, and get consistent preferences (HC% 70-76%) for homomorphic orders, strongest for Dem-Num. The PMI analysis is a clean negative result: corpus-level PMI doesn't predict item-level preferences. They also check and rule out some obvious confounds (postnominal modifiers, demonstrative-as-head numerals). The mixed-effects model is reasonable. That's solid work.\n\nSoft spots, in proportion. The big one is the filter. They report 148,367 of 6M+ sentences flagged and removed, but never validate recall against gold-standard parses. They even found one systematic misparse themselves ('that one' as SCONJ) and patched it with a pattern filter. That's honest, but it also signals other constructions could slip through—coordinated adjectives, appositives, complex numerals. If even 1% survive, thousands of multi-modifier NPs remain in a 100M-token corpus, and after 40 epochs the model sees them repeatedly. That would still allow direct exposure to 'those two dogs'. The PoS framing in the Discussion is therefore conditional. This is not a fatal flaw—the preference could still emerge from indirect evidence—but the strong 'never observing' claim needs a recall audit before it can be stated that way.\n\nMinor: evaluation is 150 items total, so the by-combination breakdown is 50 per pair; no analysis of whether training-sourced vs test-sourced items differ. Fine for a first pass, worth noting. Also no control training (e.g., unfiltered corpus) to isolate the effect of the perturbation, though the design and the PMI null give some protection.\n\nWho this is for: people working on LM learnability, ALL, and cognitive biases in language. It's a clean demonstration that a general system can induce a structure-dependent preference from impoverished input, which is exactly what the ALL literature wants to know. I'd send it to a competent referee who knows FiCT and can scrutinize the corpus pipeline. Value comes from the method and the negative PMI result as much as the headline preference.\n\nRecommendation: yes, send to peer review. Ask the authors to validate filter recall and soften the 'never' claim until they do.","headline":"The core FiCT result is real and worth engaging; the 'never observed' claim is only as strong as an unvalidated parser filter.","tokens_in":11797,"tokens_out":1788,"would_cite":true,"duration_ms":21226,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that language models trained on a corpus with every multi-modifier noun phrase removed still develop a reliable preference for scope-homomorphic modifier order, matching human artificial-language-learning results.","keywords":["language models","modifier ordering","scope-homomorphism","artificial language learning","poverty of the stimulus","filtered corpus training","pointwise mutual information","inductive bias"],"falsifier":"Audit the training corpus (or the filtering pipeline) against a gold-standard set of noun phrases with multiple modifiers and count how many multi-modifier noun phrases survive the perturbation; if the count is nonzero and large enough to provide signal, the claim that models never observed the target structure is false.","tokens_in":10765,"feed_emoji":"🧠","tokens_out":8564,"duration_ms":97410,"temperature":0.7,"pith_summary":"This paper asks whether a general-purpose language model can acquire a human-like bias for ordering modifiers inside a noun phrase when the training data contains no noun phrases with more than one modifier. The authors build a 100-million-token English corpus in which every multi-modifier noun phrase detected by a dependency parser is rewritten into single-modifier variants, train decoder-only transformers of three sizes on it, and then test on minimal pairs that contrast the homomorphic order with the swapped order. All three models reliably prefer the homomorphic order—'two fluffy dogs' over 'fluffy two dogs'—even though they never saw such phrases in training. The preference is strongest for demonstrative-numeral combinations and weakest for demonstrative-adjective combinations, and it is not explained by noun-modifier association strength measured by pointwise mutual information. If the result holds, it suggests that a widely documented human learning bias can emerge from impoverished input in a general learning system, without an innate, language-specific constraint.","feed_headline":"LMs learn human-like word order from data that hides it","feed_subtitle":"Trained with all multi-modifier NPs stripped out, three model sizes still pick scope-homomorphic order.","key_machinery":"The central mechanism is filtered corpus training (FiCT): every sentence containing a noun phrase with multiple target modifiers—demonstratives, numerals, and adjective blocks—is decomposed into variants that keep only one modifier per noun, so the training corpus contains only single-modifier noun phrases. The target generalization, scope-homomorphism, is the hypothesis that the linear order of modifiers mirrors their compositional semantic scope, so that wider-scope modifiers come first. Evaluation uses minimal pairs of the form $M_1 M_2 N$ versus $M_2 M_1 N$, scored by the log-probability difference $HC_\\Delta(M_1M_2N|c) = \\log P(M_1M_2N|c) - \\log P(M_2M_1N|c)$ and by the proportion of items for which this difference is positive ($HC\\%$). Positive values indicate a preference for the order that matches semantic scope. The same measure is then compared against per-item differences in pointwise mutual information between the noun and each modifier, which is the distributional account the paper tests and rejects.","core_discovery":"Under filtered-corpus training conditions that remove all direct evidence of multi-modifier noun phrase order, OPT-style decoder-only language models with 52M, 110M, and 350M parameters develop a statistically reliable preference for scope-homomorphic modifier orders. Across model sizes, homomorphic consistency accuracy is 70–76% and mean log-probability differences are positive, with the largest effects for demonstrative-numeral orderings. The models' modifier-type profile differs from humans—demonstrative-adjective is the strongest preference in human experiments but the weakest here—so the result is not a simple copy of the human pattern. Item-level analysis with pointwise mutual information shows that noun-modifier association strength does not predict model preferences, ruling out one distributional account of the bias. The paper concludes that the preference can be recovered from impoverished input, though the exact representational or statistical source remains open.","pith_inferences":["Editorial extension: a synthetic corpus with known noun-phrase structure would let researchers verify the filter's recall directly and test the same question without parser noise; this is the cleanest way to confirm the poverty-of-stimulus condition.","Editorial extension: if the training corpus's modifier classes were statistically permuted—for example, by making demonstratives the most noun-associated modifiers—the PMI account would predict a different preference profile; observing whether model preferences follow would distinguish structural from associative learning.","Editorial extension: applying the same procedure to a language whose dominant modifier order differs from English would test whether the learned preference tracks scope-homomorphism as a universal or merely English surface distributions.","Editorial extension: the near-chance PMI agreement leaves open the possibility that a context-sensitive association measure, rather than raw PMI, explains the item-level choices; this can be tested without changing the experimental design."],"forward_implications":["If the central claim is right, an innate language-specific constraint is not required for scope-homomorphic modifier order to emerge; a general sequence learner can retrieve the bias from indirect evidence.","The same filtered-corpus protocol can be applied to other proposed linguistic universals, turning each into a testable question about what can be learned when direct evidence is absent.","Because pointwise mutual information fails to explain model preferences, the source of the bias must lie in other distributional or structural properties of single-modifier input; identifying that property becomes the natural next step.","The cross-modifier contrast with human results (Dem-Num strongest in models, Dem-Adj strongest in humans) means any shared bias must be flexible in its surface strength, not a rigid universal ordering rule.","Model scale does not appear to be a prerequisite for this generalization: all three sizes show comparable preferences, so larger models are not needed to reproduce the effect."],"supporting_citations":[{"why":"Establishes the human artificial-language-learning preference for scope-homomorphic modifier order that this paper's models are compared against.","marker":"Culbertson and Adger (2014)"},{"why":"Introduces filtered corpus training (FiCT), the method this paper adapts to remove multi-modifier noun phrases.","marker":"Patil et al. (2024)"},{"why":"Provides the dependency parser used to detect and remove multi-modifier noun phrases, which the poverty-of-stimulus claim depends on.","marker":"Qi et al. (2020)"},{"why":"Proposes noun-modifier pointwise mutual information as the distributional source of scope-homomorphic preferences; this paper tests and rejects that explanation for models.","marker":"Culbertson et al. (2020)"},{"why":"Supplies a prior filtered-corpus demonstration that language models generalize to rare structures and the minimal-pair evaluation approach.","marker":"Misra and Mahowald (2024)"},{"why":"Provides human artificial-language-learning evidence that scope-homomorphic preferences persist, used to compare model and human modifier-type profiles.","marker":"Martin et al. (2020)"},{"why":"The English Wikipedia dump from which the 100M-token training corpus is sampled.","marker":"Wikimedia (2023)"},{"why":"Supplies cross-linguistic statistics on scope-homomorphic modifier order used to frame the universal's scope and future work.","marker":"Dryer (2018)"}],"fun_headline_variants":["LMs pick human-like word order despite zero training evidence","Filtered corpus, no modifier order: models still infer scope-homophony","AI learns word order bias from data that hides it","Three model sizes recover hidden human word order preference","No direct evidence? LMs generalize to scope-homomorphic modifier order"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole argument rests on the claim that the filtered training corpus truly contains no multi-modifier noun phrases; if the dependency parser missed any, the models could have seen direct evidence of the target ordering and the poverty-of-stimulus framing would collapse.","fun_headline_variants_meta":{"raw":{"variants":["LMs pick human-like word order despite zero training evidence","Filtered corpus, no modifier order: models still infer scope-homophony","AI learns word order bias from data that hides it","Three model sizes recover hidden human word order preference","No direct evidence? LMs generalize to scope-homomorphic modifier order"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000227,"raw_usage":{"total_tokens":1454,"prompt_tokens":908,"completion_tokens":546,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":524,"completion_tokens_details":{"reasoning_tokens":460}},"tokens_in":524,"tokens_out":546,"duration_ms":6739,"temperature":1.0,"reasoning_tokens":460,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T11:04:55.840237+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Audit the training corpus (or the filtering pipeline) against a gold-standard set of noun phrases with multiple modifiers and count how many multi-modifier noun phrases survive the perturbation; if the count is nonzero and large enough to provide signal, the claim that models never observed the target structure is false.","supporting_citations":[{"cited_title":"From the World to Word Order:","cited_arxiv_id":null,"evidence_quote":"Proposes noun-modifier pointwise mutual information as the distributional source of scope-homomorphic preferences; this paper tests and rejects that explanation for models."},{"cited_title":", year = 2018, journal =","cited_arxiv_id":null,"evidence_quote":"Supplies cross-linguistic statistics on scope-homomorphic modifier order used to frame the universal's scope and future work."}],"review_version":1}