{"id":"d30c0d7d-fdd6-451e-8b8b-84be845dadb4","arxiv_id":"2506.09218","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A new probe that bypasses the fully-connected layer shows convolutional layers of a small-bottleneck WaveGAN can reflect a phonotactic restriction learned from lexical training data.","lead":"This paper tests whether the convolutional layers of a speech-generating neural network learn sound-combination rules on their own, apart from the word-memory layer. The authors bypass the memory layer and feed random patterns into the convolution layers, and in a small-bottleneck model the generated sounds still obey a training-set rule about /s/ before vowels.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'equally biased' conclusion rests on a non-significant interaction in a regression that ignores model-level clustering with one model per condition; this cannot support the central claim.","rationale":"I agree with the reader's conditional assessment, but the single most load-bearing problem is not the distributional overlap of random feature maps. Even if random maps are somewhat out-of-distribution, the between-condition comparison of Conv-only outputs could still reveal learned differences; the immediate threat is statistical. The regression treats outputs as independent when they are nested in one model per condition, and the 'equally biased' wording requires demonstrating equivalence, not merely failing to reject an interaction. The paper's qualitative observations—8ch Conv-only outputs being spectrally structured and showing a VOT pattern consistent with the training lexicon—are genuinely suggestive and the proposed probing technique is worth pursuing. However, the central quantitative claim cannot be accepted without model-level replication and a proper equivalence analysis. Since these are addressable in revision, the conditional verdict stands. The reader's weakest-assumption pick (feature-map distribution overlap) is a real limitation but is secondary to the statistical inference problem; hence partial agreement.","tokens_in":6753,"tokens_out":5094,"duration_ms":60751,"concrete_test":"Train at least 8 model instances per condition (16 total, varying only the random seed), sample the same number of FC and Conv-only outputs from each, and fit VOT ~ condition * generation_method with model as a random intercept. Then run a two-one-sided equivalence test on the interaction with a pre-registered bound (e.g., ±5 ms). If the 90% confidence interval for the interaction excludes the bound, or if model-level random variance accounts for most of the variance, the 'equally biased' claim is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim that Conv-only outputs are 'equally biased' by the training phonotactic restriction is not established by the reported statistics. In Section III-D, the interaction term in VOT ~ condition + conv_only + condition:conv_only is non-significant (estimate = -0.0076, SE = 0.006, p = 0.177), and this non-significance is treated as evidence that the FC is not involved. But a non-significant interaction is not evidence of equal effects; the point estimate actually implies the conv-only training effect is about 36% smaller than the FC effect (0.0134 vs 0.021), and its confidence interval is not reported. More fundamentally, only one generator was trained per condition, yet 200 outputs per model are entered as independent observations. The reported p-values are therefore anti-conservative, and the between-condition difference could be driven by a single lucky or unlucky initialization. Without model-level replication and an explicit equivalence or Bayesian test, the data cannot distinguish the intended conclusion from a probe artifact or seed-specific noise.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes and tests a technique for probing lexically-independent phonetic generalizations in generative CNNs trained on raw audio waveforms. The method consists of bypassing the fully-connected (FC) layer and feeding randomized feature maps directly into the convolutional stack, under the hypothesis that a sharply reduced FC bottleneck (8 channels versus 1024) makes random feature maps more representative of those encountered during training. Using ciwGAN generators trained on two artificial lexica that differ in whether /s/ can precede a vowel, the paper reports that conv-only outputs from the 8-channel model are spectrally structured and that their /s/-vowel VOTs appear to reflect the training condition. A linear regression with a non-significant interaction is interpreted as showing that bypassing the FC does not change the model's adherence to the phonotactic restriction, leading to the abstract claim that conv-only outputs are 'equally biased' by the training phonotactics.","tokens_in":6943,"tokens_out":3004,"duration_ms":34061,"significance":"If the central claim were established, the proposed technique would be a valuable addition to the interpretability toolkit for generative speech models, offering a way to separate lexically-constrained FC representations from locally generalizable convolutional statistics. The paper also demonstrates a practically interesting result: a ciwGAN generator with fewer than 1% of the FC parameters can still produce interpretable, variable acoustic outputs in the conventional generation mode, and the qualitative spectrogram comparison between 8ch and 1024ch conv-only outputs is striking. The work is clearly relevant to computational phonology and to interpretability research on GAN-based speech synthesis. However, the statistical evidence underpinning the 'equally biased' claim is not yet load-bearing, and the validity of the probe depends on an unmeasured distributional assumption about random feature maps.","major_comments":[{"comment":"The central claim that conv-only outputs are 'equally biased' by the training phonotactic restriction is not supported by the reported regression. A non-significant interaction (estimate = -0.0076, SE = 0.006, p = 0.177) is not evidence of equality; the point estimate implies the conv-only training effect is about 36% smaller than the FC effect (0.0134 vs 0.021), and the confidence interval for the interaction is not reported. The authors should either report an explicit equivalence test (e.g., TOST with a pre-specified bound) or a Bayesian analysis that can quantify evidence for the absence of an interaction, and they should temper the abstract's wording accordingly.","section":"Abstract and §III-D"},{"comment":"The statistical analysis treats each of the 200 generated outputs per condition as an independent observation, but only one generator was trained per training condition. This is a severe form of pseudoreplication: outputs from the same model are correlated draws, and the between-condition difference could be driven by a single favorable or unlucky initialization. The authors should train multiple models per condition (different random seeds) and fit a mixed-effects model with model as a random effect, or otherwise account for the clustering. Without this, the reported p-values are anti-conservative and cannot support the paper's quantitative claims.","section":"§III-C and §III-D"},{"comment":"The load-bearing assumption of the proposed technique is that random 8x16 feature maps resemble the feature maps the convolutional stack encountered during training, so that conv-only outputs reflect learned convolutional statistics rather than out-of-distribution artifacts. The paper explicitly hedges this ('may make it likelier') but never measures the distributional overlap. The authors should quantify the distance or overlap between the distribution of FC-produced feature maps and the distribution of random feature maps sampled for the probe (e.g., using a classifier to distinguish them, or a Wasserstein distance in feature-map space), and show that the conv-only outputs lie within the support of the training-time feature-map distribution. Without such a check, the observed VOT differences could be artifacts of the probe rather than learned phonotactic generalizations.","section":"§II-B and §III-B"}],"minor_comments":[{"comment":"The reported p-value 'p < 2e-16' is inconsistent with the reported t = 5.12 for a binary predictor; for a two-sided test with approximately 800 degrees of freedom, p is about 3.5e-7. Please report the exact p-value and the degrees of freedom.","section":"§III-D"},{"comment":"There is a grammar error: 'This paper propose sampling outputs' should be 'This paper proposes sampling outputs.'","section":"§II-A"},{"comment":"The acknowledgment contains a typo: 'UC Berkely' should be 'UC Berkeley.'","section":"§VI"},{"comment":"The manual VOT annotation is described as performed by the author alone, with no report of intra-annotator reliability or a second annotator. Given that the main dependent measure is manual, the paper should report reliability statistics or show that the annotation protocol is replicable.","section":"§III-C"},{"comment":"Details of the training setup are sparse: the number of random seeds per condition, the checkpoint selection criterion (the paper reports results at 1350 epochs without explaining why this checkpoint was chosen), and the exact hyperparameters beyond 'following [16]' should be specified for reproducibility.","section":"§III-B"},{"comment":"Figure 6 shows only point patterns; adding confidence intervals, boxplots, or a jittered density would make the variability and overlap across conditions more transparent.","section":"Figure 6"}],"recommendation":"major_revision","confidential_remarks":"The paper presents a novel and potentially useful probe, and the qualitative spectrogram results are compelling. The main obstacle is that the central claim of 'equally biased' conv-only outputs rests on a statistically weak equivalence argument and on an unmeasured assumption about feature-map distribution overlap. These issues are fixable within the scope of the manuscript (with more models, better statistics, and a distribution-overlap check), so I do not recommend rejection. The author's reliance on their own prior work is understandable given the niche; however, the manuscript would benefit from acknowledging that the proposed technique works only under the narrow-bottleneck condition and cannot currently speak to the original 1024-channel architecture."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The one thing to know: this is a genuinely new probing technique for generative speech CNNs, and the qualitative result—that a narrow FC bottleneck lets you sample interpretable audio by feeding random feature maps into the conv stack—is real. But the abstract's \"equally biased\" claim is not supported. The interaction test is non-significant; that is not equivalence, and the point estimate actually puts the conv-only effect about 36% lower. On top of that, only one model per training condition exists, yet 200 outputs per model are treated as independent observations. The reported p-values are anti-conservative.\n\nWhat's good: shrinking the FC to 8 channels to make random feature maps plausible is clever and is explicitly motivated by the redundancy of the 1024-channel FC. The paper is honest about the qualitative nature of the spectrogram differences. The regression is simple and the main effect of training condition is in the expected direction. The write-up is clear, and the relevant literature is engaged.\n\nSoft spots, in order: (1) The statistical argument for \"equal bias\" needs an equivalence test or a Bayesian interaction; as it stands, the data cannot distinguish equal from merely non-significant. (2) Model-level replication is missing. Train several seeds per condition and fit a mixed model with random intercepts for model and word. (3) Manual annotation reliability is unreported: one annotator, no agreement. (4) No code or data released, which matters for a method paper. (5) The in-distribution concern about random feature maps is acknowledged in Section II-B but never checked; that's a caveat, not a fatal flaw.\n\nThe citation pattern is heavy on the author's own prior work and Beguš, but in this small niche that is legitimate—the method builds directly on ciwGAN.\n\nWho it's for: researchers studying phonological learning in deep generative models and interpretability of speech GANs. It deserves a serious referee, but the revision needs to either soften the central claim or add the missing replication and equivalence testing. I'd engage with it, not as is.","headline":"New probe, real idea, but the central claim of equal bias overreaches the statistics: non-significant interaction, no model replication.","tokens_in":7446,"tokens_out":2241,"would_cite":true,"duration_ms":24629,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A generative CNN trained on word waveforms learns a phonotactic restriction—/s/ not before a vowel—in its convolutional layers alone, not just in its fully-connected layer, provided the fully-connected bottleneck is narrow.","keywords":["phonotactics","generative CNNs","WaveGAN","fully-connected bottleneck","voice onset time","latent space probing","speech synthesis","lexical learning"],"falsifier":"Measure the distributional overlap between random 8x16 feature maps and FC-produced feature maps from training, for example by projecting both sets into PCA space and estimating density; if random maps fall largely outside the training-time support, the conv-only outputs are probes of out-of-distribution inputs and cannot establish a learned phonotactic generalization. A second check: run the same conv-only probe on an untrained 8-channel generator and test whether the /s/-vowel VOT bias persists.","tokens_in":6528,"feed_emoji":"🎙️","tokens_out":5825,"duration_ms":55491,"temperature":0.7,"pith_summary":"The paper asks whether a generative CNN's convolutional layers can learn phonetic dependencies that are independent of the lexical templates stored in its fully-connected layer. It proposes a probe: shrink the FC bottleneck from 1024 channels to 8, train on raw waveforms of words that either forbid or allow /s/ before a vowel, then generate outputs by feeding random feature maps directly into the convolutional stack. The resulting conv-only outputs from the 8-channel model are interpretable speech-like sounds, and their /s/-vowel voice onset times carry the same training-data bias as outputs generated through the FC. The paper takes this as evidence that convolutional layers can dynamically generalize local phonotactic dependencies beyond lexically-constrained FC configurations.","feed_headline":"Random-input probe shows convolutional layers learn phonotactics alone","feed_subtitle":"Squeezing the fully connected layer to 8 channels lets random feature maps reveal /s/-vowel timing learned from words.","key_machinery":"The central object is the generator's convolutional stack, treated as a translation-invariant transducer of feature maps. The paper's move is to replace the FC-produced feature map with an 8x16 random feature map drawn from a uniform distribution, forcing the convolutional layers to produce the waveform without lexical templates. The narrow 8-channel bottleneck is the enabler: it shrinks the space of feature maps so that random samples plausibly resemble training-time inputs, whereas the 1024-channel space is too sparsely sampled. The measured quantity is the /s/-vowel voice onset time (VOT) in generated waveforms, with longer VOT interpreted as a learned bias against /s/ directly before a vowel.","core_discovery":"The paper claims that WaveGAN-style generative CNNs trained on raw word waveforms encode a phonotactic restriction in the convolutional layers themselves, not only in the fully-connected layer. With the FC bottleneck reduced from 1024 to 8 channels, outputs generated by feeding uniform-random feature maps directly into the convolutional stack remain linguistically interpretable, and their /s/-vowel VOTs are biased in the same direction as outputs generated through the FC. A linear regression finds that training condition significantly predicts VOT, that bypassing the FC has no main effect on VOT, and that there is no interaction between training condition and bypassing the FC. The paper reads this as showing the convolutional layers alone reproduce the training-data phonotactic pattern, while the 1024-channel model produces only noisy, uninterpretable conv-only outputs.","pith_inferences":["The technique should transfer to other locally-defined phonetic dependencies, such as vowel harmony or nasalization, where the dependent variable would be a different acoustic measurement rather than VOT.","A stronger test would compare conv-only outputs against those of an untrained 8-channel generator; if the untrained model already shows an SV-timing bias, the result would reflect architectural priors rather than learning.","The paper's analogy to n-phone models suggests conv-only outputs could be treated as a sampled distribution over sublexical sequences, opening the door to quantitative comparisons between the model's implicit phonotactic grammar and explicit phonotactic learners.","Because the 8-channel bottleneck changes what the FC can represent, the claimed dissociation between FC and convolutional layers may not transfer to standard 1024-channel models; the claim of lexical independence is conditional on the bottleneck manipulation."],"forward_implications":["The convolutional stack of a generative CNN trained on raw waveforms can represent a local phonotactic restriction without the fully-connected layer.","An 8-channel FC bottleneck yields interpretable, variable speech-like outputs, whereas a 1024-channel bottleneck produces noisy conv-only outputs.","Bypassing the FC with uniform-random feature maps is a viable interpretability probe only in the narrow-bottleneck regime.","The absence of an interaction between conv-only generation and training condition indicates the FC is not the locus of the learned /s/-vowel restriction.","Reducing the FC to under 1% of its original parameters does not destroy the model's ability to reproduce training-like lexical outputs."],"supporting_citations":[{"why":"Defines the ciwGAN objective used to train generators that yield interpretable lexical outputs.","marker":"[16]"},{"why":"Supplies the WaveGAN generator architecture that the convolutional stack and FC bottleneck modify.","marker":"[17]"},{"why":"Motivates shrinking the FC bottleneck by showing that information is redundantly encoded across feature-map channels.","marker":"[20]"},{"why":"Provides the ciwGAN schematic and the training setup that the experiment follows.","marker":"[23]"},{"why":"Generates the 300 acoustic tokens per lexical type used as training data.","marker":"[24]"},{"why":"Provides Praat, used to annotate /s/ and vowel intervals for VOT measurement.","marker":"[25]"},{"why":"Provides R's glm, used to fit the VOT regression testing training condition and conv-only effects.","marker":"[26]"}],"fun_headline_variants":["Tiny bottleneck reveals phonotactic learning in CNN conv layers","Random features expose phonetic rules hidden in conv layers","Squeezed model proves conv layers encode sound patterns","Bypassing FC shows conv layers learn phonotactics","8-channel bottleneck isolates phonetic generalizations"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The argument assumes that uniformly random 8x16 feature maps are representative of the feature maps the convolutional stack encountered during training, so that the conv-only outputs expose learned convolutional statistics rather than responses to out-of-distribution inputs.","fun_headline_variants_meta":{"raw":{"variants":["Tiny bottleneck reveals phonotactic learning in CNN conv layers","Random features expose phonetic rules hidden in conv layers","Squeezed model proves conv layers encode sound patterns","Bypassing FC shows conv layers learn phonotactics","8-channel bottleneck isolates phonetic generalizations"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000494,"raw_usage":{"total_tokens":2383,"prompt_tokens":860,"completion_tokens":1523,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":476,"completion_tokens_details":{"reasoning_tokens":1458}},"tokens_in":476,"tokens_out":1523,"duration_ms":11243,"temperature":1.0,"reasoning_tokens":1458,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T04:53:31.005204+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure the distributional overlap between random 8x16 feature maps and FC-produced feature maps from training, for example by projecting both sets into PCA space and estimating density; if random maps fall largely outside the training-time support, the conv-only outputs are probes of out-of-distribution inputs and cannot establish a learned phonotactic generalization. A second check: run the same conv-only probe on an untrained 8-channel generator and test whether the /s/-vowel VOT bias persists.","supporting_citations":[{"cited_title":"Ciwgan and fiwgan: Encoding information in acoustic data to model lexical learning with generative adversarial networks,","cited_arxiv_id":null,"evidence_quote":"Defines the ciwGAN objective used to train generators that yield interpretable lexical outputs."},{"cited_title":"Exploring the encoding of linguistic representations in the Fully-Connected Layer of generative CNNs for Speech","cited_arxiv_id":"2501.07726","evidence_quote":"Motivates shrinking the FC bottleneck by showing that information is redundantly encoded across feature-map channels."},{"cited_title":"SpeechT5: Unified- modal encoder-decoder pre-training for spoken language processing,","cited_arxiv_id":null,"evidence_quote":"Generates the 300 acoustic tokens per lexical type used as training data."},{"cited_title":"Praat: doing phonetics by computer [Computer program],","cited_arxiv_id":null,"evidence_quote":"Provides Praat, used to annotate /s/ and vowel intervals for VOT measurement."},{"cited_title":"[Online]","cited_arxiv_id":null,"evidence_quote":"Provides R's glm, used to fit the VOT regression testing training condition and conv-only effects."}],"review_version":1}