{"id":"50f8c9b8-6b4a-49bf-80ff-cd77dc734bc4","arxiv_id":"2501.07726","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"Vowel-like columns of the fully connected layer of a speech GAN cluster by phonetic similarity across different words, and can be recombined into speech-like segments.","lead":"This paper opens up the fully connected layer of a speech-generating neural network and finds that the weights for individual words contain shared, reusable patterns for similar vowel sounds. It introduces two hands-on techniques for inspecting and recombining those weights, giving interpretability researchers a new way to probe what such models actually learn.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported vowel clustering may be a selection artifact: columns are chosen by inspecting their outputs, then those same columns are correlated, so the result could reflect the selection criterion rather than shared sublexical structure.","rationale":"The reader's weakest_assumption identifies exactly this selection issue: a single FC column chosen because it produces only a vowel is not validated as representative, and the clustering could be an artifact. I agree, and I sharpen the concern by pointing out the specific circularity: the selection criterion (vowel-only output) is the same modality used to define the vowel categories, so the selection and the outcome are not independent. Other concerns, such as the small number of columns (12) and the qualitative MDS interpretation, are secondary because more data alone would not fix the selection bias. The paper does introduce a genuinely novel manipulation of the FC layer and provides qualitative evidence that isolated columns can produce interpretable speech sounds, which is a useful methodological contribution. However, the strong claim that ciwGAN 'encodes lexical information in a linguistically principled manner' is only as strong as the selection procedure. A concrete automated selection rule and a permutation test would settle whether the observed correlation structure is specific to phonetic similarity. Since the current evidence is conditional on addressing this bias, the reader's CONDITIONAL verdict remains appropriate; my read does not change it, so I recommend UNCHANGED.","tokens_in":12753,"tokens_out":3571,"duration_ms":39808,"concrete_test":"Re-run the column-selection procedure automatically and blind to vowel labels. For each of the 9 latent codes, select the column by an objective criterion independent of auditory judgment, e.g., the column at the temporal midpoint of the highest-mean-absolute-weight region, or the column whose isolated output maximizes a periodicity/voicing-energy score. Then recompute the 12x12 FC-code correlation matrix and the spectral correlation matrix. Additionally, run a permutation test: assign the 12 vowel labels randomly to the selected columns 10,000 times, compute the mean within-category correlation under each shuffle, and compare the observed value to this null distribution. If the observed clustering falls outside the 95th percentile of the null, the result supports the claim; if not, the reported grouping is statistically indistinguishable from chance and the central claim is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim rests on Experiment 2 (Section 4). The authors extract one column per vowel from each latent code's FC weight matrix by \"pass[ing] individual feature columns ... and select[ing] the column that contains only vowel information in its output\" (Section 4, Fig. 9). This selection is manual, qualitative, and not independent of the outcome measure. The same selected columns are then used to compute the 12x12 FC-code correlation matrix (Fig. 10) and the spectral correlation matrix (Fig. 13). Because the selection criterion is 'produces only vowel information' - judged by the same acoustic output that defines vowel identity - the resulting clustering could be driven by a systematic property of the selected columns (e.g., cleaner formants, higher periodicity) rather than by the linguistic category itself. The paper provides no algorithmic selection rule, no inter-rater reliability, and no null baseline such as randomly selected columns or all 16 columns. Without such controls, the high/low vowel grouping in Figs. 10-11 is not sufficient evidence that 'similar-sounding vowels are encoded in similar ways in the FC layer' across lexically specific codes. This is a methodological circularity risk, not an accusation of intent; it is the single most load-bearing weakness because it directly undermines the paper's main conclusion.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes two interpretability techniques for the fully-connected (FC) layer of ciwGAN, a WaveGAN variant with categorical latent codes trained on TIMIT words. Experiment 1 shows that latent-code weight matrices have larger average magnitudes than noise-variable matrices and, when passed through the convolutional layers, generate the corresponding lexical items. Experiment 2 extracts individual FC columns that, in isolation, produce single vowels; comparing 12 such columns across 9 lexical codes, the paper reports that high vowels cluster together and low vowels cluster together in a correlation matrix and MDS plot, and that recombining vowel columns yields predictable /s/+vowel outputs. The central claim is that ciwGAN encodes lexical information in a linguistically principled way, with shared lexically-invariant sublexical representations in the FC layer.","tokens_in":13039,"tokens_out":3775,"duration_ms":40697,"significance":"If the central claim is substantiated, the paper makes a genuinely useful contribution to interpretability of generative speech models: it introduces weight-matrix-as-input probing and column/channel manipulation as tools that go beyond latent-space interpolation, and it provides evidence against purely holistic lexical encoding in a GAN generator. The work is exploratory and methodologically creative, and the channel-masking experiment (Fig. 17) is a nice illustration of how information can be localized. However, the main conclusion currently rests on a manual, potentially circular column-selection procedure and on descriptive statistics without significance testing, so the strength of the claim is not yet commensurate with the evidence.","major_comments":[{"comment":"The correlation and MDS analyses are purely descriptive, with no significance tests, confidence intervals, or effect sizes reported. With only 12 columns and strong dependencies among the 1024 values within each column, the observed high/low grouping could plausibly arise by chance. A permutation test (e.g., shuffling vowel-category labels or the set of columns) or a bootstrap over latent codes is needed before concluding that similar-sounding vowels are encoded similarly in the FC layer.","section":"Section 4.1, Figs. 10-13"},{"comment":"The isolated-column probe sets all other latent variables to zero and omits the trainable bias term, even though the convolutional layers were trained on feature maps that are sums of 100 scaled weight matrices plus a bias. The paper acknowledges this limitation for z-variables in Section 3.4 ('their outputs should be interpreted as mere approximations'), but it does not address the same issue in Experiment 2, where the isolated-column input is the basis for the main claim. The authors should show that the results are robust to including the bias term or to varying the other latent variables, or explicitly justify why an off-manifold single-column input is a valid probe of the FC layer's encoding.","section":"Section 2.2 and Section 4"}],"minor_comments":[{"comment":"There is a typo: 'have been frequently used for for image and audio generation' should read 'frequently used for image and audio generation.'","section":"Section 1.2"},{"comment":"The phrase 'we demonstrates that' should be 'we demonstrate that.'","section":"Section 3.2"},{"comment":"The caption states that high-front vowels have a bimodal peak 'representing a high F1 and low F2,' but high-front vowels have low F1 and high F2; mid-low vowels have higher F1 and lower F2. This should be corrected for accuracy.","section":"Section 4.1 (Fig. 12 caption)"},{"comment":"The claim that feature columns 'can function as compositional units' is asserted on the basis of /s/+vowel outputs from repeated columns, but the paper itself proposes an alternative explanation in terms of convolutional kernel interactions and channel alignment. This wording should be moderated, since the evidence distinguishes predictable contextual effects from true compositionality only weakly.","section":"Section 4.2"},{"comment":"No code, model checkpoint, or data availability statement is provided; given that the analysis relies on manual selection and qualitative inspection, releasing code and the exact column indices would substantially improve reproducibility.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The paper relies heavily on the authors' own prior work (References [1], [3], [16], [21], [22], [24], [36]), which is understandable in a niche area but makes independent validation of the novelty claim important. The 'first exploration' framing should be checked carefully against Chen and Elsner (2023) and Baas and Kamper (2024), both cited, which also analyze internal representations of GANs for speech. The central methodological concern - manual column selection feeding directly into the correlation analysis - is fixable with additional controls and would substantially strengthen the paper."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The useful part of this paper is the toolkit. Passing trained weight matrices into the convolutional layers as feature maps (Experiment 1) is a neat idea, and the result that each lexical code's weights produce the right word is a clean, concrete demonstration. The channel-slicing and column-recombination manipulations in Experiment 2 are also new for speech models — I haven't seen anyone isolate single FC columns and splice them into manually assembled feature maps before. The 64-channel zeroing experiment is a sensible way to localize information. As a methodological contribution, this is worth having.\n\nThe soft spot is exactly where the reader's stress-test lands. In Experiment 2, the authors select the vowel columns by generating outputs and picking the column that \"contains only vowel information\" (Section 4, Fig. 9). They then compute correlations among those same selected columns and report that similar vowels cluster. That is a selection on the outcome measure. If the selection procedure favors columns with clean, well-defined formants, those columns could cluster by acoustic cleanliness rather than by phonological height. There is no algorithmic selection rule, no inter-rater agreement, no baseline with randomly chosen columns or all 16 columns, and no significance testing. With N=12 and a visually interpreted MDS plot, the central claim that ciwGAN encodes lexical items in a \"linguistically principled manner\" is not yet supported. The concern is not circular parameter fitting — the correlation analysis itself does not use the vowel labels — but selection bias is enough to make the headline result fragile.\n\nA second, related issue is the input regime. Passing isolated columns or weight matrices into convolutional layers trained on full feature maps is not a validated operation. The authors note this for the z-variables but not for the vowel columns. The bias term and all other latent variables are absent, so the outputs live in a part of input space the model never saw. The fact that those outputs are interpretable is interesting, but it does not license strong conclusions about how the model actually represents vowels during normal generation.\n\nWho gets value: people working on GAN interpretability, speech representation learning, and maybe phonological acquisition. The methods are worth trying on other architectures. The main conclusion, however, should be treated as a suggestive observation, not an established result.\n\nRecommendation: send it to peer review, but with the expectation of major revision. The authors need an algorithmic column-selection rule, a null baseline, significance tests, and a validation of the isolated-column regime (e.g., comparing against full-input generations). If those arrive, the paper could be solid. As is, it is a promising methods paper with an overstated conclusion.","headline":"A genuinely novel set of FC-layer interpretability manipulations for a speech GAN, but the main claim about shared vowel encodings is undermined by output-dependent column selection and a lack of statistical controls.","tokens_in":13487,"tokens_out":1896,"would_cite":true,"duration_ms":22208,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A speech-generating CNN encodes similar vowels with similar internal weight patterns across different words, rather than storing each word as an unanalyzable whole.","keywords":["AI interpretability","generative adversarial networks","speech synthesis","fully connected layer","latent space","phonetic categories","ciwGAN","sublexical representation"],"falsifier":"Train a fresh ciwGAN on the same nine words with a different random seed, extract vowel columns by a fixed rule that does not listen to outputs (for example, the column at the temporal position matching each vowel's known duration in the training data), and compute the correlation matrix of those columns; if high-vowel columns no longer correlate more strongly with each other than with low-vowel columns, the reported structure is an artifact of output-guided column selection.","tokens_in":12541,"feed_emoji":"🔊","tokens_out":4234,"duration_ms":42483,"temperature":0.7,"pith_summary":"This paper tries to show that the fully-connected layer of ciwGAN, a generative CNN for speech, does not merely memorize each lexical item as an indivisible code. Instead, the layer's learned weights contain shared, lexically invariant structure: the same vowel quality is encoded in similar ways across distinct word-specific latent codes. The authors develop two techniques to see this: feeding a latent variable's weight matrix directly into the convolutional layers, and extracting single feature-map columns that produce an isolated vowel. If the claim holds, it matters because it indicates that sublexical sound categories can emerge inside a generative model trained only to reproduce waveforms, strengthening the case that such networks can serve as models of phonological learning.","feed_headline":"Similar vowels share hidden codes inside a speech AI","feed_subtitle":"Inside ciwGAN's fully-connected layer, the same vowel sound maps to matching weight patterns across different words.","key_machinery":"The central object is the variable-specific weight matrix $W_i$ of the fully-connected layer: for each of the 100 latent inputs (9 one-hot lexical codes plus 91 noise variables), the trained weights form a $1024 \\times 16$ feature-map matrix whose time axis mirrors the output waveform. Because the FC layer's pre-activation output is the sum of these 100 weight matrices each scaled by its latent variable value, each matrix can be treated as an independently testable blueprint of that variable's contribution. Passing a full weight matrix into the convolutional block reproduces the corresponding word, and passing a single extracted column in isolation yields a single vowel. Correlation matrices of these vowel columns, together with correlation matrices of the averaged output spectra, carry the argument that representational similarity in the FC layer tracks acoustic and phonological similarity.","core_discovery":"The paper's central discovery is that in ciwGAN each latent code's fully-connected weight matrix is not a monolithic word code but contains temporally organized columns that, passed alone through the convolutional layers, produce a single vowel. Comparing 12 such vowel columns taken from the 9 lexical items shows that phonetically similar vowels have positively correlated weight patterns, while high and low vowels are negatively correlated; multidimensional scaling places the high vowels together and the low vowels apart. The authors state this as evidence that similar-sounding vowels are encoded in similar ways in the FC layer, both within and across the weight matrices of lexically specific latent codes. They further show that these vowel columns behave compositionally: placing a vowel column beside a copy of itself or beside a vowel column from another word consistently yields an /s/ plus vowel output, with the /s/ and vowel information located in distinct channel groups.","pith_inferences":["The paper selects vowel columns by listening to which isolated column yields a clean vowel; a fully blind selection rule based only on temporal position or spectral criteria would be a stronger test that the correlation structure is not an artifact of output-guided column choice.","The demonstrated compositionality of columns suggests a concrete next experiment the paper does not run: transplanting the /i/ column from 'greasy' into the feature map of 'suit' and checking whether the output becomes something like 'seat', which would test word-level recombination from sublexical units.","If the channel-localization result generalizes, ablating the specific channel groups that the paper found to carry pure vowel or pure frication information should remove those sounds from a full word output, giving a causal test of the claimed representational dissociation.","A natural extension is to apply column extraction to consonants rather than only vowels; the prediction would be that any sublexical segment with a consistent acoustic realization forms its own correlated cluster of FC columns across codes."],"forward_implications":["If the claim is correct, ciwGAN's lexical codes are not holistic word templates; shared sublexical structure across words implies the model has learned emergent vowel categories without supervision.","The weight-matrix-as-input technique provides a label-free way to explore a latent space: each code's weight matrix alone can regenerate its lexical item's waveform, enabling bottom-up analysis of other trained models.","The column-extraction and recombination technique shows that FC columns act as compositional units, which could support segment-level manipulation in latent speech models, such as swapping vowel columns between words.","The finding that distinct channel groups specialize in frication versus vowel information suggests redundancy in the FC layer, implying the network might be compressible without losing its linguistically structured representations.","Because the methods only require a fully-connected projection from a latent space, the same techniques should transfer to other generative architectures, such as autoencoders, for probing their internal linguistic structure."],"supporting_citations":[{"why":"Defines the ciwGAN architecture, its training objective, and the one-to-one mapping of lexical items to latent codes that the paper builds on and extends.","marker":"[1]"},{"why":"Supplies the architecture schematic and prior evidence of sublexical structure in the fiwGAN variant, the baseline against which the paper's FC-layer claim is positioned.","marker":"[24]"},{"why":"WaveGAN is the base generator architecture on which ciwGAN is built, providing the convolutional block that the weight matrices are passed into.","marker":"[25]"},{"why":"InfoGAN is the source of the Q-network objective that pressures the latent codes to carry lexical information, explaining why code weights grow large.","marker":"[26]"},{"why":"TIMIT is the training corpus that supplies the nine lexical items the analysis is based on.","marker":"[27]"},{"why":"Provides the prior computer-vision technique of zeroing feature-map channels, which the paper adapts to isolate linguistic information within FC feature columns.","marker":"[29]"}],"fun_headline_variants":["Vowel similarity found in speech AI's hidden layer","Speech AI stores similar vowels in similar patterns","ciwGAN's FC layer encodes vowels by sound","Hidden layer reveals vowel structure in speech AI"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central result assumes that a single 1024-sample column of FC weights, chosen by inspecting which column produces only a vowel when run through the convolutional layers, is a valid and representative encoding of that vowel in the model; if that selection is confounded by how the convolutional layers respond to isolated columns, the clustering of similar vowels could be an artifact.","fun_headline_variants_meta":{"raw":{"variants":["Vowel similarity found in speech AI's hidden layer","Speech AI stores similar vowels in similar patterns","ciwGAN's FC layer encodes vowels by sound","Hidden layer reveals vowel structure in speech AI"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000145,"raw_usage":{"total_tokens":1185,"prompt_tokens":958,"completion_tokens":227,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":574,"completion_tokens_details":{"reasoning_tokens":168}},"tokens_in":574,"tokens_out":227,"duration_ms":2883,"temperature":1.0,"reasoning_tokens":168,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T20:36:21.622297+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a fresh ciwGAN on the same nine words with a different random seed, extract vowel columns by a fixed rule that does not listen to outputs (for example, the column at the temporal position matching each vowel's known duration in the training data), and compute the correlation matrix of those columns; if high-vowel columns no longer correlate more strongly with each other than with low-vowel columns, the reported structure is an artifact of output-guided column selection.","supporting_citations":[{"cited_title":"Ciwgan and fiwgan: Encoding information in acoustic data to model lexical learning with generative adversarial networks","cited_arxiv_id":null,"evidence_quote":"Defines the ciwGAN architecture, its training objective, and the one-to-one mapping of lexical items to latent codes that the paper builds on and extends."},{"cited_title":"Modeling speech recognition and synthesis simultaneously: Encoding and decoding lexical and sublexical semantic information into speech with no direct access to speech data","cited_arxiv_id":null,"evidence_quote":"Supplies the architecture schematic and prior evidence of sublexical structure in the fiwGAN variant, the baseline against which the paper's FC-layer claim is positioned."},{"cited_title":"InfoGAN: Interpretable representation learning by information maximizing generative adversarial nets","cited_arxiv_id":null,"evidence_quote":"InfoGAN is the source of the Q-network objective that pressures the latent codes to carry lexical information, explaining why code weights grow large."},{"cited_title":"Timit acoustic-phonetic continuous speech corpus","cited_arxiv_id":null,"evidence_quote":"TIMIT is the training corpus that supplies the nine lexical items the analysis is based on."}],"review_version":1}