{"id":"991b63a9-7b2d-4c0c-90c9-4edc15ddae84","arxiv_id":"2506.21712","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"Unsupervised k-means clusters of SSL features and i-vectors identify speaker-relevant feed-forward neurons; protecting them during pruning preserves speaker identification.","lead":"The paper shows that speaker-related neurons in self-supervised speech transformers can be found without labels, by clustering the model's own features and i-vectors. Protecting those neurons during pruning keeps speaker identification working, so the method helps both model compression and understanding where speaker information lives.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Pruning-protection evidence is confounded by neuron-count and selection effects: without a random/permuted-cluster control matched per layer, the SID recovery does not prove the identified neurons specifically encode speaker information.","rationale":"The paper is a useful extension of property-neuron analysis to unsupervised clusters, and the authors are transparent about hyperparameter choices and alternative definitions. The internal experiments are self-consistent, and the phonetic/gender cluster interpretation is plausible. However, the central claim—that specific FFN neurons carry speaker information—depends on a causal interpretation of the pruning-protection experiment: protecting those neurons preserves SID better than regular pruning. This experiment is not controlled for what is being protected: the identified sets are small and overlap is removed (Eqs. 3, 9), and the authors themselves attribute Alternative differences to the number of protected neurons (Sec. IV-C). A matched random-neuron protection condition is the minimal control needed to distinguish 'speaker-relevant neurons' from 'some protected neurons help one-shot pruning.' The reader's weakest assumption is close but not identical: the reader focuses on thresholds and distributed versus sparse encoding; the sharper issue is that even if the neuron identification is reliable, the evaluation cannot discriminate speaker-specific importance from generic activation-frequency or neuron-count effects. Thus I partially agree with the reader and maintain the conditional verdict pending this control.","tokens_in":7574,"tokens_out":4524,"duration_ms":53258,"concrete_test":"Run the one-shot pruning pipeline on MelHuBERT and wav2vec 2.0 at the same average per-layer sparsity (867/780 dims), protecting per layer: (a) the proposed SSL/i-vector cluster neurons; (b) the same number of randomly selected neurons over at least 5 seeds; (c) the same number of neurons with the highest global activation frequency, ignoring i-vector/SSL cluster assignments; (d) neurons identified using cluster labels that are randomly permuted across frames. Compute SID (and PR/ER/IC) for each condition. If any of (b)-(d) matches (a) within a few accuracy points while beating regular pruning, the pruning evidence is not specific to speaker-correlated neurons; if (a) clearly outperforms all size-matched controls, the central claim is supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that the discovered SSL-cluster and i-vector-cluster neurons encode speaker information, with pruning-protection as evidence. The strongest result (Table I, one-shot: MelHuBERT SID 62.96 vs 54.08 regular; wav2vec 2.0 70.67 vs 63.42) is only interpretable if protecting these specific neurons, rather than any comparably sized set of frequently-activated neurons, causes the recovery. That control is missing. The ablation in Table II is revealing: the authors explain that 'conditioning and intersecting' (Eq. 5) beats Alternatives A/B/C because it identifies fewer neurons, 'allowing for more efficient pruning that avoids protecting too many neurons' (Sec. IV-C). If the chosen set's advantage is its size, the pruning results are equally consistent with a neuron-count effect. The i-vector clusters are gender classes (97.23% male / 89.82% female purity), so the protected neurons are correlated with a coarse demographic attribute, not with individual speakers. Protecting such neurons (plus SSL phonetic-cluster neurons) might help SID by retaining a generally useful, frequently-activated subset of the FFN rather than by protecting speaker-identity-specific representations. A random-neuron or cluster-label-permuted control, matched per layer in count, would settle whether the identified neurons are special. No such control, code, or error bars are provided, so the causal claim is currently underdetermined.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a label-free method for identifying neurons in the feed-forward layers of self-supervised speech Transformers that are related to speaker information. The method extends the property-neuron framework of Lin et al. [20] by replacing supervised labels (phones, gender) with k-means clusters of SSL features and i-vectors. In their experiments on MelHuBERT and wav2vec 2.0, the authors protect the identified 'SSL cluster neurons' and 'i-vector cluster neurons' during iterative and one-shot pruning, and report that speaker identification (SID) performance is preserved much better than with regular pruning, rivaling the supervised property-neuron approach. They also present ablations studying the number of clusters and alternative definitions of i-vector cluster neurons. The central claim is that these cluster neurons play a significant role in encoding speaker information.","tokens_in":7905,"tokens_out":3010,"duration_ms":36154,"significance":"If the central claim is established, the paper would offer a useful unsupervised tool for interpreting speaker-related representations in SSL speech models, with potential practical benefits for pruning. The experimental design is coherent, and the comparison against regular pruning and the supervised property-neuron baseline is a strength. However, the causal interpretation hinges on the assumption that the identified neurons are special relative to other comparably sized neuron sets; this is currently not tested. The absence of error bars, the post-hoc selection of cluster numbers and layer, and the unvalidated cross-model transfer of cluster assignments all weaken the evidence. These issues are addressable with additional experiments, making the manuscript a candidate for revision rather than outright rejection.","major_comments":[{"comment":"The pruning-protection evidence does not currently demonstrate that the identified neurons specifically encode speaker information, because no control matched for neuron count and selection is provided. The authors attribute the advantage of the conditioning-and-intersection definition in Eq. (5) to the fact that it identifies fewer neurons, 'allowing for more efficient pruning that avoids protecting too many neurons' (Sec. IV-C). This makes the reported improvements consistent with a neuron-count or frequently-activated-neuron effect rather than with speaker-specific encoding. I request a control in which an equal number of neurons per layer, matched to the protected sets, is selected either randomly or by permuting cluster labels, and the pruning experiment is repeated with that control. Without such a control, the central causal claim in the abstract and conclusion is underdetermined.","section":"IV-B and IV-C (Tables I and II)"},{"comment":"Key hyperparameters are selected after seeing the clustering and pruning outcomes, which introduces circularity into the main claim. The paper sets kssl=3, kive=2, and layer 9 intentionally because they produce interpretable clusters (Sec. IV-A) and good pruning results. While the ablation in Table II shows that kssl=39 and kssl=100 degrade performance, it does not establish that the chosen values are not optimized for the reported SID numbers. The authors should either prespecify these choices or provide a robustness study across a range of kssl and kive values that shows the pruning-preservation effect is not confined to a narrowly tuned operating point.","section":"III and IV-A"},{"comment":"The claim that the method identifies speaker information is stronger than is supported by the cluster semantics. The i-vector clusters are shown to correspond to gender classes (97.23% male and 89.82% female purity, Sec. IV-A), so the protected neurons are correlated with a coarse demographic attribute rather than with individual speaker identity. In addition, the same MelHuBERT layer-9 cluster assignments are applied to wav2vec 2.0 features without validating that frame-level cluster alignment transfers across models. The authors should temper the 'speaker information' terminology to 'gender- and phone-correlated information' or provide evidence that the identified neurons capture more than these coarse attributes, e.g., by evaluating speaker verification with multiple speakers per gender.","section":"III and IV-B"}],"minor_comments":[{"comment":"There are several typographical errors, including 'thie section', 'self-sueprvised', 'Spefically', 'overlappting', 'paramameters', and 'lowℓ1' (missing space in 'low ℓ1'). These should be corrected.","section":"II and III"},{"comment":"The notation 'v1 > top λ%(v)' is ambiguous; please define precisely whether the threshold is the (100−λ)-th percentile or the value exceeded by exactly λ% of dimensions.","section":"II, Eq. (1)"},{"comment":"No error bars, confidence intervals, or significance tests are reported for any of the pruning results, despite the fact that several comparisons are close (e.g., MelHuBERT iterative SID: 52.66 vs. 54.10 for property neurons). Reporting multiple seeds or bootstrap intervals would strengthen the conclusions.","section":"III"},{"comment":"The one-shot regular-pruning SID result for MelHuBERT (54.08) is higher than the iterative regular-pruning result (51.04), which seems counterintuitive given that iterative pruning normally retains more performance. A brief explanation would help the reader.","section":"Table I"},{"comment":"The subscript differences in Table II (e.g., '49.19-3.47') are not clearly defined in the text. Please state explicitly that the subscripted number is the difference from the Proposed Method row.","section":"IV-C"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is built largely on the authors' own prior work [20], and while this is a legitimate extension, the incremental novelty would be clearer if the unsupervised discovery aspect were framed as the main contribution. The central issue for me is the missing control for neuron-count/selection effects; the authors should be encouraged to add a random or permuted-cluster baseline. I would also urge the authors to release code and pruning configurations to make the results reproducible. The paper fits the scope of the conference, but the current evidence does not yet warrant acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is a useful, label-free extension of the authors' own property-neuron framework. The new bit is replacing phone and gender labels with k-means clusters of SSL features and i-vectors, then showing that protecting these cluster neurons during pruning rivals the supervised approach and works especially well in one-shot pruning. The experimental design is coherent: two models, four tasks, ablations over three alternative neuron-selection schemes, and honest discussion of the method's own limitations. Credit where due: the unsupervised pipeline is genuinely label-free, and the practical pruning gains are real and reproducible in principle.\n\nThe soft spot is the one the stress-test note hits: the pruning-protection evidence is confounded by neuron-count and selection effects. The i-vector clusters are essentially gender classes (97% male / 90% female purity), so 'speaker information' here is largely a coarse demographic attribute plus phonetic clusters. Protecting that subset might simply keep a generally useful fraction of the FFN alive. The authors effectively concede this when they attribute the Alternative A/B/C results to neuron counts rather than to the content of the neurons—'allowing for more efficient pruning that avoids protecting too many neurons' is a size effect, not an identity effect. The missing control is load-bearing: protect a random or cluster-label-permuted set of the same size, per layer, and see if SID recovers. Without that, the causal claim that these specific neurons encode speaker identity is underdetermined.\n\nOther soft spots: no error bars, significance tests, or code/data; hyperparameters kssl=3, kive=2, and layer 9 are selected for cluster interpretability and pruning behavior, which is fine for a hypothesis but weak for a general conclusion. The authors also flag their own uncertainty about the conditioning design ('It is not entirely clear why the neurons need to be conditioned'), which I read as honest but not reassuring.\n\nWho is this for? Practitioners pruning SSL speech models who want a label-free way to preserve SID, and interpretability researchers working on FFN property neurons. It deserves a serious referee—the method is plausible and well-motivated, and the flaws are fixable with a matched random/permuted control and reproducibility artifacts. I would send it to review with the expectation that those additions be required.","headline":"A clean, label-free extension of the authors' property-neuron work, but the pruning evidence does not yet isolate whether the identified neurons specifically encode speaker information.","tokens_in":8411,"tokens_out":2235,"would_cite":false,"duration_ms":25539,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that speaker information in self-supervised speech Transformers is concentrated in specific feed-forward neurons, which can be found without labels by correlating activations with k-means clusters of speech features and…","keywords":["self-supervised speech","speaker identification","feed-forward layers","neuron pruning","i-vectors","k-means clustering","property neurons","one-shot pruning"],"falsifier":"Run one-shot pruning on the same models while protecting an equal number of randomly chosen neurons in the same layers, matched for layer and set size; if random protection recovers the same speaker-identification accuracy as the discovered cluster neurons, the cluster-conditioned selection is not what preserves the information. A complementary check is to prune away the discovered neurons and keep everything else: the central claim predicts a sharp speaker-ID collapse.","tokens_in":7361,"feed_emoji":"🗣️","tokens_out":6767,"duration_ms":65171,"temperature":0.7,"pith_summary":"This paper claims that speaker information in self-supervised speech Transformers is concentrated in a small set of feed-forward neurons, and that those neurons can be found without any labels. The method clusters self-supervised features and i-vectors, treats the clusters as coarse phonetic and gender classes, and marks the neurons that co-activate with a cluster as speaker-relevant. Protecting those neurons during one-shot pruning keeps speaker-identification accuracy close to the unpruned model after roughly 70% of the feed-forward parameters are removed, and matches or beats a supervised method that uses phone and gender labels. The result matters because it gives a label-free way to locate where speaker identity lives in these models and shows that pruning can serve as an analysis tool rather than only a compression trick.","feed_headline":"Unlabeled clusters find speaker neurons; 70% pruning safe","feed_subtitle":"Protecting neurons tied to speech-feature and i-vector clusters keeps speaker ID nearly intact at 70% pruning.","key_machinery":"The load-bearing object is the activation-pattern correlation test: each frame is binarized into neurons active in the top 1% (Eq. 1), and a neuron belongs to a cluster's set if it is active in more than 1% of that cluster's frames (Eqs. 2 and 4). Overlapping neurons across clusters are removed (Eqs. 3 and 9), giving the protected sets $P_{ssl}$ and $P_{ive}$. The k-means clusters themselves are the label-free proxy: three SSL clusters approximate vowel/consonant classes, and two i-vector clusters approximate gender, so the identified neurons capture both phonetic and speaker-level structure. During pruning, these sets are exempted from $\\ell^1$-norm-based removal of the 3072 hidden dimensions, which is what lets the paper attribute the preserved speaker-identification accuracy to the identified neurons.","core_discovery":"The central claim is that the sets the paper calls SSL cluster neurons and i-vector cluster neurons — dimensions of the first feed-forward layer whose top-1% activations co-occur with a k-means cluster of self-supervised features or of i-vectors — carry a substantial share of the speaker information. With three self-supervised clusters and two i-vector clusters, the clusters line up with broad phonetic classes and with gender (one i-vector cluster is 97.23% male, the other 89.82% female), so the identified neurons act as speaker-relevant units discovered without labels. On SUPERB speaker identification, pruning the feed-forward layers while protecting these neurons recovers most of the loss that regular pruning causes, and in the one-shot setting the protection beats the supervised property-neuron baseline on both MelHuBERT and wav2vec 2.0. The paper further reports that removing either cluster-neuron type from the protected set degrades performance, with the SSL cluster neurons more critical, and that too many clusters leave too few neurons to protect.","pith_inferences":["The same unsupervised recipe could be pointed at other properties — emotion, language, noise condition — by clustering embeddings suited to those properties, giving a label-free route to locate task-relevant neurons in any self-supervised speech model.","Because SSL cluster neurons matter more than i-vector cluster neurons for speaker identification, the paper's picture suggests speaker identity is partly carried by phonetic structure; a testable prediction is that the protected set should also transfer across languages or recording conditions if it captures speaker rather than corpus-specific units.","A matched-control test the paper does not run: protecting an equal number of randomly chosen neurons in the same layers. If random sets preserve speaker identification as well, the cluster-conditioned selection would not be doing the work.","The one-shot results that beat the unpruned model suggest protecting cluster neurons may act as structured regularization; one could test whether the same protection ratio improves generalization on held-out speakers or noisy utterances."],"forward_implications":["Pruning can remove about 70% of the feed-forward parameters in a single step without a large speaker-identification drop, provided the cluster neurons are spared.","The unsupervised identification rivals the supervised property-neuron approach, and in one-shot pruning it beats it on speaker identification for both MelHuBERT (62.96 vs. 58.27 accuracy) and wav2vec 2.0 (70.67 vs. 66.05).","Both the SSL cluster neurons and the i-vector cluster neurons are necessary; dropping either from the protected set lowers speaker-identification accuracy, and the SSL cluster neurons are the more critical of the two.","The benefit is not limited to speaker tasks: protecting cluster neurons also helps emotion recognition and intent classification under one-shot pruning, sometimes surpassing the unpruned model.","Using too many clusters backfires because each additional cluster shrinks the number of identified neurons in middle layers, leaving too few units to protect."],"supporting_citations":[{"why":"Supplies the property-neuron identification method this work extends and the supervised phone/gender baseline it must rival.","marker":"[20]"},{"why":"Introduces i-vectors, the label-free speaker representation the paper clusters.","marker":"[10]"},{"why":"Shows feed-forward compression is especially harmful to speaker tasks, motivating pruning as the evaluation tool.","marker":"[19]"},{"why":"Provides the SUPERB benchmark used to measure speaker identification, phoneme recognition, emotion recognition, and intent classification.","marker":"[32]"},{"why":"Defines wav2vec 2.0, one of the two self-supervised transformers analyzed.","marker":"[3]"},{"why":"Defines MelHuBERT, the other analyzed model and the source of the layer-9 features used for clustering.","marker":"[18]"},{"why":"Gives the phonetic-information-for-speaker-embeddings hypothesis that motivates combining SSL and i-vector clusters.","marker":"[25]"}],"fun_headline_variants":["Speaker neurons found in feed-forward layer, pruning loss avoided","Unsupervised clusters spot speaker neurons; pruning stays safe","Cluster neurons carry speaker info; protecting them aids pruning","Feed-forward neurons encode speaker, pruning-safe via clusters","No-label neuron hunt finds speaker-relevant units in SSL model"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole identification rests on assuming that a neuron whose top-1% activations co-occur with a k-means cluster more than 1% of the time is one of a few neurons that carry speaker information; if the information is spread thinly across many neurons or lives in activation magnitudes, the protected sets could be arbitrary.","fun_headline_variants_meta":{"raw":{"variants":["Speaker neurons found in feed-forward layer, pruning loss avoided","Unsupervised clusters spot speaker neurons; pruning stays safe","Cluster neurons carry speaker info; protecting them aids pruning","Feed-forward neurons encode speaker, pruning-safe via clusters","No-label neuron hunt finds speaker-relevant units in SSL model"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00023,"raw_usage":{"total_tokens":1440,"prompt_tokens":860,"completion_tokens":580,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":476,"completion_tokens_details":{"reasoning_tokens":500}},"tokens_in":476,"tokens_out":580,"duration_ms":6847,"temperature":1.0,"reasoning_tokens":500,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T22:20:20.920095+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run one-shot pruning on the same models while protecting an equal number of randomly chosen neurons in the same layers, matched for layer and set size; if random protection recovers the same speaker-identification accuracy as the discovered cluster neurons, the cluster-conditioned selection is not what preserves the information. A complementary check is to prune away the discovered neurons and keep everything else: the central claim predicts a sharp speaker-ID collapse.","supporting_citations":[{"cited_title":"Property Neurons in Self- Supervised Speech Transformers","cited_arxiv_id":null,"evidence_quote":"Supplies the property-neuron identification method this work extends and the supervised phone/gender baseline it must rival."},{"cited_title":"Front-End Factor Analysis for Speaker Verification","cited_arxiv_id":null,"evidence_quote":"Introduces i-vectors, the label-free speaker representation the paper clusters."},{"cited_title":"SUPERB: Speech processing Universal PERformance Benchmark","cited_arxiv_id":null,"evidence_quote":"Provides the SUPERB benchmark used to measure speaker identification, phoneme recognition, emotion recognition, and intent classification."},{"cited_title":"wav2vec 2.0: A framework for self-supervised learning of speech representations","cited_arxiv_id":null,"evidence_quote":"Defines wav2vec 2.0, one of the two self-supervised transformers analyzed."},{"cited_title":"MelHu- BERT: A simplified HuBERT on Mel spectrograms","cited_arxiv_id":null,"evidence_quote":"Defines MelHuBERT, the other analyzed model and the source of the layer-9 features used for clustering."},{"cited_title":"Speaker embedding extraction with pho- netic information","cited_arxiv_id":null,"evidence_quote":"Gives the phonetic-information-for-speaker-embeddings hypothesis that motivates combining SSL and i-vector clusters."}],"review_version":1}