{"id":"6bb31d90-b51a-4351-8b53-859d3439efaa","arxiv_id":"2505.16406","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Standard feature attribution methods give inconsistent explanations across retrained speech classifiers, except for word-level perturbation on word-based tasks.","lead":"This paper tests whether common methods for explaining what a speech classifier pays attention to give the same answer when the same model is retrained from a different random seed. It finds that most standard explanation methods are unreliable for speech, except when explanations are computed by mutating whole words in word-based tasks.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reliability conclusion depends on the unvalidated assumption that different seeds use comparable internal features; output agreement alone does not establish this, so low ISA may measure model variation rather than attribution unreliability.","rationale":"The reader's weakest_assumption identifies exactly the same load-bearing premise: ISA is only a measure of attribution reliability if differently seeded models are internally comparable, and the paper supports this only with output-level agreement. My independent reading of the methods and results confirms that this is the step on which the central negative conclusion depends. The accuracy and Fleiss's κ evidence in Section 3.1 is suggestive but not sufficient, because speech inputs contain many redundant predictive cues and different seeds may exploit different cues while producing identical labels. If that happens, any faithful attribution method would necessarily yield different maps for different seeds, so low ISA would be a sign of model variation rather than attribution unreliability. The proposed concrete test separates these two explanations by checking whether one seed's top features are as informative for another seed's decisions as the other seed's own top features. This directly tests the comparability assumption without requiring synthetic data or access to ground-truth features. Other concerns, such as the lack of a predefined reliability threshold or statistical tests between conditions, are secondary: they affect the strength of the quantitative claims but not the validity of the measurement design. The paper is otherwise clear and well-scoped, and the positive word-aligned perturbation result is visible in the reported distributions. Since the identified concern is real but does not invalidate the empirical findings — it only weakens the interpretation of the negative result as a statement about attribution methods rather than model variability — the existing CONDITIONAL verdict remains appropriate. No change to the reader's verdict is needed.","tokens_in":8203,"tokens_out":5641,"duration_ms":54044,"concrete_test":"Cross-seed faithfulness check using the released code: for a fixed test utterance, compute IG attribution A_i for seed i and A_j for seed j. Measure the drop in seed j's output confidence when the top-20% waveform samples identified by A_i are replaced by the silence baseline, and compare with the drop when seed j's own top-20% samples are replaced. Average over all seed pairs and test utterances. If cross-seed masking is significantly less effective than within-seed masking, the seeds use different features, and low ISA reflects model variation; if cross-seed masking is equally effective, the assumption holds and low ISA is attributable to the attribution methods. A control using randomly chosen 20% samples should be included.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3 explicitly assumes that models fine-tuned from the same checkpoint with different random seeds 'identify and employ comparable relevant features for a given input' when they achieve high accuracy. Section 3.1 attempts to support this with accuracy and Fleiss's κ, but those are output-level measures: they show that the runs make the same label decisions, not that they rely on the same internal input features. In speech, inputs are high-dimensional and full of redundant, highly correlated cues; different seeds can reach the same decisions using different features (e.g., different frequency bands or temporal segments). If so, a perfectly faithful attribution method would be required to produce different maps for different seeds, and ISA would be low even for an ideal method. Therefore the abstract's central claim — that standard attribution methods are 'generally unreliable' — conflates inter-seed model variability with attribution unreliability. This is the load-bearing premise of the paper's negative finding, and it is not independently validated. The paper itself flags the assumption, but offers no direct evidence for it. This is not an internal inconsistency; it is an unverified empirical premise that the central conclusion hinges on.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents an empirical study of the reliability of four feature attribution methods (Saliency, Integrated Gradients, LIME, and Feature Ablation) for speech classification. Using wav2vec2-base fine-tuned with nine random seeds on gender/speaker identification and intent classification (with three subtasks), the authors define the inter-seed agreement (ISA) metric as the fraction of shared top-20% attribution indices between pairs of runs. They report that ISA is generally low for waveform, spectrogram, and CNN-embedding attributions, even after frame- or word-level aggregation, but becomes high for word-aligned perturbation on intent classification subtasks. The paper concludes that standard attribution methods are generally unreliable in speech, except for word-aligned perturbation for word-based tasks.","tokens_in":8399,"tokens_out":5895,"duration_ms":46553,"significance":"If the finding is supported, it is a valuable and actionable result for the speech interpretability community: it would caution against off-the-shelf attribution methods on wav2vec2-style models and identify a concrete configuration (word-level perturbation for word-based tasks) that yields reproducible explanations. The study is well-grounded in standard datasets (Common Voice, Fluent Speech Commands) and the authors provide code, which aids reproducibility. The choice of inter-seed agreement as a reliability measure is reasonable in principle, but the paper's central negative claim hinges on an unvalidated assumption about inter-seed feature equivalence, and the lack of statistical tests and a single backbone limit the strength of the generalization.","major_comments":[{"comment":"The interpretation of ISA as a measure of attribution reliability rests entirely on the assumption, stated in Section 3, that models fine-tuned from the same checkpoint with different seeds 'identify and employ comparable relevant features for a given input' when they achieve high accuracy. The evidence offered in Table 1 (accuracy and Fleiss's kappa) is at the output level and does not establish internal feature equivalence; in a high-dimensional, redundant input space such as speech, different seeds could plausibly reach identical decisions via different features, in which case low ISA would reflect model variation rather than attribution unreliability. This is the load-bearing premise of the paper's negative conclusion, and it is not directly validated. Please add an explicit test: for example, compute a representation-similarity metric (e.g., CKA) between the fine-tuned models at the layers used for attribution, or use a synthetic task with known ground-truth features to show that the ISA gap between methods is driven by the attribution method rather than by inter-seed model differences.","section":"Section 3, Eq. (1)"},{"comment":"The paper fixes the top-p threshold at 20% and defines 'acceptable' reliability only by visual inspection of boxplots; no formal threshold, confidence interval, or significance test is reported. For instance, the statement in Section 5 that Integrated Gradients 'does not surpass 50% inter-seed agreement for most tasks' is a descriptive summary, and the claim that LIME is 'barely above the baseline' is not backed by a test of whether the distribution of ISA across the 36 seed pairs differs from the shuffled baseline. The central negative claim would be materially strengthened by reporting a permutation test against the random-shuffle baseline, confidence intervals on the medians, and a sensitivity analysis for the choice of p (e.g., 1%, 5%, 10%, 20%, 50%).","section":"Section 3.2 and Section 4"},{"comment":"All experiments use a single backbone, wav2vec2-base, and the abstract's conclusion that standard attribution methods are 'generally unreliable when applied to the speech domain' is broader than what the evidence supports. The paper should either temper the generalization to wav2vec2-base models (or to the tested conditions) or add at least one additional self-supervised backbone (e.g., HuBERT, WavLM) to test whether the reliability pattern transfers.","section":"Section 3.1"},{"comment":"The positive exception—word-aligned perturbation for intent classification—is the key result that prevents the paper from being entirely negative, but the mechanistic explanation given in Section 5 ('classification decisions for these tasks rely on specific words') is not directly tested. The lower ISA for the Location subtask is explained post hoc as due to 'redundant lexical cues,' but no supporting analysis is provided. I recommend adding a simple diagnostic: for the intent subtasks, compute whether the words identified by the word-aligned perturbation method correspond to task-specific lexical cues, or compare ISA for subsets of utterances with vs. without such cues. This would turn the explanation from a plausible conjecture into a tested claim.","section":"Section 4.3 and Section 5"}],"minor_comments":[{"comment":"In the first paragraph, 'atrribution' should be corrected to 'attribution'.","section":"Section 5"},{"comment":"Equation (1) is not typeset cleanly in the manuscript; the product of N and the sum should be spaced clearly to avoid confusion.","section":"Equation (1)"},{"comment":"The value of p is said to be fixed at 20%, but the formula uses p as a variable; please clarify whether p is a proportion (0.2) or a percentage (20).","section":"Section 3.2"},{"comment":"The red dotted lines are called 'randomly shuffled baseline' but the text does not explain how many shuffles were used or the variance of the baseline; please specify the procedure.","section":"Figures 1-3"},{"comment":"The phrasing 'We can see that the classification accuracy is near perfect' is informal; also, the two sentences beginning 'If models' architecture...' are redundant with the presentation of Table 1.","section":"Section 3.1"}],"recommendation":"major_revision","confidential_remarks":"The paper is a solid empirical contribution with a clear negative result, but the central claim is broader than the evidence. The reviewer's skepticism about inter-seed feature equivalence is the crux; if the authors can supply direct evidence (e.g., representation similarity or synthetic controls), the paper would be acceptable. Given the current state, I recommend major revision rather than rejection because the issue is fixable with additional experiments and the empirical pattern is likely to persist after validation."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe paper is a clean empirical study of inter-seed reliability for four attribution methods on wav2vec2-based speech classifiers, across three tasks and multiple input, aggregation, and perturbation conditions. The main finding—standard attribution methods give low inter-seed agreement, except for word-aligned perturbation on word-based tasks—is new and clearly presented. The authors include a random-shuffle baseline, report full distributions, and publish code. That is real work, and the field can use it.\n\nThe soft spots are real but not fatal. The conclusion generalizes from a single backbone, the top-p of 20% is arbitrary, and there are no statistical tests separating conditions. More importantly, the central premise is that different seeds 'employ comparable relevant features' when accuracy is high. The paper supports that with accuracy and Fleiss' kappa, which are output-level measures. In speech, with highly redundant cues, different seeds can make identical decisions using different internal features. If so, low ISA measures model variation, not attribution unreliability. The paper's own explanation for the word-aligned exception—'little scope for models to disagree'—shows they know this. The negative result should be phrased as 'attribution methods do not give consistent explanations across seeds for these models,' not 'attribution methods are generally unreliable.' That is a meaningful distinction for practitioners.\n\nThe assumption is not indefensible; it is a practical definition of reliability for real use. But it should be validated directly (e.g., probing internal representations) or the claims should be softened. This is a solid paper with one load-bearing soft spot. It deserves a serious referee and likely a revision before acceptance.\n\nMy recommendation: send it to peer review. The question it asks is important, the experiments are honest, and the limitations section is candid. A good referee can push on the core assumption and the generalization claims.","headline":"Useful empirical study of attribution reliability in speech, with a load-bearing assumption about inter-seed feature similarity that should be validated or the claims softened.","tokens_in":8956,"tokens_out":2204,"would_cite":true,"duration_ms":19273,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Standard feature attribution methods are unreliable on speech models, except when perturbations are word-aligned for word-based tasks.","keywords":["speech processing","feature attribution","reliability","inter-seed agreement","interpretability","perturbation methods","word alignment","intent classification"],"falsifier":"Perturb the same intent-classification inputs using random segments of the same average length as words instead of forced-aligned word boundaries; if the inter-seed agreement stays high, the reliability effect is due to segment duration rather than lexical alignment, and the paper's exception would need reinterpretation.","tokens_in":7991,"feed_emoji":"🎙️","tokens_out":5879,"duration_ms":45752,"temperature":0.7,"pith_summary":"Feature attribution methods claim to show which parts of an input drive a model's decision, but explanations that change between equally good retrained copies of the same model cannot be trusted. This paper tests four standard attribution methods on speech classifiers and measures reliability as the agreement between attributions from nine fine-tuning seeds. It finds that most methods are unreliable whether the input is a waveform, spectrogram, or CNN embedding, and that post-hoc aggregation of scores does not fix this. The exception is directly perturbing word-aligned segments on word-based intent classification, where agreement becomes high. The upshot is that off-the-shelf attribution methods should not be presumed reliable for speech, and that reliable explanation requires aligning perturbation with meaningful lexical units for tasks that depend on them.","feed_headline":"Word-aligned perturbations pass a reliability test others fail","feed_subtitle":"Across speech tasks, only directly perturbing word segments gives stable attributions between model runs.","key_machinery":"The central instrument is the inter-seed agreement (ISA) metric, a dynamic top-p overlap score that measures how much the top 20% of attribution indices from two models fine-tuned with different random seeds intersect; it operationalizes reliability as reproducibility under retraining. The word-aligned perturbation setting, where the input is masked at forced-alignment word boundaries, is the treatment that rescues reliability for lexical tasks, and it works by removing the model's freedom to rely on different redundant cues across seeds.","core_discovery":"The paper claims that when a speech classifier is fine-tuned multiple times from the same pre-trained checkpoint with different random seeds, standard feature attribution methods—Saliency, Integrated Gradients, LIME, and Feature Ablation—produce explanations that agree poorly across runs, even though the classifiers themselves agree on nearly all test inputs. Using a dynamic top-20% inter-seed agreement (ISA) score, the authors find that for waveform, spectrogram, and CNN-embedding input types, and at no, frame, and word-level aggregation, most method-task combinations fall near or below a randomly shuffled baseline. The one reliable configuration is directly perturbing word-aligned input segments, and only for the word-based intent classification subtasks; gender and speaker identification never reach acceptable reliability. The conclusion is that naively applying standard attribution methods to speech generally yields unreliable explanations, and that reliable attribution requires aligning perturbations with meaningful lexical units for tasks that depend on them.","pith_inferences":["The ISA protocol could be transferred to other continuous-input domains, such as medical time series or audio event detection, where retraining stability is similarly under-evaluated.","Because the assumption that equally accurate seeds use comparable features is untested for acoustic tasks, the low ISA on gender and speaker ID may partly reflect genuinely different feature usage, not attribution failure; separating these two causes would strengthen the conclusion.","A natural extension would zoom into phoneme-level alignment, especially for languages where words are not the meaningful unit, to see whether the reliable regime generalizes beyond lexical segmentation.","The finding suggests a design rule that may transfer to other modalities: choose attribution granularity to match the model's decision units rather than the input resolution."],"forward_implications":["Practitioners should not treat gradient-based or high-resolution perturbation attributions for speech as stable explanations, since most conditions fall near the random baseline.","Aggregating high-resolution attribution scores to frame or word level does not systematically improve reliability for non-word-based tasks.","Reliable explanations in speech are currently restricted to tasks whose decisions depend on lexical content, using direct word-level perturbation.","Reliability must be established before validity: without seed-invariant attributions, faithfulness comparisons are themselves unstable.","Speech-specific attribution methods are needed for acoustic-dependent tasks like speaker and gender identification."],"supporting_citations":[{"why":"Supplies the pre-trained wav2vec2 speech model that is fine-tuned and analyzed in all experiments.","marker":"[1]"},{"why":"Shows that restricting perturbation windows around phonemes improves the validity of LIME, motivating the idea that perturbation granularity matters.","marker":"[7]"},{"why":"Provides the method of prepending an ISTFT layer so attributions can be computed for spectrogram inputs even for waveform-native models.","marker":"[10]"},{"why":"Previous work whose word-aligned perturbation setting is reused and is the condition that achieves high reliability.","marker":"[11]"},{"why":"Documents fundamental limitations of feature attribution, motivating the need to evaluate reliability separately from validity.","marker":"[22]"},{"why":"Provides the implementations of Saliency, Integrated Gradients, LIME, and Feature Ablation used to compute attribution scores.","marker":"[30]"},{"why":"Produces the forced-alignment word time stamps used to aggregate and perturb at word level.","marker":"[31]"}],"fun_headline_variants":["Only word-aligned perturbations survive speech reliability test","Speech attribution unreliable except for word-aligned methods","Word-aligned wins as other speech explanations fail","Reliability in speech attribution hinges on word alignment"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The approach assumes that differently seeded fine-tuned models that agree on predictions also rely on the same internal features, so any disagreement in their attribution maps is due to the attribution method rather than to genuinely different model behavior.","fun_headline_variants_meta":{"raw":{"variants":["Only word-aligned perturbations survive speech reliability test","Speech attribution unreliable except for word-aligned methods","Word-aligned wins as other speech explanations fail","Reliability in speech attribution hinges on word alignment"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000171,"raw_usage":{"total_tokens":1212,"prompt_tokens":824,"completion_tokens":388,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":440,"completion_tokens_details":{"reasoning_tokens":329}},"tokens_in":440,"tokens_out":388,"duration_ms":2980,"temperature":1.0,"reasoning_tokens":329,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T15:01:00.853453+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Perturb the same intent-classification inputs using random segments of the same average length as words instead of forced-aligned word boundaries; if the inter-seed agreement stays high, the reliability effect is due to segment duration rather than lexical alignment, and the paper's exception would need reinterpretation.","supporting_citations":[{"cited_title":"On the reliability of feature attribution methods for speech classification","cited_arxiv_id":"2505.16406","evidence_quote":"Supplies the pre-trained wav2vec2 speech model that is fine-tuned and analyzed in all experiments."},{"cited_title":"Can We Trust Explainable AI Methods on ASR? An Eval- uation on Phoneme Recognition,","cited_arxiv_id":null,"evidence_quote":"Shows that restricting perturbation windows around phonemes improves the validity of LIME, motivating the idea that perturbation granularity matters."},{"cited_title":"We use a hop length of 320 for the STFT and ISTFT transformations to keep the time resolution at 20ms to be consistent with the wav2vec2 model feature extrac- tor","cited_arxiv_id":null,"evidence_quote":"Provides the method of prepending an ISTFT layer so attributions can be computed for spectrogram inputs even for waveform-native models."},{"cited_title":"Impossibil- ity theorems for feature attribution,","cited_arxiv_id":null,"evidence_quote":"Provides the implementations of Saliency, Integrated Gradients, LIME, and Feature Ablation used to compute attribution scores."},{"cited_title":"“Will you find these shortcuts?","cited_arxiv_id":null,"evidence_quote":"Produces the forced-alignment word time stamps used to aggregate and perturb at word level."}],"review_version":1}