{"id":"b918f1e4-7028-4403-a2ce-d085797c23d7","arxiv_id":"2601.03615","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"Reasoning traces in audio deepfake detectors fail in two distinct ways — incoherent panic under acoustic attacks, confident false reasoning under linguistic attacks — and those failure signatures may be detectable.","lead":"This paper introduces SARA, three diagnostics that test whether an audio deepfake detector's explanations match the sound, support its verdict, and flag problems when the verdict is wrong. It reports that acoustic attacks produce confused explanations, while text-based attacks can make models confidently justify the wrong answer.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Central 'reasoning tax vs shield' bifurcation is confounded by Qwen2-tailored data synthesis; no control rules out a data-alignment artifact.","rationale":"The reader identified the undisclosed entailment function E as the weakest assumption, and that is a legitimate concern for the coherence/dissonance metrics. However, the single most load-bearing threat to the paper's central claim is the data-synthesis confound. The 'reasoning tax vs shield' bifurcation is the paper's headline contribution and is directly supported by Table 1 and Tables 2-3, which compare RSN and NON modes. Because the training data was generated and iteratively refined using Qwen2-Audio itself, the comparison is biased: Qwen2 sees reasoning traces matched to its own inductive biases, while gemma, Phi-4, and granite see traces that are optimized for a different model's distribution. The authors acknowledge this limitation in Appendix B.1, yet still conclude a general 'reasoning tax.' This is not an internal inconsistency, but it is a severe external validity threat: the result may not generalize beyond the specific data pipeline. The reader's point (3) lists this as an issue but does not make it the weakest assumption; I argue it is the most load-bearing because it directly undermines the paper's central claim, not just the supporting metrics. The entailment-function issue is also important and would independently require conditional acceptance, but it is secondary because it affects the forensic diagnostic framing rather than the central tax/shield result. My proposed control experiment is concrete and would settle whether the bifurcation is genuine or an artifact of data alignment. The verdict should remain CONDITIONAL: the paper is promising and the limitation is acknowledged, but the central claim is not yet established without this control.","tokens_in":22765,"tokens_out":5510,"duration_ms":51664,"concrete_test":"Re-run the RSN vs NON attack comparison after fine-tuning each ALM on reasoning traces generated by that same model (self-bootstrapping), holding all attack protocols and evaluation identical. If the ASR gap between Qwen2 and gemma-3n (or the coherence/dissonance differences) narrows or disappears, the 'reasoning tax' is a data-alignment artifact, not a general property of CoT reasoning. A complementary check: use a model-agnostic dataset (e.g., GPT-5-generated, without the Qwen2 self-correction loop) to fine-tune all models; if the bifurcation persists, the original claim gains support.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper's core claim — that CoT acts as a shield for Qwen2-Audio but a tax on gemma-3n, Phi-4, and granite — is directly undermined by the training-data generation process described in Appendix B.1. The reasoning dataset was synthesized by GPT-5, then iteratively refined by fine-tuning Qwen2-Audio on the current set and using Qwen2 to re-generate all reasoning traces, with majority-voting selection repeated until convergence. The authors explicitly acknowledge: 'the resulting dataset is highly aligned with its specific processing style. Consequently... other models like Phi-4 or Gemma may encounter a reasoning tax due to this domain-specific data alignment.' This means the RSN-vs-NON comparison is not a clean test of whether explicit reasoning helps or hurts; it is a comparison of Qwen2 on in-distribution reasoning data versus other models on out-of-distribution reasoning data. The observed 'tax' for non-Qwen models could be entirely a distribution-shift artifact, not a general property of CoT or acoustic perception. No control experiment — e.g., training each model on its own self-generated reasoning traces, or training all models on a model-agnostic reasoning dataset — is reported. Without such a control, the central bifurcation is not established; it may be a 'data alignment tax' rather than a 'reasoning tax.' This concern is more load-bearing than the undisclosed entailment function E because it targets the paper's headline claim directly, and the authors themselves flag the confound while still drawing the strong conclusion.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces SARA, a diagnostic framework for evaluating the reasoning traces of Audio Language Models (ALMs) under adversarial attacks. It defines three audit dimensions — acoustic perception (Eq. 1), cognitive coherence (Eq. 2), and cognitive dissonance (Eq. 4) — and evaluates four open-source ALMs (Qwen2-Audio, Phi-4-multimodal, gemma-3n-E4B, granite-3.3-8b) fine-tuned with chain-of-thought (CoT) on ASVSpoof 2019. The models are tested under linguistic (TAPAS) and acoustic (CLAD-based) attacks. The central claims are (1) CoT acts as a \"shield\" for acoustically grounded models and a \"tax\" for others, (2) cognitive dissonance can serve as a \"silent alarm\" when the final verdict is wrong, and (3) acoustic and linguistic attacks induce distinct failure modes (panic vs. rationalization trap). The abstract also claims that coherence alone enables detection of perturbed audio with F1 = 0.78.","tokens_in":23148,"tokens_out":6286,"duration_ms":60407,"significance":"If established, the framework would be a useful contribution: it shifts the robustness question from binary prediction to reasoning-level failure, proposes a novel 'silent alarm' concept, and distinguishes attack-specific pathologies. The paper uses a standard benchmark, open-source models, and reports detailed attack hyperparameters, and the ethics statement promises release of attack and audit code. However, the current evidence does not support the headline claims. The two central metrics depend on an unspecified entailment function, the main tax/shield comparison is confounded by Qwen2-tailored data synthesis, and the F1 claim in the abstract is absent from the body. The perception audit also uses a small, LLM-labeled sample. These issues are load-bearing for the manuscript's contributions as stated.","major_comments":[{"comment":"The coherence and dissonance metrics depend on an entailment function E(ri, c) that is never operationalized. The paper does not state what implements E (a separate judge LLM? the same ALM? a human template?), how it was validated, or its error rate. Without this, the quantitative values in Tables 2–7 and all conclusions about 'panic', 'rationalization trap', and 'silent alarm' are not interpretable. Please specify the implementation, report its agreement with human judgment, and show worked examples where E is not trivial.","section":"§3.3 Eq. (2) and §3.4 Eq. (4)"},{"comment":"The RSN training data was generated by GPT-5, then iteratively regenerated by fine-tuning Qwen2-Audio and majority-voting with Qwen2-Audio. Therefore, Qwen2's RSN advantage and the other models' 'reasoning tax' may be an in-domain/out-of-domain artifact rather than a general property of CoT. The authors acknowledge this in B.1. A control is needed: e.g., train each model on its own self-generated traces, or train all models on a fixed model-agnostic reasoning corpus. Without such a control, the central tax/shield bifurcation as a function of acoustic perception is not established.","section":"Appendix B.1 and §6.3"},{"comment":"The abstract claims that coherence enables detection of perturbed audio with F1 = 0.78 'without accessing the raw acoustic signal', but no such experiment appears in the body. The 78.2% figure in Table 4 is a dissonance rate under one attack (gemma-3n, Shape Space), not a detection F1. Please add the detection experiment, or remove the claim. This is a direct discrepancy between the abstract and the results.","section":"Abstract and §5.3.2/§6.2"},{"comment":"The 'Shield Effect' text says that for Qwen2 and Granite, CoT 'effectively lowers the Attack Success Rate (ASR)'. This is true only for linguistic attacks (Table 3). Under acoustic attacks, Table 2 shows the opposite: Qwen2 ASR rises from 36.6 to 45.7 and Granite from 34.4 to 49.7 when moving from NON to RSN. The paper should either restrict the shield claim to linguistic attacks or report an aggregation that explains the discrepancy. This directly affects Contribution 1.","section":"§6.3 vs Tables 2 and 3"},{"comment":"The perception audit underpins the 'grounded vs ungrounded' explanation of the bifurcation, but its ground-truth labels come from majority vote of three frontier LLMs (Gemini-3-Pro, Gemini-3-Flash, GPT-5.2) on only N=100 samples. LLM self-labeling is not an independent oracle for acoustic properties, and the per-dimension scores in Fig. 2 are shown without confidence intervals. Please provide human-validated labels on a subset, or at least confidence intervals and inter-labeler agreement, and acknowledge the limitation more prominently in the main text.","section":"Appendix B.2 and §5.2/Fig. 2"}],"minor_comments":[{"comment":"'beyondblack-box' and 'black-boxclassifiers' are missing spaces; the text should read 'beyond black-box' and 'black-box classifiers'.","section":"Abstract"},{"comment":"The axis label renders as 'Cognitive Coherence ( Coh)%' and 'Cognitive Dissonance ( Diss)%' with broken spacing; fix the LaTeX/PDF output.","section":"Fig. 3"},{"comment":"Column headers like 'N ON' and 'RSN' appear with corrupted superscripts (e.g., 'Qwen2-Audio-7BN ON'). Please clean the table formatting.","section":"Tables 1–3"},{"comment":"The definitions of OC and ASR should state the exact test set used for each attack condition; the discrepancy between Table 1 clean accuracies and Table 3 clean OC values (e.g., Qwen2 98.0 vs 67.4) is confusing and should be explained.","section":"§4 Metrics"},{"comment":"The training set is heavily imbalanced (22,627 fake vs 2,481 real). The paper should report whether any class-balancing was applied and how this affects the real/fake F1 numbers in Table 1.","section":"Appendix B.1"}],"recommendation":"major_revision","confidential_remarks":"The manuscript's own appendices concede the two largest threats to validity: B.1 admits the Qwen2-alignment of the training data, and B.2 admits the small N and LLM label proxy. These are not fatal in principle, but the main text reads as if the conclusions are already established. I would require the entailment function to be specified and validated, the data-alignment control to be run, and the abstract's F1 claim to be either substantiated or removed before this is suitable for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague, quick read of arXiv:2601.03615. The useful core: SARA gives audio deepfake detection a three-layer audit—perception, coherence, dissonance—and the panic-versus-rationalization distinction is a genuine lens. The empirical split (acoustic attacks drop coherence, linguistic attacks keep coherence high while fooling the model) is plausible and is not in the cited ALLM4ADD or post-hoc explainability work. Tables 4 and 5 do show that pattern for these four models. That is a real contribution, and the authors are honest about several limits.\n\nThe soft spots, in order. The load-bearing one: coherence and dissonance are defined through an entailment function E(r_i,c) that is never specified or validated. Every central number—the 14.20% coherence drop, the silent alarm, the panic/rationalization taxonomy—flows through E, and the paper never says whether E is another LLM, the same ALM, or a template, nor how its error rate was checked. That is not cosmetic; the core measurement pipeline is unverifiable as written.\n\nThe stress-test concern also holds up. Appendix B.1 says the CoT training set was synthesized by GPT-5, then iteratively regenerated and majority-filtered by Qwen2-Audio until convergence. The authors themselves admit the dataset is highly aligned with Qwen2's processing style. So the RSN-vs-NON comparison is Qwen2 on in-distribution reasoning data versus other models on out-of-distribution reasoning data. The observed reasoning tax on gemma/phi/granite could simply be a data-alignment tax. Without a control—each model trained on its own self-generated traces, or all models on a model-agnostic dataset—the headline bifurcation is not established. The authors flag the confound but still draw the strong conclusion.\n\nSmaller issues: the metadata abstract claims an F1=0.78 detection result that does not appear in the body, and the perception audit's ground truth is an LLM majority vote on N=100. Fine as a pilot, but weak as the basis for the perception scores in Figure 2.\n\nWho this is for: people building LLM-based audio forensics or reasoning-robustness benchmarks. It deserves a serious referee—the framework is publishable as a diagnostic proposal, and the panic/rationalization split is worth testing under cleaner conditions. But as it stands, the central claim is conditional. I'd send it to review with the expectation that the authors operationalize E, add a self-generated reasoning control or soften the tax/shield claim, and reconcile the abstract F1 with the body. If they can do that, this becomes a solid paper; right now it is a promising one with a missing measurement core.","headline":"Useful diagnostic framing and a real empirical pattern, but the headline tax/shield split is confounded by Qwen2-tailored data and the core metrics depend on an unspecified entailment function.","tokens_in":23634,"tokens_out":1903,"would_cite":false,"duration_ms":18966,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Reasoning helps some audio deepfake detectors and hurts others.","keywords":["audio deepfake detection","audio language models","chain-of-thought reasoning","adversarial attacks","reasoning coherence","cognitive dissonance","explainable AI","forensic audio analysis"],"falsifier":"Take a random sample of the reasoning traces, have humans label whether each reasoning aspect entails the verdict, recompute coherence and dissonance, and see whether the panic/rationalization split and the 0.78-F1 text-only detector survive. A second check: run the same detector on a held-out set of real-world perturbed audio; an F1 near 0.5 would refute the claim that coherence flags adversarial input without the raw signal.","tokens_in":22577,"feed_emoji":"🎙️","tokens_out":5180,"duration_ms":49839,"temperature":0.7,"pith_summary":"This paper argues that explicitness is not a free robustness upgrade: when audio language models are forced to reason out loud before judging deepfakes, the outcome splits. For models that hear acoustic detail well, the reasoning acts as a shield; for models with weaker perception, it becomes a tax that lets attacks succeed more often and even fabricates evidence for the wrong verdict. The paper also claims that the two attack families leave different reasoning signatures — acoustic attacks erode reasoning-verdict coherence, while linguistic attacks preserve coherence while raising attack success — and that coherence alone can flag perturbed audio (F1 0.78) without touching the raw signal. These findings matter because they reframe explainable deepfake detection: the explanation trace is not just a transparency add-on but a diagnostic signal that survives some failures.","feed_headline":"Chain-of-thought shields some audio deepfake detectors, taxes others","feed_subtitle":"Coherence of the model's own reasoning flags manipulated audio at 0.78 F1, even when the final verdict is wrong.","key_machinery":"The load-bearing object is SARA, a three-tier audit built on a binary entailment function E(ri,c) that decides whether a reasoning aspect supports the final verdict. From E it derives cognitive coherence ΦCoh and, on wrong verdicts, cognitive dissonance ΨDiss; a companion perception score ΦPerc checks whether textual descriptions match ground-truth acoustic properties. Differential shifts ΔΦ/ΔΨ under perturbation sort failures into 'panic' (low coherence, high dissonance) and 'rationalization trap' (high coherence, low dissonance), and the text-only detection of attack traces rests on measured coherence.","core_discovery":"The central claim, in the authors' terms, is a bifurcation: explicit chain-of-thought reasoning lowers attack success for audio language models with strong acoustic perception (a shield) but raises it for weaker-perception models (a tax), which hallucinate coherent justifications for wrong verdicts. Across four open models, acoustic perturbations cut average reasoning-verdict coherence by 14.20% and raised dissonance, while linguistic attacks achieved higher success while keeping coherence high and dissonance near zero, a 'rationalization trap.' As a result, text-only analysis of the reasoning trace detected perturbed audio at 0.78 F1 even when the final label was wrong, and high dissonance","pith_inferences":["Because the reasoning data were synthesized and filtered through a single model's outputs (the paper concedes this alignment), the shield/tax split may partly reflect training-data alignment rather than inherent model capability; testing on independently annotated reasoning traces would separate the two.","A testable extension the authors leave implicit: the 0.78 F1 coherence detector could be evaluated as a standalone pre-filter in a pipeline where the raw audio is encrypted or bandwidth-limited, where current binary detectors cannot operate.","If the entailment judge is replaced by human annotation and the coherence/dissonance rankings shift, the 'silent alarm' claim would need to be re-anchored; this is the natural next audit.","The framework's three axes could be applied to non-adversarial hard examples, such as genuine misclassifications from missing liveness cues, to see whether panic and rationalization are attack-specific pathologies or general failure modes."],"forward_implications":["If the bifurcation is real, adding chain-of-thought prompting to an audio deepfake detector is not inherently safety-improving; it must be conditioned on the model's acoustic perception.","If coherence erosion under acoustic attacks is systematic, reasoning-verdict consistency can serve as a forensic red flag for manipulated audio even for models whose labels are wrong.","If linguistic attacks suppress dissonance, transcript-level perturbations are the more dangerous failure mode for explainable systems, since they produce confident, internally consistent wrong answers that an auditor would likely trust.","If coherence-based detection transfers beyond the test set, a lightweight text-only module could screen for adversarial audio without decoding the raw waveform, which is useful when the signal is unavailable or protected.","The coherence-dissonance tradeoff (r=-0.79) implies that current audio language models cannot simultaneously maximize logical explanation and anomaly signaling, so any deployment must choose which failure mode to tolerate."],"fun_headline_variants":["Audio deepfake reasoning: shield for some, tax for others","Coherent reasoning catches deepfake attacks at 0.78 F1","SARA: why chain-of-thought splits audio deepfake defense","Reasoning traces expose audio fakes when verdicts fail","Acoustic attacks fracture AI reasoning by 14%, text betrayals"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"Everything rests on the unspecified check that decides whether a reasoning statement actually supports the model's final verdict; if that check is biased or the model is grading itself, the coherence, dissonance, and alarm findings are not established.","fun_headline_variants_meta":{"raw":{"variants":["Audio deepfake reasoning: shield for some, tax for others","Coherent reasoning catches deepfake attacks at 0.78 F1","SARA: why chain-of-thought splits audio deepfake defense","Reasoning traces expose audio fakes when verdicts fail","Acoustic attacks fracture AI reasoning by 14%, text betrayals"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000185,"raw_usage":{"total_tokens":1178,"prompt_tokens":784,"completion_tokens":394,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":528,"completion_tokens_details":{"reasoning_tokens":305}},"tokens_in":528,"tokens_out":394,"duration_ms":4128,"temperature":1.0,"reasoning_tokens":305,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T12:13:56.682007+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a random sample of the reasoning traces, have humans label whether each reasoning aspect entails the verdict, recompute coherence and dissonance, and see whether the panic/rationalization split and the 0.78-F1 text-only detector survive. A second check: run the same detector on a held-out set of real-world perturbed audio; an F1 near 0.5 would refute the claim that coherence flags adversarial input without the raw signal.","supporting_citations":[],"review_version":1}