{"id":"bf4f5421-a9dc-439c-b60e-8001cff69425","arxiv_id":"2509.02771","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Neural audio codecs match or beat Opus for speaker verification on VoxCeleb1 below 12 kbps and stay within about 1.5 percentage points EER above it.","lead":"This paper measures how three speaker verification models perform when audio is compressed with neural codecs (Encodec, DAC) and the traditional Opus codec at bitrates from 1.5 to 24 kbps. It finds neural codecs beat Opus at low bitrates and lag only slightly at high bitrates, suggesting they are viable for bandwidth-limited speaker verification.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Opus low-bitrate conditions may be degenerate: 1.5 and 3 kbps rows are nearly identical, so the claimed 6-8% NAC advantage and ~12 kbps crossover rest on unverified actual bitrates.","rationale":"I agree with the reader's weakest assumption. The concern is load-bearing because the abstract's most actionable numbers (6-8%, 0.4-0.7%, crossover ~12 kbps) are point estimates from a single, unreported codec configuration. The nearly identical low-rate Opus EERs are a red flag that the comparison may not be at the labeled bitrates; this can be resolved empirically. I do not see a reason to reject: the trend is supported by many conditions, and DAC at 8 kbps is an independent reference point. But the paper should report measured bitrates and confidence intervals, and the abstract should scope the 6-8% claim to the rates where it actually holds, before the headline is taken at face value. Therefore I keep the CONDITIONAL verdict and do not adjust it.","tokens_in":7324,"tokens_out":11730,"duration_ms":127357,"concrete_test":"Measure actual bitrates of all compressed files (Opus Ogg payload bytes / duration; Encodec/DAC encoded bytes per second) and log Opus's negotiated bandwidth via opus_encoder_ctl(OPUS_GET_BANDWIDTH). Re-run the EER comparison only at measured bitrates that differ by <5% between codecs, adding a forced-CBR Opus condition at 3 kbps. If the 1.5 and 3 kbps Opus files have equal measured bitrate, drop duplicate conditions and recompute the 6-8% margin and the crossover; if the margin disappears or moves, revise the abstract's quantitative claims.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim - NACs beat Opus by 6-8% below 12 kbps, with a crossover near 12 kbps - depends on the codecs being compared at genuinely matched bitrates. Section II-B only says Opus was applied 'via ffmpeg at comparable bitrates'; measured bitrates, VBR/CBR settings, and Opus's internal bandwidth mode are not reported. This is not a pedantic detail: Opus has a low-bitrate floor and switches to narrowband SILK mode below a threshold, which the authors themselves cite in Section IV-D to explain Opus's low-rate degradation. The tables contain internal evidence that the 1.5 and 3 kbps Opus conditions are not distinct operating points: across all three VoxCeleb1 splits, Opus EERs at 1.5 and 3 kbps are nearly identical (e.g., VoxCeleb1-O ECAPA 11.75 vs 11.61; ERes2Net-Large 12.99 vs 12.76), while the jump from 3 to 6 kbps is large. If ffmpeg maps both nominal rates to the same effective bitrate or the same narrowband mode, the abstract's '6-8% at low bitrates' is computed against an Opus reference whose bitrate is not what the table column claims, and the crossover point and margin could shift. The monotonic degradation trend and Encodec's competitiveness are not threatened; the specific quantitative headline is.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper reports a measurement study of how neural audio codecs (Encodec and DAC) and a traditional codec (Opus) affect speaker verification performance. Three modern SV models (ECAPA-TDNN, CAM++, ERes2Net-Large) are evaluated on the VoxCeleb1-O/E/H test splits after audio is compressed and decompressed at bitrates from 1.5 to 24 kbps. The authors report EER and MinDCF values, observe monotonic degradation with decreasing bitrate, claim that NACs outperform Opus at low bitrates (the abstract says 6-8% below 12 kbps, while Section III-A specifies 3 kbps) and remain only marginally behind Opus at about 24 kbps, and provide a t-SNE embedding analysis to illustrate compression-induced loss of speaker separability.","tokens_in":7700,"tokens_out":6390,"duration_ms":67588,"significance":"If the comparisons are technically sound, the paper provides a useful practical benchmark for codec selection in bandwidth-limited speaker verification pipelines, with clear tables across three models and three VoxCeleb1 splits. The use of standard 3D-Speaker recipes and publicly available pre-trained checkpoints is a strength, as is the inclusion of both EER and MinDCF metrics. The central crossover finding near 12 kbps is a plausible and falsifiable empirical claim. However, the headline quantitative statements are broader than the data support, and the fairness of the low-bitrate Opus comparison is not established.","major_comments":[{"comment":"The Opus bitrate conditions are not verified. The text only says Opus was applied \"via ffmpeg at comparable bitrates\" without reporting the actual measured bitrate, VBR/CBR setting, or Opus bandwidth mode. The data contain internal evidence that the 1.5 and 3 kbps Opus conditions are effectively the same operating point: e.g., Table III, ECAPA-TDNN EER 11.75 vs 11.61, CAM++ 10.85 vs 10.86, ERes2Net-Large 12.99 vs 12.76; similar near-identity appears in Tables IV and V. Section IV-D itself attributes Opus's low-rate degradation to switching to narrowband SILK mode. If both nominal rates map to the same low-bitrate mode, the claimed 6-8% Encodec advantage at 3 kbps is measured against an Opus stream whose actual bitrate is not the nominal 3 kbps. Please report measured bitrates and Opus configuration, and re-evaluate or qualify the crossover accordingly.","section":"Section II-B; Tables III-V"},{"comment":"The abstract's quantitative claim \"6-8% at low-bitrates (<12 kbps)\" is an overgeneralization. The 6-8 percentage-point absolute EER gap appears at the 3 kbps condition on VoxCeleb1-O/E (e.g., Table III: 11.61-4.12=7.49 for ECAPA-TDNN, 10.86-4.27=6.59 for CAM++, 12.76-5.22=7.54 for ERes2Net-Large), but at 1.5 kbps the gap is 0.46-2.6 pp and at 6 kbps it is roughly 3-4 pp. Similarly, the abstract's \"0.4-0.7%\" gap at 24 kbps holds for VoxCeleb1-O but not for VoxCeleb1-E/H (e.g., Table V ECAPA: 3.27 vs 2.14 is a 1.13 pp gap). The claims should be stated for the specific bitrate and split where they hold, with all splits reported.","section":"Abstract; Section III-A"},{"comment":"The Encodec 1.5 kbps rows contain identical EER and MinDCF values for CAM++ and ERes2Net-Large: Table III shows both as 10.39/0.8214, and Table V shows both as 20.19/0.9286. Exact equality to four decimal places for two distinct architectures on a 37k-trial set is highly implausible and suggests a data-entry or table-transcription error. These entries should be verified and corrected; if the underlying measurements are wrong, the affected comparisons need to be re-run.","section":"Tables III and V, Encodec 1.5 kbps rows"},{"comment":"The framing that NACs \"do not fundamentally break SV performance\" needs calibration. At 1.5-3 kbps, EER increases roughly 5-10 times from baseline for both codec families (e.g., Table III baseline ECAPA 0.86% vs Encodec 10.29% at 1.5 kbps and 4.12% at 3 kbps). This is a severe degradation in absolute terms, even though NACs may remain competitive with Opus. A quantitative criterion for \"not fundamentally broken\" should be given, or the wording should be softened.","section":"Section III-A; Figure 1"}],"minor_comments":[{"comment":"The description of Opus using LPC and MDCT cites AMR-WB (G.722.2) and AAC standards, not Opus itself. Opus uses SILK (LPC-based) and CELT (MDCT-based); please cite the Opus specification or [11] for these details.","section":"Section IV-D; References [25], [26]"},{"comment":"The shaded region is described as the standard deviation across the three SV models. This conflates architectural differences with measurement uncertainty; showing per-model curves or trial-based confidence intervals would be more informative.","section":"Figure 1"},{"comment":"The t-SNE analysis is qualitative. Claims of monotonic changes in intra-speaker variance and inter-speaker separation would be stronger with quantitative embedding-space metrics (e.g., average intra/inter speaker distances or silhouette scores).","section":"Section III-B"},{"comment":"No code, codec commands, or version identifiers are provided for reproducibility. At minimum, the exact ffmpeg/opusenc invocation and Encodec/DAC model versions should be listed.","section":"General"},{"comment":"The term \"6-8%\" is used inconsistently: the abstract says \"at low-bitrates (<12 kbps)\", while Section III-A correctly limits the 6-8% figure to 3 kbps. Please reconcile these statements.","section":"Abstract and Conclusion"}],"recommendation":"major_revision","confidential_remarks":"The paper's empirical core is potentially valuable and within scope for APSIPA ASC. The main risk is not the measured trends themselves but the fairness and reporting of the low-bitrate Opus comparison; if the authors can provide measured bitrates/modes and correct the suspicious table entries, the paper could become acceptable. I would not reject based on disagreement with current consensus, but the quantitative headline must be made accurate before publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a clean, useful measurement study—three SV models, three VoxCeleb1 splits, Encodec/DAC vs Opus across bitrates—and the qualitative trend (EER rises monotonically as bitrate falls, Encodec wins at very low rates, Opus wins at 24 kbps) is clearly supported by the tables. The paper earns its place as an extension of DASB/Codec-SUPERB, with a practical bitrate sweep that those benchmarks don't quite provide. The embedding t-SNE is a nice sanity check, even if not deep.\n\nThe soft spots are in the framing, not the core measurement. The abstract's '6-8% at low-bitrates (<12 kbps)' is only true at 3 kbps; at 1.5 kbps the Encodec/Opus gap is a couple of EER points, and at 6 kbps it's larger. Similarly, 'crossover at ~12 kbps' is really a crossover somewhere between 6 and 12; at 12 kbps Opus is already better. Those claims should be re-anchored to the actual conditions.\n\nMore important, the Opus bitrate matching worry is real. Section II-B says only 'via ffmpeg at comparable bitrates'—no measured bitrate, no VBR/CBR setting, no internal bandwidth mode. The near-identical Opus EERs at 1.5 and 3 kbps on all splits (e.g., 11.75 vs 11.61 on VoxCeleb1-O) strongly suggest those two nominal rates collapsed to the same operating point, probably the narrowband SILK mode the authors themselves cite in Section IV-D. That doesn't kill the monotonic trend, but it does mean the specific NAC advantage at 'low bitrate' and the crossover point are less solid than the abstract implies. This is a measurement-fairness issue, not a logic flaw, and it's fixable by reporting the actual Opus rates.\n\nI'd also like to see error bars over trials or seeds (Figure 1's shaded region is across models, not noise) and ideally some code or config files. Without code, the exact ffmpeg/Encodec settings are hard to reproduce.\n\nWho is it for: practitioners building low-bandwidth SV pipelines, and codec researchers who want a quick read on where Encodec/DAC stand relative to Opus. It deserves peer review; I'd send it to a workshop or conference venue with a request to verify the Opus settings and tighten the abstract.","headline":"Useful empirical bitrate/performance sweep for codecs on speaker verification, but the abstract overgeneralizes the numbers and the Opus low-bitrate settings need verification.","tokens_in":8180,"tokens_out":2954,"would_cite":true,"duration_ms":29380,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Neural audio codecs preserve speaker identity well enough to beat Opus below 12 kbps and stay within 0.7% EER at 24 kbps.","keywords":["neural audio codec","speaker verification","Encodec","DAC","Opus","bitrate","equal error rate","VoxCeleb1"],"falsifier":"Measure the actual encoded bitrate of Opus at each nominal setting (bytes per second of audio), then rerun the EER comparison with Opus forced to a constant measured bitrate (VBR off). If the crossover moves or disappears, the low-bitrate advantage attributed to neural codecs is an artifact of Opus configuration.","tokens_in":7257,"feed_emoji":"🎙️","tokens_out":6670,"duration_ms":63760,"temperature":0.7,"pith_summary":"This paper asks whether neural audio codecs (NACs) such as Encodec and DAC can be used in speaker-verification pipelines without wrecking accuracy compared with the hand-engineered Opus codec. Testing three state-of-the-art verification models on the VoxCeleb1 test set, the authors find that verification error rises monotonically as bitrate falls for every codec and model. At low bitrates (below about 12 kbps) NACs beat Opus by 6–8% relative error, while at 24 kbps Opus retains only a small edge (0.4–0.7% absolute EER). The paper concludes that NACs are a feasible transmission codec for speaker verification, especially when bandwidth is scarce, and that the remaining high-bitrate gap reflects NACs' optimization for perceptual quality rather than speaker-discriminative detail.","feed_headline":"Neural audio codecs beat Opus under 12 kbps for speaker verification","feed_subtitle":"At 24 kbps the accuracy gap to Opus is under 0.7% EER, so neural codecs are viable for voice biometrics.","key_machinery":"The evaluation pipeline is the load-bearing mechanism: audio is compressed and decompressed by each codec at matched nominal bitrates, then scored by three pretrained speaker-verification models (ECAPA-TDNN, CAM++, ERes2Net-Large) using the paper's standard verification recipes with cosine similarity, reported as EER and MinDCF. The crossover near 12 kbps is the central empirical phenomenon, and t-SNE visualizations of embeddings support the mechanism: intra-speaker dispersion grows and inter-speaker separation shrinks as bitrate decreases.","core_discovery":"The central empirical discovery is a crossover in speaker-verification performance near 12 kbps: Encodec and DAC match or beat Opus when bandwidth is tight, and stay within a fraction of a percent EER of Opus at 24 kbps. The degradation curve is monotonic across all tested models and codec splits, and the effect is consistent across the three VoxCeleb1 evaluation protocols. At 1.5 kbps error rates inflate roughly tenfold regardless of model size, showing that extreme compression strips speaker-discriminative cues such as formants and high-frequency detail; in the 3–12 kbps range the neural codecs preserve enough speaker identity to outperform Opus, which drops to narrowband mode and discards","pith_inferences":["If the nominal-Opus bitrate assumption is wrong, the 12 kbps crossover could shift; a replication that measures actual Opus bitrate would bound the claim.","Speech tasks with different sensitivity to high-frequency content, such as automatic speech recognition, might show a different crossover, so the 12 kbps boundary is likely specific to speaker verification.","The residual high-bitrate gap suggests a concrete design target: speaker-aware codebooks or fine-tuning verification models on codec-degraded audio could close the 0.4–0.7% EER deficit.","The embedding-space behavior points toward using intra-speaker dispersion as a training signal for codecs that preserve identity, not just perceptual quality."],"forward_implications":["Below roughly 12 kbps, a transmission pipeline for speaker verification should prefer a neural codec over Opus; above that, Opus is slightly safer but the difference is small.","DAC at 8 kbps reaching Encodec-24-kbps-level performance shows bitrate alone does not determine verification loss; codec architecture and training objective play a large role.","The monotonic EER-versus-bitrate trend lets system designers predict verification accuracy for a given bandwidth budget.","The roughly tenfold EER increase at 1.5 kbps shows that extreme compression strips identity-critical cues regardless of model capacity."],"supporting_citations":[{"why":"Supplies the neural audio codec Encodec, tested at 1.5–24 kbps.","marker":"[4]"},{"why":"Supplies the second neural codec DAC, tested at 8 kbps as a mid-bitrate reference.","marker":"[5]"},{"why":"Defines the traditional Opus codec used as the comparison baseline.","marker":"[11]"},{"why":"One of the three speaker-verification models evaluated.","marker":"[7]"},{"why":"Second speaker-verification model, a lighter D-TDNN architecture.","marker":"[8]"},{"why":"Third speaker-verification model, a larger Res2Net variant.","marker":"[9]"},{"why":"Provides the VoxCeleb1 test splits and trials used for evaluation.","marker":"[10]"},{"why":"Supplies the standard evaluation recipe and trial configuration for scoring.","marker":"[12]"}],"fun_headline_variants":["Neural codecs beat Opus for speaker ID under 12 kbps","Speaker verification holds up with neural codecs at low bitrates","Neural codecs outshine Opus in tight-bandwidth voice ID","Under 12 kbps neural codecs win speaker verification","Neural codecs: viable alternative to Opus for voice biometrics"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The results assume that Opus was actually operating at the nominal bitrates stated in the comparison; the paper does not report measured bitrates or encoder mode, so the reported crossover boundary could shift if the true Opus bitrate differs from nominal.","fun_headline_variants_meta":{"raw":{"variants":["Neural codecs beat Opus for speaker ID under 12 kbps","Speaker verification holds up with neural codecs at low bitrates","Neural codecs outshine Opus in tight-bandwidth voice ID","Under 12 kbps neural codecs win speaker verification","Neural codecs: viable alternative to Opus for voice biometrics"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000285,"raw_usage":{"total_tokens":1534,"prompt_tokens":783,"completion_tokens":751,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":527,"completion_tokens_details":{"reasoning_tokens":659}},"tokens_in":527,"tokens_out":751,"duration_ms":7877,"temperature":1.0,"reasoning_tokens":659,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T11:23:52.076815+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure the actual encoded bitrate of Opus at each nominal setting (bytes per second of audio), then rerun the EER comparison with Opus forced to a constant measured bitrate (VBR off). If the crossover moves or disappears, the low-bitrate advantage attributed to neural codecs is an artifact of Opus configuration.","supporting_citations":[{"cited_title":"Cam++: A fast and efficient network for speaker verification using context-aware masking,","cited_arxiv_id":null,"evidence_quote":"Second speaker-verification model, a lighter D-TDNN architecture."},{"cited_title":"Vox- celeb: A large-scale speaker identification dataset,","cited_arxiv_id":null,"evidence_quote":"Provides the VoxCeleb1 test splits and trials used for evaluation."},{"cited_title":"3d- speaker-toolkit: An open source toolkit for multi- modal speaker verification and diarization,","cited_arxiv_id":null,"evidence_quote":"Supplies the standard evaluation recipe and trial configuration for scoring."}],"review_version":1}