Pith. sign in

REVIEW 3 major objections 5 minor 7 cited by

Neural Codec Source Tracing: Toward Comprehensive Attribution in Open-Set Condition

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper shows that neural codec source tracing models, while nearly perfect on in-distribution classification and out-of-distribution detection, perform very poorly on unseen real audio, with the best model reaching only 16.94% F1 on…

desk verdict Useful new open-set codec source tracing benchmark; the unseen-real claim needs a ranking metric to fully land. read the letter →

arxiv 2501.06514 v1 pith:XXCUPF6P submitted 2025-01-11 cs.SD cs.AIeess.AS

Yuankun Xie , Xiaopeng Wang , Zhiyong Wang , Ruibo Fu , Zhengqi Wen , Songjun Cao , Long Ma , Chenxing Li
show 2 more authors
Haonnan Cheng Long Ye
This is my paper · ORCID
classification cs.SDcs.AIeess.AS
keywords neuralcodecsourcetracingopen-setrecognitionaudiodeepfakedetectionout-of-distributionST-Codecfakelanguagemodel
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper defines the Neural Codec Source Tracing (NCST) task: given a piece of audio, identify which of several known neural codecs reconstructed it, while also flagging audio from unknown codecs and from non-codec synthesis. To support the task, the authors built the ST-Codecfake dataset of about 235,000 bilingual samples synthesized by 11 neural codecs plus audio-language-model test material, and they set up a three-part benchmark covering in-distribution classification, open-set out-of-distribution detection, and behavior on unseen real audio, changed codec configurations, and changed source domains. The central finding is an asymmetry: the best NCST models reach 99.99% F1 on in-distribution classification and 97.54% AUC on OOD detection, but collapse on unseen real audio, with the best model scoring only 16.94%, 77.01%, and 4.77% F1 on three real-speech test sets. The paper argues that current source tracing systems therefore cannot be trusted to attribute or reject audio in open-set conditions, and identifies the likely cause as too little real training data and underweighting of the real class.

What carries the argument

The machinery is the ST-Codecfake dataset and the accompanying evaluation protocol. ST-Codecfake pairs 23,228 real utterances from VCTK and AISHELL-3 with reconstructions from six in-distribution codecs (Mimi, SpeechTokenizer, FunCodec, EnCodec, FACodec, SNAC) and five out-of-distribution codecs (WavTokenizer, AcademicCodec, DAC, AudioDec, SoundStream), totalling 235,736 samples. The protocol then tests a seven-class classifier (real plus six codecs) under three axes of generalization failure: unseen real audio from different corpora, different generation configurations of the same codec, and different source domains; it also measures OOD detection with threshold-free metrics (AUC, FPR95, EER) and runs a back-end detection test on audio-language-model outputs. The three baselines, Mel-LCNN, AASIST, and W2V2-AASIST, are standard anti-spoofing models repurposed with a seven-head classifier; W2V2-AASIST is the strongest and is the model used for the OOD experiments.

What would settle it

A direct test: train W2V2-AASIST on ST-Codecfake with the real class oversampled to match the majority fake class and with an additional held-out speaker set from VCTK and AISHELL, then measure F1 on the 19LA, ITW, and NCSSD real subsets. If F1 climbs above 90% on all three, the paper's attribution of failure to real-class underrepresentation and insufficient real training data is confirmed; if it stays near the reported 16.94%, 77.01%, and 4.77%, the failure is due to corpus shift or codec-specific overfitting rather than class imbalance.

Watch

Extended reading notes

Core claim

The paper's central claim is that open-set neural codec source tracing, as formalized by the new task, is not yet solved: the models that nearly perfectly classify the seven in-distribution classes (real plus six codecs) and that detect out-of-distribution codecs with high AUC systematically misclassify real audio they have not been trained on. The evidence is Table 4, where every baseline model drops sharply on the 19LA real subset, the In-the-Wild real subset, and the NCSSD set, with W2V2-AASIST, the best model overall, at 16.94%, 77.01%, and 4.77% F1 respectively. The paper frames this as a failure of the real category to generalize and shows that the same high-confidence logits that make OOD detection easy are what push unseen real audio outside the real class. The authors also show that the NCST model can often identify the back-end codec of audio-language-model outputs, which is the promised interpretability benefit.

Load-bearing premise

The benchmark's validity depends on the ST-Codecfake labels: that audio reconstructed by a given neural codec carries artifacts specific to that codec, and that downsampling to 16 kHz and trimming or padding to 4 seconds preserves those artifacts well enough for a classifier to learn them.

Editorial extensions

If this is right

  • Source tracing in open-set conditions is not achieved by current models; any deployed NCST system must treat unseen real audio as an open problem rather than as an in-distribution class.
  • The high OOD detection scores (up to 97.54% AUC) mean that logits-based confidence is enough to tell known codecs from novel ones, so back-end codec detection for ALM audio can be built on top of existing in-distribution classifiers.
  • Because the failure on unseen real audio is attributed to data imbalance and real-class underrepresentation, adding more diverse real speech and reweighting the real class is a concrete next step that could close most of the gap.
  • The dataset and code release make it possible for other groups to reproduce the benchmark and to test whether alternative training strategies fix the real-class collapse.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The 4.77% F1 on the NCSSD real subset, far below chance for a seven-way classifier, suggests that studio-recorded dialogue speech is being assigned to codec classes with high confidence; it would be worth measuring whether the model's per-sample confidence correlates with codec artifacts or with speaker characteristics.
  • The benchmark's OOD detection treats all non-codec fake audio and all unseen codecs as a single 'other' class; a stricter test would require per-codec open-set identification, where the model must say which known codec is closest, not just whether it is out-of-distribution.
  • The ALM back-end detection results, with Mini-Omni at 100% F1, likely reflect that some ALMs directly expose the codec's residual artifacts; if so, NCST could be inverted as a tool for auditing which codec a deployed ALM uses, which the paper only hints at.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The manuscript defines a new task, Neural Codec Source Tracing (NCST), in which a model must classify audio by the neural codec used to reconstruct it, detect out-of-distribution (OOD) novel codecs, and attribute the back-end of audio language model (ALM) speech. The authors construct the ST-Codecfake dataset, containing bilingual real audio and fake audio reconstructed with 11 neural codecs, and evaluate three baselines (Mel-LCNN, AASIST, W2V2-AASIST) under closed-set, OOD, robustness, and ALM back-end conditions. The central empirical finding is that the models achieve near-perfect in-distribution classification (99.99% average F1) and strong OOD detection (best AUC 97.54%), but perform poorly at classifying unseen real audio from ASVspoof2019LA, In-the-Wild, and NCSSD, with the best model reaching only 16.94%, 77.01%, and 4.77% F1, respectively. The paper concludes that current source-tracing models lack robustness in classifying unseen real audio and that future work should focus on improving real-class robustness.

Significance. If the reported findings hold, the paper makes a useful contribution by formalizing the NCST task, releasing a large codec-based source-tracing dataset (ST-Codecfake), and showing a striking generalization gap to unseen real audio. The use of threshold-free OOD metrics (AUC, FPR95, EER), multiple OOD scenarios (different method/configuration/source), three diverse baseline architectures, and public code and data are clear strengths. The negative result on unseen real audio is important for the community because it indicates that strong in-distribution and OOD performance does not imply the ability to recognize real speech in the open-set setting. However, the strength of this central claim is currently limited by the threshold-based F1 metric used for unseen-real evaluation and by the very small test set for one ALM back-end, so the main conclusion needs additional validation before it can be accepted as stated.

major comments (3)
  1. [§4.1, Table 4] The "Unseen Real" robustness evaluation is reported exclusively via F1-score, which requires a decision threshold, but the manuscript does not specify how thresholds are selected for these unseen sets. Moreover, §4.2 explicitly states that the model assigns low-confidence logits to both unseen real and unseen fake samples while ID samples receive excessively high-confidence logits. This implies that the low F1 values (e.g., 4.77% on NCSSD real, 16.94% on 19LA real) may reflect an inappropriate threshold inherited from the development set rather than a lack of discriminative ranking between real and fake audio. To support the claim that models "lack robustness in classifying unseen real audio," please report a threshold-free measure such as real-vs-fake ROC-AUC or average precision on these unseen-real sets, and discuss whether the intended claim is about ranking ability or about decision-making under a fixed threshold.
  2. [§2.4, Table 6] The ALM back-end detection experiment uses only 18 SpeechGPT-Gen samples and 4,436 VALL-E samples, yet Table 6 reports per-model F1 scores for these categories (e.g., 99.35–99.48% for SpeechGPT-Gen) without confidence intervals or significance tests. With 18 samples, a single misclassification changes the F1 by several percentage points, so these numbers are not reliable evidence of accurate back-end detection. Please provide the sample count for each tested condition, bootstrap confidence intervals, or a statistical comparison, and temper the corresponding conclusions in §4.3 and the abstract.
  3. [§3.3, implementation details] All audio is downsampled to 16 kHz and trimmed/padded to 4 seconds before feature extraction, even though several codecs (Mimi, EnCodec, FACodec, SNAC, WavTokenizer, DAC, AudioDec) operate natively at 24 kHz or 44 kHz. The benchmark's validity depends on codec-specific artifacts surviving this resampling and length normalization, but the paper provides no analysis of whether codec traces are preserved. Without such validation, the reported ID and OOD results may be confounded by the preprocessing pipeline rather than reflecting codec identity. Please add an experiment or analysis demonstrating that the 16-kHz/4-s representation retains codec-specific information (e.g., by comparing ID classification with and without downsampling for a subset of codecs).
minor comments (5)
  1. [§1 and §2.1] There are several typos and inconsistencies: "categoriary" should be "category," "ALSHELL" should be "AISHELL-3," and the dataset name is inconsistently capitalized as ST-Codecfake vs. ST-codecfake.
  2. [Tables 3–4] The column heading "A VG" should be "AVG".
  3. [§2.1] The text refers to the NCSSD dataset using reference [19], but the cited title is "Generative expressive conversational speech synthesis"; please clarify the relationship between NCSSD and this reference.
  4. [Figure 2] The labels "T3-T5" appear in Figure 2 but are never defined in the text; please explain what they represent.
  5. [§4.1, Table 4] The introduction criticizes threshold-based F1 as overly subjective, but the manuscript never describes the threshold-selection procedure used to compute the F1 scores in Table 4; please clarify how the thresholds are set for both ID and unseen-real evaluations.

Circularity Check

0 steps flagged · score 1.0 of 10

No circular derivation; the benchmark conclusions are empirical and do not reduce to fitted parameters or self-citations.

full rationale

The paper's central claim—that NCST models classify in-domain codecs and detect OOD codecs well but generalize poorly to unseen real audio—is an empirical benchmark outcome, not a quantity derived from its own inputs. The ST-Codecfake dataset (Sec. 2.1) fixes labels by construction from 11 external neural codecs and external speech corpora (VCTK/AISHELL-3); no label is defined by the model's output. The OOD metrics (AUC/FPR95/EER, Sec. 2.3) are threshold-free and are computed on held-out codec/fake types the models never saw, so the strong OOD results are not fitted. The ALM back-end identification experiment (Sec. 2.4) trains on codec-reconstructed audio and tests on independently generated ALM audio, so a correct Mimi/SNAC prediction is genuine transfer. Self-citations are present—W2V2-AASIST as a baseline from [8] and NSD as an OOD comparator from [8]—but neither is load-bearing: W2V2-AASIST is re-trained and re-evaluated in Tables 3-4, and NSD is one of five comparators and is not the basis of the headline conclusion. The paper itself flags the threshold-based F1 limitation (Sec. 1 and Sec. 4.1), which is a measurement-design concern, not circularity. No equation in the paper defines a predicted quantity in terms of the quantity it claims to predict, and no fitted parameter is renamed as a prediction.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The benchmark rests on the correctness of the dataset labels, the representativeness of the 11 codecs, and the fairness of the unseen-real test sets. No additional free parameters or invented entities are used beyond standard training hyperparameters.

assumptions (3)
  • domain assumption The 11 neural codec methods used to generate ST-Codecfake audio are representative of the neural codecs used in current ALM-based deepfakes.
    Section 2.1 selects ID codecs as those 'commonly used in ALM' and includes five OOD codecs; if real-world systems use other codecs, the benchmark may not transfer.
  • domain assumption Reducing all audio to 16 kHz mono and 4-second clips preserves the codec-specific artifacts that allow source tracing.
    Implementation details in Section 3.3 down-sample and truncate all samples; if codec traces live in higher frequencies or longer context, the benchmark may be misleading.
  • domain assumption The three unseen-real test sets (ASVspoof2019LA, In-the-Wild, NCSSD) provide a fair evaluation of robustness to unseen real audio.
    Section 2.2 uses these sets without controlling for channel/speaker/recording conditions, so poor F1 may reflect domain shift rather than a general inability to handle unseen real audio.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Neural Codec Source Tracing: Toward Comprehensive Attribution in Open-Set Condition." pith.science (2026). https://pith.science/paper/XXCUPF6P

@misc{pith2026250106514,
  author       = {Pith},
  title        = {Pith review of: Neural Codec Source Tracing: Toward Comprehensive Attribution in Open-Set Condition},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XXCUPF6P}},
  note         = {Machine review of arXiv:2501.06514}
}
read the original abstract

Current research in audio deepfake detection is gradually transitioning from binary classification to multi-class tasks, referred as audio deepfake source tracing task. However, existing studies on source tracing consider only closed-set scenarios and have not considered the challenges posed by open-set conditions. In this paper, we define the Neural Codec Source Tracing (NCST) task, which is capable of performing open-set neural codec classification and interpretable ALM detection. Specifically, we constructed the ST-Codecfake dataset for the NCST task, which includes bilingual audio samples generated by 11 state-of-the-art neural codec methods and ALM-based out-ofdistribution (OOD) test samples. Furthermore, we establish a comprehensive source tracing benchmark to assess NCST models in open-set conditions. The experimental results reveal that although the NCST models perform well in in-distribution (ID) classification and OOD detection, they lack robustness in classifying unseen real audio. The ST-codecfake dataset and code are available.

Figures

Figures reproduced from arXiv: 2501.06514 by the authors.

Figure 1
Figure 1. Two abilities of the neural codec source tracing method: (a) Open-set neural codec classification, (b) Explain￾able ALM-based audio detection. real audio as ID real. Secondly, for the fake categoary, current approaches only consider a single novel fake as the OOD fake and lack a discussion on how deepfakes generated by different configurations, sources, and methods impact the performance of source tracing systems. L… view at source ↗
Figure 2
Figure 2. Open-set audio deepfake source tracing benchmark [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 7 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Towards Neural Audio Codec Source Parsing

    eess.AS 2025-06 conditional novelty 7.0 of 10

    NACSP predicts codec generation parameters from deepfake audio, and the proposed HYDRA hyperbolic model beats Euclidean baselines on most benchmark tasks.

  2. Multilingual Source Tracing of Speech Deepfakes: A First Benchmark

    eess.AS 2025-08 conditional novelty 6.0 of 10

    The first multilingual source-tracing benchmark for speech deepfakes, showing LFCC-ECAPA-TDNN generalizes best across languages.

  3. Codec-Based Deepfake Source Tracing via Neural Audio Codec Taxonomy

    cs.SD 2025-05 conditional novelty 6.0 of 10

    Taxonomy-based source tracing can identify codec components of codec-based deepfakes in-domain, but performance collapses on generated speech, and two of three tracing tasks fall below random on unseen-codec data.

  4. Open-Set Source Tracing of Audio Deepfake Systems

    eess.AS 2025-07 conditional novelty 5.0 of 10

    Softmax energy, a modified out-of-distribution score, improves open-set source tracing of audio deepfake systems, achieving a 31% relative FPR95 reduction and best FPR95 of 8.3% with augmentation.

  5. Towards Generalized Source Tracing for Codec-Based Deepfake Speech

    cs.SD 2025-06 conditional novelty 5.0 of 10

    SASTNet, which fuses Whisper semantic features with Wav2Vec2 and AudioMAE acoustic features, improves source tracing for codec-based deepfake speech on CodecFake+, while exposing that prior models overfit to silence.

  6. Towards Explainable Spoofed Speech Attribution and Detection:a Probabilistic Approach for Characterizing Speech Synthesizer Components

    eess.AS 2025-02 conditional novelty 5.0 of 10

    Probabilistic attribute embeddings derived from countermeasure embeddings match raw embedding performance on spoofed speech detection and attack attribution while providing component-level explanations.

  7. Unveiling Audio Deepfake Origins: A Deep Metric learning And Conformer Network Approach With Ensemble Fusion

    cs.SD 2025-06 conditional novelty 4.0 of 10

    An XLSR-Conformer system trained with Real Emphasis, Fake Dispersion, and multi-class N-pair loss reaches 95.6% in-domain and up to 44.8% out-of-domain source tracing accuracy on MLAAD, versus 83.4% and 26.5% for the ...

Reference graph

Works this paper leans on

42 extracted references · 19 canonical work pages · cited by 7 Pith papers

  1. [1]

    Introduction In recent years, there has been rapid advancement in the field of text-to-speech (TTS) and voice conversion (VC), which called deepfake audio. Diverse endeavors and competitions, such as ASVspoof [1, 2] and Audio Deepfake Detection (ADD) chal- lenge [3, 4], have been instituted to promote research aimed at developing deepfake countermeasure (...

    work page arXiv 2023

  2. [2]

    During testing, in addition to ID classifica- tion, one unseen fake audio was evaluate for OOD detection experiments

    Source tracing benchmark Previous source tracing models, as shown on the left side of Figure 2, were trained on real audio and seen fake audio for ID classification. During testing, in addition to ID classifica- tion, one unseen fake audio was evaluate for OOD detection experiments. To comprehensively evaluate the performance of an NCST model, we first pr...

    work page

  3. [3]

    Baseline model This section introduces the baseline models used for the NCST task, including three representative models in the field of ADD: Mel-LCNN, AASIST, and W2V2-AASIST

    Neural codec source tracing method 3.1. Baseline model This section introduces the baseline models used for the NCST task, including three representative models in the field of ADD: Mel-LCNN, AASIST, and W2V2-AASIST. We first con- sider the impact of different features in the NCST scenario. Therefore, we selected handcrafted feature Mel-Spectrogram with L...

    work page

  4. [4]

    ID close-set evaluation After training with ST-codecfake, we conducted evaluations as described in Section 2

    Results and Discussion 4.1. ID close-set evaluation After training with ST-codecfake, we conducted evaluations as described in Section 2. First, we evaluated the NCST model on the ST-Codecfake test set (0-6) for ID closed-set classification using the F1-score. The results are shown in Table 3. It can be observed that in the ID classification experiments, ...

    work page

  5. [5]

    Among these, MSP achieved the best overall per- formance across all conditions, with an AUC of 97.54%, an FPR95 of 9.72%, and an EER of 6.94%

    For OOD method average score (calculated horizontally in Table 5), the logits-based methods, MSP and Energy, outper- formed the feature-based and logits-based NSD, which in turn performed better than the feature-based KNN and Mahalanobis methods. Among these, MSP achieved the best overall per- formance across all conditions, with an AUC of 97.54%, an FPR9...

    work page

  6. [6]

    To this end, we first constructed the ST-codecfake dataset and used it to train three baseline NCST models, which were then tested in ID close-set and OOD open-set conditions

    Conclusion In this paper, we focus on the NCST task and establish a com- prehensive benchmark for evaluating source tracing models. To this end, we first constructed the ST-codecfake dataset and used it to train three baseline NCST models, which were then tested in ID close-set and OOD open-set conditions. The experimen- tal results demonstrate that altho...

    work page

  7. [7]

    Asvspoof 2021: Towards spoofed and deepfake speech detection in the wild,

    X. Liu, X. Wang, M. Sahidullah, J. Patino, H. Delgado, T. Kin- nunen, M. Todisco, J. Yamagishi, N. Evans, A. Nautsch et al. , “Asvspoof 2021: Towards spoofed and deepfake speech detection in the wild,”IEEE/ACM Transactions on Audio, Speech, and Lan- guage Processing, 2023

    work page 2021

  8. [8]

    ASVspoof 5: Crowdsourced speech data, deepfakes, and adversarial attacks at scale,

    X. Wang, H. Delgado, H. Tak, J.-w. Jung, H.-j. Shim, M. Todisco, I. Kukanov, X. Liu, M. Sahidullah, T. Kinnunen, N. Evans, K. A. Lee, and J. Yamagishi, “ASVspoof 5: Crowdsourced speech data, deepfakes, and adversarial attacks at scale,” in ASVspoof Work- shop 2024 (accepted), 2024

    work page 2024

Show all 42 references
  1. [9]

    Add 2022: the first audio deep synthesis detection challenge,

    J. Yi, R. Fu, J. Tao, S. Nie, H. Ma, C. Wang, T. Wang, Z. Tian, Y . Bai, C. Fan et al., “Add 2022: the first audio deep synthesis detection challenge,” in Proceedings of ICASSP . IEEE, 2022, pp. 9216–9220

    2022

  2. [10]

    Add 2023: the second audio deep- fake detection challenge,

    J. Yi, J. Tao, R. Fu, X. Yan, C. Wang, T. Wang, C. Zhang, X. Zhang, Z. Yan, Y . Ren, L. Xu, J. Zhou, H. Gu, Z. Wen, S. Liang, Z. Lian, and H. Li, “Add 2023: the second audio deep- fake detection challenge,”ADD 2023: the Second Audio Deepfake Detection Challenge, accepted by IJ...

    2023

  3. [11]

    Learning A Self- Supervised Domain-Invariant Feature Representation for General- ized Audio Deepfake Detection,

    Y . Xie, H. Cheng, Y . Wang, and L. Ye, “Learning A Self- Supervised Domain-Invariant Feature Representation for General- ized Audio Deepfake Detection,” in Proc. INTERSPEECH 2023, 2023, pp. 2808–2812

    2023

  4. [12]

    Genuine-focused learning using mask au- toencoder for generalized fake audio detection,

    X. Wang, R. Fu, Z. Wen, Z. Wang, Y . Xie, Y . Liu, J. Tao, X. Liu, Y . Li, X. Qi et al. , “Genuine-focused learning using mask au- toencoder for generalized fake audio detection,” arXiv preprint arXiv:2406.03247, 2024

    2024 arXiv

  5. [13]

    Generalized fake audio detection via deep stable learning,

    Z. Wang, R. Fu, Z. Wen, Y . Xie, Y . Liu, X. Wang, X. Liu, Y . Li, J. Tao, Y . Lu et al., “Generalized fake audio detection via deep stable learning,” arXiv preprint arXiv:2406.03237, 2024

    2024 arXiv

  6. [14]

    Generalized source tracing: Detecting novel audio deep- fake algorithm with real emphasis and fake dispersion strategy,

    Y . Xie, R. Fu, Z. Wen, Z. Wang, X. Wang, H. Cheng, L. Ye, and J. Tao, “Generalized source tracing: Detecting novel audio deep- fake algorithm with real emphasis and fake dispersion strategy,” arXiv preprint arXiv:2406.03240, 2024

    2024 arXiv

  7. [15]

    Source trac- ing of audio deepfake systems,

    N. Klein, T. Chen, H. Tak, R. Casal, and E. Khoury, “Source trac- ing of audio deepfake systems,”arXiv preprint arXiv:2407.08016, 2024

    2024 arXiv

  8. [16]

    The codecfake dataset and counter- measures for the universally detection of deepfake audio,

    Y . Xie, Y . Lu, R. Fu, Z. Wen, Z. Wang, J. Tao, X. Qi, X. Wang, Y . Liu, H. Cheng et al. , “The codecfake dataset and counter- measures for the universally detection of deepfake audio,” arXiv preprint arXiv:2405.04880, 2024

    2024 arXiv

  9. [17]

    Codecfake: Enhancing anti-spoofing models against deepfake audios from codec-based speech synthesis systems,

    H. Wu, Y . Tseng, and H.-y. Lee, “Codecfake: Enhancing anti-spoofing models against deepfake audios from codec-based speech synthesis systems,” arXiv preprint arXiv:2406.07237 , 2024

    2024 arXiv

  10. [18]

    Moshi: a speech-text foundation model for real-time dialogue,

    A. D ´efossez, L. Mazar´e, M. Orsini, A. Royer, P. P´erez, H. J´egou, E. Grave, and N. Zeghidour, “Moshi: a speech-text foundation model for real-time dialogue,” arXiv preprint arXiv:2410.00037, 2024

    2024 arXiv

  11. [19]

    Mini-omni: Language models can hear, talk while thinking in streaming,

    Z. Xie and C. Wu, “Mini-omni: Language models can hear, talk while thinking in streaming,” arXiv preprint arXiv:2408.16725 , 2024

    2024 arXiv

  12. [20]

    Snac: Multi- scale neural audio codec,

    H. Siuzdak, F. Gr ¨otschla, and L. A. Lanzend ¨orfer, “Snac: Multi- scale neural audio codec,” arXiv preprint arXiv:2410.14411 , 2024

    2024 arXiv

  13. [21]

    Cstr vctk corpus: English multi-speaker corpus for cstr voice cloning toolkit,

    C. Veaux, J. Yamagishi, and K. MacDonald, “Cstr vctk corpus: English multi-speaker corpus for cstr voice cloning toolkit,” 2017

    2017

  14. [22]

    AISHELL-3: A Multi-Speaker Mandarin TTS Corpus,

    Y . Shi, H. Bu, X. Xu, S. Zhang, and M. Li, “AISHELL-3: A Multi-Speaker Mandarin TTS Corpus,” inProc. Interspeech 2021, 2021, pp. 2756–2760

    2021

  15. [23]

    Asvspoof 2019: spoofing countermeasures for the detection of synthesized, converted and replayed speech,

    A. Nautsch, X. Wang, N. Evans, T. H. Kinnunen, V . Vestman, M. Todisco, H. Delgado, M. Sahidullah, J. Yamagishi, and K. A. Lee, “Asvspoof 2019: spoofing countermeasures for the detection of synthesized, converted and replayed speech,” IEEE Transac- tions on Biometrics, Behavio...

    2019

  16. [24]

    Does Audio Deepfake Detection Generalize?

    N. M ¨uller, P. Czempin, F. Diekmann, A. Froghyar, and K. B ¨ottinger, “Does Audio Deepfake Detection Generalize?” in Proc. Interspeech 2022, 2022, pp. 2783–2787

    2022

  17. [25]

    Generative expressive conversational speech synthesis,

    R. Liu, Y . Hu, Y . Ren, X. Yin, and H. Li, “Generative expressive conversational speech synthesis,” inProceedings of the 32nd ACM International Conference on Multimedia, 2024, pp. 4187–4196

    2024

  18. [26]

    Speechtok- enizer: Unified speech tokenizer for speech language models,

    X. Zhang, D. Zhang, S. Li, Y . Zhou, and X. Qiu, “Speechtok- enizer: Unified speech tokenizer for speech language models,” in The Twelfth International Conference on Learning Representa- tions, 2024

    2024

  19. [27]

    Funcodec: A funda- mental, reproducible and integrable open-source toolkit for neural speech codec,

    Z. Du, S. Zhang, K. Hu, and S. Zheng, “Funcodec: A funda- mental, reproducible and integrable open-source toolkit for neural speech codec,” arXiv preprint arXiv:2309.07405, 2023

    2023 arXiv

  20. [28]

    High fidelity neural audio compression,

    A. D ´efossez, J. Copet, G. Synnaeve, and Y . Adi, “High fidelity neural audio compression,” arXiv preprint arXiv:2210.13438 , 2022

    2022 arXiv

  21. [29]

    Naturalspeech 3: Zero-shot speech syn- thesis with factorized codec and diffusion models,

    Z. Ju, Y . Wang, K. Shen, X. Tan, D. Xin, D. Yang, Y . Liu, Y . Leng, K. Song, S. Tang et al., “Naturalspeech 3: Zero-shot speech syn- thesis with factorized codec and diffusion models,”arXiv preprint arXiv:2403.03100, 2024

    2024 arXiv

  22. [30]

    Wavtokenizer: an efficient acoustic discrete codec tokenizer for audio language modeling,

    S. Ji, Z. Jiang, W. Wang, Y . Chen, M. Fang, J. Zuo, Q. Yang, X. Cheng, Z. Wang, R. Li et al. , “Wavtokenizer: an efficient acoustic discrete codec tokenizer for audio language modeling,” arXiv preprint arXiv:2408.16532, 2024

    2024 arXiv

  23. [31]

    Hifi- codec: Group-residual vector quantization for high fidelity audio codec,

    D. Yang, S. Liu, R. Huang, J. Tian, C. Weng, and Y . Zou, “Hifi- codec: Group-residual vector quantization for high fidelity audio codec,” arXiv preprint arXiv:2305.02765, 2023

    2023 arXiv

  24. [32]

    High-fidelity audio compression with improved rvqgan,

    R. Kumar, P. Seetharaman, A. Luebs, I. Kumar, and K. Kumar, “High-fidelity audio compression with improved rvqgan,” Ad- vances in Neural Information Processing Systems, vol. 36, 2024

    2024

  25. [33]

    Audiodec: An open-source streaming high-fidelity neural audio codec,

    Y .-C. Wu, I. D. Gebru, D. Markovi´c, and A. Richard, “Audiodec: An open-source streaming high-fidelity neural audio codec,” in ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2023, pp. 1–5

    2023

  26. [34]

    Soundstream: An end-to-end neural audio codec,

    N. Zeghidour, A. Luebs, A. Omran, J. Skoglund, and M. Tagliasacchi, “Soundstream: An end-to-end neural audio codec,” IEEE/ACM Transactions on Audio, Speech, and Lan- guage Processing, vol. 30, pp. 495–507, 2021

    2021

  27. [35]

    Speechgpt-gen: Scaling chain-of-information speech genera- tion,

    D. Zhang, X. Zhang, J. Zhan, S. Li, Y . Zhou, and X. Qiu, “Speechgpt-gen: Scaling chain-of-information speech genera- tion,” arXiv preprint arXiv:2401.13527, 2024

    2024 arXiv

  28. [36]

    Neural codec language mod- els are zero-shot text to speech synthesizers,

    C. Wang, S. Chen, Y . Wu, Z. Zhang, L. Zhou, S. Liu, Z. Chen, Y . Liu, H. Wang, J. Li et al. , “Neural codec language mod- els are zero-shot text to speech synthesizers,” arXiv preprint arXiv:2301.02111, 2023

    2023 arXiv

  29. [37]

    Stc antispoofing systems for the asvspoof2019 chal- lenge,

    G. Lavrentyeva, A. Novoselov, S., M. V olkova, A. Gorlanov, and A. Kozlov, “Stc antispoofing systems for the asvspoof2019 chal- lenge,” arXiv preprint arXiv:1904.05576, 2019

    1904 arXiv

  30. [38]

    Aasist: Audio anti-spoofing using in- tegrated spectro-temporal graph attention networks,

    J.-w. Jung, H.-S. Heo, H. Tak, H.-j. Shim, J. S. Chung, B.-J. Lee, H.-J. Yu, and N. Evans, “Aasist: Audio anti-spoofing using in- tegrated spectro-temporal graph attention networks,” in Proceed- ings of the ICASSP, 2022, pp. 6367–6371

    2022

  31. [39]

    A baseline for detecting misclas- sified and out-of-distribution examples in neural networks,

    D. Hendrycks and K. Gimpel, “A baseline for detecting misclas- sified and out-of-distribution examples in neural networks,”arXiv preprint arXiv:1610.02136, 2016

    2016 arXiv

  32. [40]

    Energy-based out-of- distribution detection,

    W. Liu, X. Wang, J. Owens, and Y . Li, “Energy-based out-of- distribution detection,” Advances in neural information process- ing systems, vol. 33, pp. 21 464–21 475, 2020

    2020

  33. [41]

    Out-of-distribution detec- tion with deep nearest neighbors,

    Y . Sun, Y . Ming, X. Zhu, and Y . Li, “Out-of-distribution detec- tion with deep nearest neighbors,” inInternational Conference on Machine Learning. PMLR, 2022, pp. 20 827–20 840

    2022

  34. [42]

    A simple unified framework for detecting out-of-distribution samples and adversarial attacks,

    K. Lee, K. Lee, H. Lee, and J. Shin, “A simple unified framework for detecting out-of-distribution samples and adversarial attacks,” Advances in neural information processing systems, vol. 31, 2018

    2018

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.