Pith. sign in

REVIEW 3 major objections 6 minor 71 references

AI-Generated Song Detection via Lyrics Transcripts

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read AI-generated songs can be caught by reading their lyrics, even when the lyrics come from a shaky audio transcript.

desk verdict Real practical gain in transcribed-lyrics detection, but the data cannot separate lyric content from Whisper artifacts on AI vocals, so the strong claim overreaches. read the letter →

arxiv 2506.18488 v2 pith:ZEV2QMC7 submitted 2025-06-23 cs.SD cs.AIcs.CL

Markus Frohmann , Elena V. Epure , Gabriel Meseguer-Brocal , Markus Schedl , Romain Hennequin This is my paper
classification cs.SDcs.AIcs.CL
keywords AI-generatedmusicdetectionlyricstranscriptionautomaticspeechrecognitionWhisperLLM2Vecsyntheticaudiorobustnesscross-generatorgeneralization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that AI-generated songs can be reliably detected by reading their lyrics, and that this still holds when the lyrics are not available as clean text but have to be transcribed from the audio by a speech recognizer. The authors build a pipeline that turns a song's waveform into a transcript with Whisper large-v2, embeds the transcript with LLM2Vec, and classifies it with a small multi-layer perceptron, reaching 90.7% macro recall on an in-domain multilingual test set. That is within 3.6 points of the 94.3% reached with clean, perfectly formatted lyrics, so the transcription step costs little. The same detector stays above 85% macro recall when the audio is distorted or when the songs come from an unseen music generator, while an audio-artifact CNN falls to near chance on the unseen generator. If this survives further testing, streaming platforms can monitor for fully AI-generated songs from ordinary audio, without waiting for metadata or watermarks.

What carries the argument

The load-bearing object is the lyrics transcript itself, treated as the invariant representation of a song. The machinery is a fixed pipeline: faster-whisper with Whisper large-v2 transcribes the audio into raw text (the paper reports that punctuation cleanup and normalization did not help), LLM2Vec converts that text into a single fixed vector by adapting a Llama-3-8B decoder into a bidirectional encoder, and a two-layer MLP classifies the vector as human-written or AI-generated. The paper's explanation is that the AI-versus-human signal lives in the words, which are nearly unaffected by audio perturbations and by which music generator produced the recording, whereas the artifacts used by audio CNNs are tied to one generator and break under perturbation.

What would settle it

Generate songs with AI vocals singing human-written lyrics, or have a human sing AI-written lyrics, and run the trained detector; if it still separates the classes, the cue is the lyric text, and if it does not, the cue is audio-to-text artifacts of AI vocals.

Watch

Extended reading notes

Core claim

On its own terms, the paper's discovery is that the textual content of lyrics is a usable and transferable cue for detecting fully AI-generated songs, and that the cue survives automatic transcription. The authors create paired data in which every AI-written lyric is sung by a commercial generator (Suno for training, Udio for held-out evaluation) and every human lyric is sung by a human, then show that a frozen text encoder trained on transcripts separates the classes almost as well as it does on clean lyrics. The best combination, Whisper large-v2 plus LLM2Vec on a Llama-3-8B base, reaches 90.7% macro recall in-domain against a 94.3% upper bound with clean lyrics; under audio modifications (stretching, pitch shift, EQ, noise, reverb) it stays above 85%, and on Udio it reaches 85.9% versus 56.9% for the spectrogram CNN. The paper reads this as evidence that lyrics are a stable, generator-invariant channel for detecting fully AI-generated content.

Load-bearing premise

The classifier is reacting to the words themselves, not to systematic transcription errors caused by AI-generated voices, and the dataset never separates those two cues.

Editorial extensions

If this is right

  • AI-generated music detection can be deployed without clean lyric metadata: transcribing the audio is enough, and the cost is under four macro-recall points.
  • Because the signal sits in the text, detectors are robust to common audio edits that break artifact-based detectors, including time stretch, pitch shift, EQ, noise, and reverb.
  • The method transfers to a new music generator: trained on Suno audio, it keeps 85.9% macro recall on Udio songs, where a spectrogram CNN is near chance at 56.9%.
  • Training on lyrics from several text generators matters; leave-one-generator-out experiments show meaningfully higher macro recall when the detector has seen multiple LLMs.
  • The pipeline offers a practical route to copyright monitoring and transparency because it runs on the audio alone and does not depend on signed metadata or watermarking.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the paper is right, the boundary case the authors set aside—AI-generated audio performed on human-written lyrics—is the natural next test; the current dataset never pairs those, so the detector's reliance on lyric content versus transcription artifacts is not yet fully separated.
  • The same frozen text encoder could be reused as new music generators appear, with only the transcription step or a small adapter retrained, which would make detector maintenance cheaper than artifact-based audio models.
  • The reported weak spots (Japanese, and some genre-language pairs) suggest per-language thresholds or lightweight domain adaptation could close most of the remaining gap without changing the architecture.
  • In a production moderation stack, a lyrics-transcript detector would complement an audio-artifact detector: one captures generator-specific traces, the other a generator-invariant textual trace, and their errors are unlikely to overlap.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes a pipeline for detecting AI-generated songs from audio only, by transcribing the audio with Whisper large-v2, embedding the transcripts with frozen text encoders (best: LLM2Vec-LLaMA), and classifying the resulting vector with a small MLP. Using a multilingual, multi-genre lyrics corpus from the authors' prior work, the authors create fake songs by generating Suno v3.5 audio conditioned on AI-generated lyrics, while real songs serve as human examples. The best model reaches 90.7% macro recall in-domain, within 3.6 points of the ground-truth-lyrics upper bound (94.3%), and it remains above 85% under audio attacks and on Udio-generated audio, whereas a spectrogram CNN drops to 56.9% on Udio. The paper concludes that lyrics are a robust and practical cue for detecting fully AI-generated music, with code released.

Significance. If the detector's signal is genuinely the lyric content, this is a practically valuable result: it removes the need for clean lyrics, shows robustness to audio perturbations, and demonstrates cross-generator transfer, with public code. The experimental design is strong in several respects: nine languages, multiple genres, leave-one-generator-out text evaluation, audio attacks, and an out-of-domain audio generator. The main caveat is that the data confound AI-generated lyrics with AI-generated audio, so the mechanism that drives classification is not identified; this limits the central claim until the missing experimental cell (AI audio with human-written lyrics) is tested.

major comments (3)
  1. [Sec. 4.1, Table 4] The training and test data contain only two combinations: human lyrics with human audio and AI-generated lyrics with Suno-generated audio, plus a Udio cell that reuses AI lyrics. Because the classifier sees Whisper transcripts, it cannot be distinguished whether the model uses lyric content or systematic misrecognition of AI-generated vocals. The small gap between the ground-truth-lyrics upper bound (94.3%) and the transcribed model (90.7%) is equally consistent with either signal. A decisive experiment would generate Suno/Udio songs with human-written lyrics from the test set, transcribe them, and report the detector's recall; if performance collapses, the method fails exactly on the hybrid case that Section 1 and the Ethics Statement identify as possible.
  2. [Sec. 5.1, Table 5] The robustness claim in the abstract is scoped to 'different music generators,' but the Udio evaluation uses only AI-generated lyrics, and the audio-attack evaluation also uses only the fake songs from the confounded dataset. The observed stability under attacks and on Udio is therefore compatible with the artifact hypothesis, since Whisper might transcribe synthetic vocals consistently across generators. The authors should either add the missing human-lyrics/AI-audio cell or narrow the abstract and conclusion to 'fully AI-generated songs with AI-written lyrics.'
  3. [Sec. 5.3] The paper reports 0% recall for Italian Jazz and Turkish Folk and 32–35% for three Japanese genres, yet the abstract and conclusion state 'generally strong detection performance across languages and genres.' These zero-recall cells need per-cell sample sizes and a demonstration that the class-imbalance explanation actually holds; otherwise the generalization claim is overstated. At minimum, the conclusion should acknowledge these failure cases explicitly.
minor comments (6)
  1. [Sec. 3] The text says the 'complete lyrics transcript' is used, but the context window is truncated to 512 tokens; please rephrase to avoid the inconsistency.
  2. [Sec. 4.3] Only macro-recall is reported; adding precision, F1, or a confusion matrix would help assess false-positive behavior, especially for the English genres mentioned in Section 5.3.
  3. [Sec. 4.1] The phrase 'we use the original audio' for human-generated songs is vague; please specify the source and any preprocessing applied.
  4. [Sec. 5.1, Table 5] The Stretch column for the CNN (98.1%) is not a drop; the text correctly says 'except time-stretching,' but a table note would make this easier for readers to see.
  5. [Sec. 4.2] The claim that the CNN baseline is 'state-of-the-art' is not substantiated; the cited work [10] is one architecture, so the abstract's 'state-of-the-art audio-based ones' should be qualified accordingly.
  6. [Title] The full-text header contains a typo: 'AI-GENERA TED SONG DETECTION' should be 'AI-GENERATED SONG DETECTION.'

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation: the core robustness and generalization claims are new out-of-sample predictions; the only self-citations are to the authors' prior lyrics-detection work and are not load-bearing.

full rationale

The paper's central chain is: transcribe audio with Whisper, embed transcripts with frozen text encoders, train a small MLP on the [11] dataset labels, then evaluate on held-out languages, on the unseen Udio generator, under audio perturbations, and in leave-one-generator-out settings. None of these predictions is fitted to the test condition: the Udio and perturbation results are obtained by models trained only on Suno-based data, and the leave-one-out text experiments hold out an entire LLM. The GT LYRICS LLM2Vec baseline from the authors' prior work [11] is used as an upper reference, not as a source of fitted parameters for the proposed method, so the small gap between clean and transcribed lyrics is an empirical measurement rather than a construction. The confound that AI lyrics always co-occur with generated audio (and human lyrics with human audio) is a real validity limitation—the classifier could exploit ASR artifacts rather than lyric content—but it is not a circularity: the paper explicitly acknowledges the hybrid case in Section 1 ('AI-generated audio and AI-generated lyrics are not perfectly correlated') and in the Ethics Statement ('our method addresses fully AI-generated content'), and the reported Udio and attack results are genuine out-of-distribution measurements. The self-citations to [9,10,11] motivate the task and supply baselines, but the central robustness evidence is generated by this paper's own experiments, so the derivation is not reduced to its inputs.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The central claim rests on standard pre-trained components and a trained MLP. No new physical entities or fitted constants are introduced. The main unexamined choices are the MLP hyperparameters and the ASR model, which are plausible but not swept. The key domain assumption, that the detector uses lyric content rather than transcription artifacts, is not directly tested.

free parameters (4)
  • MLP hidden sizes = 256, 128
    Chosen without reported sensitivity analysis; the paper states these sizes in Section 3.
  • MLP learning rate = 1e-3
    Set to 1e-3 with decay; standard choice, no ablation reported.
  • Context window truncation = 512 tokens
    Set to 512 tokens for fair comparison across encoders; longer lyrics are truncated, which could affect detection for long songs.
  • ASR model choice = Whisper large-v2
    Selected based on prior robustness literature; no comparison with other Whisper sizes or ASR models.
assumptions (3)
  • domain assumption The [11] lyrics corpus is a valid proxy for real-world human and AI lyrics across nine languages and genres.
    The dataset reused from [11] was created by the same research group and is the only available AI-lyrics corpus; its representativeness is assumed.
  • domain assumption The signal used by the classifier is the lyric content in the transcript, not systematic transcription artifacts of AI vocals.
    The experiments never separate AI lyrics from AI audio, so this assumption is untested and load-bearing.
  • domain assumption Tracks with entirely AI-generated lyrics are likely to be fully AI-generated, making AI-lyrics detection an appropriate proxy for AI-song detection.
    Stated in the Introduction: 'tracks with entirely AI-generated lyrics are likely to be fully AI-generated.' This justifies the task but is not validated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of AI-Generated Song Detection via Lyrics Transcripts." pith.science (2026). https://pith.science/paper/ZEV2QMC7

@misc{pith2026250618488,
  author       = {Pith},
  title        = {Pith review of: AI-Generated Song Detection via Lyrics Transcripts},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZEV2QMC7}},
  note         = {Machine review of arXiv:2506.18488}
}
read the original abstract

The recent rise in capabilities of AI-based music generation tools has created an upheaval in the music industry, necessitating the creation of accurate methods to detect such AI-generated content. This can be done using audio-based detectors; however, it has been shown that they struggle to generalize to unseen generators or when the audio is perturbed. Furthermore, recent work used accurate and cleanly formatted lyrics sourced from a lyrics provider database to detect AI-generated music. However, in practice, such perfect lyrics are not available (only the audio is); this leaves a substantial gap in applicability in real-life use cases. In this work, we instead propose solving this gap by transcribing songs using general automatic speech recognition (ASR) models. We do this using several detectors. The results on diverse, multi-genre, and multi-lingual lyrics show generally strong detection performance across languages and genres, particularly for our best-performing model using Whisper large-v2 and LLM2Vec embeddings. In addition, we show that our method is more robust than state-of-the-art audio-based ones when the audio is perturbed in different ways and when evaluated on different music generators. Our code is available at https://github.com/deezer/robust-AI-lyrics-detection.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

71 extracted references · 53 canonical work pages

  1. [11]

    Simple and control- lable music generation,

    J. Copet, F. Kreuk, I. Gat, T. Remez, D. Kant, G. Syn- naeve, Y . Adi, and A. Défossez, “Simple and control- lable music generation,” in Thirty-seventh Conference on Neural Information Processing Systems , 2023

    work page 2023

  2. [1]

    AI-Generated Song De- tection via Lyrics Transcripts

    INTRODUCTION In recent years, the full generation of musical audio with artificial intelligence systems has matured [1–3] and is now widely deployed in commercial systems such as Suno, Udio, Stable Audio, and Riffusion. The generation of this content presents new challenges for the music industry: Revenue dilution for real artists due to AI-generated trac...

    arXiv 2025

  3. [2]

    Current AI music generation models typically rely on two main components working in sequence

    RELA TED WORK AI-generated music generation. Current AI music generation models typically rely on two main components working in sequence. The first is an autoencoder (AE) trained to compress raw audio into a more manageable representation, which can then be reconstructed into an audio signal. Today, advanced neural audio codecs such as Encodec [13], DAC ...

    work page

  4. [3]

    First, the audio is processed by a transcription model to generate a lyrics tran- script

    METHOD The proposed pipeline to identify directly from au- dio if lyrics are AI-generated or human-written is il- lustrated in Figure 1. First, the audio is processed by a transcription model to generate a lyrics tran- script. Building upon previous research showing their effectiveness with lyrics [41, 42] and robust to audio modifications [43], we use pr...

    work page

  5. [4]

    Then, we briefly describe the baselines and the evaluation metrics used

    EXPERIMENTAL SETUP We present further how we created the dataset starting from the lyrics-only corpus proposed by [11]. Then, we briefly describe the baselines and the evaluation metrics used. 4 As of the time of writing, the detailed adaptation of Gemma in BGE- ML-G EMMA for semantic text similarity has not been fully disclosed 4.1 Data While several dat...

    work page

  6. [5]

    As a reminder, the GT L YRICS LLM2Vec baseline refers to the method proposed by [11], which takes ground-truth lyrics as input and can be considered an upper baseline

    RESULTS We present the main experimental results in Table 4. As a reminder, the GT L YRICS LLM2Vec baseline refers to the method proposed by [11], which takes ground-truth lyrics as input and can be considered an upper baseline. The 5 Such models could also be trained on other input representations, but the findings of [10] are consistent across them. Hen...

    work page

  7. [6]

    To achieve this, we first transcribe the songs, overcoming the reliance on perfect ground-truth lyrics

    CONCLUSION In this work, we proposed a robust and practical method to detect AI-generated music focused on lyrics, but us- ing only audio. To achieve this, we first transcribe the songs, overcoming the reliance on perfect ground-truth lyrics. Features are then extracted using various text en- coders, and a lightweight MLP classifier is trained on top of t...

    work page

  8. [7]

    ETHICS STA TEMENT Although designed for beneficial purposes such as safe- guarding copyright and promoting transparency, the de- velopment and disclosure of AI detection systems pose complex ethical challenges. We acknowledge the dynamic landscape of this field and the need to catch up; as gen- erative models evolve, their outputs may become statisti- cal...

    work page

Show all 71 references
  1. [8]

    55776/COE12, https://doi.org/10.55776/ DFH23, https://doi.org/10.55776/P36413

    ACKNOWLEDGEMENTS This research was funded in whole or in part by the Aus- trian Science Fund (FWF): https://doi.org/10. 55776/COE12, https://doi.org/10.55776/ DFH23, https://doi.org/10.55776/P36413. The authors would like to thank Aurelien Herault, Manuel Moussallam, Yanis Lab...

    doi

  2. [9]

    Detecting music deepfakes is easy but actually hard,

    D. Afchar, G. Meseguer-Brocal, and R. Hennequin, “Detecting music deepfakes is easy but actually hard,” ArXiv, vol. abs/2405.04181, 2024. [Online]. Available: https://api.semanticscholar.org/CorpusID:269614314

    2024 arXiv

  3. [10]

    Musiclm: Generating music from text,

    A. Agostinelli, T. I. Denk, Z. Borsos, J. Engel, M. Verzetti, A. Caillon, Q. Huang, A. Jansen, A. Roberts, M. Tagliasacchi, M. Sharifi, N. Zeghidour, and C. H. Frank, “Musiclm: Generating music from text,” ArXiv, vol. abs/2301.11325, 2023. [Online]. Available: https://api.sema...

    2023 arXiv

  4. [12]

    DiffRhythm: Blazingly fast and embarrassingly sim- ple end-to-end full-length song generation with latent diffusion,

    N. Ziqian, C. Huakang, J. Yuepeng, H. Chunbo, M. Guobin, W. Shuai, Y . Jixun, and X. Lei, “DiffRhythm: Blazingly fast and embarrassingly sim- ple end-to-end full-length song generation with latent diffusion,” arXiv preprint arXiv:2503.01183, 2025

    2025 arXiv

  5. [13]

    AI-generated song charts in Ger- many, amid controversy,

    Joe Sparrow, “AI-generated song charts in Ger- many, amid controversy,” https://musically.com/2024/ 08/13/ai-generated-song-charts-in-germany-amid- controversy/, 2024, [Online; accessed 26-March- 2025]

    2024

  6. [14]

    Record Companies Bring Landmark Cases for Responsible AI Against Suno and Udio in Boston and New York Federal Courts, Respectively,

    RIAA Press statements, “Record Companies Bring Landmark Cases for Responsible AI Against Suno and Udio in Boston and New York Federal Courts, Respectively,” https: //www.riaa.com/record-companies-bring-landmark- cases-for-responsible-ai-againstsuno-and-udio-in- boston-and-new-...

    2024

  7. [15]

    10,000 AI tracks uploaded daily to Deezer, platform reveals, as it files two patents for new AI detection tool,

    Daniel Tencer, “10,000 AI tracks uploaded daily to Deezer, platform reveals, as it files two patents for new AI detection tool,” https://www. musicbusinessworldwide.com/10000-ai-tracks-are- uploaded-daily-to-deezer-platform-reveals-as-it- files-two-patents-for-new-ai-detection...

    2025

  8. [16]

    C2pa specficiations,

    T. C. for Content Provenance and A. (C2PA), “C2pa specficiations,” 2024. [Online]. Available: https: //c2pa.org/specifications/specifications/1.3/index.html

    2024

  9. [17]

    Atsc standard: Audio watermark emission,

    A. T. S. Committee, “Atsc standard: Audio watermark emission,” 2024

    2024

  10. [18]

    Stable audio open,

    Z. Evans, J. Parker, C. Carr, Z. Zukowski, J. Taylor, and J. Pons, “Stable audio open,” ArXiv, vol. abs/2407.14358, 2024. [Online]. Available: https: //api.semanticscholar.org/CorpusID:271310050

    2024 arXiv

  11. [19]

    Ai-generated music detection and its chal- lenges,

    ——, “Ai-generated music detection and its chal- lenges,” ICASSP 2025 - 2025 IEEE International Con- ference on Acoustics, Speech and Signal Processing (ICASSP), 2025

    2025

  12. [20]

    Synthetic lyrics detection across languages and genres,

    Y . Labrak, M. Frohmann, G. Meseguer-Brocal, and E. V . Epure, “Synthetic lyrics detection across languages and genres,” inProceedings of the 5th Work- shop on Trustworthy NLP (TrustNLP 2025) , T. Cao, A. Das, T. Kumarage, Y . Wan, S. Krishna, N. Mehrabi, J. Dhamala, A. Ramakr...

    2025

  13. [21]

    Copyright act of 1976,

    “Copyright act of 1976,” 1976

    1976

  14. [22]

    High fidelity neural audio compression,

    A. Défossez, J. Copet, G. Synnaeve, and Y . Adi, “High fidelity neural audio compression,” ArXiv, vol. abs/2210.13438, 2022. [Online]. Available: https: //api.semanticscholar.org/CorpusID:253097788

    2022 arXiv

  15. [23]

    High-fidelity audio compression with improved rvqgan,

    R. Kumar, P. Seetharaman, A. Luebs, I. Kumar, and K. Kumar, “High-fidelity audio compression with improved rvqgan,” in Advances in Neural Information Processing Systems , vol. 36. Curran Associates, Inc., 2023, pp. 27 980–27 993. [Online]. Available: https://arxiv.org/abs/2306.06546

    2023 arXiv

  16. [24]

    Soundstream: An end-to-end neural audio codec,

    N. Zeghidour, A. Luebs, A. Omran, J. Skoglund, and M. Tagliasacchi, “Soundstream: An end-to-end neural audio codec,” IEEE/ACM Transactions on Audio, Speech, and Language Processing , vol. 30, pp. 495–507, 2021. [Online]. Available: https: //api.semanticscholar.org/CorpusID:236149944

    2021

  17. [25]

    Music2latent: Consistency autoencoders for latent audio compres- sion,

    M. Pasini, S. Lattner, and G. Fazekas, “Music2latent: Consistency autoencoders for latent audio compres- sion,” in Proceedings of the 25th International Society for Music Information Retrieval Conference . IS- MIR, Nov. 2024, pp. 111–119. [Online]. Available: https://doi.org/1...

    2024 doi

  18. [26]

    Fast timing-conditioned latent audio diffusion,

    Z. Evans, C. Carr, J. Taylor, S. H. Hawley, and J. Pons, “Fast timing-conditioned latent audio diffusion,” ArXiv, vol. abs/2402.04825, 2024. [Online]. Available: https://api.semanticscholar.org/CorpusID: 267523339

    2024 arXiv

  19. [27]

    A watermark for large language models,

    J. Kirchenbauer, J. Geiping, Y . Wen, J. Katz, I. Miers, and T. Goldstein, “A watermark for large language models,” in International Conference on Machine Learning. PMLR, 2023, pp. 17 061–17 084

    2023

  20. [28]

    Musi- cldm: Enhancing novelty in text-to-music gen- eration using beat-synchronous mixup strategies,

    K. Chen, Y . Wu, H. Liu, M. Nezhurina, T. Berg-Kirkpatrick, and S. Dubnov, “Musi- cldm: Enhancing novelty in text-to-music gen- eration using beat-synchronous mixup strategies,” ICASSP 2024 - 2024 IEEE International Confer- ence on Acoustics, Speech and Signal Processing (ICAS...

    2024

  21. [29]

    Singfake: Singing voice deepfake detection,

    Y . Zang, Y . Zhang, M. Heydari, and Z. Duan, “Singfake: Singing voice deepfake detection,” in Proc. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2024, pp. 1–5

    2024

  22. [30]

    From real to cloned singer identifica- tion,

    D. Desblancs, G. Meseguer-Brocal, R. Hennequin, and M. Moussallam, “From real to cloned singer identifica- tion,” in Proceedings of the 25th International Society for Music Information Retrieval Conference . ISMIR, Nov. 2024

    2024

  23. [31]

    Sonics: Synthetic or not - identify- ing counterfeit songs,

    M. A. Rahman, Z. I. A. Hakim, N. H. Sarker, B. Paul, and S. A. Fattah, “Sonics: Synthetic or not - identify- ing counterfeit songs,” in International Conference on Learning Representations (ICLR), 2025

    2025

  24. [32]

    Coco: Coherence-enhanced machine-generated text detection under low resource with contrastive learn- ing,

    X. Liu, Z. Zhang, Y . Wang, H. Pu, Y . Lan, and C. Shen, “Coco: Coherence-enhanced machine-generated text detection under low resource with contrastive learn- ing,” in Proceedings of the 2023 Conference on Empir- ical Methods in Natural Language Processing , 2023, pp. 16 167–16 188

    2023

  25. [33]

    Toblend: Token- level blending with an ensemble of llms to at- tack ai-generated text detection,

    F. Huang, H. Kwak, and J. An, “Toblend: Token- level blending with an ensemble of llms to at- tack ai-generated text detection,” arXiv preprint arXiv:2402.11167, 2024

    2024 arXiv

  26. [34]

    Adversarial watermark- ing transformer: Towards tracing text provenance with data hiding,

    S. Abdelnabi and M. Fritz, “Adversarial watermark- ing transformer: Towards tracing text provenance with data hiding,” in2021 IEEE Symposium on Security and Privacy (SP). IEEE, 2021, pp. 121–140

    2021

  27. [35]

    Counter turing test (ct2): Ai-generated text detection is not as easy as you may think-introducing ai detectability index (adi),

    M. Chakraborty, S. T. I. Tonmoy, S. M. Zaman, S. Gau- tam, T. Kumar, K. Sharma, N. Barman, C. Gupta, V . Jain, A. Chadha et al. , “Counter turing test (ct2): Ai-generated text detection is not as easy as you may think-introducing ai detectability index (adi),” in Pro- ceedings...

    2023

  28. [36]

    Detectllm: Leveraging log rank information for zero-shot detec- tion of machine-generated text,

    J. Su, T. Zhuo, D. Wang, and P. Nakov, “Detectllm: Leveraging log rank information for zero-shot detec- tion of machine-generated text,” in Findings of the Association for Computational Linguistics: EMNLP 2023, 2023, pp. 12 395–12 412

    2023

  29. [37]

    MAGE: Machine- generated text detection in the wild,

    Y . Li, Q. Li, L. Cui, W. Bi, Z. Wang, L. Wang, L. Yang, S. Shi, and Y . Zhang, “MAGE: Machine- generated text detection in the wild,” in Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (V olume 1: Long Papers) , L.-W. Ku, A. Martins, an...

    2024

  30. [38]

    Multitude: Large-scale multilingual machine- generated text detection benchmark,

    D. Macko, R. Moro, A. Uchendu, J. S. Lucas, M. Ya- mashita, M. Pikuliak, I. Srba, T. Le, D. Lee, J. Simko et al. , “Multitude: Large-scale multilingual machine- generated text detection benchmark,” in 2023 Confer- ence on Empirical Methods in Natural Language Pro- cessing, EMN...

    2023

  31. [39]

    From text to source: Results in detecting large language model- generated content,

    W. Antoun, D. Seddah, and B. Sagot, “From text to source: Results in detecting large language model- generated content,” in The 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024) , 2024

    2024

  32. [40]

    Token prediction as implicit classification to identify llm-generated text,

    Y . Chen, H. Kang, V . Zhai, L. Li, R. Singh, and B. Raj, “Token prediction as implicit classification to identify llm-generated text,” inProceedings of the 2023 Confer- ence on Empirical Methods in Natural Language Pro- cessing, 2023, pp. 13 112–13 120

    2023

  33. [41]

    How reliable are ai-generated-text detectors? an assessment framework using evasive soft prompts,

    T. S. Kumarage, P. Sheth, R. Moraffah, J. Garland et al., “How reliable are ai-generated-text detectors? an assessment framework using evasive soft prompts,” in The 2023 Conference on Empirical Methods in Natural Language Processing, 2023

    2023

  34. [42]

    We then sample 260 real songs to maintain balance across lan- Model en de tr fr pt es it ar ja Macro A vg

    We leave the other settings at their default value. We then sample 260 real songs to maintain balance across lan- Model en de tr fr pt es it ar ja Macro A vg. BASELINES GT LYRICS LLM2Vec † 91.3 97.4 95.3 99.4 97.5 95.7 94.3 91.5 85.9 94.3 CNN Spectrogram ‡ 97.5 96.3 97.5 98.7 ...

  35. [43]

    Authorship attribution for neural text generation,

    A. Uchendu, T. Le, K. Shu, and D. Lee, “Authorship attribution for neural text generation,” in Proceedings of the 2020 conference on empirical methods in natural language processing (EMNLP), 2020, pp. 8384–8395

    2020

  36. [44]

    Real or fake? learning to discriminate machine from human generated text,

    A. Bakhtin, S. Gross, M. Ott, Y . Deng, M. Ranzato, and A. Szlam, “Real or fake? learning to discriminate machine from human generated text,” arXiv preprint arXiv:1906.03351, 2019

    1906 arXiv

  37. [45]

    Detectgpt: Zero-shot machine-generated text detection using probability curvature,

    E. Mitchell, Y . Lee, A. Khazatsky, C. D. Manning, and C. Finn, “Detectgpt: Zero-shot machine-generated text detection using probability curvature,” inInternational Conference on Machine Learning , 2023. [Online]. Available: https://api.semanticscholar.org/CorpusID: 256274849

    2023

  38. [46]

    Beat llms at their own game: Zero-shot llm-generated text detection via querying chatgpt,

    B. Zhu, L. Yuan, G. Cui, Y . Chen, C. Fu, B. He, Y . Deng, Z. Liu, M. Sun, and M. Gu, “Beat llms at their own game: Zero-shot llm-generated text detection via querying chatgpt,” in Proceedings of the 2023 Confer- ence on Empirical Methods in Natural Language Pro- cessing, 2023...

    2023

  39. [47]

    Can ai-generated text be re- liably detected?

    V . S. Sadasivan, A. Kumar, S. Balasubramanian, W. Wang, and S. Feizi, “Can ai-generated text be re- liably detected?” arXiv preprint arXiv:2303.11156 , 2023

    2023 arXiv

  40. [48]

    Gpt-4 technical report,

    J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Alt- man, S. Anadkat et al., “Gpt-4 technical report,” arXiv preprint arXiv:2303.08774, 2023

    2023 arXiv

  41. [49]

    RAID: A shared benchmark for robust evaluation of machine-generated text detectors,

    L. Dugan, A. Hwang, F. Trhlík, A. Zhu, J. M. Ludan, H. Xu, D. Ippolito, and C. Callison-Burch, “RAID: A shared benchmark for robust evaluation of machine-generated text detectors,” in Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (V ol...

    2024

  42. [50]

    Lyricwhiz: Robust multilingual zero-shot lyrics transcription by whispering to chatgpt,

    L. Zhuo, R. Yuan, J. Pan, Y . Ma, Y . Li, G. Zhang, S. Liu, R. B. Dannenberg, J. Fu, C. Lin, E. Benetos, W. Chen, W. Xue, and Y .-T. Guo, “Lyricwhiz: Robust multilingual zero-shot lyrics transcription by whispering to chatgpt,” ArXiv, vol. abs/2306.17103,

    arXiv

  43. [51]

    BERT: Pre-training of deep bidirectional transformers for language understanding,

    J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologi...

    2019

  44. [52]

    Lyrics transcription for humans: A readability-aware benchmark,

    O. Cífka, H. Schreiber, L. Miner, and F.-R. Stöter, “Lyrics transcription for humans: A readability-aware benchmark,” arXiv preprint arXiv:2408.06370, 2024

    2024 arXiv

  45. [53]

    Benchmark- ing whisper under diverse audio transformations and real-time constraints,

    S. Katkov, A. Liotta, and A. Vietti, “Benchmark- ing whisper under diverse audio transformations and real-time constraints,” in International Conference on Speech and Computer. Springer, 2024, pp. 82–91

    2024

  46. [54]

    Robust speech recognition via large-scale weak supervision,

    A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust speech recognition via large-scale weak supervision,” ArXiv, vol. abs/2212.04356, 2022. [Online]. Available: https: //api.semanticscholar.org/CorpusID:252923993

    2022 arXiv

  47. [55]

    faster- whisper: A reimplementation of openai’s whisper model using ctranslate2,

    G. Klein, J. W. Kim, Y . Kim, and C. Delangue, “faster- whisper: A reimplementation of openai’s whisper model using ctranslate2,” nov 2023. [Online]. Avail- able: https://github.com/SYSTRAN/faster-whisper

    2023

  48. [56]

    Sentence-BERT: Sentence embeddings using Siamese BERT-networks,

    N. Reimers and I. Gurevych, “Sentence-BERT: Sentence embeddings using Siamese BERT-networks,” in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) , K....

    2019

  49. [57]

    SimCSE: Simple contrastive learning of sentence embeddings,

    T. Gao, X. Yao, and D. Chen, “SimCSE: Simple contrastive learning of sentence embeddings,” in Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing , M.-F. Moens, X. Huang, L. Specia, and S. W.-t. Yih, Eds. Online and Punta Cana, Dominican Repu...

    2021

  50. [58]

    M3-embedding: Multi-linguality, multi-functionality, multi-granularity text embeddings through self- knowledge distillation,

    J. Chen, S. Xiao, P. Zhang, K. Luo, D. Lian, and Z. Liu, “M3-embedding: Multi-linguality, multi-functionality, multi-granularity text embeddings through self- knowledge distillation,” in Findings of the Association for Computational Linguistics: ACL 2024 , L.-W. Ku, A. Martins...

    2024

  51. [59]

    Llm2vec: Large language models are secretly powerful text encoders,

    P. BehnamGhader, V . Adlakha, M. Mosbach, D. Bahdanau, N. Chapados, and S. Reddy, “Llm2vec: Large language models are secretly powerful text encoders,” 2024. [Online]. Available: https://arxiv.org/abs/2404.05961

    2024 arXiv

  52. [60]

    Learning universal authorship representations,

    R. A. Rivera-Soto, O. E. Miano, J. Ordonez, B. Y . Chen, A. Khan, M. Bishop, and N. Andrews, “Learning universal authorship representations,” in Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing , M.-F. Moens, X. Huang, L. Specia, and S. W....

    2021

  53. [63]

    Mpnet: Masked and permuted pre-training for language under- standing,

    K. Song, X. Tan, T. Qin, J. Lu, and T.-Y . Liu, “Mpnet: Masked and permuted pre-training for language under- standing,” Advances in neural information processing systems, vol. 33, pp. 16 857–16 867, 2020

    2020

  54. [64]

    Minilm: Deep self-attention distillation for task-agnostic compression of pre-trained transform- ers,

    W. Wang, F. Wei, L. Dong, H. Bao, N. Yang, and M. Zhou, “Minilm: Deep self-attention distillation for task-agnostic compression of pre-trained transform- ers,” Advances in neural information processing sys- tems, vol. 33, pp. 5776–5788, 2020

    2020

  55. [65]

    Unsupervised cross-lingual representation learning at scale,

    A. Conneau, K. Khandelwal, N. Goyal, V . Chaud- hary, G. Wenzek, F. Guzmán, E. Grave, M. Ott, L. Zettlemoyer, and V . Stoyanov, “Unsupervised cross-lingual representation learning at scale,” in Proceedings of the 58th Annual Meeting of the Association for Computational Linguis...

    2020

  56. [66]

    Gemma 2: Improving open language models at a practical size,

    G. Team, M. Riviere, S. Pathak, P. G. Sessa, C. Hardin, S. Bhupatiraju, L. Hussenot, T. Mesnard, B. Shahri- ari, A. Ramé et al. , “Gemma 2: Improving open language models at a practical size,” arXiv preprint arXiv:2408.00118, 2024

    2024 arXiv

  57. [67]

    Few-shot detection of machine-generated text using style representations,

    R. A. R. Soto, K. Koch, A. Khan, B. Y . Chen, M. Bishop, and N. Andrews, “Few-shot detection of machine-generated text using style representations,” in The Twelfth International Conference on Learning Representations, 2024

    2024

  58. [68]

    Decoupled weight decay regularization,

    I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” in International Conference on Learning Representations , 2017. [Online]. Available: https://api.semanticscholar.org/CorpusID:53592270

    2017

  59. [69]

    PyTorch Lightning,

    W. Falcon and The PyTorch Lightning team, “PyTorch Lightning,” Mar. 2019. [Online]. Available: https: //github.com/Lightning-AI/lightning

    2019

  60. [70]

    SemEval-2013 task 2: Sentiment analysis in Twitter,

    P. Nakov, S. Rosenthal, Z. Kozareva, V . Stoyanov, A. Ritter, and T. Wilson, “SemEval-2013 task 2: Sentiment analysis in Twitter,” in Second Joint Conference on Lexical and Computational Semantics (*SEM), V olume 2: Proceedings of the Seventh International Workshop on Semantic...

    2013

  61. [320]

    Available: https://aclanthology.org/S13- 2052

    [Online]. Available: https://aclanthology.org/S13- 2052

  62. [2023]

    Available: https://api.semanticscholar

    [Online]. Available: https://api.semanticscholar. org/CorpusID:259287024

  63. [4186]

    Available: https://aclanthology.org/ N19-1423

    [Online]. Available: https://aclanthology.org/ N19-1423

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.