{"id":"32bac1a7-f59b-40e6-a156-b4945e3bf615","arxiv_id":"2501.10111","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A simple convolutional classifier detects autoencoder-generated music with 99.8% accuracy, but accuracy collapses under audio manipulation and on unseen generator families.","lead":"Researchers at Deezer trained a simple audio classifier that tells real music apart from AI-generated music with 99.8% accuracy in a controlled test. The same detector fails badly under pitch shifts, added noise, re-encoding, and on codec families it has not seen, a warning for commercial AI-content checkers.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The MusicGen transfer test does not establish AI-music detection: MusicGen shares the Encodec decoder used in training, so the 99.9% result may simply reflect Encodec-fingerprint detection; only a non-Encodec full-generator test can settle the central claim.","rationale":"I agree with the reader's weakest assumption. The paper is transparent about its limitations and the negative results are valuable; but the abstract's claim of 'a AI-music detector' and the 99.8%/99.9% numbers are repeatedly cited as the main finding. The single most load-bearing gap is that the only end-to-end generator tested uses the same Encodec decoder as the training reconstructions. A falsifiable test with a non-Encodec full generator would either support or refute the central claim. Therefore the verdict remains CONDITIONAL pending that experiment; if the test fails, the paper should be reframed as a study of decoder fingerprinting and its challenges, not AI-music detection.","tokens_in":8110,"tokens_out":4611,"duration_ms":48068,"concrete_test":"Download or generate 50 tracks (approx. 25 min) from VampNet (DAC decoder) and, if accessible, MusicLM (SoundStream decoder), plus 50 tracks from MusicGen as a positive control. Use the published amplitude-spectrogram model and the same evaluation protocol as Section II.D (2500 random 0.8s snippets, report per-class accuracy). Also evaluate on real FMA tracks that have been re-encoded with Encodec 24kbps but not generated. If accuracy on VampNet/MusicLM is near chance while Encodec re-encoded real audio is flagged synthetic at high rate, the central claim fails: the model detects the Encodec decoder, not AI-generated music. If accuracy is high on the non-Encodec generators, the proxy assumption receives genuine support.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that a classifier trained on real audio versus matched autoencoder reconstructions detects AI-generated music (Section II.A). The only full-generator validation (Section II.D) uses MusicGen, whose audio decoder is Encodec—a codec whose reconstructions dominate the training set. Consequently, the 99.9% transfer is the expected outcome for a model that has learned Encodec-specific artefacts; it provides no evidence about detection of generation itself. The paper's own Table III shows that models trained on one decoder family achieve ~0% accuracy on other families (e.g., Encodec→DAC, GriffinMel→Encodec), and Section III.A shows scores dropping to near zero under manipulations with the model defaulting to 'real'. Thus the detector is best described as an Encodec fingerprint detector, not a general AI-music detector. The framework's proxy assumption—that AE reconstruction artefacts equal AI-generation artefacts—is therefore the load-bearing premise, and it is only validated within the Encodec family. Without at least one full text-to-music generator built on a different decoder (e.g., VampNet/DAC, MusicLM/SoundStream, Jukebox/VQ-VAE), the headline accuracy cannot be interpreted as AI-music detection. The missing Table II values and absence of confidence intervals compound the issue, but the confound is the core concern.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes detecting AI-generated music by training simple CNNs to distinguish real audio from autoencoder reconstructions of the same tracks, using nine reconstructions from Encodec, DAC, GriffinMel, and Musika applied to the FMA dataset. The amplitude-spectrogram model reaches 99.8% held-out accuracy, and the authors report 99.9% accuracy on 50 MusicGen tracks. The second half documents that accuracy collapses under common audio manipulations and that models trained on one decoder family do not generalize to other families, which the authors present as caveats for deployed detectors. The paper is framed both as a first AI-music detector and as a position paper on the challenges of such detectors.","tokens_in":8397,"tokens_out":3499,"duration_ms":37091,"significance":"If the core result stands, it is a valuable proof-of-concept that known decoder families leave detectable artefacts, and the paper's self-reported negative results—especially the cross-family collapse in Table III and the \"default to real\" behavior—are useful cautionary evidence for the emerging field of AI-music detection. The authors also make code and trained weights available, which supports reproducibility. However, the central generalization claim from autoencoder reconstructions to real AI-generated music is only validated within the Encodec family, since MusicGen uses Encodec as its decoder; the advertised \"AI-music detector\" is therefore broader than the evidence supports. The missing numerical content of Table II and the absence of confidence intervals further weaken the robustness discussion.","major_comments":[{"comment":"The 99.9% MusicGen transfer result does not establish detection of AI-generated music in general, because MusicGen uses Encodec as its audio decoder, as the authors themselves note in Section II.B. Since Encodec reconstructions are in the training set, the MusicGen test is an in-family generalization test, not a test of detection across generator families. Table III makes this concern concrete: an Encodec-trained model scores 0% on DAC and GriffinMel. The claim in the abstract and Section II.D should be narrowed to \"detection of autoencoder artefacts\" and \"transfer to an Encodec-based generator,\" or the paper should add a full-generator evaluation using a non-Encodec decoder (e.g., VampNet/DAC, MusicLM/SoundStream, or Jukebox/VQ-VAE).","section":"II.D and II.B"},{"comment":"Table II, which supports the robustness claims in Section III.A, contains no readable numeric values in the manuscript: only the caption and a table frame appear, with the actual accuracy numbers missing. The claims about pitch-shift, noise, and codec-reencoding performance collapsing to near zero cannot be checked without those values. Please provide the complete table, including per-class breakdowns and confidence intervals or per-run variability.","section":"Table II"},{"comment":"The statement that \"fine-tuning on these manipulations could reliably restore high accuracy scores\" and \"the same is true about fine-tuning to a new decoder\" is not supported by any experiment shown in the paper. Since this is offered as a mitigation direction, either include the corresponding experiments with numbers, or explicitly label it as an unreported preliminary observation.","section":"Section III.C"},{"comment":"No confidence intervals or statistical significance tests are reported for the accuracies in Table I. With a test split of 20% of 25,000 tracks, the differences among 99.8%, 99.7%, and 99.6% may be within noise; reporting standard errors or confidence intervals would clarify whether the amplitude representation is genuinely better than phase or polar representations.","section":"Section II.C and Table I"}],"minor_comments":[{"comment":"\"a AI-music detector\" should be \"an AI-music detector.\"","section":"Abstract"},{"comment":"The name \"Mo ˆusai\" appears with corrupted formatting; it should be rendered as \"Mousai\" or \"Moûsai\" consistently.","section":"Section II.B"},{"comment":"\"sampled with a 1 2 probability\" should read \"sampled with probability 1/2.\"","section":"Section II.C"},{"comment":"The MusicGen test uses only 50 tracks (25 minutes of audio). While the snippet count is 2500, these snippets are not independent because they come from only 50 generated tracks; a track-level analysis or a larger corpus would strengthen the claim.","section":"Section II.D"},{"comment":"The table is difficult to parse in text form; a heatmap or a matrix with explicit row/column labels would improve readability.","section":"Table III"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This is the first paper I know of that tackles instrumental-music deepfake detection in a controlled way. The setup is sensible: take real FMA tracks, autoencode them with nine decoder configurations, and train a simple CNN to tell original from reconstruction. The 99.8% number checks out internally, and the cross-decoder transfer matrix (Table III) is the most informative thing in the paper. It shows intra-family generalization (Encodec 24kbps to 3kbps works) and inter-family collapse (everything goes to zero). The authors deserve credit for publishing that—most papers would bury it.\n\nThe paper is also refreshingly self-critical. They explicitly say the model is probably detecting decoder artefacts, that the real class acts as a default, and that fine-tuning will always be chasing an unseen generator. That framing is more honest than the abstract, which claims \"the first AI-music detector\" and implies general detection.\n\nNow the soft spots. The stress-test concern about MusicGen is valid: MusicGen uses Encodec, which dominates the training set, so the 99.9% transfer is the expected result if the model has learned Encodec-specific fingerprints. It does not validate the proxy assumption for non-Encodec generators. The authors list VampNet, MusicLM, and Jukebox as motivating models but test none of them. One non-Encodec full generator would have settled the question. Related: Table II is rendered without any numbers in the arXiv PDF, so the robustness results cannot actually be read. That is an editorial fix, but a referee should insist on it. There are no confidence intervals either, though with 2500 snippets the CIs would be tight except where accuracy is near zero or one.\n\nThe bigger conceptual issue is that the central claim of AI-music detection is only supported within the Encodec family. The paper's own evidence shows that a detector trained on Encodec does not detect DAC, GriffinMel, or Musika outputs. So the title and abstract overstate. That said, the authors already say most of this in Sections III.A and III.B—the problem is mostly presentation.\n\nWho is this for? Anyone building or evaluating music deepfake detectors, and anyone studying shortcut learning in audio classification. It is a proof of concept with strong negative results, not a deployable tool. I would send it to a serious referee, and I would ask the authors to add one cross-family generator test, fix Table II, and temper the abstract. I would cite it for the transfer matrix and the confound-controlled benchmark.\n\nMy verdict: conditional accept, with the Table II repair and a non-Encodec generalization test as conditions. The core honesty of the paper makes it a useful contribution even if the headline claim does not fully survive.","headline":"An honest, useful first study whose real contributions are the negative results, not the 99.8% headline; the MusicGen transfer test is confounded by the shared Encodec decoder.","tokens_in":8899,"tokens_out":2711,"would_cite":true,"duration_ms":26310,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A simple convolutional classifier trained on real tracks versus their autoencoder reconstructions detects AI-made music at 99.8% accuracy, but the detector collapses under pitch shifts, added noise, and re-encoding, and only generalizes…","keywords":["AI-generated music detection","deepfake audio forensics","neural audio codecs","autoencoder fingerprinting","text-to-music generation","robustness to audio manipulation","detector generalization","convolutional neural network"],"falsifier":"Train the released detector on real audio versus reconstructions from only GriffinMel and DAC (neither is Encodec), then evaluate it on full MusicGen generations; if accuracy falls to near chance, the Section II.D 99.9% score came from codec familiarity rather than from a general generation artefact, and the central claim is autoencoder fingerprinting, not AI-music detection.","tokens_in":7927,"feed_emoji":"🎧","tokens_out":11148,"duration_ms":95411,"temperature":0.7,"pith_summary":"AI-generated music can be detected with surprising ease: a basic convolutional network that compares a real track with its own reconstruction through a generator's autoencoder reaches 99.8% accuracy on amplitude spectrograms. The paper's second, cautionary finding is that this ability is narrow. The same detector scores near zero when audio is pitch-shifted, mixed with white noise, or re-encoded, and a model trained on one autoencoder family fails almost completely on another. The authors present this as the first published AI-music detector while arguing that a high test score does not make a deployable tool: performance must be measured against manipulations, unseen generators, calibration, and the risk of a model that simply defaults to 'real' whenever it does not find a familiar artefact.","feed_headline":"AI-music detector scores 99.8%, then near zero after tweaks","feed_subtitle":"Same CNN separates real tracks from autoencoder reconstructions but breaks on pitch shifts and new codecs.","key_machinery":"The carrying object is the decoder fingerprint: the small reconstruction artefacts left by the autoencoder stage of waveform-based music generators, such as the checkerboard patterns associated with transposed convolutions. The protocol controls for confounds by taking a real track and passing it through the generator's autoencoder at the same bitrate, so the only difference between the two examples is the decoder's artefact. A six-layer convolutional network with 16 to 512 filters, trained on random 0.8-second amplitude-spectrogram snippets with a one-in-two real-versus-synthetic sampling ratio, is enough to exploit this difference. The same mechanism explains the failures: each autoencoder family leaves a different fingerprint, and the model's bias toward simple solutions turns the real class into a default category rather than a learned manifold of authentic music.","core_discovery":"On the paper's own terms, the central discovery is that AI-music detection can be reduced to autoencoder fingerprinting. By autoencoding 25,000 real tracks from the Free Music Archive (FMA) with nine configurations of four decoders (the neural codecs Encodec and DAC, the GriffinMel mel-spectrogram pipeline, and the Musika decoder) and training a six-layer CNN on 0.8-second amplitude-spectrogram snippets, the authors obtain 99.8% test accuracy and 99.9% accuracy on 2,500 snippets from the full text-to-music generator MusicGen. The catch documented in the same paper is that the fingerprint is not a universal marker of machine generation: detection transfers within the Encodec family (24 kbps training covers 6 and 3 kbps) but is near zero across families, such as GriffinMel to DAC, and common manipulations like a two-semitone pitch shift and 64 kbps re-encoding drop scores to near zero because the model predicts 'real' when it encounters no known artefact. The authors' claim is therefore twofold: forged music is easy to detect in a matched setting, and that ease is precisely why the detector should not be trusted outside it.","pith_inferences":["Our inference: the matched-reconstruction protocol could become a standard evaluation scheme in which every reported accuracy is paired with a cross-family score, so a headline number like 99.8% is never quoted without its generalization envelope.","Our inference: because inter-family scores are near zero, combining many codec families in one training set may or may not yield a detector that learns a common neural-codec artefact; the paper does not test this, and it is the natural next experiment.","Our inference: the near-zero performance under pitch shift and re-encoding suggests that resistance to manipulation and to new generators will be bought with data augmentation and continual fine-tuning, and a practical detector should be evaluated under a partial-knowledge protocol that simulates an attacker who knows the detector.","Our inference: the paper's framing implies that content provenance measures such as watermarking and platform-level regulation may prove more scalable than chasing every new generator with a detector, even though the paper notes that watermarking is itself brittle."],"forward_implications":["A detector trained on one bitrate of a neural codec family transfers to other bitrates of the same family, so not every codec configuration needs to be seen during training.","Common user-level manipulations, including pitch shifting by two semitones, adding white noise, or re-encoding to mp3, AAC, or Opus at 64 kbps, can reduce accuracy to near zero, making a naively deployed detector unsafe.","Fine-tuning on manipulations and on new decoders can restore high accuracy, but since unseen manipulations and generators will keep appearing, detection needs a continual patching process rather than a one-time model.","The model's tendency to default to the real class means its output probabilities should not be interpreted as a percentage of AI content, which matters for calibration and for mixed real-and-synthetic stems.","The high score on MusicGen supports autoencoder fingerprinting rather than proof of generalization, because MusicGen shares the Encodec family seen during training."],"supporting_citations":[{"why":"Supplies the 25,000 real FMA tracks that are autoencoded to build the detection dataset.","marker":"[25]"},{"why":"Defines the Encodec codec used as one training decoder family and as the decoder inside MusicGen.","marker":"[21]"},{"why":"Defines the DAC codec (LAC variant) used as a second training decoder family and in the cross-family tests.","marker":"[22]"},{"why":"Provides the MusicGen text-to-music generator used for the 99.9% generalization check.","marker":"[10]"},{"why":"Documents the checkerboard artefacts of transposed-convolution decoders that motivate detecting the autoencoder fingerprint.","marker":"[16]"},{"why":"Justifies the matched-reconstruction protocol as a way to control for confounding by musical content and file encoding.","marker":"[24]"},{"why":"Supplies prior evidence that CNN-based forgery detectors are not robust to out-of-distribution shifts, framing the paper's caveats.","marker":"[29]"}],"fun_headline_variants":["99.8% AI-music detection, then near zero after tweaks","99.8% AI-music detection, zero after pitch shift","AI music detector: 99.8% accurate, but fragile to edits","Detecting AI music: easy to train, easy to trick"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that running a real track through a generator's autoencoder leaves exactly the same artefacts as that generator's actual output, so a classifier trained on these reconstructions is a valid detector of real AI-generated music; the paper checks this against only one full generator, MusicGen, which shares Encodec with the training data.","fun_headline_variants_meta":{"raw":{"variants":["99.8% AI-music detection, then near zero after tweaks","99.8% AI-music detection, zero after pitch shift","AI music detector: 99.8% accurate, but fragile to edits","Detecting AI music: easy to train, easy to trick"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000798,"raw_usage":{"total_tokens":3524,"prompt_tokens":970,"completion_tokens":2554,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":586,"completion_tokens_details":{"reasoning_tokens":2476}},"tokens_in":586,"tokens_out":2554,"duration_ms":20348,"temperature":1.0,"reasoning_tokens":2476,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T19:23:12.752064+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the released detector on real audio versus reconstructions from only GriffinMel and DAC (neither is Encodec), then evaluate it on full MusicGen generations; if accuracy falls to near chance, the Section II.D 99.9% score came from codec familiarity rather than from a general generation artefact, and the central claim is autoencoder fingerprinting, not AI-music detection.","supporting_citations":[{"cited_title":"FMA: A dataset for music analysis,","cited_arxiv_id":null,"evidence_quote":"Supplies the 25,000 real FMA tracks that are autoencoded to build the detection dataset."},{"cited_title":"High-fidelity audio compression with improved rvqgan,","cited_arxiv_id":null,"evidence_quote":"Defines the DAC codec (LAC variant) used as a second training decoder family and in the cross-family tests."},{"cited_title":"Simple and controllable music generation,","cited_arxiv_id":null,"evidence_quote":"Provides the MusicGen text-to-music generator used for the 99.9% generalization check."},{"cited_title":"Melgan: Generative adversarial networks for conditional waveform synthesis,","cited_arxiv_id":null,"evidence_quote":"Documents the checkerboard artefacts of transposed-convolution decoders that motivate detecting the autoencoder fingerprint."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Justifies the matched-reconstruction protocol as a way to control for confounding by musical content and file encoding."},{"cited_title":"Cnn-generated images are surprisingly easy to spot... for now,","cited_arxiv_id":null,"evidence_quote":"Supplies prior evidence that CNN-based forgery detectors are not robust to out-of-distribution shifts, framing the paper's caveats."}],"review_version":1}