REVIEW 2 major objections 6 minor 34 references
Hidden Echoes Survive Training in Audio To Audio Generative Instrument Models
T0 review · 2 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read Embedding a faint echo into every clip of a training set makes DDSP, RAVE, and Dance Diffusion reproduce that echo in their synthesized audio, so a watermark as old as echo hiding can tag what a generative model was trained on.
desk verdict Useful empirical result, but the detection statistic as defined cannot detect the peaks it reports; fix Eq. (5) and this deserves full review. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The workhorse is the echo-hiding transform, a one-tap feedforward filter $\hat{x}[n] = x[n] + \alpha x[n-\delta]$ whose short delay (tens of samples at 44.1 kHz) is masked by temporal aliasing of human hearing, so the watermark is imperceptible at moderate $\alpha$. Detection runs through the cepstrum $c = \mathrm{ifft}(\log(|\mathrm{fft}(x_w)|))$: a time-domain echo is a convolution, so it separates additively in the log-magnitude spectrum and shows up as a peak at lag $\delta$. The paper's evaluation metric is a z-score over cepstral lags 25 to 125 (Eq. 5) on model outputs, comparing models trained on watermarked data against clean-trained baselines over 100 test stems per instrument from MUSDB18-HQ. For the high-capacity variant, the machinery is the time-spread echo $\hat{x} = x \ast \alpha p_\delta$ with a pseudorandom bit pattern mapped to $\pm 1$, and detection cross-correlates the output cepstrum with the pattern to obtain $c^\ast$, then z-scores that correlation. The three architectures — DDSP at about 5 million parameters, RAVE at about 32 million, and Dance Diffusion at about 222 million — are chosen to span very different inductive biases, so a positive result across all three is what supports the generality of the tag.
What would settle it
Run the released code and model outputs and test the detection claim directly: for each synthesized clip, check whether the cepstrum has a genuine local maximum at the embedded lag ($c[\delta] > c[j]$ for every $j$ in the band from 25 to 125 samples), comparing models trained on watermarked data with models trained on clean data. If a corrected peak-based metric no longer separates the two distributions the way Figures 3–6 report, the claim that echoes survive training fails; if the separation survives, the printed z-score formula (Eq. 5) is a typo and the underlying claim stands.
Extended reading notes
Core claim
The central claim is that a single echo embedded in every training clip acts as a persistent, detectable tag on a trained audio-to-audio model. Concretely, the authors take a training corpus and form $\hat{x}[n] = x[n] + \alpha x[n-\delta]$ with $\alpha = 0.4$ and $\delta$ between 50 and 100 samples at a 44.1 kHz rate; they train DDSP, RAVE, or Dance Diffusion on the watermarked corpus; and they then feed unseen audio into the model and compute the cepstrum $c = \mathrm{ifft}(\log(|\mathrm{fft}(x_w)|))$ of the output. The synthesized audio shows a cepstral peak at the embedded delay $\delta$ that a clean-trained model does not produce, across three corpora (Groove drums, VocalSet singing, GuitarSet guitar). The paper reports this for all three architectures, with the echo reproduced most strongly by DDSP, clearly but more weakly by RAVE, and by Dance Diffusion with some confusion between adjacent lags. The same procedure with a pseudorandom time-spread pattern $p_\delta[n] = 2p[n-\delta] - 1$ at a much smaller $\alpha = 0.01$, detected by cross-correlating the output cepstrum with the pattern, also works for RAVE and DDSP and carries far more bits. The paper additionally reports that the single echo survives fine-tuning a diffusion model, survives mixing the outputs of three differently tagged models and demixing them with a source-separation network, and survives pitch-shift augmentation up to a 90% shift probability, with degraded but still detectable separation.
Load-bearing premise
The load-bearing premise of the evaluation is that the z-score of Eq. 5 detects a cepstral peak at the embedded echo lag, but as printed that formula divides the mean of the surrounding cepstrum by its standard deviation and contains no term involving the cepstral value at the lag itself, so it cannot by itself indicate a peak; if the released code computes a corrected metric, the printed equation is a typo, and this discrepancy must be resolved for the experimental conclusions to hold.
Editorial extensions
If this is right
- A dataset owner can tag a corpus before training with two lines of signal processing, then later probe any released black-box audio-to-audio model by checking its output cepstrum for the embedded lag — no access to weights or training logs needed.
- The tag's survival of fine-tuning and of mixing/demixing means the watermark travels through realistic production pipelines: a model that is fine-tuned by a third party or whose outputs are embedded in a mix still reveals its training data's tag.
- Because 1024-bit time-spread patterns survive in RAVE and DDSP with detection confidence that grows with output length, the scheme can carry far more than a single bit-pattern signature — enough for per-artist, per-instrument, or per-licensor tags within one dataset.
- Detection strength orders by architecture (DDSP most reliable, RAVE next, Dance Diffusion weakest and prone to confusing adjacent lags), so the practical embedding strength and lag spacing must be chosen with the target model's complexity in mind.
Reading between the lines
- The most plausible mechanism consistent with these results is statistical: the echo is a linear, stationary operation applied to every clip, so any waveform-domain model that matches the marginal statistics of its training distribution has an incentive to reproduce the resulting autocorrelation structure; if that is the mechanism, the same tag should appear in other waveform-domain generators, inc
- The paper leaves implicit a design rule for practitioners: since Dance Diffusion confused adjacent lags 75 and 76, tags should use well-separated lags for large diffusion models, while the simpler models can afford denser lag spacing and hence more bits.
- The per-gender tagging experiment suggests the tag can be attribute-specific within one dataset; extended to per-licensor or per-artist patterns, the scheme becomes a forensic fingerprint for screening a model's outputs before release to detect unlicensed training data, an audit use case the paper describes but does not develop into a procedure.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes embedding an imperceptible echo (a single delayed copy or a time-spread pseudorandom pattern) into every clip of a training set for audio-to-audio generative models, specifically DDSP, RAVE, and Dance Diffusion. It reports that trained models reproduce the embedded echo in their synthesized outputs, as detected by a peak in the cepstrum at the embedded delay, and that this effect survives fine-tuning, mixing/demixing, and partial pitch-shift augmentation. The central claim is that classical echo-hiding watermarks can serve as persistent tags for generative audio models and for tracing training data.
Significance. If the central claim holds, the paper offers an unusually simple and practical method for tracing generative audio models to their training data, with potential applications in licensing, copyright, and model forensics. The experimental scope is broad: three architecturally distinct models, three datasets, several echo delays, time-spread patterns with ROC evaluation, and multiple transfer scenarios. The authors provide code and supplementary audio, which is commendable and supports reproducibility. However, the detection statistic defined in Eq. (5) is internally inconsistent, and because every quantitative conclusion is expressed in terms of that statistic, the paper's experimental support is currently unverifiable as written. This issue is likely fixable, but it is load-bearing.
major comments (2)
- [Section 2.2, Eq. (5)] The z-score is defined as z = μ/σ, where μ and σ are the mean and standard deviation of the cepstrum over [a,b] with the test index i excluded. As written, this statistic does not include the cepstral value c[i], so it cannot measure the height of a peak at the embedded echo lag δ. The prose states that the z-score 'scores the cepstrum value at δ', and every experimental figure (Figures 3–12) reports separations in these z-scores. With the equation literally implemented, the reported detections could not be obtained; in fact, excluding a large positive peak from μ and σ would tend to produce a lower value at the peak location than elsewhere. The authors must correct Eq. (5) (e.g., to (c[i]−μ)/σ or a similar peak-normalized form), verify the corrected formula against the released code, and regenerate the affected results.
- [Section 2.3] The same defect carries over to the time-spread echo detection, where the z-score on the cross-correlation c* is computed using 'an equation analogous to Eq. 5'. As written, that metric also lacks any dependence on the value of c* at the candidate lag, so the ROC curves and AUROC values in Section 3.2 are not meaningful under the stated definition. Both equations need to be corrected, and the time-spread analyses must be re-run with a peak-sensitive statistic.
minor comments (6)
- [Section 2.1] The text says 'we train DDSP for 500,000 samples on all models', but the other models are trained for a number of steps; please clarify the units (presumably 500,000 steps) for consistency.
- [Section 2.2] In the paragraph after Eq. (1), 'andx1 is fully mixed in if the payload contains a 0' should read 'a 1'.
- [Section 2.2] The list of experimental echo delays appears as 'δ ∈ {50, 76, 76, 100}', which contains a duplicated 76; Section 3.1 correctly lists {50, 75, 76, 100}.
- [Section 4.1] The phrasing 'we then embed echoes in the test set and fine tune the corresponding Dance Diffusion trained on the clean training sets' is ambiguous; it is unclear whether the fine-tuning subset is drawn from the training set or a held-out set, and what role the validation set plays in preventing overfitting. Please clarify the experimental setup.
- [Abstract and Section 2.2] The paper repeatedly describes the echoes as 'imperceptible', but it provides no perceptual evaluation. In particular, the single-echo experiments use α = 0.4, which places the echo roughly 8 dB below the carrier; at such a level audibility is a real concern. A listening test or an objective perceptual metric would substantiate the claim; otherwise, the wording should be tempered.
- [Section 3] The paper reports z-score distributions but does not provide formal statistical comparisons (e.g., p-values or confidence intervals) between embedded and clean models, or between different echo delays. Adding such tests would strengthen the conclusions, especially for the smaller separation observed for Dance Diffusion.
Circularity Check
No significant circularity found: the echo-embedding/detection loop is a standard watermark measurement, not a fitted prediction; the flagged Eq. (5) issue is a correctness/reproducibility concern, not circularity.
full rationale
The paper's derivation chain is self-contained: training data are watermarked by Eq. (1) for single echoes or Eq. (6) for time-spread patterns, and the detection statistic is the cepstral z-score or cross-correlation defined in Sections 2.2 and 2.3. The relationship between embedder and detector is the standard invertible watermark construction: the same delay delta that is inserted is later searched for in the output cepstrum. This is an inverse measurement, not an assumed conclusion; the central claim is the empirical fact that trained synthesizers reproduce the learned echo, which is not entailed by the definition of the detector and is tested against clean-model false positives. No parameter is fitted and then renamed as a prediction, and no load-bearing premise rests on a self-citation: the authors' own DDSP implementation is used, but the DDSP architecture itself is external, and the other models and datasets are external. One non-circular concern must be flagged: Eq. (5) defines the z-score as z = mu/sigma with no explicit dependence on the cepstral value c[i], which as written cannot by itself score a peak at the embedded delay and conflicts with the prose describing the z-score as scoring the cepstrum value at delta. This is an internal inconsistency that affects reproducibility and should be corrected, likely to (c[i]-mu)/sigma, but it is a typo/correctness issue, not a circularity of the kind that would make the watermark-survival claims reduce to their inputs.
Assumptions & free parameters
free parameters (7)
- echo amplitude α (single echo) =
0.4
- echo amplitude α (time-spread) =
0.01
- echo delays δ =
50, 75, 76, 100 samples at 44.1 kHz
- cepstral z-score window bounds (a,b) =
a=25, b=125 for single echo; a=3, b=L+δ for time-spread
- Dance Diffusion style transfer noise η =
0.2
- time-spread pattern length L =
1024
- pitch shift augmentation probability =
varied (default 50%, up to 90%)
assumptions (4)
- standard math The cepstrum of a windowed signal exposes single echoes as peaks (Eq. 2), and log-magnitude DFT additively separates echo impulse response from carrier.
- domain assumption The z-score metric as defined in Eq. (5) is a valid detector for the embedded echo.
- domain assumption Echo amplitude α=0.4 is perceptually transparent (imperceptible).
- domain assumption Model outputs should be evaluated on MUSDB18-HQ stems of the same instrument type as training data; these inputs are representative of the intended style transfer use case.
Cite this review
Pith. "Pith review of Hidden Echoes Survive Training in Audio To Audio Generative Instrument Models." pith.science (2026). https://pith.science/paper/TIJYH5AP
@misc{pith2026241210649,
author = {Pith},
title = {Pith review of: Hidden Echoes Survive Training in Audio To Audio Generative Instrument Models},
year = {2026},
howpublished = {\url{https://pith.science/paper/TIJYH5AP}},
note = {Machine review of arXiv:2412.10649}
}
read the original abstract
As generative techniques pervade the audio domain, there has been increasing interest in tracing back through these complicated models to understand how they draw on their training data to synthesize new examples, both to ensure that they use properly licensed data and also to elucidate their black box behavior. In this paper, we show that if imperceptible echoes are hidden in the training data, a wide variety of audio to audio architectures (differentiable digital signal processing (DDSP), Realtime Audio Variational autoEncoder (RAVE), and ``Dance Diffusion'') will reproduce these echoes in their outputs. Hiding a single echo is particularly robust across all architectures, but we also show promising results hiding longer time spread echo patterns for an increased information capacity. We conclude by showing that echoes make their way into fine tuned models, that they survive mixing/demixing, and that they survive pitch shift augmentation during training. Hence, this simple, classical idea in watermarking shows significant promise for tagging generative audio models.
Figures
Figures from the paper (8 more)
Reference graph
Works this paper leans on
-
[1]
Harmonycloak: Making music unlearnable for gen- erative ai
Syed Irfan Ali Meerza, Lichao Sun, and Jian Liu. Harmonycloak: Making music unlearnable for gen- erative ai. Proceedings of the 46th IEEE Symposium on Security and Privacy , 2025
work page 2025
-
[2]
Audio data defenses: Protecting music and speech data from targeted attacks
Julia Barnett, William Agnew, Robin Netzorg, Patrick O’Reilly, Ezra Awumey, Chris Donahue, and Sauvik Das. Audio data defenses: Protecting music and speech data from targeted attacks. Late Breaking Session At The 25th Conference of the In- ternational Society for Music Information Retrieval (ISMIR 2024) , 2024
work page 2024
-
[3]
Julia Barnett, Hugo Flores Garcia, and Bryan Pardo. Exploring musical roots: Applying audio embeddings to empower influence attribution for a generative music model. Proceedings of the 25th Conference of the International Society for Music Information Retrieval (ISMIR 2024) , 2024
work page 2024
-
[4]
Roser Batlle-Roca, Wei-Hsiang Liao, Xavier Serra, Yuki Mitsufuji, and Emilia G´ omez Guti´ errez. To- wards assessing data replication in music genera- tion with music similarity metrics on raw audio. Proceedings of the 25th Conference of the Interna- tional Society for Music Information Retrieval (IS- MIR 2024) , 2024
work page 2024
-
[5]
Rave: A vari- ational autoencoder for fast and high-quality neural audio synthesis
Antoine Caillon and Philippe Esling. Rave: A vari- ational autoencoder for fast and high-quality neural audio synthesis. arXiv preprint arXiv:2111.05011 , 2021. 7
arXiv 2021
-
[6]
Chris Cannam. Rubberband library. https:// github.com/breakfastquay/rubberband, 2024
work page 2024
-
[7]
Invisible watermarking for audio generation diffusion mod- els
Xirong Cao, Xiang Li, Divyesh Jadav, Yanzhao Wu, Zhehui Chen, Chen Zeng, and Wenqi Wei. Invisible watermarking for audio generation diffusion mod- els. In 2023 5th IEEE International Conference on Trust, Privacy and Security in Intelligent Systems and Applications (TPS-ISA), pages 193–202. IEEE, 2023
work page 2023
-
[8]
Guangke Chen, Yedi Zhang, Fu Song, Ting Wang, Xiaoning Du, and Yang Liu. A proactive and dual prevention mechanism against illegal song cov- ers empowered by singing voice conversion. arXiv preprint arXiv:2401.17133, 2024
arXiv 2024
Show all 34 references
-
[9]
Music source separa- tion in the waveform domain
Alexandre D´ efossez, Nicolas Usunier, L´ eon Bot- tou, and Francis Bach. Music source separa- tion in the waveform domain. arXiv preprint arXiv:1911.13254, 2019
1911 arXiv
-
[10]
Hey that’s mine imperceptible watermarks are preserved in diffusion generated outputs
Luke Ditria and Tom Drummond. Hey that’s mine imperceptible watermarks are preserved in diffusion generated outputs. arXiv preprint arXiv:2308.11123, 2023
2023 arXiv
-
[11]
Receiving data hidden in music
Manuel Eichelberger, Simon Tanner, Gabriel Voirol, and Roger Wattenhofer. Receiving data hidden in music. In Proceedings of the 20th International Workshop on Mobile Computing Systems and Ap- plications, pages 33–38. ACM, 2019
2019
-
[12]
Ddsp: Differentiable digital signal processing
Jesse Engel, Chenjie Gu, Adam Roberts, et al. Ddsp: Differentiable digital signal processing. In International Conference on Learning Representa- tions, 2020
2020
-
[13]
Dance diffusion
Zach Evans. Dance diffusion. https://github. com/harmonai-org/sample-generator, 2022
2022
-
[14]
Haw- ley, and Jordi Pons
Zach Evans, CJ Carr, Josiah Taylor, Scott H. Haw- ley, and Jordi Pons. Fast timing-conditioned latent audio diffusion. In Forty-first International Confer- ence on Machine Learning , 2024
2024
-
[15]
Long- form music generation with latent diffusion
Zach Evans, Julian D Parker, CJ Carr, Zack Zukowski, Josiah Taylor, and Jordi Pons. Long- form music generation with latent diffusion. Late Breaking Session At The 25th Conference of the In- ternational Society for Music Information Retrieval (ISMIR 2024) , 2024
2024
-
[16]
Learning to groove with inverse sequence transformations
Jon Gillick, Adam Roberts, Jesse Engel, Douglas Eck, and David Bamman. Learning to groove with inverse sequence transformations. In International Conference on Machine Learning (ICML) , 2019
2019
-
[17]
Echo hiding
Daniel Gruhl, Anthony Lu, and Walter Bender. Echo hiding. In Information Hiding: First Interna- tional Workshop Cambridge, UK, May 30–June 1, 1996 Proceedings 1, pages 295–315. Springer, 1996
1996
-
[18]
Multi-instrument music synthesis with spec- trogram diffusion
Curtis Hawthorne, Ian Simon, Adam Roberts, Neil Zeghidour, Josh Gardner, Ethan Manilow, and Jesse Engel. Multi-instrument music synthesis with spec- trogram diffusion. In Proceedings of the 23th Con- ference of the International Society for Music Infor- mation Retrieval (ISMIR...
2022
-
[19]
Variable-dimensional vector modulation for perceptual-based dwt blind audio watermarking with adjustable payload capacity
Hwai-Tsu Hu, Ling-Yuan Hsu, and Hsien-Hsin Chou. Variable-dimensional vector modulation for perceptual-based dwt blind audio watermarking with adjustable payload capacity. Digital Signal Processing, 31:115–123, 2014
2014
-
[20]
Crepe: A convolutional represen- tation for pitch estimation
Jong Wook Kim, Justin Salamon, Peter Li, and Juan Pablo Bello. Crepe: A convolutional represen- tation for pitch estimation. In 2018 IEEE Interna- tional Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 161–165. IEEE, 2018
2018
-
[21]
Ro- bust spread-spectrum audio watermarking
Darko Kirovski and Henrique Malvar. Ro- bust spread-spectrum audio watermarking. In 2001 IEEE international conference on acoustics, speech, and signal processing. Proceedings (Cat. No. 01CH37221), volume 3, pages 1345–1348. IEEE, 2001
2001
-
[22]
Time-spread echo method for digital audio watermarking
Byeong-Seob Ko, Ryouichi Nishimura, and Yˆ oiti Suzuki. Time-spread echo method for digital audio watermarking. IEEE Transactions on Multimedia , 7(2):212–221, 2005
2005
-
[23]
Hafiz M. A. Malik, Rashid Ansari, and Ashfaq A. Khokhar. Robust data hiding in audio using allpass filters. IEEE Transactions on Audio, Speech and Language Processing, 15(4):1296–1304, 2007
2007
-
[24]
Musdb18-hq - an uncompressed version of musdb18, August 2019
Zafar Rafii, Antoine Liutkus, Fabian-Robert St¨ oter, Stylianos Ioannis Mimilakis, and Rachel Bittner. Musdb18-hq - an uncompressed version of musdb18, August 2019
2019
-
[25]
Pesto: Pitch estimation with self- supervised transposition-equivariant objective
Alain Riou, Stefan Lattner, Ga¨ etan Hadjeres, and Geoffroy Peeters. Pesto: Pitch estimation with self- supervised transposition-equivariant objective. In International Society for Music Information Re- trieval Conference (ISMIR 2023) , 2023
2023
-
[26]
Glaze: Protecting artists from style mimicry by {Text-to- Image} models
Shawn Shan, Jenna Cryan, Emily Wenger, Haitao Zheng, Rana Hanocka, and Ben Y Zhao. Glaze: Protecting artists from style mimicry by {Text-to- Image} models. In 32nd USENIX Security Sym- posium (USENIX Security 23) , pages 2187–2204, 2023
2023
-
[27]
Prompt-specific poisoning attacks on text-to-image generative mod- els
Shawn Shan, Wenxin Ding, Josephine Passananti, Haitao Zheng, and Ben Y Zhao. Prompt-specific poisoning attacks on text-to-image generative mod- els. arXiv e-prints , pages arXiv–2310, 2023. 8
2023
-
[28]
Deep unsuper- vised learning using nonequilibrium thermodynam- ics
Jascha Sohl-Dickstein, Eric Weiss, Niru Mah- eswaranathan, and Surya Ganguli. Deep unsuper- vised learning using nonequilibrium thermodynam- ics. In International conference on machine learn- ing, pages 2256–2265. PMLR, 2015
2015
-
[29]
Getty images is suing the creators of ai art tool stable diffusion for scraping its content
James Vincent. Getty images is suing the creators of ai art tool stable diffusion for scraping its content. The Verge, 3, 2023
2023
-
[30]
Tree-ring watermarks: Fingerprints for diffusion images that are invisible and robust
Yuxin Wen, John Kirchenbauer, Jonas Geiping, and Tom Goldstein. Tree-ring watermarks: Fingerprints for diffusion images that are invisible and robust. arXiv preprint arXiv:2305.20030 , 2023
2023 arXiv
-
[31]
Vocalset: A singing voice dataset
Julia Wilkins, Prem Seetharaman, Alison Wahl, and Bryan Pardo. Vocalset: A singing voice dataset. In 19th International Society for Music Information Retrieval (ISMIR), Paris, France, 2018
2018
-
[32]
Guitarset: A dataset for guitar transcription
Qingyang Xi, Rachel M Bittner, Johan Pauwels, Xuzhou Ye, and Juan Pablo Bello. Guitarset: A dataset for guitar transcription. In 19th Interna- tional Society for Music Information Retrieval (IS- MIR), Paris, France, pages 453–460, 2018
2018
-
[33]
Effective pseudonoise sequence and decoding function for imperceptibility and robustness enhancement in time-spread echo- based audio watermarking
Yong Xiang, Dezhong Peng, Iynkaran Natgu- nanathan, and Wanlei Zhou. Effective pseudonoise sequence and decoding function for imperceptibility and robustness enhancement in time-spread echo- based audio watermarking. IEEE Transactions on Multimedia, 13(1):2–13, 2010
2010
-
[34]
Bocko, and Z
Xiaoxiao Dong, M.F. Bocko, and Z. Ignjatovic. Data hiding via phase manipulation of audio signals. In 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing , volume 5, pages V– 377–80. IEEE, 2004. 9
2004
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.