Pith. sign in

REVIEW 4 major objections 4 minor 48 references

A single vector addition in a neural codec's latent space rivals large diffusion models for music bandwidth extension.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

A single mean shift in the latent space of several neural codecs achieves competitive music bandwidth extension on some metrics, implying a largely linear structure.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection Interesting latent-geometry probe, but the headline 'competitive with diffusion' rests on baseline rows that look copied from the degraded-roundtrip row. the 4 major comments →

arxiv 2608.03721 v1 pith:W64L7MLB submitted 2026-08-04 cs.SD

On the Geometry of Music Bandwidth Extension in Latent Spaces of Audio Codecs

Hendrik Vincent Koops , Hao Hao Tan , Elio Quinton This is my paper
classification cs.SD
keywords music bandwidth extensionlatent space geometryneural audio codecsmean shift transportaudio restorationzero-parameter baselinediffusion modelslatent arithmetic
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether large generative models are necessary for music bandwidth extension. It shows that in several neural codec latent spaces, a single global vector—the mean difference between clean and degraded latents over a reference set—added back to degraded latents restores audio almost as well as state-of-the-art diffusion and Schrödinger bridge models that have up to a billion parameters. The result is zero-parameter: no training, no learned denoiser, just one arithmetic addition per latent frame. The paper interprets this as evidence that band-limiting maps to a coherent, nearly dataset-independent linear direction in these codec spaces, and proposes this 'mean shift' as a cheap sanity baseline for future bandwidth-extension research.

Core claim

The central claim is that estimating a single transport vector T = (1/N) Σ (z_clean_i − z_degraded_i) on a training set, then applying it uniformly as z_hat_i = z_degraded_i + T, yields bandwidth-extension results competitive with large diffusion models on several metrics and datasets. Across four codecs (Stable Audio OpenVAE, CodiCodec, DAC, Encodec) and three datasets (MTD, MAESTRO, ccMixter), the mean-shift transport often matches or exceeds the LSD, ViSQOL, and sometimes SiSpec scores of AudioSR, A2SB, IBAR, and CQTDiff, despite using zero learnable parameters. The paper further shows that these transport vectors align strongly across datasets at the same cutoff frequency, and that as fe

What carries the argument

The central object is the mean-shift transport vector T, defined as the average per-sample latent-space displacement between clean and degraded audio (z_clean − z_degraded) computed on a reference set. Restoration is a single vector addition to each degraded latent frame, followed by codec decoding. This probe isolates the linear, global part of the degradation-to-clean mapping and is compared with state-of-the-art learned restorers; its performance and stability (measured by cosine alignment across datasets, sample efficiency, and identity-preservation margins) quantify how much of bandwidth extension is already encoded as a simple direction in the latent geometry.

Load-bearing premise

The baseline numbers for AudioSR, CQTDiff, IBAR, and A2SB are taken verbatim from reference [13]; the paper's competitiveness claim assumes those numbers were produced under the same degradation simulation, data splits, chunking, and metric implementations as the mean-shift evaluation.

What would settle it

Run the mean-shift transport and the published baselines under a single shared evaluation harness—identical resampling-to-8/16/24-kHz pipeline, identical 1.5-second chunking, identical versions of LSD/SiSpec/ViSQOL—on the same test files. If the mean-shift gaps widen to non-competitive levels (e.g., LSD or ViSQOL clearly worse than the baselines across all datasets), the central claim of competitiveness is overturned. A simpler check: re-evaluate the mean-shift on the exact test split used for the baselines reported in reference [13].

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Zero-parameter latent translation is a strong, reproducible baseline for music bandwidth extension; future BWE papers should report it alongside learned models.
  • Large generative models appear to spend capacity relearning an average correction that a codec's latent space already supports; redirecting capacity to sample-specific details may improve efficiency and quality.
  • The choice of codec changes restoration behavior (e.g., CodiCodec low LSD but low spectral detail, Encodec high spectral detail but weaker LSD); codec selection or design is a lever for restoration performance.
  • The transport direction transfers across musical datasets for a given cutoff, so a correction estimated on one corpus can be applied to another.
  • Bandwidth degradation keeps a sample's latent near its clean counterpart (identity preservation), suggesting restoration is a local correction rather than regeneration from noise.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If confirmed in a shared evaluation harness, the competitiveness claim implies that some published gains of large restoration models over simple baselines may be overstated; the field should adopt zero-parameter latent arithmetic as a control.
  • The near dataset-independence of the transport vector suggests a codec-specific, per-cutoff correction could be precomputed once per codec and reused across tasks, possibly including speech if the structure transfers.
  • The failure of mean-shift for declipping and dereverberation suggests the linear-latent phenomenon is specific to band-limiting; a codec trained to linearize other degradations might enable similarly cheap restoration.
  • The identity-preservation margin could be promoted during codec training; a codec that keeps degraded versions of a track close to the clean latent would make any subsequent learned restorer's job easier.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper studies latent-space geometry of four neural audio codecs for musical bandwidth extension. It computes a single mean-shift vector T by averaging differences between clean and degraded latents over paired training chunks, then adds T to test degraded latents before decoding. Across MAESTRO, MTD, and ccMixter at 4/8/12 kHz cutoffs, this translation is reported to be competitive with large diffusion/Schrödinger-bridge baselines on some objective metrics (LSD, ViSQOL), though not SiSpec. The paper further analyzes cross-dataset cosine similarity of estimated shift vectors, sample efficiency of estimating T, an identity-preservation margin, and behavior on denoising/declipping/dereverberation. It concludes that bandwidth degradation is encoded as a coherent linear direction in codec latent spaces and proposes mean shift as a baseline.

Significance. The observation is potentially useful for the MIR/audio-restoration community: it identifies a simple, reproducible baseline and suggests that codec choice and latent geometry matter for restoration difficulty. The paper ships a concrete method and quantitative geometry probes. However, the headline comparison to large generative models currently rests on externally reported baseline numbers, one of which is implausibly identical to the degraded input; this must be fixed before the central claim can be assessed. With corrected comparisons, the paper would be a valuable empirical contribution, though the conclusions are qualitative and metric-dependent.

major comments (4)
  1. [§3.1, Tables 2–3] The AudioSR rows are numerically identical to the 'Degraded roundtrip' row in both MTD and CCMIXTER (e.g., Table 2, 4 kHz: LSD 1.75, SiSpec 21.74, ViSQOL 3.39; Table 3 likewise). This implies AudioSR was not evaluated under the authors' resampling/chunking/metric pipeline, or the entries are placeholders. Since the central claim of competitiveness with large diffusion models is based on comparing mean-shift numbers to these external numbers, this is load-bearing. Please either run the baselines under the identical protocol or clearly mark them as non-comparable and restrict the claims accordingly.
  2. [§3.2, §4.1, Tables 1–3] The label 'zero-parameter' is inaccurate: T is a fitted statistic (a vector of the latent dimension) estimated from up to N=8192 paired training chunks. Tables listing 'Parameters 0' and the section title 'Zero-parameter Bandwidth Extension' overstate simplicity. It is a single fixed translation learned from data, not a parameter-free method. Please rename to something like 'no-learned-parameters' or 'fixed mean shift', and state explicitly that T is estimated on a reference set.
  3. [§4.1, Tables 2–3] The abstract and Section 4.1 use 'competitive' without always specifying that this holds only on some metrics. For example, in Table 2 at 4 kHz, mean-shift SiSpec values are 7.03–13.37, far below A2SB 4-partitioning's 27.56 and CQTDiff's 10.62 (where applicable); in Table 3, CodiCodec mean-shift SiSpec is 6.02 vs A2SB 4-part 18.00. The claim should be qualified as metric- and dataset-dependent in the abstract, not just in the body.
  4. [Tables 1–3] No confidence intervals, standard deviations, or significance tests are reported for the objective metrics. Differences of 0.1–0.2 in LSD (e.g., Table 2, MTD 4 kHz: VAE mean shift 1.29 vs A2SB no-partition 1.33) may be within metric variability, especially when baselines come from external pipelines. Please add error bars or paired significance tests, or at least report variability over test-set chunks.
minor comments (4)
  1. [Figure 3 and §3.2] The caption says 'Results are close to full-dataset results at 8 samples' while the text reports N=8192 for the full estimate. Clarify the relationship between the fit-subset sizes in Figure 3 and the N=8192 used for Tables 1–3, and specify whether the curves are evaluated on the test set or on the same fit data.
  2. [Equation (1) and Table 4] State whether the cosine alignment values are computed on the same training pairs used to estimate T or on held-out test pairs. Table 4 also should state explicitly that negative ΔLSD means improvement.
  3. [Throughout] There are minor formatting inconsistencies: 'V .' (extra space), 'CODICODEC' vs 'CodiCodec', and 'VISQOL' vs 'ViSQOL'. Please harmonize.
  4. [References] Since the baseline numbers are central, give the exact version/date of the Kong et al. preprint [13] and, if possible, provide scripts or command lines to reproduce the baseline evaluations under the paper's protocol.

Circularity Check

0 steps flagged

Mean-shift transport is fit on paired training data and evaluated on held-out test examples; baseline comparability is a risk but no derivation reduces to its inputs.

full rationale

The central derivation is not circular. In Sec. 3.2 the mean-shift vector is defined as T = (1/N) Σ(z_clean_i − z_deg_i) on a training subset, and it is then applied to test-set degraded latents as \hat z_i = z_deg_i + T. The objective restoration metrics in Tables 1–3 are therefore out-of-sample; the fitted T is not a renamed version of the test prediction. The 'zero-parameter' wording understates the fact that T is estimated from paired data, but that is a labeling issue rather than a definitional collapse. The alignment probe in Eq. 1 is a descriptive statistic and could be mildly optimistic if evaluated on the same vectors used to estimate T, and the paper is not explicit about the split used for this probe; however, the headline restoration result does not depend on Eq. 1. The external baselines (AudioSR, CQTDiff, IBAR, A2SB) are taken from Kong et al. and the rows for AudioSR/CQTDiff equal the 'Degraded roundtrip' row, which is a serious comparability/reporting concern for the competitive claim, but it is not a circularity: the paper does not derive those baseline numbers from its own method. No self-citations or imported uniqueness theorems are used. Therefore no step in the paper reduces, by construction, to its inputs.

Axiom & Free-Parameter Ledger

1 free parameters · 3 axioms · 0 invented entities

The central claim rests on the mean-shift vector T fitted to training data, the comparability of external baseline results, and the validity of the objective metrics. No new physical entities or forces are introduced; the 'transport' is an algebraic operation, not a new object.

free parameters (1)
  • Global mean-shift vector T = latent-dimension vector per codec, dataset, and cutoff (numerical values not reported in the paper)
    Estimated as the average of clean-degraded latent differences over up to N=8192 training chunks (Section 3.2). It is the only learned quantity in the proposed 'zero-parameter' baseline, despite the paper calling it zero-parameter.
axioms (3)
  • domain assumption The selected objective metrics (LSD, SiSpec, ViSQOL) are valid proxies for perceptual restoration quality.
    Section 3.3 states these metrics 'capture spectral fidelity, spectral consistency, and perceptual quality' but does not validate them against listening tests for this specific task.
  • domain assumption The degradation simulation (resampling to twice the cutoff then back to 44.1 kHz) matches the degradation used to train or evaluate the cited baselines.
    Section 3.1 describes the resampling procedure; the paper assumes this is the same protocol as in Kong et al. [13] when copying baseline numbers.
  • domain assumption The latent spaces of the four codecs, sampled before the quantization bottleneck, are suitable for linear transport.
    Section 3.1 uses continuous latent embeddings without quantized codes; the paper assumes the unquantized space has meaningful structure, which is itself the hypothesis under test.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of On the Geometry of Music Bandwidth Extension in Latent Spaces of Audio Codecs." pith.science (2026). https://pith.science/paper/W64L7MLB

@misc{pith2026260803721,
  author       = {Pith},
  title        = {Pith review of: On the Geometry of Music Bandwidth Extension in Latent Spaces of Audio Codecs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/W64L7MLB}},
  note         = {Machine review of arXiv:2608.03721}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Recent audio restoration increasingly relies on large-scale conditional latent generative modeling, including diffusion, Schrodinger Bridges, and Flow Matching variants, to invert degradations such as bandwidth limitation or noise. We present an analysis of the performance of various state-of-the-art methods compared to simple arithmetic transformations in the latent spaces of multiple neural codecs for musical bandwidth extension. We show that estimating a single transport vector between the clean and degraded latent centroids on a reference set, and adding it to degraded latents, can yield restoration performance competitive with large diffusion models. This suggests, first, that some neural codec latent spaces exhibit structure aligned with audio bandwidth; and second, that in such cases complex conditional models may offer only limited gains over a simple vector addition. We argue that these findings reveal an interesting avenue for future research whereby models could take advantage of the latent space structure in order to offer greater training and parameter efficiency, and overall better performance. Additionally, we propose to consider this simple arithmetic transformation as a baseline for music bandwidth extension research, as it allows an assessment of the contribution of learnable parameters towards restoration performance.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

48 extracted references · 38 canonical work pages · 2 internal anchors

  1. [1]

    On the Geometry of Music Bandwidth Extension in Latent Spaces of Audio Codecs

    INTRODUCTION Many audio restoration tasks, such as bandwidth exten- sion, require regenerating information and content that is not present in the original signal, which makes genera- tive models an attractive approach for solving this type of problem. Following the recent developments in gen- erative modeling and the very significant improvement in the qu...

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  2. [2]

    RELATED WORK Recent audio restoration systems are increasingly built as large generative models. Early neural bandwidth- extension work already treated super-resolution as a learned conditional mapping [4, 20], and more recent ap- proaches have pushed this paradigm further with large- scale diffusion models for audio super-resolution and blind restoration...

    work page

  3. [3]

    We wish to evaluate whether audio bandwidth is somehow represented in the geometry of the latent space of various neural codecs

    EXPERIMENTS For brevity, our study focuses on bandwidth extension (BWE). We wish to evaluate whether audio bandwidth is somehow represented in the geometry of the latent space of various neural codecs. For that we propose to use sim- ple algebraic operations in the latent space. Letz clean i and zdeg i denote the clean and degraded latents of a paired au-...

    work page

  4. [4]

    identity preservation

    RESULTS 4.1 Zero-parameter Bandwidth Extension Tables 1, 2 and 3 compare zero-parameter latent transports with high-capacity generative models forBWE. Although mean shift is not intended as a practical replacement for these models, we were surprised to find that the perfor- mance gap between large models and a simple latent trans- lation is much smaller t...

    work page

  5. [5]

    We do not interpret this as an end in itself, or as a replace- ment for learned restoration systems

    DISCUSSION Our results show that, for music bandwidth extension, a simple latent-space transformation can be surprisingly competitive with large generative restoration models. We do not interpret this as an end in itself, or as a replace- ment for learned restoration systems. Rather, it sug- gests that some neural codec latent spaces already contain resto...

    work page

  6. [6]

    CONCLUSION We studied audio restoration through simple deterministic probes in neural codec latent spaces. Our results show that musicBWEcan appear as a surprisingly coherent latent di- rection, making zero-parameter transport competitive with much larger learned restorers in several settings. This find- ing is not meant to replace generative restoration ...

    work page

  7. [7]

    Scaling rectified flow transformers for high-resolution image synthesis,

    P. Esser, S. Kulal, A. Blattmann, R. Entezari, J. Müller, H. Saini, Y . Levi, D. Lorenz, A. Sauer, F. Boeselet al., “Scaling rectified flow transformers for high-resolution image synthesis,” inProceedings of the 41st Inter- national Conference on Machine Learning, 2024, pp. 12 606–12 633

    work page 2024

  8. [8]

    FLUX. 1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Space,

    B. F. Labs, S. Batifol, A. Blattmann, F. Boesel, S. Con- sul, C. Diagne, T. Dockhorn, J. English, Z. English, P. Esseret al., “FLUX. 1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Space,”arXiv preprint arXiv:2506.15742, 2025

    Pith/arXiv arXiv 2025

  9. [9]

    Long-Form Music Generation With Latent Diffusion,

    Z. Evans, J. D. Parker, C. Carr, Z. Zukowski, J. Taylor, and J. Pons, “Long-Form Music Generation With Latent Diffusion,” inProceedings of the 25th International Society for Music Information Retrieval Conference (ISMIR), 2024, pp. 429–437. [Online]. Available: https://arxiv.org/abs/2404.10301

    Pith/arXiv arXiv 2024

  10. [10]

    Audio Super Resolution using Neural Networks,

    V . Kuleshov, S. Z. Enam, and S. Ermon, “Audio Super Resolution using Neural Networks,” ICLR Workshop / arXiv preprint arXiv:1708.00853, 2017. [Online]. Available: https://arxiv.org/abs/1708.00853

    Pith/arXiv arXiv 2017

  11. [11]

    Audiosr: Versatile audio super-resolution at scale,

    H. Liu, K. Chen, Q. Tian, W. Wang, and M. D. Plumbley, “Audiosr: Versatile audio super-resolution at scale,” inICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Process- ing (ICASSP 2024). Institute of Electrical and Elec- tronics Engineers (IEEE), 2024, pp. 1076–1080

    work page 2024

  12. [12]

    Blind Au- dio Bandwidth Extension: A Diffusion-Based Zero- Shot Approach,

    E. Moliner, F. Elvander, and V . Välimäki, “Blind Au- dio Bandwidth Extension: A Diffusion-Based Zero- Shot Approach,”IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2024

    work page 2024

  13. [13]

    Diffusion Schrödinger Bridge with Applications to Score-Based Generative Modeling,

    V . D. Bortoli, J. Thornton, J. Heng, and A. Doucet, “Diffusion Schrödinger Bridge with Applications to Score-Based Generative Modeling,” inAdvances in Neural Information Processing Systems (NeurIPS), 2021

    work page 2021

  14. [14]

    Flow Matching for Generative Modeling,

    Y . Lipman, R. T. Q. Chen, H. Ben-Hamu, M. Nickel, and M. Le, “Flow Matching for Generative Modeling,” inInternational Conference on Learning Representa- tions (ICLR), 2023

    work page 2023

  15. [15]

    V oice- bridge: Designing latent bridge models for gen- eral speech restoration at scale,

    C. Zhang, Z. Chen, K. Zheng, and J. Zhu, “V oice- bridge: Designing latent bridge models for gen- eral speech restoration at scale,”arXiv preprint arXiv:2509.25275, 2025

    arXiv 2025

  16. [16]

    Diffusion models for image restoration and enhancement: A comprehensive sur- vey,

    X. Li, Y . Ren, X. Jin, C. Lan, X. Wang, W. Zeng, X. Wang, and Z. Chen, “Diffusion models for image restoration and enhancement: A comprehensive sur- vey,”International Journal of Computer Vision, vol. 133, no. 11, pp. 8078–8108, 2025

    work page 2025

  17. [17]

    Diffusion models for audio restoration: A review,

    J.-M. Lemercier, J. Richter, S. Welker, E. Moliner, V . Välimäki, and T. Gerkmann, “Diffusion models for audio restoration: A review,”IEEE Signal Processing Magazine, vol. 41, no. 6, pp. 72–84, 2025

    work page 2025

  18. [18]

    High-resolution speech restoration with la- tent diffusion model,

    T. Dhyani, F. Lux, M. Mancusi, G. Fabbro, F. Hohl, and N. T. Vu, “High-resolution speech restoration with la- tent diffusion model,” inICASSP 2025-2025 IEEE In- ternational Conference on Acoustics, Speech and Sig- nal Processing (ICASSP). IEEE, 2025, pp. 1–5

    work page 2025

  19. [19]

    A2sb: Audio-to-audio schrodinger bridges,

    Z. Kong, K. J. Shih, W. Nie, A. Vahdat, S.-g. Lee, J. F. Santos, A. Jukic, R. Valle, and B. Catanzaro, “A2sb: Audio-to-audio schrodinger bridges,”arXiv preprint arXiv:2501.11311, 2025

    Pith/arXiv arXiv 2025

  20. [20]

    Mu- sichifi: Fast high-fidelity stereo vocoding,

    G. Zhu, J.-P. Caceres, Z. Duan, and N. J. Bryan, “Mu- sichifi: Fast high-fidelity stereo vocoding,”IEEE Sig- nal Processing Letters, vol. 31, pp. 2365–2369, 2024

    work page 2024

  21. [21]

    Learning to Upsample and Upmix Audio in the Latent Domain,

    D. Bralios, P. Smaragdis, and J. Casebeer, “Learning to Upsample and Upmix Audio in the Latent Domain,” in IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), 2025

    work page 2025

  22. [22]

    Stable audio open,

    Z. Evans, J. D. Parker, C. Carr, Z. Zukowski, J. Taylor, and J. Pons, “Stable audio open,” in ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2025, pp. 1–5. [Online]. Available: https: //arxiv.org/abs/2407.14358

    Pith/arXiv arXiv 2025

  23. [23]

    CoDiCodec: Unifying Continuous and Discrete Compressed Representations of Audio

    M. Pasini, S. Lattner, and G. Fazekas, “CoD- iCodec: Unifying Continuous and Discrete Compressed Representations of Audio,” arXiv preprint arXiv:2509.09836, 2025. [Online]. Avail- able: https://arxiv.org/abs/2509.09836

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  24. [24]

    High-Fidelity Audio Compression with Improved RVQGAN,

    R. Kumar, P. Seetharaman, A. Luebs, I. Ku- mar, and K. Kumar, “High-Fidelity Audio Compression with Improved RVQGAN,” arXiv preprint arXiv:2306.06546, 2023. [Online]. Avail- able: https://arxiv.org/abs/2306.06546

    Pith/arXiv arXiv 2023

  25. [25]

    High Fidelity Neural Audio Compression,

    A. Défossez, J. Copet, G. Synnaeve, and Y . Adi, “High Fidelity Neural Audio Compression,” arXiv preprint arXiv:2210.13438, 2022. [Online]. Available: https://arxiv.org/abs/2210.13438

    Pith/arXiv arXiv 2022

  26. [26]

    Band- width Extension is All You Need,

    J. Su, Y . Wang, A. Finkelstein, and Z. Jin, “Band- width Extension is All You Need,” inICASSP 2021 - IEEE International Conference on Acoustics, Speech and Signal Processing, 2021, pp. 696–700

    work page 2021

  27. [27]

    FLowHigh: To- wards efficient and high-quality audio super-resolution with single-step flow matching,

    J.-H. Yun, S.-B. Kim, and S.-W. Lee, “FLowHigh: To- wards efficient and high-quality audio super-resolution with single-step flow matching,” inICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2025, pp. 1–5

    work page 2025

  28. [28]

    Latent Rep- resentation Learning for Artificial Bandwidth Exten- sion Using a Conditional Variational Auto-encoder,

    P. Bachhav, M. Todisco, and N. Evans, “Latent Rep- resentation Learning for Artificial Bandwidth Exten- sion Using a Conditional Variational Auto-encoder,” inICASSP 2019 - IEEE International Conference on Acoustics, Speech and Signal Processing, 2019

    work page 2019

  29. [29]

    SoundStream: An End-to-End Neu- ral Audio Codec,

    N. Zeghidour, A. Luebs, A. Omran, J. Skoglund, and M. Tagliasacchi, “SoundStream: An End-to-End Neu- ral Audio Codec,”IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 30, pp. 495– 507, 2021

    work page 2021

  30. [30]

    Music2Latent: Consistency Autoencoders for Latent Audio Com- pression,

    M. Pasini, S. Lattner, and G. Fazekas, “Music2Latent: Consistency Autoencoders for Latent Audio Com- pression,” inProceedings of the International Society for Music Information Retrieval Conference (ISMIR), 2024

    work page 2024

  31. [31]

    Generative timbre spaces with variational audio syn- thesis,

    P. Esling, A. Chemla-Romeu-Santos, and A. Bitton, “Generative timbre spaces with variational audio syn- thesis,” inProceedings of the International Conference on Digital Audio Effects (DAFx), 2018, pp. 175–181

    work page 2018

  32. [32]

    Learning and Controlling the Source- Filter Representation of Speech with a Variational Au- toencoder,

    S. Sadok, S. Leglaive, L. Girin, X. Alameda-Pineda, and R. Séguier, “Learning and Controlling the Source- Filter Representation of Speech with a Variational Au- toencoder,”Speech Communication, vol. 148, pp. 1– 13, 2023

    work page 2023

  33. [33]

    Learning dis- entangled representations of timbre and pitch for mu- sical instrument sounds using gaussian mixture varia- tional autoencoders,

    Y .-J. Luo, K. Agres, and D. Herremans, “Learning dis- entangled representations of timbre and pitch for mu- sical instrument sounds using gaussian mixture varia- tional autoencoders,” in20th International Society for Music Information Retrieval Conference, ISMIR 2019. International Society for Music Information Retrieval, 2019

    work page 2019

  34. [34]

    Latent timbre syn- thesis: Audio-based variational auto-encoders for mu- sic composition and sound design applications,

    K. Tatar, D. Bisig, and P. Pasquier, “Latent timbre syn- thesis: Audio-based variational auto-encoders for mu- sic composition and sound design applications,”Neu- ral Computing and Applications, vol. 33, no. 1, pp. 67– 84, 2021

    work page 2021

  35. [35]

    GANSpace: Discovering Interpretable GAN Con- trols,

    E. Härkönen, A. Hertzmann, J. Lehtinen, and S. Paris, “GANSpace: Discovering Interpretable GAN Con- trols,” inAdvances in Neural Information Processing Systems (NeurIPS), 2020

    work page 2020

  36. [36]

    Interpreting the Latent Space of GANs for Semantic Face Editing,

    Y . Shen, C. Yang, X. Tang, and B. Zhou, “Interpreting the Latent Space of GANs for Semantic Face Editing,” inProceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR), 2020

    work page 2020

  37. [37]

    Codec Does Matter: Exploring the Semantic Shortcoming of Codec for Audio Language Model,

    Z. Ye, P. Sun, J. Lei, H. Lin, X. Tan, Z. Dai, Q. Kong, J. Chen, J. Pan, Q. Liu, Y . Guo, and W. Xue, “Codec Does Matter: Exploring the Semantic Shortcoming of Codec for Audio Language Model,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 39, no. 24, 2025, pp. 25 697–25 705

    work page 2025

  38. [38]

    MTD: A Multimodal Dataset of Musical Themes for MIR Research,

    F. Zalkow, S. Balke, V . Arifi-Müller, and M. Müller, “MTD: A Multimodal Dataset of Musical Themes for MIR Research,”Transactions of the International So- ciety for Music Information Retrieval (TISMIR), vol. 3, no. 1, pp. 180–192, 2020

    work page 2020

  39. [39]

    ccMixter,

    “ccMixter,” Creative Commons music remix commu- nity and dataset source, 2004. [Online]. Available: https://ccmixter.org/

    work page 2004

  40. [40]

    Enabling Factorized Piano Music Modeling and Generation with the MAESTRO Dataset,

    C. Hawthorne, A. Stasyuk, A. Roberts, I. Simon, C.- Z. A. Huang, S. Dieleman, E. Elsen, J. Engel, and D. Eck, “Enabling Factorized Piano Music Modeling and Generation with the MAESTRO Dataset,” inIn- ternational Conference on Learning Representations (ICLR), 2019

    work page 2019

  41. [41]

    Estimation using log- spectral-distance criterion for noise-robust speech recognition,

    A. Erell and M. Weintraub, “Estimation using log- spectral-distance criterion for noise-robust speech recognition,” inInternational conference on acoustics, speech, and signal processing. IEEE, 1990, pp. 853– 856

    work page 1990

  42. [42]

    V oiceFixer: Toward general speech restoration with neural vocoder,

    H. Liu, Q. Kong, Q. Tian, Y . Zhao, D. Wang, C. Huang, and Y . Wang, “V oiceFixer: Toward general speech restoration with neural vocoder,”arXiv preprint arXiv:2109.13731, 2021

    Pith/arXiv arXiv 2021

  43. [43]

    ViSQOL v3: An open source production ready objective speech and audio metric,

    M. Chinen, F. S. Lim, J. Skoglund, N. Gureev, F. O’Gorman, and A. Hines, “ViSQOL v3: An open source production ready objective speech and audio metric,” in2020 twelfth international conference on quality of multimedia experience (QoMEX). IEEE, 2020, pp. 1–6

    work page 2020

  44. [44]

    ETTA: Elucidating the Design Space of Text-to-Audio Models,

    S.-G. Lee, Z. Kong, A. Goel, S. Kim, R. Valle, and B. Catanzaro, “ETTA: Elucidating the Design Space of Text-to-Audio Models,” inInternational Conference on Machine Learning. PMLR, 2025, pp. 33 157– 33 180

    work page 2025

  45. [45]

    Audit: Audio editing by following instructions with latent diffusion models,

    Y . Wang, Z. Ju, X. Tan, L. He, Z. Wu, J. Bianet al., “Audit: Audio editing by following instructions with latent diffusion models,”Advances in Neural Informa- tion Processing Systems, vol. 36, pp. 71 340–71 357, 2023

    work page 2023

  46. [46]

    eDiff-I: Text- to-Image Diffusion Models with Ensemble of Expert Denoisers,

    Y . Balaji, S. Nah, X. Huang, A. Vahdat, J. Song, Q. Zhang, K. Kreis, M. Aittala, T. Aila, S. Laine, B. Catanzaro, T. Karras, and M.-Y . Liu, “eDiff-I: Text- to-Image Diffusion Models with Ensemble of Expert Denoisers,”arXiv preprint arXiv:2211.01324, 2022

    Pith/arXiv arXiv 2022

  47. [47]

    Solving Audio Inverse Problems with a Diffusion Model,

    E. Moliner Juanpere, J. Lehtinen, and V . Välimäki, “Solving Audio Inverse Problems with a Diffusion Model,” inICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Process- ing (ICASSP). IEEE, 2023, pp. 1–5

    work page 2023

  48. [48]

    Speech Enhancement and Derever- beration with Diffusion-based Generative Models,

    J. Richter, S. Welker, J.-M. Lemercier, B. Lay, and T. Gerkmann, “Speech Enhancement and Derever- beration with Diffusion-based Generative Models,” IEEE/ACM Transactions on Audio, Speech, and Lan- guage Processing, vol. 31, pp. 2351–2364, 2023

    work page 2023

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.