Pith. sign in

REVIEW 4 major objections 3 minor 1 cited by

Learning to Upsample and Upmix Audio in the Latent Domain

T0 review · 4 major / 3 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper establishes that audio-to-audio operations can be trained and run entirely inside a frozen autoencoder's latent space, matching raw-waveform post-processing at roughly 100–200× lower FLOPs.

desk verdict A useful latent-domain audio processing recipe with a real training simplicity win, but the headline efficiency numbers overstate end-to-end gains. read the letter →

arxiv 2506.00681 v2 pith:XBAZYRAJ submitted 2025-05-31 cs.SD cs.LGeess.AS

Dimitrios Bralios , Paris Smaragdis , Jonah Casebeer This is my paper
classification cs.SDcs.LGeess.AS
keywords latentaudioprocessingbandwidthextensionmono-to-stereoupmixingneuralautoencodersadversarialdiscriminatorvariationalconditioningefficientpipelinesL1reconstruction
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Neural audio autoencoders compress sound into small latent codes, but most audio enhancement tasks still decode to raw waveforms and process there, which is expensive. This paper proposes doing the processing entirely in the latent space of a pretrained, frozen autoencoder, training only a lightweight module with a plain L1 loss on the latent codes, optionally aided by a single latent discriminator or a variational control branch. In bandwidth extension and mono-to-stereo upmixing, the latent module matches or beats raw-audio baselines that operate on autoencoder-transmitted audio, while needing about 100–200× fewer FLOPs at inference. The result matters for pipelines that already encode or decode audio with an autoencoder, because enhancement can then happen on the compressed representation instead of the waveform.

What carries the argument

The central object is the Re-Encoder, a task-specific feedforward module $F_\theta$ that maps an input latent $z_{\mathrm{in}} = A_E(x_{\mathrm{in}})$ to a target code $\hat{z}_{\mathrm{tgt}}$, trained in the latent domain with the reconstruction loss $\mathcal{L}_{\mathrm{rec}} = \mathbb{E}[\|A_E(x_{\mathrm{tgt}}) - F_\theta(A_E(x_{\mathrm{in}}), c)\|_1]$. The pretrained autoencoder $(A_E, A_D)$ is frozen; only $F_\theta$ is trained, so no decoder pass is needed during training. A latent adversarial discriminator (least-squares GAN plus feature matching) is added for bandwidth extension, and a variational encoder $G_\phi$ producing a sampled control vector $c$ from $\mathcal{N}(\mu, \sigma)$ is added for mono-to-stereo controllability. The modules are small stacked-convolution blocks, giving the 0.4–1.6 GFLOP/s operating range that drives the efficiency result.

What would settle it

A direct test would measure the correlation between latent L1 distance and waveform-level STFT/mel distance across many real audio pairs under the frozen autoencoder; if there are common pairs with near-zero latent distance but large spectral or perceptual distance, the latent-only loss can be minimized without improving what listeners hear. The paper's own comparison showing that adding a decoder-propagated mel loss improves STFT distance (row 7 versus row 6 of Table 1) already indicates the surrogate is not exact.

Watch

Extended reading notes

Core claim

The central claim is that a pretrained audio autoencoder's latent space is itself a sufficient domain for audio-to-audio processing: target latents can be predicted from input latents directly. With latent L1 reconstruction alone, a 4.3M-parameter module reaches 0.4 GFLOP/s and 1.38 STFT distance on bandwidth extension, improving on the transmitted raw-audio baselines (1.43 and 1.47) in the paper's simulated transmission scenario, and approaching the autoencoder's own reconstruction ceiling of 1.13. For mono-to-stereo, sampling a conditioning vector from a latent variational encoder yields 1.31 STFT distance versus 1.34 for the transmitted raw baseline, and an oracle conditioning vector improves this to 1.22, while needing 138× fewer FLOPs. The authors interpret these results as evidence that L1 proximity in the latent code corresponds to waveform perceptual similarity, which is why such a simple latent-only objective suffices.

Load-bearing premise

The load-bearing premise is that L1 distance in the frozen autoencoder's latent space is a faithful proxy for perceptual and spectral distance between decoded waveforms; if that correspondence fails, the latent-only objective will not optimize audible quality and the method's quality claim collapses.

Editorial extensions

If this is right

  • Audio enhancement tasks that can be framed as input-latent to target-latent prediction can reuse an existing autoencoder and a lightweight head; the paper demonstrates this for bandwidth extension and mono-to-stereo upmixing.
  • Training is simplified to a latent L1 term plus at most one latent discriminator, removing multi-scale waveform losses and decoder backpropagation, and completing in under two days on a single GPU.
  • Inference FLOPs drop from 85–222 GFLOP/s for the raw-audio baselines to 0.4–1.6 GFLOP/s for the latent modules, with quality at or above the baselines in the autoencoder-transmission scenario.
  • A single latent discriminator recovers most of the benefit that a decoder-propagated mel loss would provide, closing the gap from 1.41 to 1.38 STFT distance in bandwidth extension.
  • The variational conditioning branch makes mono-to-stereo output controllable and diverse: interpolating the control vector $c$ between a prior sample and the ground-truth stereo code changes the output channel-energy ratio in a nearly linear way.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the latent-smoothness assumption transfers to other audio autoencoders, the same recipe should apply to tasks such as speech enhancement, source separation, or dereverberation, with the latent L1 surrogate as the main property to validate for each new codec.
  • The 100–200× FLOP figure compares only the processing module; for audio that is not already in latent form, the fixed encode/decode cost of the autoencoder (about 51 GFLOP/s per second of audio in the tables) must be included, so the net advantage is largest in pipelines that cross the autoencoder bottleneck once.
  • The control-vector interpolation experiment measures channel energy ratios, not perceived spatial quality; a listening test on interpolated or style-transferred control vectors would test whether the variational branch controls spatial attributes perceptually, not just statistically.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 3 minor

Summary. The paper proposes Re-Encoder, a framework for audio-to-audio processing in which a lightweight ConvNeXt-V2 module operates entirely on latent representations of a frozen, pre-trained VOCOS autoencoder. For bandwidth extension, training uses a latent L1 reconstruction loss optionally augmented by a latent discriminator with feature matching; for mono-to-stereo upmixing, a variational conditioning encoder provides a control vector. The experiments compare against raw-audio baselines (Aero and MusicHiFi) in a simulated VAE transmission scenario and report STFT/mel distances and FLOPs, claiming up to 100x or 200x efficiency gains while maintaining quality comparable to raw-audio post-processing.

Significance. If the claims are properly scoped, this is a useful contribution: latent-domain training removes decoder passes from the training loop, simplifies the objective to L1 plus a single discriminator, and can replace expensive raw-audio modules in pipelines that already use an autoencoder. The transmission-scenario comparison is a legitimate experimental design, and the inclusion of a latent discriminator and a variational control mechanism goes beyond a simple reimplementation. The central ideas are clear and the empirical comparison is informative. However, the paper's headline efficiency and quality claims are broader than the experiments support, and several load-bearing claims need additional evidence or restatement.

major comments (4)
  1. [Abstract; §4.1; Table 1; Table 2] The claimed 'up to 100x' or '200x' efficiency gain is computed from module-only FLOPs: 0.4 GFLOPS for the small BWE module versus 85/111 GFLOPS for the raw baselines (Table 1 rows 2, 3, 9). In the transmission scenario described in §3.3.1, both the baseline and the proposed method require a VAE decode to produce audible output, and the paper's own Table 1 row 1 gives the VAE cost as 51 GFLOPS. Including this shared cost, the BWE end-to-end comparison is roughly (51+85)=136 GFLOPS for VAE+Aero versus (51+0.4)=51.4 GFLOPS for the proposed small module, a factor of about 2.6x, not 100x; for M2S the corresponding factor is about 5x. The abstract and conclusion should either report end-to-end FLOPs or explicitly justify why the mandatory decoder pass is excluded from the cost model.
  2. [Abstract; §4.1; Table 1; §4.2; Table 2] The abstract and introduction claim quality 'comparable to post-processing on raw audio,' but the clean raw baselines perform substantially better than the proposed method: Table 1 rows 2 and 3 report 0.94 and 0.99 STFT-D for Aero and MusicHiFi without the VAE transmission step, while the proposed models achieve 1.35-1.41 STFT-D (rows 6-9); Table 2 rows 2 and 4-5 show 0.85 versus 1.22-1.31 for M2S. The experiments support a more limited claim: latent-only training beats or matches the VAE-degraded raw baselines (1.38 vs 1.43/1.47 in Table 1 rows 8-9 vs 4-5; 1.31 vs 1.34 in Table 2 rows 4 vs 3). The wording should be scoped to the transmission scenario.
  3. [§2.1; Eq. (1); §3.1.1] The method relies on the assertion that L1 distance in the frozen latent space corresponds to waveform perceptual similarity (§2.1), but this property is not verified. The authors do not provide evidence that latent L1 correlates with the STFT/mel distances used in evaluation under the perturbations relevant to BWE and M2S, and the autoencoder-agnostic claim in §1 is supported by experiments on only one VOCOS variant. Adding a latent-metric smoothness analysis or at least a second autoencoder would strengthen the central claim that a pure latent L1 objective is a reliable proxy for audio quality.
  4. [§4.1; Table 1; Table 2] Several claims of superiority rest on small metric gaps, e.g., 1.38 vs 1.43/1.47 STFT-D in Table 1 rows 8-9 versus rows 4-5, and 1.31 vs 1.34 in Table 2 row 4 versus row 3. No confidence intervals, standard deviations over the evaluation subset, or significance tests are provided. Given the size of these differences, the conclusion that the latent modules 'surpass' the VAE-degraded baselines should be supported by variability estimates or a larger evaluation set.
minor comments (3)
  1. [Abstract; §5] The abstract reports 'up to 100x' efficiency gains while the conclusion reports 'up to a 200x speedup'; these numbers should be reconciled.
  2. [Eq. (3)] The feature matching loss divides by ||D_i(F_theta(z_in))||_1; consider adding a small epsilon to the denominator to avoid numerical instability if a feature map is near zero.
  3. [Fig. 1] The caption lists 'Bottom' before 'Top' in a way that may not match the figure layout; please ensure the caption labels correspond to the actual placement of the two panels.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central claims rest on external benchmarks and generic reconstruction/adversarial losses, not on fitted quantities or self-citations.

full rationale

The paper's derivation chain is empirical rather than definitional. Equation (1) trains F_theta to minimize ||AE(xtgt) - F_theta(AE(xin), c)||_1; this is the intended objective, not a quantity fitted to the evaluation metric. The target and input latents both come from the same frozen VAE, but that is the explicitly stated design of a Re-Encoder and not a hidden reuse of the output as an input. The latent-space smoothness premise ('proximity corresponds to waveform perceptual similarity') is an assumption, but the paper validates it indirectly through STFT/mel comparisons against external baselines Aero and MusicHiFi, so it is empirically checkable rather than true by construction. No uniqueness theorem is imported from the authors' prior work; the only self-citation, [33], is used as an example of fine-tuning codecs for enhancement and is not load-bearing. The one notable weakness, the headline 100-200x efficiency claim, compares only task-module FLOPs and omits the mandatory VAE decode pass (e.g., the 51 GFLOPS VAE rows in Tables 1 and 2); this is an accounting/correctness caveat, not circularity, because the reported numbers are measured rather than derived from the method's own assumptions. Accordingly, no circular step can be quoted and reduced, and the score is 0.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The framework does not introduce new physical entities or parameters fitted to the evaluation benchmark. The listed free parameters are training loss weights; the listed axioms are domain assumptions about the VAE latent and evaluation metrics.

free parameters (2)
  • BWE loss weights = 10.0 (L1), 0.5 (adversarial), 1.0 (feature matching)
    Hand-chosen scalar weights for the latent L1, adversarial, and feature-matching terms (Sec 3.2). They are not fitted to the evaluation metric, but the claimed recipe depends on them.
  • M2S loss weights = 10.0 (L1), 5e-4 (KL)
    Hand-chosen scalar weights for the latent L1 and KL terms (Sec 3.2).
assumptions (4)
  • domain assumption Latent L1 distance approximates waveform perceptual similarity
    Explicitly stated in Sec 2.1: 'proximity corresponds to waveform perceptual similarity.' This is the core premise enabling latent-only training.
  • domain assumption The frozen VAE reconstruction error is the quality ceiling
    The paper treats the VAE's full-band reconstruction error (1.13 STFT-D) as the empirical upper bound on any method whose output is decoded by that VAE (Table 1 row 1).
  • domain assumption STFT and mel distances are adequate proxies for audio quality
    All quality claims are evaluated with STFT/mel distance from the auraloss library; no listening tests are reported (Sec 3.3.2).
  • domain assumption A single 64-dim Gaussian sample at inference covers the target stereo distribution
    The variational M2S model samples c from N(0,I) for blind upmixing; this assumes the posterior over spatial images is well approximated by a Gaussian with a 64-dim latent (Sec 2.3).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Learning to Upsample and Upmix Audio in the Latent Domain." pith.science (2026). https://pith.science/paper/XBAZYRAJ

@misc{pith2026250600681,
  author       = {Pith},
  title        = {Pith review of: Learning to Upsample and Upmix Audio in the Latent Domain},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XBAZYRAJ}},
  note         = {Machine review of arXiv:2506.00681}
}
read the original abstract

Neural audio autoencoders create compact latent representations that preserve perceptually important information, serving as the foundation for both modern audio compression systems and generation approaches like next-token prediction and latent diffusion. Despite their prevalence, most audio processing operations, such as spatial and spectral up-sampling, still inefficiently operate on raw waveforms or spectral representations rather than directly on these compressed representations. We propose a framework that performs audio processing operations entirely within an autoencoder's latent space, eliminating the need to decode to raw audio formats. Our approach dramatically simplifies training by operating solely in the latent domain, with a latent L1 reconstruction term, augmented by a single latent adversarial discriminator. This contrasts sharply with raw-audio methods that typically require complex combinations of multi-scale losses and discriminators. Through experiments in bandwidth extension and mono-to-stereo up-mixing, we demonstrate computational efficiency gains of up to 100x while maintaining quality comparable to post-processing on raw audio. This work establishes a more efficient paradigm for audio processing pipelines that already incorporate autoencoders, enabling significantly faster and more resource-efficient workflows across various audio tasks.

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Re-Bottleneck: Latent Re-Structuring for Neural Audio Autoencoders

    cs.SD 2025-07 conditional novelty 5.0 of 10

    A post-hoc Re-Bottleneck network trained only in latent space can impose ordering, semantic alignment, or equivariance on pre-trained audio autoencoder latents with little extra compute.

Reference graph

Works this paper leans on

40 extracted references · 36 canonical work pages · cited by 1 Pith paper

  1. [1]

    Soundstream: An end-to-end neural audio codec,

    N. Zeghidour, A. Luebs, A. Omran, J. Skoglund, and M. Tagliasacchi, “Soundstream: An end-to-end neural audio codec,”IEEE/ACM Trans. Audio, Speech, Lang. Process., vol. 30, pp. 495–507, 2021

    work page 2021

  2. [2]

    High fidelity neural audio compression,

    A. D ´efossez, J. Copet, G. Synnaeve, and Y . Adi, “High fidelity neural audio compression,”Transactions on Machine Learning Research, 2023

    2023

  3. [3]

    High- fidelity audio compression with improved rvqgan,

    R. Kumar, P. Seetharaman, A. Luebs, I. Kumar, and K. Kumar, “High- fidelity audio compression with improved rvqgan,”Proc. NeurIPS, vol. 36, 2024

    work page 2024

  4. [4]

    Rave: A variational autoencoder for fast and high-quality neural audio synthesis,

    A. Caillon and P. Esling, “Rave: A variational autoencoder for fast and high-quality neural audio synthesis,”arXiv preprint arXiv:2111.05011, 2021

    arXiv 2021

  5. [5]

    Audioldm: Text-to-audio generation with latent diffusion models,

    H. Liu, Z. Chen, Y . Yuan, X. Mei, X. Liu, D. Mandic, W. Wang, and M. D. Plumbley, “Audioldm: Text-to-audio generation with latent diffusion models,” inProc. ICML. PMLR, 2023, pp. 21 450–21 474

    work page 2023

  6. [6]

    Make-an-audio: Text-to-audio generation with prompt-enhanced diffusion models,

    R. Huang, J. Huang, D. Yang, Y . Ren, L. Liu, M. Li, Z. Ye, J. Liu, X. Yin, and Z. Zhao, “Make-an-audio: Text-to-audio generation with prompt-enhanced diffusion models,” inProc. ICML. PMLR, 2023, pp. 13 916–13 932

    work page 2023

  7. [7]

    Stable audio open,

    Z. Evans, J. D. Parker, C. Carr, Z. Zukowski, J. Taylor, and J. Pons, “Stable audio open,” inProc. ICASSP. IEEE, 2025, pp. 1–5

    work page 2025

  8. [8]

    Fast timing- conditioned latent audio diffusion,

    Z. Evans, C. Carr, J. Taylor, S. H. Hawley, and J. Pons, “Fast timing- conditioned latent audio diffusion,” inProc. ICML. PMLR, 2024, pp. 12 652–12 665

    work page 2024

Show all 40 references
  1. [9]

    A review on score-based generative models for audio applications,

    G. Zhu, Y . Wen, and Z. Duan, “A review on score-based generative models for audio applications,”arXiv preprint arXiv:2506.08457, 2025

    2025

  2. [10]

    Audioldm 2: Learning holistic audio generation with self-supervised pretraining,

    H. Liu, Y . Yuan, X. Liu, X. Mei, Q. Kong, Q. Tian, Y . Wang, W. Wang, Y . Wang, and M. D. Plumbley, “Audioldm 2: Learning holistic audio generation with self-supervised pretraining,”IEEE/ACM Trans. Audio, Speech, Lang. Process., 2024

    2024

  3. [11]

    Audiogen: Textually guided audio generation,

    F. Kreuk, G. Synnaeve, A. Polyak, U. Singer, A. D ´efossez, J. Copet, D. Parikh, Y . Taigman, and Y . Adi, “Audiogen: Textually guided audio generation,” inProc. ICLR, 2023

    2023

  4. [12]

    Simple and controllable music generation,

    J. Copet, F. Kreuk, I. Gat, T. Remez, D. Kant, G. Synnaeve, Y . Adi, and A. D ´efossez, “Simple and controllable music generation,” inProc. NeurIPS, 2023

    2023

  5. [13]

    Audiolm: a language modeling approach to audio generation,

    Z. Borsos, R. Marinier, D. Vincent, E. Kharitonov, O. Pietquin, M. Sharifi, D. Roblek, O. Teboul, D. Grangier, M. Tagliasacchiet al., “Audiolm: a language modeling approach to audio generation,”IEEE/ACM Trans. Audio, Speech, Lang. Process., vol. 31, pp. 2523–2533, 2023

    2023

  6. [14]

    Musiclm: Generating music from text,

    A. Agostinelli, T. I. Denk, Z. Borsos, J. Engel, M. Verzetti, A. Caillon, Q. Huang, A. Jansen, A. Roberts, M. Tagliasacchiet al., “Musiclm: Generating music from text,”arXiv preprint arXiv:2301.11325, 2023

    2023 arXiv

  7. [15]

    Uniaudio: Towards universal audio generation with large language models,

    D. Yang, J. Tian, X. Tan, R. Huang, S. Liu, H. Guo, X. Chang, J. Shi, sheng zhao, J. Bian, Z. Zhao, X. Wu, and H. M. Meng, “Uniaudio: Towards universal audio generation with large language models,” inProc. ICML, 2024

    2024

  8. [16]

    Soundstorm: Efficient parallel audio generation,

    Z. Borsos, M. Sharifi, D. Vincent, E. Kharitonov, N. Zeghidour, and M. Tagliasacchi, “Soundstorm: Efficient parallel audio generation,”arXiv preprint arXiv:2305.09636, 2023

    2023 arXiv

  9. [17]

    Masked audio generation using a single non-autoregressive transformer,

    A. Ziv, I. Gat, G. L. Lan, T. Remez, F. Kreuk, J. Copet, A. D ´efossez, G. Synnaeve, and Y . Adi, “Masked audio generation using a single non-autoregressive transformer,” inProc. ICLR, 2024

    2024

  10. [18]

    Vampnet: Music generation via masked acoustic token modeling,

    H. Flores Garcia, P. Seetharaman, R. Kumar, and B. Pardo, “Vampnet: Music generation via masked acoustic token modeling,” inProc. ISMIR, 2023

    2023

  11. [19]

    Aero: Audio super resolution in the spectral domain,

    M. Mandel, O. Tal, and Y . Adi, “Aero: Audio super resolution in the spectral domain,” inProc. ICASSP. IEEE, 2023, pp. 1–5

    2023

  12. [20]

    Musichifi: Fast high- fidelity stereo vocoding,

    G. Zhu, J.-P. Caceres, Z. Duan, and N. J. Bryan, “Musichifi: Fast high- fidelity stereo vocoding,”IEEE Signal Process. Lett., 2024

    2024

  13. [21]

    Behm-gan: Bandwidth extension of historical music using generative adversarial networks,

    E. Moliner and V . V ¨alim¨aki, “Behm-gan: Bandwidth extension of historical music using generative adversarial networks,”IEEE/ACM Trans. Audio, Speech, Lang. Process., vol. 31, pp. 943–956, 2022

    2022

  14. [22]

    Blind audio bandwidth extension: A diffusion-based zero-shot approach,

    E. Moliner, F. Elvander, and V . V ¨alim¨aki, “Blind audio bandwidth extension: A diffusion-based zero-shot approach,”IEEE/ACM Trans. Audio, Speech, Lang. Process., 2024

    2024

  15. [23]

    Apollo: Band-sequence modeling for high-quality audio restoration,

    K. Li and Y . Luo, “Apollo: Band-sequence modeling for high-quality audio restoration,” inProc. ICASSP. IEEE, 2025, pp. 1–5

    2025

  16. [24]

    Multi-scale spectral loss revisited,

    S. Schw ¨ar and M. M ¨uller, “Multi-scale spectral loss revisited,”IEEE Signal Process. Lett., vol. 30, pp. 1712–1716, 2023

    2023

  17. [25]

    Audiosr: Versatile audio super-resolution at scale,

    H. Liu, K. Chen, Q. Tian, W. Wang, and M. D. Plumbley, “Audiosr: Versatile audio super-resolution at scale,” inProc. ICASSP. IEEE, 2024, pp. 1076–1080

    2024

  18. [26]

    Vector quantized diffusion model based speech bandwidth extension,

    Y . Fang, J. Bai, J. Wang, and X. Zhang, “Vector quantized diffusion model based speech bandwidth extension,” inProc. ICASSP. IEEE, 2025, pp. 1–5

    2025

  19. [27]

    Tokensplit: Using discrete speech representations for direct, refined, and transcript-conditioned speech separation and recognition,

    H. Erdogan, S. Wisdom, X. Chang, Z. Borsos, M. Tagliasacchi, N. Zeghi- dour, and J. Hershey, “Tokensplit: Using discrete speech representations for direct, refined, and transcript-conditioned speech separation and recognition,” inProc. Interspeech, 2023, pp. 3462–3466

    2023

  20. [28]

    Genhancer: High-fidelity speech enhancement via generative modeling on discrete codec tokens,

    H. Yang, J. Su, M. Kim, and Z. Jin, “Genhancer: High-fidelity speech enhancement via generative modeling on discrete codec tokens,” inProc. Interspeech, 2024, pp. 1170–1174

    2024

  21. [29]

    Towards audio codec-based speech separation,

    J. Q. Yip, S. Zhao, D. Ng, E. S. Chng, and B. Ma, “Towards audio codec-based speech separation,” inProc. Interspeech, 2024, pp. 2190– 2194

    2024

  22. [30]

    Speech enhancement using continuous embeddings of neural audio codec,

    H. Li, J. Q. Yip, T. Fan, and E. S. Chng, “Speech enhancement using continuous embeddings of neural audio codec,” inProc. ICASSP. IEEE, 2025, pp. 1–5

    2025

  23. [31]

    Speech separation using neural audio codecs with embedding loss,

    J. Q. Yip, C. Y . Kwok, B. Ma, and E. S. Chng, “Speech separation using neural audio codecs with embedding loss,” inProc. APSIPA ASC. IEEE, 2024, pp. 1–6

    2024

  24. [32]

    High-fidelity music vocoder using neural audio codecs,

    L. A. Lanzend ¨orfer, F. Gr ¨otschla, M. Ungersb ¨ock, and R. Wattenhofer, “High-fidelity music vocoder using neural audio codecs,” inProc. ICASSP. IEEE, 2025, pp. 1–5

    2025

  25. [33]

    Enhancing into the codec: Noise robust speech coding with vector- quantized autoencoders,

    J. Casebeer, V . Vale, U. Isik, J.-M. Valin, R. Giri, and A. Krishnaswamy, “Enhancing into the codec: Noise robust speech coding with vector- quantized autoencoders,” inProc. ICASSP. IEEE, 2021, pp. 711–715

    2021

  26. [34]

    Melgan: Generative adversarial networks for conditional waveform synthesis,

    K. Kumar, R. Kumar, T. De Boissiere, L. Gestin, W. Z. Teoh, J. Sotelo, A. De Brebisson, Y . Bengio, and A. C. Courville, “Melgan: Generative adversarial networks for conditional waveform synthesis,”Proc. NeurIPS, vol. 32, 2019

    2019

  27. [35]

    Upmixing via style transfer: a variational autoencoder for disentangling spatial images and musical content,

    H. Yang, S. Wager, S. Russell, M. Luo, M. Kim, and W. Kim, “Upmixing via style transfer: a variational autoencoder for disentangling spatial images and musical content,” inProc. ICASSP. IEEE, 2022, pp. 426– 430

    2022

  28. [36]

    Least squares generative adversarial networks,

    X. Mao, Q. Li, H. Xie, R. Y . Lau, Z. Wang, and S. Paul Smolley, “Least squares generative adversarial networks,” inProc. ICCV, 2017, pp. 2794–2802

    2017

  29. [37]

    V ocos: Closing the gap between time-domain and fourier- based neural vocoders for high-quality audio synthesis,

    H. Siuzdak, “V ocos: Closing the gap between time-domain and fourier- based neural vocoders for high-quality audio synthesis,” inProc. ICLR, 2024

    2024

  30. [38]

    Convnext v2: Co-designing and scaling convnets with masked autoencoders,

    S. Woo, S. Debnath, R. Hu, X. Chen, Z. Liu, I. S. Kweon, and S. Xie, “Convnext v2: Co-designing and scaling convnets with masked autoencoders,” inProc. CVPR, 2023, pp. 16 133–16 142

    2023

  31. [39]

    Fma: A dataset for music analysis,

    M. Defferrard, K. Benzi, P. Vandergheynst, and X. Bresson, “Fma: A dataset for music analysis,” inProc. ISMIR, 2017

    2017

  32. [40]

    auraloss: Audio focused loss functions in pytorch,

    C. J. Steinmetz and J. D. Reiss, “auraloss: Audio focused loss functions in pytorch,” inDigital music research network one-day workshop (DMRN+ 15), 2020

    2020

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.