Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Inference-time Scaling for Diffusion-based Audio Super-resolution

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Generating 120 upsampled candidates and selecting the best one by a verifier improves audio super-resolution across speech, music, and sound effects.

desk verdict A genuine transfer of inference-time search to audio SR, but the headline gains are partly selection artifacts because the verifiers doing the selecting are the same models doing the evaluating. read the letter →

arxiv 2508.02391 v1 pith:NXYTOJOU submitted 2025-08-04 cs.SD cs.AIeess.AS

Yizhu Jin , Zhen Ye , Zeyue Tian , Haohe Liu , Qiuqiang Kong , Yike Guo , Wei Xue This is my paper
classification cs.SDcs.AIeess.AS
keywords audiosuper-resolutioninference-timescalingdiffusionmodelsverifier-guidedsearchbest-of-Nselectionrandomzero-orderverifierensembling
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Audio super-resolution is a one-to-many problem: one low-resolution clip can map to many plausible high-resolution versions, and a single diffusion pass samples one of them without control over which attributes survive. This paper argues that, instead of raising the number of denoising steps, one should spend extra compute on generating many candidates and selecting the best one with task-specific verifiers. Using the AudioSR diffusion model and 120 candidates per input, the authors report consistent gains, with speech upsampled from 4 kHz to 24 kHz improving by up to 9.70% in aesthetics, 5.88% in speaker similarity, 15.20% in word error rate, and 46.98% in spectral distance relative to the low-resolution input. The paper further shows that ensembling verifiers mitigates overfitting to any single scoring model, and that random search covers a wider candidate space than local zero-order search. If correct, this gives studios and post-production pipelines a training-free way to trade compute for audio quality.

What carries the argument

The mechanism is verifier-guided search over the latent noise space of a diffusion sampler. Random search draws $N=120$ fresh Gaussian noises, generates one high-resolution output per noise, and returns the candidate with the highest verifier score; zero-order search starts from one noise and iteratively probes $K=2$ neighbors at a fixed distance $\lambda=0.99$, pivoting to the best-scoring neighbor. Verifiers are pretrained, off-the-shelf models mapping a candidate waveform (plus optional conditioning such as a transcript or caption) to a scalar, with an ensemble variant that averages per-verifier ranks to avoid scale mismatches. Search-space range is measured by the average log-spectrogram distance between each candidate and the mean spectrogram of the candidate set, and per-bin sampling variance yields uncertainty maps over time-frequency regions.

What would settle it

On a held-out set of speech, music, and sound-effect clips, generate the full N=120 candidate pool and compare the verifier-selected top-1 against the candidate with the lowest true log-spectrogram distance to the ground-truth high-resolution signal; if the verifier's pick agrees with the ground-truth-best no more often than a random pick does, the reported gains are artifacts of verifier bias rather than genuine audio-quality improvements.

Watch

Extended reading notes

Core claim

The central claim is that verifier-guided best-of-N selection over the diffusion sampling noise space converts inference-time randomness from a nuisance into a resource. From the same low-resolution input, the method draws N independent initial noises, generates N high-resolution candidates with the pretrained AudioSR model, scores them with supervised verifiers (aesthetics, speaker similarity, word error rate, CLAP alignment, or a rank-averaged ensemble), and keeps the top-scoring one. The authors report that this consistently outperforms vanilla AudioSR across speech, music, and sound effects at both 4 kHz and 8 kHz input cutoffs, with the ensemble verifier plus random search giving the best overall trade-off at 4 kHz input. They also find that optimizing a single verifier can degrade other metrics beyond a compute budget of roughly $2^3$–$2^4$ candidates, and that ensemble ranking mitigates this verifier hacking.

Load-bearing premise

The load-bearing premise is that the off-the-shelf verifiers, adopted without calibration, rank audio candidates correctly in the 4 kHz and 8 kHz-to-24 kHz upsampled regime, even though that regime is far from their training data.

Editorial extensions

If this is right

  • Best-of-N selection recovers intelligibility and speaker identity that vanilla AudioSR degrades: for 4 kHz-to-24 kHz speech, WER improves by 15.20% and speaker similarity by 5.88% relative to the low-resolution input, while vanilla generation makes both worse.
  • Ensemble verifier plus random search is the recommended default configuration, giving the most balanced gains across metrics for all three audio types at 4 kHz input.
  • At 8 kHz input, zero-order search becomes competitive on spectral distance because the candidate space is narrower, suggesting the optimal search strategy depends on input bandwidth.
  • The measured LSD variance of the search space rises from speech to music to sound effects and is larger at 4 kHz than 8 kHz, meaning the stochasticity the method exploits is domain- and bandwidth-dependent.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the verifier rankings are biased toward their training distributions, part of the reported gain could be metric-specific rather than perceptual; a human listening test comparing selected versus vanilla outputs would settle whether the 120x compute buys audible quality.
  • The same verifier-guided best-of-N recipe could transfer to other diffusion audio tasks with available objective verifiers, such as text-to-audio generation or speech enhancement, where the one-to-many ambiguity is equally present.
  • The uncertainty maps suggest a testable extension: allocate more denoising steps or higher-order solvers to high-variance time-frequency bins rather than treating all regions uniformly, which could reduce the candidate budget needed for the same quality.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes inference-time scaling for diffusion-based audio super-resolution: instead of increasing DDIM steps, it generates N=120 candidate high-resolution outputs from AudioSR and selects the top-1 candidate according to task-specific verifiers (AES, WavLM speaker similarity, ASR word error rate, CLAP, and an ensemble), using either random search or zero-order search. The authors evaluate on curated 200-sample benchmarks for speech (VCTK), music (MusicCaps), and sound effects (ESC-50) at 4 kHz and 8 kHz input cutoff frequencies, upsampled to 24 kHz. They report improvements over vanilla AudioSR and over the LR input on the optimized metrics, analyze search-space variance, visualize uncertainty maps, and discuss verifier hacking. The central claim is that verifier-guided top-1 selection consistently improves audio SR quality, with the Ensemble Verifier plus Random Search providing the best overall trade-off.

Significance. If the central claim were supported by independent evidence, this would be a useful contribution: vanilla AudioSR is shown to degrade attributes such as speaker similarity, intelligibility, and CLAP alignment, and inference-time selection is a practical way to recover them without retraining. The paper is among the first to transfer inference-time scaling ideas from LLMs and image diffusion to audio SR, and it includes useful auxiliary analyses such as verifier ensembling, search-space variance estimation, and uncertainty visualization. However, the reported evidence is currently weakened by a circularity between the search objectives and the evaluation metrics, and by the complete absence of error bars or significance tests. The contribution is therefore conditional on the authors providing a non-circular evaluation, such as human listening or metrics not used in the search, together with statistical grounding.

major comments (4)
  1. [§3.2, §4.3, Table 1, Figure 3] The supervised verifiers used for selection (AES, WavLM SpkSim, ASR WER, CLAP) are the same model families as the evaluation metrics. For any scoring function V, selecting argmax_i V(x_i) over N=120 independent samples mechanically raises the expected maximum relative to a single random sample, even if V is only weakly related to true perceptual quality. The one metric that is not the objective of the supervised runs, LSD, improves only slightly in Table 1 (e.g., speech 4 kHz: 1.73 to 1.67), so the non-circular evidence for the central claim is thin. The authors should add an independent judge, such as human listening ratings or metrics not used in search, or at minimum a calibration study showing that the verifiers rank 4 kHz and 8 kHz upsampled audio in the same order as human judgments. Without this, the reported 'consistent performance gains' may be a selection artifact.
  2. [Tables 1, 2, and 3] No confidence intervals, standard errors, or significance tests are reported anywhere. Several claimed differences between configurations are very small, such as the 0.08 AES difference between AES Verifier+Random and Ensemble Verifier+Random for 4 kHz music (7.05 vs. 6.97) or the 0.001-level CLAP differences at 8 kHz for sound effects. With 200 evaluation samples, these differences could easily be noise. The authors should report per-sample distributions, standard errors, and paired significance tests (or equivalent) for the main comparisons in Table 1, and in particular for the claims in §4.3 about verifier-algorithm-task alignment and about zero-order search being better for LSD at 8 kHz.
  3. [§3.3, §4.2] The zero-order search hyperparameters are fixed at λ=0.99, K=2, and N=120 without any sensitivity analysis. The conclusions that zero-order search achieves better LSD at 8 kHz and that music prefers the AES verifier while sound effects prefer CLAP at 8 kHz rest on differences in Table 1 that may depend on these specific hyperparameter choices. The authors should provide a sensitivity sweep over λ and K, and ideally over N, to show that the qualitative findings are robust.
  4. [§4.1] The evaluation benchmarks are constructed with procedures that are themselves not validated: MusicCaps is filtered by keyword matching on captions, and ESC-50 captions are generated by Qwen2-Audio rather than written by humans. Since CLAP scores are both a search objective and an evaluation metric, the validity of the CLAP results depends directly on the quality of these captions. The authors should provide a human-validated subset, report caption correctness, or otherwise show that the automatically generated captions are trustworthy enough to support the claimed CLAP improvements.
minor comments (5)
  1. [Abstract] The abstract states an improvement of '46.98% in spectral distance,' but this number is a relative reduction from the LR input, not from the vanilla AudioSR baseline. The baseline should be stated explicitly to avoid misleading readers.
  2. [Figure 2 and Figure 3] Figure 2 reports enhancement relative to the LR input, while Figure 3 reports relative improvement over vanilla AudioSR. The text in §4.3 switches between these baselines; each figure caption and the surrounding text should state the baseline explicitly.
  3. [Equation (3)] The definition of Sλ_{n,i} as the set {y : d(y,n)=λ} describes a sphere, not a neighborhood of size K, and the index i on the left-hand side is unused. The sampling procedure for the K neighbors should be clarified.
  4. [Algorithm 2] The notation ϵ(i) is used both for the k-th neighbor sample and for the selected neighbor on line 9, which is confusing. Rename the selected index and sample to avoid ambiguity.
  5. [Table 2] The layout of Table 2, with the audio types repeated across columns, is difficult to read; a tidier arrangement would improve clarity.

Circularity Check

2 steps flagged · score 6.0 of 10

Reported gains on AES/SpkSim/WER/CLAP are order-statistic effects because the verifiers used for search are identical to the evaluation metrics; partial circularity remains despite some independent LSD evidence.

  1. self definitional [Section 4.3, 'Effectiveness of Scaling Inference-Time Compute' paragraph]
    "Crucially, we find that when the verifier used for search directly aligns with the evaluation metric—such as using the Speaker Similarity Verifier to optimize Speaker Similarity, or the CLAP Verifier to improve CLAP Score—Random Search consistently outperforms Zero-Order Search in recovering the lost performance."

    Section 3.2 defines the supervised verifiers as WavLM-based Speaker Similarity, ASR-based WER, AudioBox-Aesthetics, and CLAP, and Table 1 evaluates exactly AES, SpkSim, WER, and CLAP. Algorithm 1 selects x* = argmax V(x_i). For the direct-verifier rows, the reported metric is the search objective itself, so the selected candidate's score is the maximum over N=120 draws and exceeds a single vanilla draw by construction (order statistics). Presenting these mechanically forced increases as evidence that inference-time scaling 'recovers lost performance' is circular: the metric being reported was the metric being maximized. The only columns not forced in this way are LSD and cross-metric effects, which the paper itself treats separately as verifier hacking.

  2. fitted input called prediction [Table 1 caption; Section 4.3, 'Verifier-Algorithm-Task Alignment']
    "The Ensemble Verifier aggregates AES, SpkSim, WER scores for speech, and AES, CLAP scores for music and sound effects."

    The Ensemble Verifier's selection score is the average rank of AES, SpkSim, WER (speech) or AES, CLAP (music/sound), and the same metrics are then used as the evaluation columns in Table 1. Claiming that the Ensemble Verifier 'consistently yields the best trade-off' on these metrics is therefore partially a restatement of the optimization objective: the selection rule is defined as maximizing the average rank of the reported metrics. A held-out judge not included in the verifier set—human ratings or a different metric family—would be required to confirm that the reported trade-off reflects genuine perceptual improvement rather than optimization of the evaluation metric.

full rationale

The paper's central claim has real non-circular content: LSD improvements in Table 1 are not directly optimized by the supervised search verifiers, and the verifier-hacking analysis (e.g., optimizing LSD or AES does not improve WER) is an empirical, non-tautological finding. The self-citations to AudioSR and related audio-generation papers are not load-bearing in a circular way: AudioSR is used as a fixed pretrained base model, and no uniqueness theorem is imported from the authors' own work. However, the headline attribute gains (AES, SpkSim, WER, CLAP) are compromised by construction: the search verifiers defined in Section 3.2 are the same model families used as evaluation metrics in Table 1 and Figure 2/3, and Algorithm 1 explicitly selects by argmax of the verifier score. For direct-verifier configurations, the reported improvement is an order-statistic effect of best-of-N selection, not independent evidence of quality. The ensemble configuration weakens but does not remove this issue, because its objective is a rank average of exactly the reported metrics. Thus the paper is partially circular: several 'predictions' reduce by construction, while the LSD results and cross-metric analyses provide independent but thinner support. Score 6 reflects this partial circularity, not an accusation of deliberate misrepresentation.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim rests on hand-chosen hyperparameters (N=120, K=2, lambda=0.99, no sensitivity analysis anywhere), on off-the-shelf verifiers assumed valid in the band-limited SR regime where they were not trained, and on curated 200-sample benchmarks with machine-generated captions. Every reported improvement on an optimized metric inherits uncertainty from the verifier-validity assumption, because the same models select and evaluate. No invented entities are introduced; the uncertainty map is a visualization technique, not a postulated entity. The search is an empirical procedure, not a derivation, so the ledger is short but every entry is load-bearing.

free parameters (5)
  • Number of candidates N = 120
    Fixed for every algorithm-verifier configuration (Sec. 4.2); no sensitivity analysis or compute-quality Pareto curve is reported.
  • Zero-order search distance lambda = 0.99
    Chosen without justification in Sec. 4.2; the distance metric d in Eq. 3 is never defined, so the neighborhood is unspecified.
  • Neighborhood size K = 2
    Set in Sec. 4.2 without a sweep; interacts with lambda to determine local exploration.
  • MusicCaps pre-filter keywords = mediocre, low quality, low fidelity
    Sec. 4.1 discards samples whose captions contain these strings, shaping the music benchmark before evaluation.
  • Evaluation subset size = 200 samples per domain
    Curated subsets of VCTK, MusicCaps, and ESC-50 (Sec. 4.1); composition and seeds are not released.
assumptions (4)
  • domain assumption AudioSR (latent diffusion + HiFiGAN vocoder) is a competent base SR model whose 50-step DDIM samples span a useful range of HR solutions
    The whole framework wraps AudioSR [16] as the generator (Sec. 3.1); if its samples are poor or low-diversity, verifier-guided search cannot recover quality.
  • domain assumption Verifiers (WavLM SpkSim, FunASR/Whisper WER, AudioBox AES, CLAP) rank audio quality faithfully on band-limited upsampled audio
    Sec. 3.2 adopts pretrained verifiers without calibration for 4/8 kHz-reconstructed audio, an out-of-distribution regime; all reported gains are scored by these same model families.
  • standard math Standard DDIM reverse sampling and classifier-free guidance theory
    Algorithms 1-2 and Sec. 3.1 rely on DDIM and CFG from [11, 12, 31] without proof or modification.
  • domain assumption Curated 200-sample benchmarks and machine-generated ESC-50 captions are representative and accurate
    Sec. 4.1 defines pre-filtering of MusicCaps entries containing 'mediocre', 'low quality', or 'low fidelity', and Qwen2-Audio captions for ESC-50; CLAP scores inherit the accuracy of these captions.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Inference-time Scaling for Diffusion-based Audio Super-resolution." pith.science (2026). https://pith.science/paper/NXYTOJOU

@misc{pith2026250802391,
  author       = {Pith},
  title        = {Pith review of: Inference-time Scaling for Diffusion-based Audio Super-resolution},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NXYTOJOU}},
  note         = {Machine review of arXiv:2508.02391}
}
read the original abstract

Diffusion models have demonstrated remarkable success in generative tasks, including audio super-resolution (SR). In many applications like movie post-production and album mastering, substantial computational budgets are available for achieving superior audio quality. However, while existing diffusion approaches typically increase sampling steps to improve quality, the performance remains fundamentally limited by the stochastic nature of the sampling process, leading to high-variance and quality-limited outputs. Here, rather than simply increasing the number of sampling steps, we propose a different paradigm through inference-time scaling for SR, which explores multiple solution trajectories during the sampling process. Different task-specific verifiers are developed, and two search algorithms, including the random search and zero-order search for SR, are introduced. By actively guiding the exploration of the high-dimensional solution space through verifier-algorithm combinations, we enable more robust and higher-quality outputs. Through extensive validation across diverse audio domains (speech, music, sound effects) and frequency ranges, we demonstrate consistent performance gains, achieving improvements of up to 9.70% in aesthetics, 5.88% in speaker similarity, 15.20% in word error rate, and 46.98% in spectral distance for speech SR from 4kHz to 24kHz, showcasing the effectiveness of our approach. Audio samples are available at: https://racerk.github.io/tt-scale-audiosr/.

Figures

Figures reproduced from arXiv: 2508.02391 by the authors.

Figure 1
Figure 1. Overview of our inference-time scaling framework for audio SR. Given a LR input, multiple [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Performance improvements over the LR input across different audio types using inference￾time Random Search with various verifiers from 4 kHz to 24 kHz. The enhancement denotes the relative improvement over LR under each evaluation metric. For Aesthetics (AES), Speaker Similarity (SpkSim) and CLAP Score, enhancements reflect relative increases. For Word Error Rate (WER) and Log Spectrogram Distance (LSD), improvement… view at source ↗
Figure 3
Figure 3. Relative performance improvements over the default generation (vanilla AudioSR) for [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Visualization of uncertainty maps over STFT [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Inference-Time Scaling for Joint Audio-Video Generation

    cs.MM 2026-06 unverdicted novelty 7.0 of 10

    Presents multi-verifier framework and Adaptive Reward Weighting (ARW) for inference-time scaling in joint audio-video generation, reporting gains in alignment and synchronization on VGGSound and JavisBench-mini.

Reference graph

Works this paper leans on

47 extracted references · 12 canonical work pages · cited by 1 Pith paper

  1. [21]

    Scaling inference time compute for diffusion models

    Nanye Ma, Shangyuan Tong, Haolin Jia, Hexiang Hu, Yu-Chuan Su, Mingda Zhang, Xuan Yang, Yandong Li, Tommi Jaakkola, Xuhui Jia, et al. Scaling inference time compute for diffusion models. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 2523–2534, 2025

    2025

  2. [1]

    Musiclm: Generating music from text.arXiv preprint arXiv:2301.11325, 2023

    Andrea Agostinelli, Timo I Denk, Zalán Borsos, Jesse Engel, Mauro Verzetti, Antoine Caillon, Qingqing Huang, Aren Jansen, Adam Roberts, Marco Tagliasacchi, et al. Musiclm: Generating music from text.arXiv preprint arXiv:2301.11325, 2023

    arXiv 2023

  3. [2]

    Large language monkeys: Scaling inference compute with repeated sampling.arXiv preprint arXiv:2407.21787, 2024

    Bradley Brown, Jordan Juravsky, Ryan Ehrlich, Ronald Clark, Quoc V Le, Christopher Ré, and Azalia Mirhoseini. Large language monkeys: Scaling inference compute with repeated sampling.arXiv preprint arXiv:2407.21787, 2024

    arXiv 2024

  4. [3]

    Musicldm: Enhancing novelty in text-to-music generation using beat-synchronous mixup strategies

    Ke Chen, Yusong Wu, Haohe Liu, Marianna Nezhurina, Taylor Berg-Kirkpatrick, and Shlomo Dubnov. Musicldm: Enhancing novelty in text-to-music generation using beat-synchronous mixup strategies. InICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1206–1210. IEEE, 2024

    work page 2024

  5. [4]

    Wavlm: Large-scale self-supervised pre- training for full stack speech processing.IEEE Journal of Selected Topics in Signal Processing, 16(6):1505–1518, 2022

    Sanyuan Chen, Chengyi Wang, Zhengyang Chen, Yu Wu, Shujie Liu, Zhuo Chen, Jinyu Li, Naoyuki Kanda, Takuya Yoshioka, Xiong Xiao, et al. Wavlm: Large-scale self-supervised pre- training for full stack speech processing.IEEE Journal of Selected Topics in Signal Processing, 16(6):1505–1518, 2022

    2022

  6. [5]

    Qwen2-audio technical report.arXiv preprint arXiv:2407.10759, 2024

    Yunfei Chu, Jin Xu, Qian Yang, Haojie Wei, Xipin Wei, Zhifang Guo, Yichong Leng, Yuan- jun Lv, Jinzheng He, Junyang Lin, et al. Qwen2-audio technical report.arXiv preprint arXiv:2407.10759, 2024

    arXiv 2024

  7. [6]

    Directly fine-tuning diffusion models on differentiable rewards.arXiv preprint arXiv:2309.17400, 2023

    Kevin Clark, Paul Vicol, Kevin Swersky, and David J Fleet. Directly fine-tuning diffusion models on differentiable rewards.arXiv preprint arXiv:2309.17400, 2023

    arXiv 2023

  8. [7]

    Clap learning audio concepts from natural language supervision

    Benjamin Elizalde, Soham Deshmukh, Mahmoud Al Ismail, and Huaming Wang. Clap learning audio concepts from natural language supervision. InICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEEE, 2023

    2023

Show all 47 references
  1. [8]

    Funasr: A fundamental end-to-end speech recognition toolkit.arXiv preprint arXiv:2305.11013, 2023

    Zhifu Gao, Zerui Li, Jiaming Wang, Haoneng Luo, Xian Shi, Mengzhe Chen, Yabin Li, Lingyun Zuo, Zhihao Du, Zhangyu Xiao, et al. Funasr: A fundamental end-to-end speech recognition toolkit.arXiv preprint arXiv:2305.11013, 2023

    2023 arXiv

  2. [9]

    Animatediff: Animate your personalized text-to-image diffusion models without specific tuning.arXiv preprint arXiv:2307.04725, 2023

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized text-to-image diffusion models without specific tuning.arXiv preprint arXiv:2307.04725, 2023. 10

    2023 arXiv

  3. [10]

    Nu-wave 2: A general neural audio upsampling model for various sampling rates.arXiv preprint arXiv:2206.08545, 2022

    Seungu Han and Junhyeok Lee. Nu-wave 2: A general neural audio upsampling model for various sampling rates.arXiv preprint arXiv:2206.08545, 2022

    2022 arXiv

  4. [11]

    Denoising diffusion probabilistic models.Advances in neural information processing systems, 33:6840–6851, 2020

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models.Advances in neural information processing systems, 33:6840–6851, 2020

    2020

  5. [12]

    Classifier-free diffusion guidance.arXiv preprint arXiv:2207.12598, 2022

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance.arXiv preprint arXiv:2207.12598, 2022

    2022 arXiv

  6. [13]

    Hifi-gan: Generative adversarial networks for efficient and high fidelity speech synthesis.Advances in neural information processing systems, 33:17022–17033, 2020

    Jungil Kong, Jaehyeon Kim, and Jaekyoung Bae. Hifi-gan: Generative adversarial networks for efficient and high fidelity speech synthesis.Advances in neural information processing systems, 33:17022–17033, 2020

    2020

  7. [14]

    Nu-wave: A diffusion probabilistic model for neural audio upsampling.arXiv preprint arXiv:2104.02321, 2021

    Junhyeok Lee and Seungu Han. Nu-wave: A diffusion probabilistic model for neural audio upsampling.arXiv preprint arXiv:2104.02321, 2021

    2021 arXiv

  8. [15]

    Noise-free optimization in early training steps for image super- resolution

    MinKyu Lee and Jae-Pil Heo. Noise-free optimization in early training steps for image super- resolution. InProceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 2920–2928, 2024

    2024

  9. [16]

    Audiosr: Versatile audio super-resolution at scale

    Haohe Liu, Ke Chen, Qiao Tian, Wenwu Wang, and Mark D Plumbley. Audiosr: Versatile audio super-resolution at scale. InICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1076–1080. IEEE, 2024

    2024

  10. [17]

    Audioldm: Text-to-audio generation with latent diffusion models.arXiv preprint arXiv:2301.12503, 2023

    Haohe Liu, Zehua Chen, Yi Yuan, Xinhao Mei, Xubo Liu, Danilo Mandic, Wenwu Wang, and Mark D Plumbley. Audioldm: Text-to-audio generation with latent diffusion models.arXiv preprint arXiv:2301.12503, 2023

    2023 arXiv

  11. [18]

    Neural vocoder is all you need for speech super-resolution.arXiv preprint arXiv:2203.14941, 2022

    Haohe Liu, Woosung Choi, Xubo Liu, Qiuqiang Kong, Qiao Tian, and DeLiang Wang. Neural vocoder is all you need for speech super-resolution.arXiv preprint arXiv:2203.14941, 2022

    2022 arXiv

  12. [19]

    V oicefixer: Toward general speech restoration with neural vocoder.arXiv preprint arXiv:2109.13731, 2021

    Haohe Liu, Qiuqiang Kong, Qiao Tian, Yan Zhao, DeLiang Wang, Chuanzeng Huang, and Yuxuan Wang. V oicefixer: Toward general speech restoration with neural vocoder.arXiv preprint arXiv:2109.13731, 2021

    2021 arXiv

  13. [20]

    Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps.Advances in Neural Information Processing Systems, 35:5775–5787, 2022

    Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps.Advances in Neural Information Processing Systems, 35:5775–5787, 2022

    2022

  14. [22]

    Uncertainty-driven loss for single image super-resolution.Advances in Neural Information Processing Systems, 34:16398–16409, 2021

    Qian Ning, Weisheng Dong, Xin Li, Jinjian Wu, and Guangming Shi. Uncertainty-driven loss for single image super-resolution.Advances in Neural Information Processing Systems, 34:16398–16409, 2021

    2021

  15. [23]

    The effects of reward misspecification: Mapping and mitigating misaligned models.arXiv preprint arXiv:2201.03544, 2022

    Alexander Pan, Kush Bhatia, and Jacob Steinhardt. The effects of reward misspecification: Mapping and mitigating misaligned models.arXiv preprint arXiv:2201.03544, 2022

    2022 arXiv

  16. [24]

    Esc: Dataset for environmental sound classification

    Karol J Piczak. Esc: Dataset for environmental sound classification. InProceedings of the 23rd ACM international conference on Multimedia, pages 1015–1018, 2015

    2015

  17. [25]

    Robust speech recognition via large-scale weak supervision

    Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. Robust speech recognition via large-scale weak supervision. InInternational conference on machine learning, pages 28492–28518. PMLR, 2023

    2023

  18. [26]

    Fastspeech 2: Fast and high-quality end-to-end text to speech.arXiv preprint arXiv:2006.04558, 2020

    Yi Ren, Chenxu Hu, Xu Tan, Tao Qin, Sheng Zhao, Zhou Zhao, and Tie-Yan Liu. Fastspeech 2: Fast and high-quality end-to-end text to speech.arXiv preprint arXiv:2006.04558, 2020

    2006 arXiv

  19. [27]

    Fastspeech: Fast, robust and controllable text to speech.Advances in neural information processing systems, 32, 2019

    Yi Ren, Yangjun Ruan, Xu Tan, Tao Qin, Sheng Zhao, Zhou Zhao, and Tie-Yan Liu. Fastspeech: Fast, robust and controllable text to speech.Advances in neural information processing systems, 32, 2019. 11

    2019

  20. [28]

    High- resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022

    2022

  21. [29]

    Naturalspeech 2: Latent diffusion models are natural and zero-shot speech and singing synthesizers.arXiv preprint arXiv:2304.09116, 2023

    Kai Shen, Zeqian Ju, Xu Tan, Yanqing Liu, Yichong Leng, Lei He, Tao Qin, Sheng Zhao, and Jiang Bian. Naturalspeech 2: Latent diffusion models are natural and zero-shot speech and singing synthesizers.arXiv preprint arXiv:2304.09116, 2023

    2023 arXiv

  22. [30]

    Scaling llm test-time compute opti- mally can be more effective than scaling model parameters.arXiv preprint arXiv:2408.03314, 2024

    Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling llm test-time compute opti- mally can be more effective than scaling model parameters.arXiv preprint arXiv:2408.03314, 2024

    2024 arXiv

  23. [31]

    Denoising diffusion implicit models.arXiv preprint arXiv:2010.02502, 2020

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models.arXiv preprint arXiv:2010.02502, 2020

    2010 arXiv

  24. [32]

    Score-based generative modeling through stochastic differential equations.arXiv preprint arXiv:2011.13456, 2020

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations.arXiv preprint arXiv:2011.13456, 2020

    2011 arXiv

  25. [33]

    Audiox: Diffusion transformer for anything-to-audio generation.arXiv preprint arXiv:2503.10522, 2025

    Zeyue Tian, Yizhu Jin, Zhaoyang Liu, Ruibin Yuan, Xu Tan, Qifeng Chen, Wei Xue, and Yike Guo. Audiox: Diffusion transformer for anything-to-audio generation.arXiv preprint arXiv:2503.10522, 2025

    2025 arXiv

  26. [34]

    Meta audiobox aesthetics: Unified automatic quality assessment for speech, music, and sound.arXiv preprint arXiv:2502.05139, 2025

    Andros Tjandra, Yi-Chiao Wu, Baishan Guo, John Hoffman, Brian Ellis, Apoorv Vyas, Bowen Shi, Sanyuan Chen, Matt Le, Nick Zacharov, et al. Meta audiobox aesthetics: Unified automatic quality assessment for speech, music, and sound.arXiv preprint arXiv:2502.05139, 2025

    2025 arXiv

  27. [35]

    Towards robust speech super-resolution.IEEE/ACM transactions on audio, speech, and language processing, 29:2058–2066, 2021

    Heming Wang and DeLiang Wang. Towards robust speech super-resolution.IEEE/ACM transactions on audio, speech, and language processing, 29:2058–2066, 2021

    2021

  28. [36]

    Esrgan: Enhanced super-resolution generative adversarial networks

    Xintao Wang, Ke Yu, Shixiang Wu, Jinjin Gu, Yihao Liu, Chao Dong, Yu Qiao, and Chen Change Loy. Esrgan: Enhanced super-resolution generative adversarial networks. InProceed- ings of the European conference on computer vision (ECCV) workshops, pages 0–0, 2018

    2018

  29. [37]

    Difix3d+: Improving 3d reconstructions with single-step diffusion models

    Jay Zhangjie Wu, Yuxuan Zhang, Haithem Turki, Xuanchi Ren, Jun Gao, Mike Zheng Shou, Sanja Fidler, Zan Gojcic, and Huan Ling. Difix3d+: Improving 3d reconstructions with single-step diffusion models. InProceedings of the Computer Vision and Pattern Recognition Conference, page...

    2025

  30. [38]

    Sana 1.5: Efficient scaling of training-time and inference-time compute in linear diffusion transformer.arXiv preprint arXiv:2501.18427, 2025

    Enze Xie, Junsong Chen, Yuyang Zhao, Jincheng Yu, Ligeng Zhu, Chengyue Wu, Yujun Lin, Zhekai Zhang, Muyang Li, Junyu Chen, et al. Sana 1.5: Efficient scaling of training-time and inference-time compute in linear diffusion transformer.arXiv preprint arXiv:2501.18427, 2025

    2025 arXiv

  31. [39]

    Flashspeech: Efficient zero-shot speech synthesis

    Zhen Ye, Zeqian Ju, Haohe Liu, Xu Tan, Jianyi Chen, Yiwen Lu, Peiwen Sun, Jiahao Pan, Weizhen Bian, Shulin He, et al. Flashspeech: Efficient zero-shot speech synthesis. InProceed- ings of the 32nd ACM International Conference on Multimedia, pages 6998–7007, 2024

    2024

  32. [40]

    Comospeech: One-step speech and singing voice synthesis via consistency model

    Zhen Ye, Wei Xue, Xu Tan, Jie Chen, Qifeng Liu, and Yike Guo. Comospeech: One-step speech and singing voice synthesis via consistency model. InProceedings of the 31st ACM International Conference on Multimedia, pages 1831–1839, 2023

    2023

  33. [41]

    Llasa: Scaling train-time and inference-time compute for llama-based speech synthesis.arXiv preprint arXiv:2502.04128, 2025

    Zhen Ye, Xinfa Zhu, Chi-Min Chan, Xinsheng Wang, Xu Tan, Jiahe Lei, Yi Peng, Haohe Liu, Yizhu Jin, Zheqi DAI, et al. Llasa: Scaling train-time and inference-time compute for llama-based speech synthesis.arXiv preprint arXiv:2502.04128, 2025

    2025 arXiv

  34. [42]

    Gan vocoder: Multi-resolution discriminator is all you need.arXiv preprint arXiv:2103.05236, 2021

    Jaeseong You, Dalhyun Kim, Gyuhyeon Nam, Geumbyeol Hwang, and Gyeongsu Chae. Gan vocoder: Multi-resolution discriminator is all you need.arXiv preprint arXiv:2103.05236, 2021

    2021 arXiv

  35. [43]

    Conditioning and sampling in variational diffusion models for speech super-resolution

    Chin-Yun Yu, Sung-Lin Yeh, György Fazekas, and Hao Tang. Conditioning and sampling in variational diffusion models for speech super-resolution. InICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEEE, 2023. 12

    2023

  36. [44]

    Uncertainty-guided perturbation for image super-resolution diffusion model

    Leheng Zhang, Weiyi You, Kexuan Shi, and Shuhang Gu. Uncertainty-guided perturbation for image super-resolution diffusion model. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 17980–17989, 2025

    2025

  37. [45]

    Flashvideo: Flowing fidelity to detail for efficient high-resolution video generation.arXiv preprint arXiv:2502.05179, 2025

    Shilong Zhang, Wenbo Li, Shoufa Chen, Chongjian Ge, Peize Sun, Yida Zhang, Yi Jiang, Zehuan Yuan, Binyue Peng, and Ping Luo. Flashvideo: Flowing fidelity to detail for efficient high-resolution video generation.arXiv preprint arXiv:2502.05179, 2025

    2025

  38. [46]

    Inference-time scaling of diffusion models through classical search.arXiv preprint arXiv:2505.23614, 2025

    Xiangcheng Zhang, Haowei Lin, Haotian Ye, James Zou, Jianzhu Ma, Yitao Liang, and Yilun Du. Inference-time scaling of diffusion models through classical search.arXiv preprint arXiv:2505.23614, 2025

    2025

  39. [47]

    In-context edit: Enabling instructional image editing with in-context generation in large scale diffusion transformer.arXiv preprint arXiv:2504.20690, 2025

    Zechuan Zhang, Ji Xie, Yu Lu, Zongxin Yang, and Yi Yang. In-context edit: Enabling instructional image editing with in-context generation in large scale diffusion transformer.arXiv preprint arXiv:2504.20690, 2025. 13

    2025 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.