Pith. sign in

REVIEW 3 major objections 4 minor 4 cited by

A2SB: Audio-to-Audio Schrodinger Bridges

T0 review · 3 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A single vocoder-free Schrödinger Bridge model claims state-of-the-art quality for both bandwidth extension and inpainting of 44.1kHz music, on out-of-distribution test sets.

desk verdict The short-segment SOTA claim is solid and the system is useful, but the hour-long restoration contribution is only supported by a memory benchmark, not by quality. read the letter →

arxiv 2501.11311 v2 pith:5P2XZYPN submitted 2025-01-20 cs.SD cs.LGeess.AS

Zhifeng Kong , Kevin J Shih , Weili Nie , Arash Vahdat , Sang-gil Lee , Joao Felipe Santos , Ante Jukic , Rafael Valle
show 1 more author
Bryan Catanzaro
This is my paper · ORCID
classification cs.SDcs.LGeess.AS
keywords audiorestorationSchrödingerBridgebandwidthextensioninpaintingdiffusionmodelsphaseestimationmusicvocoder-free
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that one end-to-end generative model can restore two kinds of degraded 44.1kHz music — missing high-frequency content (bandwidth extension) and missing time segments (inpainting) — directly to waveforms, with no vocoder or codec in the loop. The model, A2SB, casts both tasks as spectrogram inpainting inside the Schrödinger Bridge framework: high bands are gaps along the frequency axis, missing frames are gaps along the time axis. The paper claims A2SB beats state-of-the-art task-specific baselines on several out-of-distribution music test sets on both objective metrics and human listening scores, and that sliding-window fusion sampling lets it restore hour-long recordings. A sympathetic reader would care because most prior music restoration is task-specific, speech-oriented, or depends on a vocoder to convert restored magnitude spectra into audio.

What carries the argument

The load-bearing object is the masked Schrödinger Bridge: a forward process from clean audio $X_0$ to a corrupted sample $X_1$ defined as $X_0\odot(1-M)+\eta_{\mathrm{fill}}\odot M$, with the network scored only on the corrupted region, and sampling following the analytic Gaussian posterior of the bridge. The named mechanisms that make it work are the three-channel factorized representation (magnitude to the power $0.25$, cosine phase, sine phase, giving $X\in\mathbb{R}^{N\times W\times 3}$), the phase-orthogonalization projection $\mathrm{SVDO}_{+}$ that rescales each $(\cos\theta,\sin\theta)$ pair to unit norm as the least-squares nearest rotation, and MultiDiffusion sliding-window averaging (width $W$, hop $W/2$) that extends a model trained on 2.97-second segments to arbitrarily long audio. The $t$-range partitioning of the noise schedule into two or four expert denoisers is what the paper credits for most of the perceptual gains.

What would settle it

Restore a full-length, say 60-minute, recording, split the output into consecutive windows, and compare each window's spectral distance (LSD) and listening scores against the clean ground truth; if error grows monotonically with distance from the window boundaries, or if periodic artefacts appear at the hop-size cadence of the sliding windows, the long-audio contribution fails.

Watch

Extended reading notes

Core claim

The central claim is that degradation-inpainting in the spectrogram, learned as a Schrödinger Bridge between corrupted and clean audio, subsumes both bandwidth extension and audio inpainting in a single vocoder-free model. A2SB is trained on a three-channel factorized representation — power-compressed magnitude (exponent 0.25) plus the cosine and sine of the phase — and the paper argues this separation stabilizes learning compared with modeling the raw two-channel complex STFT, because magnitude varies smoothly across frequency while phase is periodic and unstable at low magnitudes. At synthesis time, whatever the network emits for the phase channels is orthogonally projected onto SO(2) via the SVD/Procrustes solution, guaranteeing a valid phase and invertible STFT. On the AAM, CCMixter, MTD, and Maestro test sets, A2SB reports the best bandwidth-extension and inpainting scores on SiSpec, ViSQOL, MIDI F1, and human MOS, and the paper concludes the 4-partitioned model has the best overall quality while the 2-partitioned model is the best cost-performance point.

Load-bearing premise

The hour-long restoration claim is backed only by a GPU-memory benchmark, not by any measurement of output quality, so the real assumption is that sliding-window averaging stays artefact-free over minute-scale horizons.

Editorial extensions

If this is right

  • One A2SB checkpoint performs both bandwidth extension at 4, 8, and 12 kHz cutoffs and inpainting of 300 ms to 1000 ms gaps at 44.1 kHz, replacing task-specific models.
  • Because restoration is end-to-end and preserves the observed phase, outputs are direct waveforms with the original intact phase retained, unlike magnitude-only methods that need a vocoder.
  • Splitting the noise schedule into expert denoisers (2- or 4-partitioning) improves objective and subjective quality, with the 2-partitioned model giving the best quality per compute.
  • MultiDiffusion sampling makes memory grow linearly with audio length, so over an hour of audio fits on a professional GPU and several minutes on a consumer card.
  • Generation quality is essentially flat down to 25 sampling steps, so fast inference is available without retraining.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same mask-centric bridge could plausibly absorb other degradations — denoising, declipping, source separation — by changing the mask pattern, which the paper lists as future work but does not demonstrate.
  • The magnitude/phase factorization with an SO(2) projection is a modular trick that could transfer to other phase-sensitive generative audio tasks, such as speech enhancement or codec post-filtering, where part of the phase is observed.
  • A direct test of the long-audio claim would be telling: if segment-level quality degrades with overall track length, the sliding-window fusion, not the bridge itself, is the bottleneck.
  • The out-of-distribution gains are on benchmark collections; a stronger generalization test would be uploading user-recorded or streaming-quality music with unknown degradations.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes A2SB, an end-to-end, vocoder-free Schrödinger bridge model for 44.1kHz music restoration that handles both bandwidth extension and inpainting with a single model. The method trains on a factorized magnitude-phase STFT representation with power compression and trigonometric phase encoding, applies SVD-based phase orthogonalization at synthesis time, and uses a MultiDiffusion-style sliding-window sampler for arbitrarily long audio. The authors curate a 2.3K-hour permissively licensed music dataset, adopt a two-stage training procedure with t-range partitioning, and evaluate against AudioSR, MAID, CQTDiff, and an instruction-based baseline (IBAR) on three out-of-distribution music test sets plus Maestro. The main empirical claims are consistent improvements over baselines in SiSpec, ViSQOL, and human MOS for both tasks, with ablations supporting the factorized representation, phase orthogonalization, and t-range partitioning.

Significance. If the results hold, A2SB is a meaningful step forward: it is the first vocoder-free single model to address both bandwidth extension and inpainting at 44.1kHz, it reports consistent SOTA-level quality on OOD music test sets with human MOS evidence, and it ships code, checkpoints, and a demo website, which aids reproducibility. The factorized magnitude-phase representation and the phase-orthogonalization safeguard are simple and well-motivated, and the t-range partitioning and MultiDiffusion adaptations are practical contributions that can transfer to other audio restoration models. The main caveat is that the 'hour-long audio' contribution is currently supported only by a GPU-memory plot, not by any quality evaluation of long outputs.

major comments (3)
  1. [Section 4.7, Figure 7; Contribution 4 (Section 1); Abstract] The central claim that A2SB 'can coherently restore hour-long audio without boundary artifacts' is not supported by the evidence in Section 4.7, which reports only GPU memory versus input length. No objective metric, listening test, or comparison against non-overlapping stitching is provided for hour-long outputs. The inpainting evaluation in Section 4.1 runs the model on a short receptive field centered on each masked region, so it does not exercise the MultiDiffusion sliding-window path at long horizons, and the bandwidth-extension evaluation likewise does not verify long-range coherence. Because the MultiDiffusion adaptation averages nonlinear epsilon predictions over overlapping windows, the absence of boundary artifacts and spectral consistency at hour scale is an empirical question that the paper leaves open. I recommend adding either an objective evaluation on long inputs (e.g., comparing MultiDiffusion against non-overlapping windowing on a long test set), a human evaluation of long excerpts, or at minimum a clear statement that the hour-long claim currently refers to memory scalability rather than verified restoration quality.
  2. [Section 4.1, 'Baselines'] The baseline comparison is not fully fair: AudioSR is used as a pre-trained model without retraining on the authors' dataset, while MAID and CQTDiff are explicitly retrained on the same training data and IBAR is trained from scratch on that data. This asymmetry confounds the reported quality differences, since AudioSR's training distribution and segment length differ from the other baselines. The paper should either retrain AudioSR on the same permissively licensed dataset, report AudioSR results after fine-tuning, or justify why the pre-trained comparison is the appropriate one for the stated SOTA claim.
  3. [Section 3.2, Eq. (10); Section 3.6, Algorithm 1] The training loss is computed only on the masked region, yet the sampler denoises the entire spectrogram, including unmasked regions, without any explicit mechanism to preserve the known content (e.g., replacing the known region with the input at each step). The paper states in Section 3.2 that 'the rest of the spectrogram should exactly match the input,' but no analysis or ablation shows that the network's epsilon predictions in unmasked regions are accurate enough to prevent drift or artifacts. The strong objective and subjective results suggest this works in practice, but the mechanism is not explained; please clarify whether the unmasked region is conditioned in some way, or provide an ablation demonstrating that the masked-only loss is sufficient to preserve unmasked content.
minor comments (4)
  1. [Section 3.5] The description of the two-stage training procedure is duplicated almost verbatim in two consecutive paragraphs; please remove the redundancy and present the pretraining/fine-tuning details once.
  2. [Section 4.1, 'Baselines'] AudioSR is absent from the Maestro bandwidth-extension table (Table 5) and the Maestro inpainting table (Table 9), even though it is listed as a bandwidth-extension baseline elsewhere; the omission is not explained and weakens the completeness of the comparison on that dataset.
  3. [Table 1] The 'Long audio restoration' checkmark for A2SB is based solely on the GPU-memory experiment in Section 4.7; given the major concern above, either the table entry should be qualified or the quality evidence should be added.
  4. [Section 3.6, Algorithm 1] The uniform averaging of overlapping epsilon predictions is stated without discussion of alternative weighting (e.g., distance-based blending) or an analysis of sensitivity to hop size H; the paper reports results only for H=128, so a brief sensitivity check or a note on H selection would be helpful.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: A2SB's Schrödinger bridge formulation, factorized representation, phase orthogonalization, and MultiDiffusion inference are either derived from first principles or cited from prior work, and the state-of-the-art claims are evaluated on held-out OOD test sets.

full rationale

The paper's central derivation chain is self-contained with respect to the claimed predictions. The Schrödinger bridge training objective (Eq. 10) and sampling posterior (Eq. 19) are adopted from Liu et al. (2023a), a peer-reviewed external source; although two A2SB authors overlap with that work, the cited formulas are mathematical identities with stated assumptions (e.g., ft = 0, Dirac delta assumption) and do not assume the target result. The factorized magnitude-phase representation (Eq. 7) and phase orthogonalization (Eqs. 13-16) are derived from trigonometric identities and the least-squares Procrustes/SVD solution cited to Schönemann (1966) and Levinson et al. (2020), not from the paper's own fitted values. The MultiDiffusion adaptation is explicitly attributed to Bar-Tal et al. (2023), and the objective and subjective metrics are computed on held-out OOD test sets against independently re-trained baselines. No parameter is fitted to a subset of the evaluation data and then reported as a prediction of that same data. The main weakness is the claim that A2SB can 'coherently restore hour-long audio without boundary artifacts' (Contribution 4), which is supported only by the GPU memory benchmark in Section 4.7 and Figure 7 with no perceptual or objective evaluation of long-horizon output quality. That is an evidence gap and a correctness risk, not a circularity: the claim does not reduce by construction to a fitted input, a self-citation, or a renamed known result. Therefore the circularity score is 0.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim depends on several hand-chosen hyperparameters and domain assumptions inherited from prior work. The model is a large-scale application of existing SB theory, so the ledger is dominated by training choices and representative-data assumptions rather than new theoretical constructs.

free parameters (5)
  • magnitude power compression rho = 0.25
    Hand-chosen in Section 3.1 to compress magnitude range; all results depend on this representation choice.
  • beta_max = 1
    Noise schedule scale, chosen in Section 3.2 as most stable; affects the bridge process.
  • inpainting gap range = 0.1 to 1.6 s
    Sampled uniformly in Section 3.2 following Liu et al. 2023b; defines the training task distribution.
  • MultiDiffusion hop size H = 128 frames (50% overlap)
    Chosen in Section 3.6 for sliding-window averaging; affects long-audio output.
  • sampling steps = 50 for BWE, 200 for inpainting
    Used at inference (Section 4.1); quality insensitive to steps per Figures 8-9.
assumptions (5)
  • standard math Schrödinger bridge posterior formulas (Eq 9, Eq 19) from Liu et al. 2023a are valid under the Gaussian Dirac-delta assumption.
    Used without proof in Section 3.2 and 3.6; the paper relies on this cited derivation.
  • domain assumption The factorized representation (magnitude, cos phase, sin phase) is invertible and the phase orthogonalization yields a valid waveform.
    Assumed in Section 3.3; the inverse STFT with phase orthogonalization is used to synthesize audio.
  • domain assumption The curated 2.3K-hour permissively licensed music training set is representative enough to generalize to the OOD test sets.
    The SOTA claim is based on generalization to AAM, CCMixter, MTD, and Maestro; no direct measurement of distribution shift.
  • domain assumption MultiDiffusion sliding-window averaging produces coherent long-audio outputs without boundary artifacts.
    Stated in Section 3.6, but only memory usage is evaluated (Section 4.7), not output quality.
  • domain assumption Masked regions are corrupted with Gaussian noise (Eq 8) to define pdeg; this is a modeling choice inherited from I2SB.
    Defines the forward process for the bridge; any deviation from real corruption patterns could limit real-world use.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A2SB: Audio-to-Audio Schrodinger Bridges." pith.science (2026). https://pith.science/paper/5P2XZYPN

@misc{pith2026250111311,
  author       = {Pith},
  title        = {Pith review of: A2SB: Audio-to-Audio Schrodinger Bridges},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5P2XZYPN}},
  note         = {Machine review of arXiv:2501.11311}
}
read the original abstract

Real-world audio is often degraded by numerous factors. This work presents an audio restoration model tailored for high-res music at 44.1kHz. Our model, Audio-to-Audio Schr\"odinger Bridges (A2SB), is capable of both bandwidth extension (predicting high-frequency components) and inpainting (re-generating missing segments). Critically, A2SB is end-to-end requiring no vocoder to predict waveform outputs, able to restore hour-long audio inputs, and trained on permissively licensed music data. A2SB is capable of achieving state-of-the-art band-width extension and inpainting quality on several out-of-distribution music test sets.

Figures

Figures reproduced from arXiv: 2501.11311 by the authors.

Figure 1
Figure 1. A2SB targets music restoration with a focus on inpainting and bandwidth extension, each corresponding to a specific corruption pattern in spectrogram. The model is then trained to fit the diffusion Schrödinger Bridge process from the corrupted distribution to the clean distribution. The second challenge is to support both restoration tasks in a single model. We frame both tasks as the generative spectrogram inpainti… view at source ↗
Figure 2
Figure 2. Qualitative comparison between different bandwidth extension methods with cutoff = [PITH_FULL_IMAGE:figures/full_fig_p011_2.png] view at source ↗
Figure 3
Figure 3. Qualitative comparison between different inpainting methods with inpainting gap = 1 sec. [PITH_FULL_IMAGE:figures/full_fig_p012_3.png] view at source ↗
Figures from the paper (18 more)
Figure 4
Figure 4. Figure 4: Qualitative comparison between A2SB trained with two-channel STFT representation (S) and our proposed three-channel factorized representation (7). The model trained with the two-channel STFT representation has artifacts around the cutoff frequency and predicts too much…
Figure 5
Figure 5. Figure 5: This plot compares the average spectrogram magnitude of outputs from models trained with [PITH_FULL_IMAGE:figures/full_fig_p015_5.png]
Figure 6
Figure 6. Figure 6: These box-plots visualize the distribution of the ( [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]
Figure 7
Figure 7. Figure 7: GPU memory usage verses input audio length (in minutes) at inference time with [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]
Figure 8
Figure 8. Figure 8: Objective evaluation results with different sampling steps in [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]
Figure 9
Figure 9. Figure 9: Objective evaluation results with different sampling steps in [PITH_FULL_IMAGE:figures/full_fig_p017_9.png]
Figure 10
Figure 10. Figure 10: Qualitative comparison between different bandwidth extension methods with cutoff = [PITH_FULL_IMAGE:figures/full_fig_p023_10.png]
Figure 11
Figure 11. Figure 11: Qualitative comparison between different bandwidth extension methods with cutoff = [PITH_FULL_IMAGE:figures/full_fig_p023_11.png]
Figure 12
Figure 12. Figure 12: Qualitative comparison between different bandwidth extension methods with cutoff = [PITH_FULL_IMAGE:figures/full_fig_p024_12.png]
Figure 13
Figure 13. Figure 13: Qualitative comparison between different bandwidth extension methods with cutoff = [PITH_FULL_IMAGE:figures/full_fig_p024_13.png]
Figure 14
Figure 14. Figure 14: Qualitative comparison between different bandwidth extension methods with cutoff = [PITH_FULL_IMAGE:figures/full_fig_p025_14.png]
Figure 15
Figure 15. Figure 15: Qualitative comparison between different bandwidth extension methods with cutoff = [PITH_FULL_IMAGE:figures/full_fig_p025_15.png]
Figure 16
Figure 16. Figure 16: Qualitative comparison between different inpainting methods with inpainting gap = 1 sec. [PITH_FULL_IMAGE:figures/full_fig_p025_16.png]
Figure 17
Figure 17. Figure 17: Qualitative comparison between different inpainting methods with inpainting gap = 1 sec. [PITH_FULL_IMAGE:figures/full_fig_p026_17.png]
Figure 18
Figure 18. Figure 18: Qualitative comparison between different inpainting methods with inpainting gap = 1 sec. [PITH_FULL_IMAGE:figures/full_fig_p026_18.png]
Figure 19
Figure 19. Figure 19: Qualitative comparison between different inpainting methods with inpainting gap = 1 sec. [PITH_FULL_IMAGE:figures/full_fig_p026_19.png]
Figure 20
Figure 20. Figure 20: Qualitative comparison between different inpainting methods with inpainting gap = 1 sec. [PITH_FULL_IMAGE:figures/full_fig_p027_20.png]
Figure 21
Figure 21. Figure 21: Qualitative comparison between different inpainting methods with inpainting gap = 1 sec. [PITH_FULL_IMAGE:figures/full_fig_p027_21.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. On the Geometry of Music Bandwidth Extension in Latent Spaces of Audio Codecs

    cs.SD 2026-08 conditional novelty 6.0 of 10

    A single mean shift in the latent space of several neural codecs achieves competitive music bandwidth extension on some metrics, implying a largely linear structure.

  2. AnyBand: Unified Multi-Bandwidth Speech Extension via Frequency-Aware In-Context Spectral Infilling

    cs.SD 2026-08 conditional novelty 6.0 of 10

    A single flow-matching model performs speech bandwidth extension across continuously varying cutoff frequencies by treating the observed low-band spectrum as an in-context prompt and infilling the masked high band.

  3. Inverse Bridge Matching Distillation

    cs.LG 2025-02 conditional novelty 6.0 of 10

    Inverse Bridge Matching Distillation converts a trained diffusion bridge model into a one-step or few-step generator by matching the teacher's drift through a tractable inverse bridge matching objective.

  4. Schr\"odinger Bridge Mamba for One-Step Speech Enhancement

    cs.SD 2025-10 conditional novelty 5.0 of 10

    A Mamba-based speech enhancer trained with Schrödinger Bridge objectives produces strong denoising and dereverberation in one inference step with a low real-time factor.

Reference graph

Works this paper leans on

83 extracted references · 35 canonical work pages · cited by 4 Pith papers

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    Musiclm: Generating music from text

    Andrea Agostinelli, Timo I Denk, Zal \'a n Borsos, Jesse Engel, Mauro Verzetti, Antoine Caillon, Qingqing Huang, Aren Jansen, Adam Roberts, Marco Tagliasacchi, et al. Musiclm: Generating music from text. arXiv preprint arXiv:2301.11325, 2023

    arXiv 2023

  3. [3]

    Stochastic interpolants: A unifying framework for flows and diffusions

    Michael S Albergo, Nicholas M Boffi, and Eric Vanden-Eijnden. Stochastic interpolants: A unifying framework for flows and diffusions. arXiv preprint arXiv:2303.08797, 2023

    arXiv 2023

  4. [4]

    Reverse-time diffusion equation models

    Brian DO Anderson. Reverse-time diffusion equation models. Stochastic Processes and their Applications, 12 0 (3): 0 313--326, 1982

    1982

  5. [5]

    Fill in the gap! combining self-supervised representation learning with neural audio synthesis for speech inpainting

    Ihab Asaad, Maxime Jacquelin, Olivier Perrotin, Laurent Girin, and Thomas Hueber. Fill in the gap! combining self-supervised representation learning with neural audio synthesis for speech inpainting. arXiv preprint arXiv:2405.20101, 2024

    arXiv 2024

  6. [6]

    ediff-i: Text-to-image diffusion models with an ensemble of expert denoisers

    Yogesh Balaji, Seungjun Nah, Xun Huang, Arash Vahdat, Jiaming Song, Qinsheng Zhang, Karsten Kreis, Miika Aittala, Timo Aila, Samuli Laine, et al. ediff-i: Text-to-image diffusion models with an ensemble of expert denoisers. arXiv preprint arXiv:2211.01324, 2022

    arXiv 2022

  7. [7]

    Multidiffusion: Fusing diffusion paths for controlled image generation

    Omer Bar-Tal, Lior Yariv, Yaron Lipman, and Tali Dekel. Multidiffusion: Fusing diffusion paths for controlled image generation. arXiv preprint arXiv:2302.08113, 2023

    arXiv 2023

  8. [8]

    The mtg-jamendo dataset for automatic music tagging

    Dmitry Bogdanov, Minz Won, Philip Tovstogan, Alastair Porter, and Xavier Serra. The mtg-jamendo dataset for automatic music tagging. In Machine Learning for Music Discovery Workshop, International Conference on Machine Learning (ICML 2019), Long Beach, CA, United States, 2019. URL http://hdl.handle.net/10230/42015

    2019

Show all 83 references
  1. [9]

    Speechpainter: Text-conditioned speech inpainting

    Zal \'a n Borsos, Matt Sharifi, and Marco Tagliasacchi. Speechpainter: Text-conditioned speech inpainting. arXiv preprint arXiv:2202.07273, 2022

    2022 arXiv

  2. [10]

    Riemannian flow matching on general geometries

    Ricky TQ Chen and Yaron Lipman. Riemannian flow matching on general geometries. arXiv preprint arXiv:2302.03660, 2023

    2023 arXiv

  3. [11]

    Likelihood training of schr " odinger bridge using forward-backward sdes theory

    Tianrong Chen, Guan-Horng Liu, and Evangelos A Theodorou. Likelihood training of schr " odinger bridge using forward-backward sdes theory. arXiv preprint arXiv:2110.11291, 2021

    2021 arXiv

  4. [12]

    Visqol v3: An open source production ready objective speech and audio metric

    Michael Chinen, Felicia SC Lim, Jan Skoglund, Nikita Gureev, Feargus O'Gorman, and Andrew Hines. Visqol v3: An open source production ready objective speech and audio metric. In 2020 twelfth international conference on quality of multimedia experience (QoMEX), pp.\ 1--6. IEEE, 2020

    2020

  5. [13]

    Diffusion posterior sampling for general noisy inverse problems

    Hyungjin Chung, Jeongsol Kim, Michael T Mccann, Marc L Klasky, and Jong Chul Ye. Diffusion posterior sampling for general noisy inverse problems. arXiv preprint arXiv:2209.14687, 2022 a

    2022 arXiv

  6. [14]

    Improving diffusion models for inverse problems using manifold constraints

    Hyungjin Chung, Byeongsu Sim, Dohoon Ryu, and Jong Chul Ye. Improving diffusion models for inverse problems using manifold constraints. Advances in Neural Information Processing Systems, 35: 0 25683--25696, 2022 b

    2022

  7. [15]

    Diffusion schr \"o dinger bridge with applications to score-based generative modeling

    Valentin De Bortoli, James Thornton, Jeremy Heng, and Arnaud Doucet. Diffusion schr \"o dinger bridge with applications to score-based generative modeling. Advances in Neural Information Processing Systems, 34: 0 17695--17709, 2021

    2021

  8. [16]

    Fma: A dataset for music analysis

    Micha \"e l Defferrard, Kirell Benzi, Pierre Vandergheynst, and Xavier Bresson. Fma: A dataset for music analysis. arXiv preprint arXiv:1612.01840, 2016

    2016 arXiv

  9. [17]

    Diffusion models beat gans on image synthesis

    Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34: 0 8780--8794, 2021

    2021

  10. [18]

    Neural audio synthesis of musical notes with wavenet autoencoders

    Jesse Engel, Cinjon Resnick, Adam Roberts, Sander Dieleman, Mohammad Norouzi, Douglas Eck, and Karen Simonyan. Neural audio synthesis of musical notes with wavenet autoencoders. In International Conference on Machine Learning, pp.\ 1068--1077. PMLR, 2017

    2017

  11. [19]

    Estimation using log-spectral-distance criterion for noise-robust speech recognition

    Adoram Erell and Mitch Weintraub. Estimation using log-spectral-distance criterion for noise-robust speech recognition. In International Conference on Acoustics, Speech, and Signal Processing, pp.\ 853--856. IEEE, 1990

    1990

  12. [20]

    Enabling factorized piano music modeling and generation with the MAESTRO dataset

    Curtis Hawthorne, Andriy Stasyuk, Adam Roberts, Ian Simon, Cheng-Zhi Anna Huang, Sander Dieleman, Erich Elsen, Jesse Engel, and Douglas Eck. Enabling factorized piano music modeling and generation with the MAESTRO dataset. In International Conference on Learning Representation...

    2019

  13. [21]

    Denoising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33: 0 6840--6851, 2020

    2020

  14. [22]

    Video diffusion models

    Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video diffusion models. Advances in Neural Information Processing Systems, 35: 0 8633--8646, 2022

    2022

  15. [23]

    Schrödinger bridge for generative speech enhancement

    Ante Juki \'c , Roman Korostik, Jagadeesh Balam, and Boris Ginsburg. Schrödinger bridge for generative speech enhancement. arXiv preprint arXiv:2407.16074, 2024

    2024 arXiv

  16. [24]

    Audio super-resolution with robust speech representation learning of masked autoencoder

    Seung-Bin Kim, Sang-Hoon Lee, Ha-Yeong Choi, and Seong-Whan Lee. Audio super-resolution with robust speech representation learning of masked autoencoder. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2024

    2024

  17. [25]

    Diffwave: A versatile diffusion model for audio synthesis

    Zhifeng Kong, Wei Ping, Jiaji Huang, Kexin Zhao, and Bryan Catanzaro. Diffwave: A versatile diffusion model for audio synthesis. In International Conference on Learning Representations, 2021

    2021

  18. [26]

    Liu, Roman Korostik, Sung-Feng Huang, Szu-Wei Fu, and Ante Jukić

    Pin-Jui Ku, Alexander H. Liu, Roman Korostik, Sung-Feng Huang, Szu-Wei Fu, and Ante Jukić. Generative speech foundation model pretraining for high-quality speech extraction and restoration, 2024. URL https://arxiv.org/abs/2409.16117

    2024 arXiv

  19. [27]

    Nu-wave: A diffusion probabilistic model for neural audio upsampling

    Junhyeok Lee and Seungu Han. Nu-wave: A diffusion probabilistic model for neural audio upsampling. arXiv preprint arXiv:2104.02321, 2021

    2021 arXiv

  20. [28]

    Bigvgan: A universal neural vocoder with large-scale training

    Sang-gil Lee, Wei Ping, Boris Ginsburg, Bryan Catanzaro, and Sungroh Yoon. Bigvgan: A universal neural vocoder with large-scale training. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=iTtGCMDEzS_

    2023

  21. [29]

    Etta: Elucidating the design space of text-to-audio models

    Sang-gil Lee, Zhifeng Kong, Arushi Goel, Sungwon Kim, Rafael Valle, and Bryan Catanzaro. Etta: Elucidating the design space of text-to-audio models. arXiv preprint arXiv:2412.19351, 2024

    2024 arXiv

  22. [30]

    Diffusion models for audio restoration

    Jean-Marie Lemercier, Julius Richter, Simon Welker, Eloi Moliner, Vesa V \"a lim \"a ki, and Timo Gerkmann. Diffusion models for audio restoration. arXiv preprint arXiv:2402.09821, 2024

    2024 arXiv

  23. [31]

    A survey of the schr\"odinger problem and some of its connections with optimal transport

    Christian L \'e onard. A survey of the schr\"odinger problem and some of its connections with optimal transport. arXiv preprint arXiv:1308.0215, 2013

    2013 arXiv

  24. [32]

    An analysis of svd for deep rotation estimation

    Jake Levinson, Carlos Esteves, Kefan Chen, Noah Snavely, Angjoo Kanazawa, Afshin Rostamizadeh, and Ameesh Makadia. An analysis of svd for deep rotation estimation. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neural Information Processi...

    2020

  25. [33]

    Bridge-sr: Schr " odinger bridge for efficient sr

    Chang Li, Zehua Chen, Fan Bao, and Jun Zhu. Bridge-sr: Schr " odinger bridge for efficient sr. arXiv preprint arXiv:2501.07897, 2025

    2025 arXiv

  26. [34]

    Flow matching for generative modeling

    Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arXiv preprint arXiv:2210.02747, 2022

    2022 arXiv

  27. [35]

    Theodorou, Weili Nie, and Anima Anandkumar

    Guan-Horng Liu, Arash Vahdat, De-An Huang, Evangelos A. Theodorou, Weili Nie, and Anima Anandkumar. I2sb: image-to-image schrödinger bridge. In Proceedings of the 40th International Conference on Machine Learning, ICML'23. JMLR.org, 2023 a

    2023

  28. [36]

    Voicefixer: Toward general speech restoration with neural vocoder

    Haohe Liu, Qiuqiang Kong, Qiao Tian, Yan Zhao, DeLiang Wang, Chuanzeng Huang, and Yuxuan Wang. Voicefixer: Toward general speech restoration with neural vocoder. arXiv preprint arXiv:2109.13731, 2021

    2021 arXiv

  29. [37]

    Neural vocoder is all you need for speech super-resolution

    Haohe Liu, Woosung Choi, Xubo Liu, Qiuqiang Kong, Qiao Tian, and DeLiang Wang. Neural vocoder is all you need for speech super-resolution. arXiv preprint arXiv:2203.14941, 2022

    2022 arXiv

  30. [38]

    Audiosr: Versatile audio super-resolution at scale

    Haohe Liu, Ke Chen, Qiao Tian, Wenwu Wang, and Mark D Plumbley. Audiosr: Versatile audio super-resolution at scale. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp.\ 1076--1080. IEEE, 2024

    2024

  31. [39]

    Maid: A conditional diffusion model for long music audio inpainting

    Kaiyang Liu, Wendong Gan, and Chenchen Yuan. Maid: A conditional diffusion model for long music audio inpainting. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp.\ 1--5. IEEE, 2023 b

    2023

  32. [40]

    Kernel additive models for source separation

    Antoine Liutkus, Derry Fitzgerald, Zafar Rafii, Bryan Pardo, and Laurent Daudet. Kernel additive models for source separation. IEEE Transactions on Signal Processing, 62 0 (16): 0 4298--4310, 2014

    2014

  33. [41]

    Deep convolutional networks on the pitch spiral for musical instrument recognition

    Vincent Lostanlen and Carmine-Emanuele Cella. Deep convolutional networks on the pitch spiral for musical instrument recognition. arXiv preprint arXiv:1605.06644, 2016

    2016 arXiv

  34. [42]

    Cutting music source separation some Slakh : A dataset to study the impact of training data quality and quantity

    Ethan Manilow, Gordon Wichern, Prem Seetharaman, and Jonathan Le Roux. Cutting music source separation some Slakh : A dataset to study the impact of training data quality and quantity. In Proc. IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA)....

    2019

  35. [43]

    A context encoder for audio inpainting

    Andr \'e s Marafioti, Nathana \"e l Perraudin, Nicki Holighaus, and Piotr Majdak. A context encoder for audio inpainting. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 27 0 (12): 0 2362--2372, 2019

    2019

  36. [44]

    Gacela: A generative adversarial context encoder for long audio inpainting of music

    Andr \'e s Marafioti, Piotr Majdak, Nicki Holighaus, and Nathana \"e l Perraudin. Gacela: A generative adversarial context encoder for long audio inpainting of music. IEEE Journal of Selected Topics in Signal Processing, 15 0 (1): 0 120--131, 2020

    2020

  37. [45]

    Behm-gan: Bandwidth extension of historical music using generative adversarial networks

    Eloi Moliner and Vesa V \"a lim \"a ki. Behm-gan: Bandwidth extension of historical music using generative adversarial networks. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 31: 0 943--956, 2022

    2022

  38. [46]

    Diffusion-based audio inpainting

    Eloi Moliner and Vesa V \"a lim \"a ki. Diffusion-based audio inpainting. arXiv preprint arXiv:2305.15266, 2023

    2023 arXiv

  39. [47]

    Solving audio inverse problems with a diffusion model

    Eloi Moliner, Jaakko Lehtinen, and Vesa V \"a lim \"a ki. Solving audio inverse problems with a diffusion model. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp.\ 1--5. IEEE, 2023

    2023

  40. [48]

    Music instrument sounds for classification

    Abdulvahap Mutlu. Music instrument sounds for classification. Kaggle, 2024

    2024

  41. [49]

    On the design of deep priors for unsupervised audio restoration

    Vivek Sivaraman Narayanaswamy, Jayaraman J Thiagarajan, and Andreas Spanias. On the design of deep priors for unsupervised audio restoration. arXiv preprint arXiv:2104.07161, 2021

    2021 arXiv

  42. [50]

    Aam: a dataset of artificial audio multitracks for diverse music information retrieval tasks

    Fabian Ostermann, Igor Vatolkin, and Martin Ebeling. Aam: a dataset of artificial audio multitracks for diverse music information retrieval tasks. EURASIP Journal on Audio, Speech, and Music Processing, 2023 0 (1): 0 13, 2023

    2023

  43. [51]

    Training language models to follow instructions with human feedback

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35: 0 2...

    2022

  44. [52]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.\ 4195--4205, 2023

    2023

  45. [53]

    Phase-aware deep speech enhancement: It's all about the frame length

    Tal Peer and Timo Gerkmann. Phase-aware deep speech enhancement: It's all about the frame length. JASA Express Letters, 2 0 (10), 2022

    2022

  46. [54]

    Movie gen: A cast of media foundation models

    Adam Polyak, Amit Zohar, Andrew Brown, Andros Tjandra, Animesh Sinha, Ann Lee, Apoorv Vyas, Bowen Shi, Chih-Yao Ma, Ching-Yao Chuang, et al. Movie gen: A cast of media foundation models. arXiv preprint arXiv:2410.13720, 2024

    2024 arXiv

  47. [55]

    mir\_eval

    Colin Raffel, Brian McFee, Eric J Humphrey, Justin Salamon, Oriol Nieto, Dawen Liang, and Daniel PW Ellis. mir\_eval

  48. [56]

    Speech enhancement and dereverberation with diffusion-based generative models

    Julius Richter, Simon Welker, Jean-Marie Lemercier, Bunlong Lay, and Timo Gerkmann. Speech enhancement and dereverberation with diffusion-based generative models. IEEE/ACM Trans. on Audio, Speech, and Language Process., 31: 0 2351--2364, 2023

    2023

  49. [57]

    Piano triads wavset

    David Roberts. Piano triads wavset. Kaggle, 2022

    2022

  50. [58]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj \"o rn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.\ 10684--10695, 2022

    2022

  51. [59]

    U-net: Convolutional networks for biomedical image segmentation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In Medical image computing and computer-assisted intervention--MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part...

    2015

  52. [60]

    Image super-resolution via iterative refinement

    Chitwan Saharia, Jonathan Ho, William Chan, Tim Salimans, David J Fleet, and Mohammad Norouzi. Image super-resolution via iterative refinement. IEEE transactions on pattern analysis and machine intelligence, 45 0 (4): 0 4713--4726, 2022

    2022

  53. [61]

    A generalized solution of the orthogonal procrustes problem

    Peter H Sch \"o nemann. A generalized solution of the orthogonal procrustes problem. Psychometrika, 31 0 (1): 0 1--10, 1966

    1966

  54. [62]

    Sur la th \'e orie relativiste de l' \'e lectron et l'interpr \'e tation de la m \'e canique quantique

    Erwin Schr \"o dinger. Sur la th \'e orie relativiste de l' \'e lectron et l'interpr \'e tation de la m \'e canique quantique. In Annales de l'institut Henri Poincar \'e , volume 2, pp.\ 269--310, 1932

    1932

  55. [63]

    Universal speech enhancement with score-based diffusion

    Joan Serr \`a , Santiago Pascual, Jordi Pons, R Oguz Araz, and Davide Scaini. Universal speech enhancement with score-based diffusion. arXiv preprint arXiv:2206.03065, 2022

    2022 arXiv

  56. [64]

    mdctgan: Taming transformer-based gan for speech super-resolution with modified dct spectra

    Chenhao Shuai, Chaohua Shi, Lu Gan, and Hongqing Liu. mdctgan: Taming transformer-based gan for speech super-resolution with modified dct spectra. arXiv preprint arXiv:2305.11104, 2023

    2023 arXiv

  57. [65]

    Musan: A music, speech, and noise corpus

    David Snyder, Guoguo Chen, and Daniel Povey. Musan: A music, speech, and noise corpus. arXiv preprint arXiv:1510.08484, 2015

    2015 arXiv

  58. [66]

    Deep unsupervised learning using nonequilibrium thermodynamics

    Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International conference on machine learning, pp.\ 2256--2265. PMLR, 2015

    2015

  59. [67]

    Pseudoinverse-guided diffusion models for inverse problems

    Jiaming Song, Arash Vahdat, Morteza Mardani, and Jan Kautz. Pseudoinverse-guided diffusion models for inverse problems. In International Conference on Learning Representations, 2023

    2023

  60. [68]

    Score-based generative modeling through stochastic differential equations

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, 2021

    2021

  61. [69]

    The gtzan dataset: Its contents, its faults, their effects on evaluation, and its future use

    Bob L Sturm. The gtzan dataset: Its contents, its faults, their effects on evaluation, and its future use. arXiv preprint arXiv:1306.1461, 2013

    2013 arXiv

  62. [70]

    Roformer: Enhanced transformer with rotary position embedding

    Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding. Neurocomputing, 568: 0 127063, 2024

    2024

  63. [71]

    John Thickstun, Zaid Harchaoui, and Sham M. Kakade. Learning features of music from scratch. In International Conference on Learning Representations (ICLR), 2017

    2017

  64. [72]

    Conditional flow matching: Simulation-free dynamic optimal transport

    Alexander Tong, Nikolay Malkin, Guillaume Huguet, Yanlei Zhang, Jarrid Rector-Brooks, Kilian Fatras, Guy Wolf, and Yoshua Bengio. Conditional flow matching: Simulation-free dynamic optimal transport. arXiv preprint arXiv:2302.00482, 2023

    2023 arXiv

  65. [73]

    Diffusion-based speech enhancement with schr " odinger bridge and symmetric noise schedule

    Siyi Wang, Siyi Liu, Andrew Harper, Paul Kendrick, Mathieu Salzmann, and Milos Cernak. Diffusion-based speech enhancement with schr " odinger bridge and symmetric noise schedule. arXiv preprint arXiv:2409.05116, 2024

    2024 arXiv

  66. [74]

    Audit: Audio editing by following instructions with latent diffusion models

    Yuancheng Wang, Zeqian Ju, Xu Tan, Lei He, Zhizheng Wu, Jiang Bian, et al. Audit: Audio editing by following instructions with latent diffusion models. Advances in Neural Information Processing Systems, 36: 0 71340--71357, 2023

    2023

  67. [75]

    Gebru, and Alexander Richard

    Yi-Chiao Wu, Dejan Marković, Steven Krenn, Israel D. Gebru, and Alexander Richard. Scoredec: A phase-preserving high-fidelity audio codec with a generalized score-based diffusion post-filter. In ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal P...

    2024

  68. [76]

    Large-scale contrastive language-audio pretraining with feature fusion and keyword-to-caption augmentation

    Yusong Wu, Ke Chen, Tianyu Zhang, Yuchen Hui, Taylor Berg-Kirkpatrick, and Shlomo Dubnov. Large-scale contrastive language-audio pretraining with feature fusion and keyword-to-caption augmentation. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Sign...

    2023

  69. [77]

    Byt5: Towards a token-free future with pre-trained byte-to-byte models

    Linting Xue, Aditya Barua, Noah Constant, Rami Al-Rfou, Sharan Narang, Mihir Kale, Adam Roberts, and Colin Raffel. Byt5: Towards a token-free future with pre-trained byte-to-byte models. Transactions of the Association for Computational Linguistics, 10: 0 291--306, 2022

    2022

  70. [78]

    Conditioning and sampling in variational diffusion models for speech super-resolution

    Chin-Yun Yu, Sung-Lin Yeh, Gy \"o rgy Fazekas, and Hao Tang. Conditioning and sampling in variational diffusion models for speech super-resolution. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp.\ 1--5. IEEE, 2023

    2023

  71. [79]

    Flowhigh: Towards efficient and high-quality audio super-resolution with single-step flow matching

    Jun-Hak Yun, Seung-Bin Kim, and Seong-Whan Lee. Flowhigh: Towards efficient and high-quality audio super-resolution with single-step flow matching. arXiv preprint arXiv:2501.04926, 2025

    2025 arXiv

  72. [80]

    u ller, and Meinard M \

    Frank Zalkow, Stefan Balke, Vlora Arifi-M \"u ller, and Meinard M \"u ller. Mtd: A multimodal dataset of musical themes for mir research. Trans. Int. Soc. Music. Inf. Retr., 3 0 (1): 0 180--192, 2020

    2020

  73. [81]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  74. [82]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  75. [83]

    t4 ٺ _ ; ׆6H^ )-79UDDz \_nGН

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.