Pith. sign in

REVIEW 5 major objections 5 minor 42 references

VS-Singer: Vision-Guided Stereo Singing Voice Synthesis with Consistency Schr\"odinger Bridge

T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read VS-Singer generates stereo singing that matches a scene image in one sampling step

desk verdict Novel task and plausible empirical results, but the score derivation in Eq. (4) is wrong, which undermines the one-step consistency claim. read the letter →

arxiv 2506.16020 v1 pith:CANZ6TIL submitted 2025-06-19 cs.SD eess.AS

Zijing Zhao , Kai Wang , Hao Huang , Ying Hu , Liang He , Jichen Yang This is my paper · ORCID
classification cs.SDeess.AS
keywords stereosingingvoicesynthesisvisualacousticmatchingconsistencySchrödingerbridgeone-stepgenerationbinauralaudioroomreverberationmultimodalsceneperspective
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

VS-Singer sets out to prove that a single model can look at a picture of a room and synthesize a song with stereo placement and reverberation that match that room, and can do it in one sampling step. Today the same result requires chaining a singing synthesis system, a visual acoustic matching system, and a mono-to-stereo converter, which is slow and lets errors pile up. The paper's central claim is that a consistency Schrödinger bridge decoder makes this unified one-step generation possible without a separately trained teacher model, roughly doubling inference speed compared with the cascaded alternative. If the claim holds, scene-aware stereo singing becomes practical for real-time and immersive applications.

What carries the argument

The load-bearing object is the consistency Schrödinger bridge (CSB), a decoder that joins consistency training with a Schrödinger bridge whose endpoints are the linguistic representation $x_1$ and the clean binaural audio $x_0 = (x_0^{\mathrm{left}}, x_0^{\mathrm{right}})$. The bridge defines a tractable conditional Gaussian $q(x_t \mid x_0, x_1) = \mathcal{N}(x_t; \mu_t(x_0, x_1), \Sigma(t)^2)$, with $\mu_t$ a variance-weighted interpolation of the two endpoints, and this yields a probability-flow ODE whose score can be estimated without a teacher model. The consistency loss trains an online network and an EMA target network to send any two states on the same ODE trajectory to the same $x_0$, which is what allows one-step sampling. Around the bridge, a modal interaction network injects image and position information into the text hidden sequence, and a spatially-aware feature enhancement module adds an L2 loss that pulls each generated channel toward its ground-truth channel while pushing the left and right channels apart.

What would settle it

Record a singer in one of the rooms shown in the test images with a binaural microphone, then compare VS-Singer's generated audio with that real recording on left-right energy ratio and RT60 error; if the model trained on synthetic convolution is not closer to the real binaural recording than a mono baseline, the central scene-matching claim is not supported.

Watch

Extended reading notes

Core claim

VS-Singer is proposed as the first framework to combine visual acoustic matching and stereo singing voice synthesis in one network. Given a scene image, it extracts a left-eye and right-eye view, adds the singer's 3D position relative to the viewpoint, fuses this spatial information into the lyric text encoding, and feeds the enriched representation to a decoder built on a consistency Schrödinger bridge. The decoder maps Gaussian noise straight to the clean binaural mel-spectrogram, so generation takes one function evaluation instead of the dozens or hundreds used by diffusion baselines. The paper reports that the resulting left-right energy ratio and reverberation-time errors are better than those of the cascaded systems on both seen and unseen scenes, while inference is about twice as fast, and its ablations show each module—the interaction network, the bridge, and the spatial enhancement—contributes to the final result.

Load-bearing premise

The load-bearing premise is that binaural singing audio created by convolving clean mono vocals with room impulse responses taken from a large binaural audiovisual corpus is valid ground-truth stereo audio; if that artificial reverberation does not match real room acoustics, the claimed scene-perspective matching may not transfer to real recordings.

Editorial extensions

If this is right

  • A single image can condition the one-step generation of stereo singing with room acoustics, removing the need to cascade three separate models.
  • The one-step decoder cuts inference time by about half relative to cascaded systems, making real-time or interactive singing synthesis plausible.
  • Training no longer requires a pretrained teacher model, because the Schrödinger bridge supplies a tractable path that keeps independently trained consistency models from losing quality.
  • The model generalizes spatial cues to rooms not seen in training, based on its lower left-right energy and reverberation-time errors on the unseen test split.
  • Each of the three modules is load-bearing: removing the interaction network, the bridge, or the spatial enhancement degrades either spatial accuracy or audio quality.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The strongest untested consequence is about real rooms: because training uses synthetic binaural audio made by convolving clean vocals with impulse responses from a binaural corpus, the model's scene-matching ability should be checked against real measured room responses, which the paper does not report.
  • The CSB conditioning recipe is not singing-specific, so the same vision-plus-text-to-binaural design could be carried over to speech synthesis or audio-visual dialogue for AR/VR, provided matched binaural speech data exist.
  • The two-viewpoint mask used to simulate left and right eyes points to an easy extension: adding more viewpoints or depth information could let the model track head motion and moving sources, which the current static image setup does not cover.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes VS-Singer, a unified framework for stereo singing voice synthesis that takes a scene image, a musical score, and lyrics as input, and produces binaural singing audio with room reverberation. The system has three components: a modal interaction network (MIN) that injects spatial cues derived from a dual-eye masking of the image into the text encoding; a decoder based on a consistency Schrödinger bridge (CSB) that is intended to generate the clean binaural mel-spectrogram from its linguistic representation in one step; and a spatially-aware feature enhancement (SFE) module that is designed to strengthen left/right channel differences and improve audio-visual consistency. The method is evaluated on a synthetic stereo corpus obtained by transferring impulse responses from NVAS-SoundSpace to Opencpop mono singing audio. The reported experiments compare against cascade-style baselines in terms of MOS, RTF, MCD, LRE, and RTE, and report a claimed 2x inference speed-up over cascaded systems. The central theoretical claim is that the CSB enables one-step generation without a teacher model while preserving quality.

Significance. If the theoretical derivation were correct, VS-Singer would be a novel and practically interesting contribution: it is, to my knowledge, the first attempt to integrate visual acoustic matching and stereo singing voice synthesis in one trainable model, and the one-step decoder would avoid the two-stage teacher-based consistency distillation used by prior work. The paper also has positive experimental features: it uses public datasets, standard objective and subjective metrics, an ablation study, and provides a public audio demo page. These are strengths that should be credited. However, the evaluation as presented is entangled with a synthetic data construction and with an ambiguous cascade baseline, and, more importantly, the Schrödinger-bridge derivation in Section 2.2 appears to contain a load-bearing mathematical error. Because the one-step generation claim rests on that derivation, the contribution cannot be accepted in its current form; substantial revision is required.

major comments (5)
  1. [§2.2, Eq. (4)] The asserted unbiased estimator is not the score of the marginal density induced by Eq. (3). With a_t = \barσ_t^2 / (σ_t^2 + \barσ_t^2), b_t = σ_t^2 / (σ_t^2 + \barσ_t^2), and Σ_t^2 = σ_t^2 \barσ_t^2 / (σ_t^2 + \barσ_t^2), the transition q(x_t | x_0, x_1) = N(a_t x_0 + b_t x_1, Σ_t^2 I) gives the marginal score ∇ log p_t(x_t) = E_{x_0,x_1|x_t}[(a_t x_0 + b_t x_1 - x_t) / Σ_t^2]. Equation (4) instead asserts the score is (E[x_1 | x_t] - x_t) / Σ_t^2. The difference is a_t (E[x_0 | x_t] - E[x_1 | x_t]) / Σ_t^2, which is not zero in general; for example, with x_0 and x_1 independent standard normals and β_t = 1, the two expressions differ by a factor (1-t)/t except at t = 1/2. Since the PF-ODE in Eq. (5) is built on this incorrect score, the states (x_{t_n}, x_{t_{n+1}}) used in the consistency loss Eq. (6) are not generally on a common ODE trajectory, so enforcing f_θ(x_{t_{n+1}}) = f_{θ^-}(x_{t_n}) does not provably drive the model to the clean audio x_0. The one-step consistency Schrödinger bridge claim is therefore unsupported by the present derivation.
  2. [§3.2, Table 1] The baseline used for the 'cascaded systems' comparison is not actually a cascade. The text says 'we will only use the singing synthesis model to represent the cascade model', and Table 1 reports only mono singing synthesis models (DiffSinger, VISinger2, CoMoSpeech), with no outputs from LeMARA or SepStereo. Consequently, the RTF, MOS, MCD, LRE, and RTE numbers do not evaluate a pipeline consisting of singing synthesis plus visual acoustic matching plus mono-to-stereo conversion. The claimed 2x inference speed-up over the cascaded baseline is therefore not substantiated. Please report the actual cascade's metrics. In addition, the one-step system 'Ours 1' has MOS 3.48 on test-unseen, lower than VISinger2's 3.72, so the statement that 'our model achieves the best results in each metric' is not correct for the one-step model; the claim should specify which row of Table 1 supports each conclusion.
  3. [§2.3, Eq. (7)] The proposed enhancement loss L_enh is unbounded below because of the negative squared term -||x'_left - x'_right||^2. For fixed ground truth x_0, one can drive this term to -∞ by making the two estimated channels increasingly different while keeping the first two terms bounded, so the loss does not define a well-posed training objective. The paper does not specify how this term is weighted in the total loss or whether any regularization prevents divergence. This issue affects the interpretation of the ablation for the SFE module and should be fixed by using a bounded form (e.g., a hinge or softplus variant) and explicitly stating the full objective.
  4. [§2.2, Eq. (3)] The Schrödinger bridge is not fully specified. The distribution q(x_t | x_0, x_1) depends on the coupling p_B(x_1 | x_0), but p_B is never defined; the displayed formula for μ_t and Σ_t^2 is garbled by missing overbars on σ_t^2 in the PDF; and no forward SDE with drift f is given, only the PF-ODE in Eq. (5). Without these definitions, Eqs. (4)-(6) cannot be verified or reproduced.
  5. [§3.1] All training and test data are synthesized by convolving impulse responses extracted from NVAS-SoundSpace with Opencpop mono audio. The validity of this synthetic binaural audio as ground truth for the claimed 'scene perspective matching' is not assessed. The paper does not describe how impulse responses are selected and matched to images, how many are used, or whether the resulting reverberation is validated against real binaural recordings. As the experiments rest entirely on this synthetic data, the generalization claim is not established and should be either explicitly qualified or supported by additional validation.
minor comments (5)
  1. [General] Please unify the metric name: the tables use LRE, while the text in Section 3.4 uses 'LER'.
  2. [§2.2, §2.3] There are several typos: 'between between' in Section 2.2, 'desigened' for 'designed', and 'the the' in Section 2.3. Please proofread the manuscript.
  3. [Abstract] The audio demo URL in the abstract reads 'VS-Signer' while the model is named 'VS-Singer'; please verify the link and the spelling.
  4. [§3.2] For the baseline models LeMARA and SepStereo, please report their configurations and how they were integrated into the cascade; as written, the description does not allow the comparison to be reproduced.
  5. [Figure 2] The y-axis of Figure 2 is not labeled and the plotted quantity is not defined in the caption. Please state whether it is MOS, MCD, loss, or another metric, and specify the error bars if any.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular reduction; the one-step and bridge properties are imported from external prior work, with only minor non-load-bearing self-citations.

full rationale

VS-Singer's derivation is not circular. The one-step generation property is inherited from the external consistency model framework [16], and the Schrödinger bridge trajectory and score formalism are taken from external bridge/diffusion results [27,29,14]; none of these are the present authors' own claims being used to prove themselves. The network is trained with the CSB consistency loss (6) and the enhancement loss (7) against a fixed synthetic binaural corpus, and no fitted constant or subset fit is later renamed as a prediction. The only author-overlap citations are routine and non-load-bearing: the variance adapter from FastSpeech 2 [17] and the DiffSinger [1] train/test split. The construction of binaural targets by convolving Opencpop audio with NVAS-SoundSpace impulse responses (Section 3.1) raises external-validity questions, but it is not circularity: those targets are not outputs of the model's own fitted parameters. The mathematical concern in Section 2.2—whether Eq. (4) is the correct score of the Eq. (3) marginal—is a correctness/consistency issue rather than a reduction of a prediction to its inputs, so it does not affect the circularity score.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

No physical entities are invented. The main free parameters are the data selection threshold and the eye-mask ratio, both chosen by hand. The central assumptions concern the validity of artificial reverberation as ground truth and the sufficiency of image plus location for acoustics.

free parameters (2)
  • binaural selection threshold = 0.01
    Videos with average left-right channel difference exceeding 0.01 were selected to build the training data (Section 3.1). This threshold is chosen by hand and affects which NVAS-SoundSpace videos are used.
  • eye mask ratio = 1/4
    The left and right sides of the scene image are masked by 1/4 to simulate left and right eye viewpoints (Section 2.1). This ratio is a design choice.
assumptions (3)
  • domain assumption The impulse responses extracted from NVAS-SoundSpace can be transferred to Opencpop mono audio to create realistic binaural singing data.
    Section 3.1: 'applied convolutional reverberation by transferring the impulse responses extracted from NVAS-SoundSpace audio to the Opencpop audio, generating binaural audio.' This assumes that the RIRs are valid for singing voices and that the resulting audio is a good proxy for real stereo singing.
  • domain assumption A 2D image plus the speaker's position (distance and angle) contains sufficient information to predict the acoustics of a room.
    Section 2.1: The model uses image features and V_loc (distance and angle) to infer spatial features. This assumes that these cues are sufficient for room acoustics estimation.
  • standard math The Schrödinger bridge equations from [27] hold when adapted to the singing voice synthesis setting with linguistic representations as endpoints.
    Section 2.2: Equations (3)-(5) directly use the forward-backward SDE theory from prior work. The paper does not re-derive these equations for its specific setting.

how reviews work

0 comments
Cite this review

Pith. "Pith review of VS-Singer: Vision-Guided Stereo Singing Voice Synthesis with Consistency Schr\"odinger Bridge." pith.science (2026). https://pith.science/paper/CANZ6TIL

@misc{pith2026250616020,
  author       = {Pith},
  title        = {Pith review of: VS-Singer: Vision-Guided Stereo Singing Voice Synthesis with Consistency Schr\"odinger Bridge},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CANZ6TIL}},
  note         = {Machine review of arXiv:2506.16020}
}
read the original abstract

To explore the potential advantages of utilizing spatial cues from images for generating stereo singing voices with room reverberation, we introduce VS-Singer, a vision-guided model designed to produce stereo singing voices with room reverberation from scene images. VS-Singer comprises three modules: firstly, a modal interaction network integrates spatial features into text encoding to create a linguistic representation enriched with spatial information. Secondly, the decoder employs a consistency Schr\"odinger bridge to facilitate one-step sample generation. Moreover, we utilize the SFE module to improve the consistency of audio-visual matching. To our knowledge, this study is the first to combine stereo singing voice synthesis with visual acoustic matching within a unified framework. Experimental results demonstrate that VS-Singer can effectively generate stereo singing voices that align with the scene perspective in a single step.

Figures

Figures reproduced from arXiv: 2506.16020 by the authors.

Figure 1
Figure 1. Overview of our proposed model structure. perspective in one step, particularly achieving a 2x infer￾ence speed-up over the comparative cascaded systems. 2. Methodology The overall architecture of the proposed VS-Singer model is illustrated in [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Convergence speed of the models at 800k training steps in the ablation experiment. trend [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

42 extracted references · 37 canonical work pages

  1. [1]

    As the continuous development of diffusion models [6–12], the naturalness and fluency of syn- thesized singing speech have now approached those of human performances

    Introduction Singing voice synthesis (SVS) involves generating singing voice from musical scores and lyrics, which include features such as notes and pitch [1–5]. As the continuous development of diffusion models [6–12], the naturalness and fluency of syn- thesized singing speech have now approached those of human performances. Grad-TTS [13] introduces st...

    work page

  2. [2]

    Methodology The overall architecture of the proposed VS-Singer model is illustrated in Fig. 1. Our model consists of three parts: 1) The MIN is used to enable the interactive aggregation of multimodal information sources, models environmental acoustics, generates spatial cues, interacts with text encoding, and generates text hidden sequences with spatial ...

    work page

  3. [3]

    test- seen

    Experiments 3.1. Data Preparation This paper evaluated the proposed method using the public datasets Opencpop [24] and NV AS-SoundSpace [21]. The Opencpop dataset contains 100 Chinese songs performed by a female singer. The NV AS-SoundSpace dataset contains 13,000 hours of binaural audiovisual data. The average difference was calculated between the left a...

    work page

  4. [4]

    VS-Singer consists of a modal interaction network, a decoder based on consistency Schr ¨odinger bridge and a spatially-aware feature enhancement module

    Conclusion This paper proposes VS-Singer, a novel framework that unifies spatial acoustic matching and stereo singing voice generation. VS-Singer consists of a modal interaction network, a decoder based on consistency Schr ¨odinger bridge and a spatially-aware feature enhancement module. The modal interaction network is introduced to add the spatial infor...

    work page

  5. [5]

    CoMoSpeech [15], a speech synthesis model using consis- tency distillation [16] with DiffSinger [1] as the teacher model

    work page

  6. [6]

    7) Sep- Stereo [35], a model that converts mono audio to binaural audio using scene images

    LeMARA [34], a visual acoustic matching model that adds room reverberation to a given audio based on images. 7) Sep- Stereo [35], a model that converts mono audio to binaural audio using scene images. 3.3. Experimental Results and Analysis In the objective evaluation, this paper measures performance from four aspects: 1) Real-Time Factor (RTF), the time r...

    work page

  7. [7]

    Video diffusion models,

    J. Ho, T. Salimans, A. Gritsenko, W. Chan, M. Norouzi, and D. J. Fleet, “Video diffusion models,”Advances in Neural Information Processing Systems, vol. 35, pp. 8633–8646, 2022

    2022

  8. [8]

    Diffsinger: Singing voice synthesis via shallow diffusion mechanism,

    J. Liu, C. Li, Y . Ren, F. Chen, and Z. Zhao, “Diffsinger: Singing voice synthesis via shallow diffusion mechanism,” inAAAI, 2022, pp. 11 020–11 028

    work page 2022

Show all 42 references
  1. [9]

    Visinger2: High-fidelity end-to-end singing voice synthesis en- hanced by digital signal processing synthesizer,

    Y . Zhang, H. Xue, H. Li, L. Xie, T. Guo, R. Zhang, and C. Gong, “Visinger2: High-fidelity end-to-end singing voice synthesis en- hanced by digital signal processing synthesizer,” inInterspeech, 2023, pp. 4444–4448

    2023

  2. [10]

    Audiogpt: Understanding and generating speech, music, sound, and talking head,

    R. Huang, M. Li, D. Yang, J. Shi, X. Chang, Z. Ye, Y . Wu, Z. Hong, J. Huang, J. Liuet al., “Audiogpt: Understanding and generating speech, music, sound, and talking head,” inAAAI, 2024, pp. 23 802–23 804

    2024

  3. [11]

    An end-to-end approach for chord- conditioned song generation,

    S. Gao, S. Lei, F. Zhuo, H. Liu, F. Liu, B. Tang, Q. Huang, S. Kang, and Z. Wu, “An end-to-end approach for chord- conditioned song generation,”arXiv preprint arXiv:2409.06307, 2024

    2024 arXiv

  4. [12]

    Unisyn: an end-to-end unified model for text-to-speech and singing voice synthesis,

    Y . Lei, S. Yang, X. Wang, Q. Xie, J. Yao, L. Xie, and D. Su, “Unisyn: an end-to-end unified model for text-to-speech and singing voice synthesis,” inAAAI, 2023, pp. 13 025–13 033

    2023

  5. [13]

    Elucidating the design space of diffusion-based generative models,

    T. Karras, M. Aittala, T. Aila, and S. Laine, “Elucidating the design space of diffusion-based generative models,”Advances in neural information processing systems, vol. 35, pp. 26 565– 26 577, 2022

    2022

  6. [14]

    Score-based generative modeling through stochas- tic differential equations,

    Y . Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole, “Score-based generative modeling through stochas- tic differential equations,” inICLR, 2021

    2021

  7. [15]

    Learning the beauty in songs: Neural singing voice beautifier,

    J. Liu, C. Li, Y . Ren, Z. Zhu, and Z. Zhao, “Learning the beauty in songs: Neural singing voice beautifier,” inACL, 2022, pp. 7970– 7983

    2022

  8. [16]

    Align your latents: High-resolution video synthesis with latent diffusion models,

    A. Blattmann, R. Rombach, H. Ling, T. Dockhorn, S. W. Kim, S. Fidler, and K. Kreis, “Align your latents: High-resolution video synthesis with latent diffusion models,” inCVPR, 2023, pp. 22 563–22 575

    2023

  9. [17]

    An image is worth one word: Personalizing text-to-image generation using textual inversion,

    R. Gal, Y . Alaluf, Y . Atzmon, O. Patashnik, A. H. Bermano, G. Chechik, and D. Cohen-Or, “An image is worth one word: Personalizing text-to-image generation using textual inversion,” inICLR, 2023

    2023

  10. [18]

    Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation,

    N. Ruiz, Y . Li, V . Jampani, Y . Pritch, M. Rubinstein, and K. Aber- man, “Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation,” inCVPR, 2023, pp. 22 500–22 510

    2023

  11. [19]

    Scalable diffusion models with transform- ers,

    W. Peebles and S. Xie, “Scalable diffusion models with transform- ers,” inICCV, 2023, pp. 4195–4205

    2023

  12. [20]

    Grad-tts: A diffusion probabilistic model for text-to-speech,

    V . Popov, I. V ovk, V . Gogoryan, T. Sadekova, and M. Kudinov, “Grad-tts: A diffusion probabilistic model for text-to-speech,” in ICML, 2021, pp. 8599–8608

    2021

  13. [21]

    Novel-view acoustic synthesis,

    C. Chen, A. Richard, R. Shapovalov, V . K. Ithapu, N. Neverova, K. Grauman, and A. Vedaldi, “Novel-view acoustic synthesis,” in CVPR, 2023, pp. 6409–6419

    2023

  14. [22]

    Como- speech: One-step speech and singing voice synthesis via consis- tency model,

    Z. Ye, W. Xue, X. Tan, J. Chen, Q. Liu, and Y . Guo, “Como- speech: One-step speech and singing voice synthesis via consis- tency model,” inMM, 2023, pp. 1831–1839

    2023

  15. [23]

    Consistency models,

    Y . Song, P. Dhariwal, M. Chen, and I. Sutskever, “Consistency models,” inICML, 2023, pp. 32 211–32 252

    2023

  16. [24]

    Results demonstrate VS-Singer’s superior capability in inference speed and immer- sive stereo singing voice synthesis

    and NV AS-SoundSpace [21] corpora. Results demonstrate VS-Singer’s superior capability in inference speed and immer- sive stereo singing voice synthesis. Audio samples are available at:https://usinger1.github.io/VS-Signer/. The key contributions of our work are summarized as f...

    2025 arXiv

  17. [25]

    Fastspeech 2: Fast and high-quality end-to-end text to speech,

    Y . Ren, C. Hu, X. Tan, T. Qin, S. Zhao, Z. Zhao, and T.-Y . Liu, “Fastspeech 2: Fast and high-quality end-to-end text to speech,” arXiv preprint arXiv:2006.04558, 2020

    2006 arXiv

  18. [26]

    2.5 d visual sound,

    R. Gao and K. Grauman, “2.5 d visual sound,” inCVPR, 2019, pp. 324–333

    2019

  19. [27]

    Enhancing spatial audio generation with source separation and channel panning loss,

    W. Lim and J. Nam, “Enhancing spatial audio generation with source separation and channel panning loss,” inICASSP, 2024, pp. 8321–8325

    2024

  20. [28]

    Multi-source spatial knowledge understanding for immersive visual text-to-speech,

    H. L. Rui Liu, Shuwei He, “Multi-source spatial knowledge understanding for immersive visual text-to-speech,” inICASSP, 2025

    2025

  21. [29]

    Visually guided binaural audio generation with cross-modal consistency,

    M. Liu, J. Wang, X. Qian, and X. Xie, “Visually guided binaural audio generation with cross-modal consistency,” inICASSP, 2024, pp. 7980–7984

    2024

  22. [30]

    Multi-modal and multi-scale spatial environment understanding for immersive visual text-to-speech,

    H. L. Rui Liu, Shuwei He, “Multi-modal and multi-scale spatial environment understanding for immersive visual text-to-speech,” inAAAI, 2025

    2025

  23. [31]

    Opencpop: A high-quality open source chinese popu- lar song corpus for singing voice synthesis,

    Y . Wang, X. Wang, P. Zhu, J. Wu, H. Li, H. Xue, Y . Zhang, L. Xie, and M. Bi, “Opencpop: A high-quality open source chinese popu- lar song corpus for singing voice synthesis,” inInterspeech, 2022, pp. 4242–4246

    2022

  24. [32]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” inCVPR, 2016, pp. 770–778

    2016

  25. [33]

    Diffusion schr¨odinger bridge matching,

    Y . Shi, V . De Bortoli, A. Campbell, and A. Doucet, “Diffusion schr¨odinger bridge matching,”Advances in Neural Information Processing Systems, vol. 36, 2024

    2024

  26. [34]

    Likelihood training of schr ¨odinger bridge using forward-backward sdes theory,

    T. Chen, G. Liu, and E. A. Theodorou, “Likelihood training of schr ¨odinger bridge using forward-backward sdes theory,” in ICLR, 2022

    2022

  27. [35]

    Simplified dif- fusion schr\

    Z. Tang, T. Hang, S. Gu, D. Chen, and B. Guo, “Simplified dif- fusion schr\” odinger bridge,”arXiv preprint arXiv:2403.14623, 2024

    2024 arXiv

  28. [36]

    I 2sb: Image-to-image schr ¨odinger bridge,

    G. Liu, A. Vahdat, D. Huang, E. A. Theodorou, W. Nie, and A. Anandkumar, “I 2sb: Image-to-image schr ¨odinger bridge,” in ICML, 2023, pp. 22 042–22 062

    2023

  29. [37]

    The unreasonable effectiveness of deep features as a perceptual met- ric,

    R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang, “The unreasonable effectiveness of deep features as a perceptual met- ric,” inICLR, 2018, pp. 586–595

    2018

  30. [38]

    Visual acoustic matching,

    C. Chen, R. Gao, P. Calamia, and K. Grauman, “Visual acoustic matching,” inCVPR, 2022, pp. 18 858–18 868

    2022

  31. [39]

    Ima- genet: A large-scale hierarchical image database,

    J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Ima- genet: A large-scale hierarchical image database,” inCVPR, 2009, pp. 248–255

    2009

  32. [40]

    Conditional variational autoencoder with adversarial learning for end-to-end text-to-speech,

    J. Kim, J. Kong, and J. Son, “Conditional variational autoencoder with adversarial learning for end-to-end text-to-speech,” inICML, 2021, pp. 5530–5540

    2021

  33. [41]

    Self-supervised vi- sual acoustic matching,

    A. Somayazulu, C. Chen, and K. Grauman, “Self-supervised vi- sual acoustic matching,” inNeurIPS, 2023

    2023

  34. [42]

    Sep-stereo: Visu- ally guided stereophonic audio generation by associating source separation,

    H. Zhou, X. Xu, D. Lin, X. Wang, and Z. Liu, “Sep-stereo: Visu- ally guided stereophonic audio generation by associating source separation,” inECCV, 2020, pp. 52–69

    2020

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.