Pith. sign in

REVIEW 4 major objections 6 minor 35 references

MeanFlow-Accelerated Multimodal Video-to-Audio Synthesis via One-Step Generation

T0 review · 4 major / 6 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read One-step video-to-audio synthesis from silent video is claimed at RTF 0.007 on an H800 GPU by modeling average velocity instead of instantaneous velocity.

desk verdict A plausible one-step VTA/TTA speedup built on MeanFlow, but the training objective's degeneracy is unaddressed and the empirical evidence is thinner than the 'no compromise' claim. read the letter →

arxiv 2509.06389 v1 pith:GXFOWO25 submitted 2025-09-08 cs.SD cs.AI

Xiaoran Yang , Jianxuan Yang , Xinyue Guo , Haoyu Wang , Ningning Pan , Gongping Huang This is my paper
classification cs.SDcs.AI
keywords video-to-audiosynthesisMeanFlowone-stepgenerationmatchingclassifier-freeguidancemultimodaljointtrainingtext-to-audioinferenceefficiency
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Video-to-audio synthesis normally pays a steep inference cost: flow-matching models estimate instantaneous velocity and need many sampling steps to turn noise into sound. This paper claims that modeling the flow field with average velocity instead removes the need for iteration, giving a native one-step map from noise to audio. On top of a multimodal joint-training backbone, the authors add MeanFlow and a scalar rescaling for classifier-free guidance, and they report a real-time factor of 0.007 on an H800 GPU for both video-to-audio and text-to-audio generation, with no loss in perceptual quality, semantic alignment, or temporal synchronization relative to multi-step baselines. The significance, if correct, is that real-time interactive dubbing and audio generation from silent video become practical on a single GPU.

What carries the argument

The MeanFlow identity is the engine: u(z_t,r,t) = (1/(t-r))∫_r^t v(z_τ,τ)dτ, and differentiating it gives u = v - (t-r) du/dt. The network u_theta is trained to satisfy this identity via a stop-gradient loss, and at inference the identity is used as an exact map, z_r = z_t - (t-r)u(z_t,r,t), which for (r,t)=(0,1) is a single step from noise to audio. The CFG-scaled scalar s projects the conditional velocity onto the unconditional direction to restrain guidance.

What would settle it

A direct test: take the trained MF-MJT model, sample z_t at several (r,t) pairs, and compute both sides of the MeanFlow identity u_theta(z_t,r,t) ≈ v_theta(z_t,t) - (t-r)(v_t ∂_z u_theta + ∂_t u_theta). If the residual is large, the stop-gradient objective has not reached its fixed point. Alternatively, train the same architecture with the standard flow-matching loss (regressing v_t) but keep one-step sampling; if quality metrics stay similar, the average-velocity parametrization is not what drives the one-step result.

Watch

Extended reading notes

Core claim

The paper's central claim is that the average velocity field u(z_t,r,t), defined as the time-average of the instantaneous velocity along the flow between r and t, satisfies a self-consistent identity, and a network trained with a stop-gradient objective to respect that identity can generate audio in one step: z0 = z1 - u(z1,0,1). The authors instantiate this in MF-MJT, a multimodal diffusion-transformer network jointly trained on video, text, and audio, and show that one-step generation reaches RTF 0.007 on VGGSound and AudioCaps while matching or beating Frieren in one step and remaining competitive with MMAudio in multi-step settings. They also introduce an adaptive scalar rescaling of the

Load-bearing premise

The load-bearing premise is that the stop-gradient objective in Eq. (5) actually trains u_theta to approximate the true average velocity field, and that after 400,000 training steps this fixed point is reached; the paper presents this objective without derivation or convergence analysis.

Editorial extensions

If this is right

  • One-step inference at RTF 0.007 on an H800 GPU, with 2x to 500x speedups over the compared VTA and TTA baselines, makes real-time audio synthesis from silent video feasible on a single GPU.
  • The same MeanFlow formulation supports variable numbers of sampling steps: multi-step MF-MJT remains competitive with MMAudio on VGGSound and beats it on several TTA metrics on AudioCaps.
  • Classifier-free guidance can be used in one-step generation without the usual artifacts because the scalar rescaling adaptively damps the unconditional component.
  • Because the network is jointly trained with multimodal conditions, the VTA-trained model transfers to text-to-audio synthesis without fine-tuning.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: if the stop-gradient objective reliably reaches the MeanFlow fixed point, the same one-step trick could be applied to other latent diffusion backbones beyond MMAudio, e.g., image or long-form video generation.
  • Editorial inference: the ablation showing that low ratios of r≠t pairs help semantic and temporal alignment suggests a curriculum that anneals the r≠t ratio during training could improve both one-step fidelity and alignment.
  • Editorial inference: RTF 0.007 is measured on an H800 GPU with fixed sample lengths; the paper does not report latency on consumer GPUs or CPU, so the practical 'real-time' claim beyond H800 is untested.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes MF-MJT, a video-to-audio (VTA) and text-to-audio (TTA) synthesis model built on the MMAudio backbone, replacing instantaneous flow-matching velocity with a MeanFlow average-velocity formulation to enable native one-step generation. A CFG-scaled mechanism adjusts the unconditional prediction with an adaptive scalar to mitigate overshooting under one-step classifier-free guidance. Experiments on VGGSound and AudioCaps report one-step RTF = 0.007 on an H800 GPU, with perceptual metrics (FAD, FD, KL, IS, CLAP/ImageBind, DeSync) claimed to be competitive with or better than baselines including Frieren, MMAudio, and AudioLCM. The central claim is that Eq. (6), z0 = z1 - u(z1,0,1), trained via the stop-gradient objective Eq. (5), yields valid one-step audio generation without distillation or iterative refinement.

Significance. If the MeanFlow formulation is sound and Eq. (5) indeed trains the network to approximate the average velocity field, the contribution is practically significant: native one-step generation with an RTF of 0.007, a 2x speedup over the fastest one-step baseline and orders-of-magnitude speedup over multi-step diffusion/flow baselines, with competitive quality on standard public benchmarks. The paper also proposes a simple adaptive scalar for CFG that is plausible and easy to adopt. However, the central training objective is not derived, and the empirical evidence is limited to single-run automatic metrics without uncertainty quantification. The novelty relative to prior MeanFlow methods (reference [13]) and to the MMAudio backbone is incremental, but the engineering contribution (joint training with MeanFlow for VTA/TTA) could be useful if the theoretical gap is closed.

major comments (4)
  1. [§2.2, Eq. (5)] The stop-gradient training objective LMF = E ||u_theta(zt,r,t) - sg(u_tgt)||^2, where u_tgt = v_t - (t-r)(v_t ∂_z u_theta + ∂_t u_theta), is stated without derivation, proof, or convergence analysis. The MeanFlow identity (4) is only a necessary condition, not a characterization of the average velocity. Indeed, if u_char solves (4), then u_char + K(r)/(t-r) also solves it, since d/dt[K/(t-r)] = -K/(t-r)^2 and (t-r) times that derivative cancels. This degeneracy means the self-consistency target does not uniquely determine the true average velocity field. At inference r=0,t=1, a spurious component K(0) directly shifts z0 = z1 - u(z1,0,1), potentially invalidating the generated audio. No boundary condition, initialization argument, or theorem is given to show that the proposed training selects K=0. The paper cites [13] without reproducing the essential result. This gap is load-bearing for
  2. [§3.3 and Tables 1-2] The experimental evidence for the 'no compromise' claim rests on single-run automatic metrics with no error bars, confidence intervals, or significance testing. Table 1 reports FAD, FD, KL, IS, IB, and DeSync for each method as point estimates; Table 2 similarly. Given that the proposed method often improves metrics by small margins (e.g., IS 9.39 vs 9.14 for Frieren one-step; CLAP 0.20 vs 0.19 for AudioLCM one-step), it is impossible to assess whether these differences are meaningful or noise. The paper should provide at least repeated evaluations (e.g., over multiple seeds or test subsets) or statistical tests. The absence of uncertainty quantifies particularly weakens the claim that MF-MJT preserves quality while accelerating.
  3. [§4.2, Fig. 4] The ablation on the r≠t sampling ratio only tests 10%, 30%, 60%, and 90%, and observes monotonic improvement as the ratio decreases. This does not demonstrate that 10% is optimal; 0% may be even better, which would undermine the necessity of MeanFlow training pairs. The explanation that r=t pairs provide 'direct supervision' is post hoc and not quantitatively supported. Since the r≠t ratio is a free hyperparameter, the choice of 10% should be justified with a more complete sweep or a clear saturation argument.
  4. [§2.3, Eq. (9)] The scalar s = (u_theta(c)^T u_theta(∅)) / ||u_theta(∅)||^2 is an adaptive projection of the conditional velocity onto the unconditional direction. The paper states this 'reflects their alignment' and 'stabilizes guidance,' but provides no analysis of why this scalar mitigates overshooting in one-step generation. Since CFG-scaled is a stated contribution and is used in all main results, a derivation or at least a geometric intuition with a controlled comparison (e.g., with CFG-Zero* or other rescaling heuristics) is needed. Without it, the mechanism remains an ad hoc fix, and the reported gains in Fig. 3 could be attributed to other factors.
minor comments (6)
  1. [§3.4] Typo: 'Notebly' should be 'Notably'.
  2. [Table 2 caption] Typo: 'oth single-step and multi-step' should be 'both single-step and multi-step'.
  3. [§2.1] The text renders 'V AE' with a space; likely a formatting issue. Also, 'Meanflow' versus 'MeanFlow' capitalization is inconsistent across the paper.
  4. [§3.2] The sentence 'Notably, the network does not condition directly on (r,t), but rather on (t,Δt)' is unclear: the model is given t and Δt, but the logit-normal sampling described above draws r and t. How are these encoded and combined? A clearer description of the conditioning mechanism would help reproducibility.
  5. [§2.2] The claim 'this objective reduces to the standard FM loss when r=t' is stated without showing the algebra. Since r=t makes the denominator in Eq. (3) zero and Eq. (4) degenerate, the limiting case requires explicit handling. Please clarify.
  6. [§4.1, Table 1] The baseline parameter counts and sampling steps are given, but no wall-clock times are reported for baselines beyond RTF. For multi-step methods, RTF depends on the number of steps (fixed at 25); reporting RTF as a function of steps would make the efficiency comparison more transparent.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: MeanFlow is applied from external prior work, and all headline results are measured against external benchmark metrics.

full rationale

The central one-step map z0 = z1 - u(z1,0,1) is a direct rearrangement of the average-velocity definition in Eq. (3); it is not fitted or defined in terms of the model output, so it is not circular by construction. The training objective Eq. (5) is a stop-gradient self-consistency loss inherited from the external MeanFlow paper [13], not from the authors' own prior work. While the paper does not reproduce the convergence or correctness proof of Eq. (5), that is an unsupported assumption or correctness risk, not a circularity: the target utgt is not the same as the model output u_theta, and the loss does not reduce to a prediction of a fitted value. The CFG scalar s in Eq. (9) is an adaptive projection computed from the model's own conditional and unconditional outputs at inference time, not a fitted constant, so it is not a "fitted input called prediction." All reported quality, alignment, and synchronization numbers are evaluated with external metrics (FAD, FD, KL, IS, ImageBind, CLAP, DeSync) on held-out VGGSound and AudioCaps test sets, so the empirical claims do not rely on the paper's own outputs as ground truth. There are no load-bearing self-citations: the cited backbone [8], MeanFlow [13], and CFG-Zero [15] are all external works with no author overlap. The skeptical concern about spurious K(r)/(t-r) solutions to the MeanFlow identity is a legitimate theoretical gap in justifying Eq. (5), but it is not a circular reduction—Eq. (6) does not assume the conclusion by definition. Therefore the derivation chain is self-contained up to an external theoretical premise, and no circular step is exhibited.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

No new physical entities are introduced; the scalar rescaling is an algorithmic mechanism. The central claim rests on the external MeanFlow objective, pretrained encoders, and hyperparameters tuned on the test set.

free parameters (4)
  • CFG guidance strength omega = 1.5 (one-step), 4.5 (multi-step)
    Chosen for best IS in Fig 3, i.e., tuned on the VGGSound test set shown; no held-out validation reported (Sec 3.2, Sec 4.2).
  • Ratio of r!=t training pairs = 10%
    Selected because Fig 4 shows 10% is best among {10,30,60,90}%; 0% is not tested. This directly affects semantic alignment (IB) and synchronization (DeSync).
  • Logit-normal parameters (mu, sigma) for sampling (r,t) = mu=-2.0, sigma=2.0
    Used to sample two time steps; no ablation or justification is given in Sec 3.2.
  • Condition dropout probability = 0.1
    Standard CFG dropout from [14]; not ablated in this paper (Sec 2.3).
assumptions (5)
  • domain assumption The MeanFlow identity Eq (4) and the stop-gradient training loss Eq (5) from [13] are correct and converge to the true average velocity field.
    The paper states Eq (5) without derivation or convergence analysis; all one-step generation claims depend on it.
  • domain assumption Pretrained CLIP, Synchformer, and VAE encoders provide embeddings whose alignment is sufficient for cross-modal generation.
    No fine-tuning or validation of encoder alignment is reported (Sec 2.1).
  • domain assumption Automatic metrics (FAD, FD, KL, IS, IB, CLAP, DeSync) are valid proxies for perceptual audio quality and synchronization.
    No human listening test is reported; 'perceptual quality' claims rest on these automatic metrics (Sec 3.3).
  • domain assumption Using empty visual tokens for audio-text data during joint training transfers to TTA without fine-tuning.
    Sec 3.1; the TTA results depend on this transfer working as described.
  • domain assumption Omitting absolute positional encodings allows variable-length synthesis.
    Sec 3.2 states this enables 8s and 10s audio, but no length-generalization evidence is shown.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MeanFlow-Accelerated Multimodal Video-to-Audio Synthesis via One-Step Generation." pith.science (2026). https://pith.science/paper/GXFOWO25

@misc{pith2026250906389,
  author       = {Pith},
  title        = {Pith review of: MeanFlow-Accelerated Multimodal Video-to-Audio Synthesis via One-Step Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GXFOWO25}},
  note         = {Machine review of arXiv:2509.06389}
}
read the original abstract

A key challenge in synthesizing audios from silent videos is the inherent trade-off between synthesis quality and inference efficiency in existing methods. For instance, flow matching based models rely on modeling instantaneous velocity, inherently require an iterative sampling process, leading to slow inference speeds. To address this efficiency bottleneck, we introduce a MeanFlow-accelerated model that characterizes flow fields using average velocity, enabling one-step generation and thereby significantly accelerating multimodal video-to-audio (VTA) synthesis while preserving audio quality, semantic alignment, and temporal synchronization. Furthermore, a scalar rescaling mechanism is employed to balance conditional and unconditional predictions when classifier-free guidance (CFG) is applied, effectively mitigating CFG-induced distortions in one step generation. Since the audio synthesis network is jointly trained with multimodal conditions, we further evaluate it on text-to-audio (TTA) synthesis task. Experimental results demonstrate that incorporating MeanFlow into the network significantly improves inference speed without compromising perceptual quality on both VTA and TTA synthesis tasks.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

35 extracted references · 29 canonical work pages

  1. [13]

    MMAudio: Tam- ing multimodal joint training for high-quality video-to-audio synthesis,

    H. K. Cheng, M. Ishii, A. Hayakawa,et al, “MMAudio: Tam- ing multimodal joint training for high-quality video-to-audio synthesis,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025

    work page 2025

  2. [1]

    MeanFlow-Accelerated Multimodal Video-to-Audio Synthesis via One-Step Generation

    INTRODUCTION Video-to-audio (VTA) synthesis focus on generating semantically aligned and temporally synchronized audios given silent video in- puts, with applications such as video dubbing. Existing approaches can be broadly divided into three paradigms. The first trains mod- els solely on paired audio-visual data from scratch [1, 2]. How- ever, paired da...

    work page Pith review arXiv 2025

  3. [2]

    On top of this backbone, MeanFlow formulation is introduced which directly models the average velocity to enable native one-step generation

    METHOD The proposed MF-MJT builds upon a joint training backbone, known as MMAudio [8], that integrates video, audio, and text modalities within a unified framework. On top of this backbone, MeanFlow formulation is introduced which directly models the average velocity to enable native one-step generation. Further- more, to stabilize CFG under one-step gen...

    work page

  4. [3]

    Multimodal Dataset The proposed MF-MJT is trained on multimodal datasets comprising both audio-video-text and audio-text pairs

    EXPERIMENTAL SETUP 3.1. Multimodal Dataset The proposed MF-MJT is trained on multimodal datasets comprising both audio-video-text and audio-text pairs. Specifically, VGGSound (approximately 500 hours) [22] and Kling-Audio-Eval (around 58 hours) [9] provide audio-video-text triplets. To expand the train- ing data coverage, we also include audio-text datase...

    work page

  5. [4]

    Comparison with Baselines Table 1 summarizes the performance of the proposed MF-MJT against representative VTA synthesis baselines on the VGGSound test set

    RESULTS 4.1. Comparison with Baselines Table 1 summarizes the performance of the proposed MF-MJT against representative VTA synthesis baselines on the VGGSound test set. For one-step generation, the MF-MJT outperforms Frieren on the majority of metrics, demonstrating its ability to generate per- ceptually high-quality audio across diverse evaluation dimen...

    work page arXiv

  6. [5]

    CONCLUSION We propose MF-MJT, the first MeanFlow-accelerated framework for multimodal video-to-audio synthesis, enabling native one-step gen- eration by modeling average velocity. To enhance the controllability of classifier-free guidance, we introduce a scalar rescaling mecha- nism (CFG-scaled) that mitigates overshooting without iterative re- finement a...

    work page

  7. [6]

    Frieren: Efficient video- to-audio generation network with rectified flow matching,

    Y . Wang, W. Guo, R. Huang,et al, “Frieren: Efficient video- to-audio generation network with rectified flow matching,”Ad- vances in Neural Information Processing Systems (NeurIPS), vol. 37, pp. 128118–128138, 2024

    work page 2024

  8. [7]

    LoV A: Long-form video- to-audio generation,

    X. Cheng, X. Wang, Y . Wu,et al, “LoV A: Long-form video- to-audio generation,” inIEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1–5, 2025

    work page 2025

Show all 35 references
  1. [8]

    AudioLDM: Text-to-audio generation with latent diffusion models,

    H. Liu, Z. Chen, Y . Yuan,et al, “AudioLDM: Text-to-audio generation with latent diffusion models,” inProceedings of the International Conference on Machine Learning (ICML), pp. 21450–21474, 2023

    2023

  2. [9]

    ImageBind: One embedding space to bind them all,

    R. Girdhar, A. El-Nouby, Z. Liu,et al, “ImageBind: One embedding space to bind them all,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition (CVPR), 2023

    2023

  3. [10]

    Seeing and Hearing: Open- domain visual-audio generation with diffusion latent aligners,

    Y . Xing, Y . He, Z. Tian,et al, “Seeing and Hearing: Open- domain visual-audio generation with diffusion latent aligners,” inProceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition (CVPR), 2024

    2024

  4. [11]

    FoleyCrafter: Bring silent videos to life with lifelike and synchronized sounds,

    Y . Zhang, Y . Gu, Y . Zeng,et al, “FoleyCrafter: Bring silent videos to life with lifelike and synchronized sounds,”arXiv preprint arXiv:2407.01494, 2024

    2024 arXiv

  5. [12]

    TA-V2A: Textually assisted video- to-audio generation,

    Y . You, X. Wu, and T. Qu, “TA-V2A: Textually assisted video- to-audio generation,” inIEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1–5, 2025

    2025

  6. [14]

    Kling-Foley: Multimodal diffusion transformer for high-quality video-to-audio genera- tion,

    J. Wang, X. Zeng, C. Qiang,et al,“Kling-Foley: Multimodal diffusion transformer for high-quality video-to-audio genera- tion,”arXiv preprint arXiv:2506.19774, 2025

    2025

  7. [15]

    Denoising diffusion probabilis- tic models,

    J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilis- tic models,”Advances in Neural Information Processing Sys- tems (NeurIPS), 2020

    2020

  8. [16]

    Flow straight and fast: Learn- ing to generate and transfer data with rectified flow,

    X. Liu, C. Gong, and qiang liu, “Flow straight and fast: Learn- ing to generate and transfer data with rectified flow,” inPro- ceedings of the International Conference on Learning Repre- sentations (ICLR), 2023

    2023

  9. [17]

    Instaflow: One step is enough for high-quality diffusion-based text-to-image generation,

    X. Liu, X. Zhang, J. Ma,et al, “Instaflow: One step is enough for high-quality diffusion-based text-to-image generation,” in Proceedings of the International Conference on Learning Rep- resentations (ICLR), 2024

    2024

  10. [18]

    Mean flows for one-step generative modeling,

    Z. Geng, M. Deng, X. Bai,et al, “Mean flows for one-step generative modeling,”arXiv preprint arXiv:2505.13447, 2025

    2025 arXiv

  11. [19]

    Classifier-free diffusion guidance,

    J. Ho and T. Salimans, “Classifier-free diffusion guidance,” arXiv preprint arXiv:2207.12598, 2022

    2022 arXiv

  12. [20]

    CFG-Zero*: Im- proved classifier-free guidance for flow matching models,

    W. Fan, A. Y . Zheng, R. A. Yeh,et al, “CFG-Zero*: Im- proved classifier-free guidance for flow matching models,” arXiv preprint arXiv:2503.18886, 2025

    2025 arXiv

  13. [21]

    Scaling rectified flow transformers for high-resolution image synthesis,

    P. Esser, S. Kulal, A. Blattmann,et al, “Scaling rectified flow transformers for high-resolution image synthesis,” inProceed- ings of the International Conference on Machine Learning (ICML), 2024

    2024

  14. [22]

    Scalable diffusion models with trans- formers,

    W. Peebles and S. Xie, “Scalable diffusion models with trans- formers,” inProceedings of the IEEE/CVF International Con- ference on Computer Vision (ICCV), pp. 4172–4182, 2023

    2023

  15. [23]

    Learning transfer- able visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy,et al, “Learning transfer- able visual models from natural language supervision,” inPro- ceedings of the International Conference on Machine Learning (ICML), 2021

    2021

  16. [24]

    A versatile diffusion transformer with mixture of noise levels for audiovisual gen- eration,

    G. Kim, A. Martinez, Y .-C. Su,et al, “A versatile diffusion transformer with mixture of noise levels for audiovisual gen- eration,”Advances in Neural Information Processing Systems (NeurIPS), vol. 378, p. 29, 2025

    2025

  17. [25]

    Synchformer: Efficient syn- chronization from sparse cues,

    V . Iashin, W. Xie, E. Rahtu,et al, “Synchformer: Efficient syn- chronization from sparse cues,” inInternational Conference on Acoustics, Speech, and Signal Processing (ICASSP), pp. 5325– 5329, 2024

    2024

  18. [26]

    Score-based generative modeling through stochastic differential equations,

    Y . Song, J. Sohl-Dickstein, D. P. Kingma,et al, “Score-based generative modeling through stochastic differential equations,” inProceedings of the International Conference on Learning Representations (ICLR), 2021

    2021

  19. [27]

    VGGSound: A large-scale audio-visual dataset,

    H. Chen, W. Xie, A. Vedaldi,et al, “VGGSound: A large-scale audio-visual dataset,” inInternational Conference on Acous- tics, Speech, and Signal Processing (ICASSP), 2020

    2020

  20. [28]

    AudioCaps: Generating cap- tions for audios in the wild,

    C. D. Kim, B. Kim, H. Lee,et al, “AudioCaps: Generating cap- tions for audios in the wild,” inProceedings of the Conference of the North American Chapter of the Association for Compu- tational Linguistics: Human Language Technologies, 2019

    2019

  21. [29]

    WavCaps: A ChatGPT- assisted weakly-labelled audio captioning dataset for audio- language multimodal research,

    X. Mei, C. Meng, H. Liu,et al, “WavCaps: A ChatGPT- assisted weakly-labelled audio captioning dataset for audio- language multimodal research,”IEEE/ACM Transactions on Audio, Speech, and Language Processing, pp. 1–15, 2024

    2024

  22. [30]

    Decoupled weight decay regular- ization,

    I. Loshchilov and F. Hutter, “Decoupled weight decay regular- ization,”arXiv preprint arXiv:1711.05101, 2019

    2019 arXiv

  23. [31]

    Audio Set: An ontology and human-labeled dataset for audio events,

    J. F. Gemmeke, D. P. W. Ellis, D. Freedman,et al, “Audio Set: An ontology and human-labeled dataset for audio events,” inIEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 776–780, 2017

    2017

  24. [32]

    PANNs: Large-scale pre- trained audio neural networks for audio pattern recognition,

    Q. Kong, Y . Cao, T. Iqbal,et al, “PANNs: Large-scale pre- trained audio neural networks for audio pattern recognition,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 28, pp. 2880–2894, 2020

    2020

  25. [33]

    Efficient train- ing of audio transformers with patchout,

    K. Koutini, J. Schl ¨uter, H. Eghbal-zadeh,et al, “Efficient train- ing of audio transformers with patchout,” inProceedings of the International Speech Communication Association, pp. 2753– 2757, 2022

    2022

  26. [34]

    CLAP: Learn- ing audio concepts from natural language supervision,

    B. Elizalde, S. Deshmukh, M. A. Ismail,et al, “CLAP: Learn- ing audio concepts from natural language supervision,” inIn- ternational Conference on Acoustics, Speech, and Signal Pro- cessing (ICASSP), pp. 1–5, 2023

    2023

  27. [35]

    AudioLCM: Efficient and high-quality text-to-audio generation with minimal inference steps,

    H. Liu, R. Huang, Y . Liu,et al, “AudioLCM: Efficient and high-quality text-to-audio generation with minimal inference steps,” inACM International Conference on Multimedia (ACM MM), p. 7008–7017, 2024

    2024

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.