Pith. sign in

REVIEW 4 major objections 6 minor 50 references

AI-Based Sound Effect Generation: A Narrative Review of Generative Models Across Input Modalities

T0 review · 4 major / 6 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A review of 30 studies finds AI sound-effect generation now reaches high fidelity and prompt alignment, while timing, metric-perception gaps, and control-versus-diversity trade-offs persist.

desk verdict A useful taxonomy, but the Auffusion misdescription and uncritical SOTA claims mean the synthesis needs corrections before I'd trust it. read the letter →

arxiv 2608.03742 v1 pith:C4KUYBMI submitted 2026-08-04 cs.SD cs.AI

Sandy Abdo , Bill Kapralos , Priyamvada Tripathi , KC Collins , Adam Dubrowski This is my paper
classification cs.SDcs.AI
keywords soundeffectgenerationgenerativeAIlatentdiffusionmodelstext-to-audiovideo-to-audiomultimodaltemporalalignmentnarrativereview
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The chapter surveys 30 peer-reviewed AI models that generate sound effects, organized by what drives them: text, visuals, audio, or several inputs at once. Its central claim is that the field has genuinely converged — latent diffusion models conditioned by language or vision encoders now produce high-fidelity sound effects that match their prompts semantically, and temporal coherence is improving. The review also argues that the remaining hard problems are specific and measurable: synchronizing sound to complex multi-event scenes, the gap between automated scores and what human listeners report, and the tug-of-war between user control and generative variety. The stakes are practical: sound design for games, film, VR, and interactive media needs thousands of varied, context-adaptive sounds, and these tools promise to automate much of that labor and lower the barrier for small studios.

What carries the argument

The argument is carried by two organizing devices. First, a taxonomy of input modalities (text, visual, audio, multimodal) that sorts the 30 models into comparability groups and lets the review trace how each modality conditions generation. Second, a two-track evaluation grid: objective distribution metrics (FD/FAD, FID, KID, IS, KL, CLAPScore, F1) contrasted with subjective human ratings (OVL/OVR, REL, MOS, AQ, SA, TA). The load-bearing mechanism is the latent diffusion model (LDM) — a diffusion process run in a compressed audio-latent space — paired with a pretrained cross-modal encoder, the architecture that recurs across all four categories and is credited for the fidelity-and-alignment

What would settle it

Audit the 30 model summaries against their cited papers. One concrete check already exists: the review describes Auffusion as generating an image that is 'denormalized' into a mel-spectrogram, a pipeline the cited Auffusion paper does not use. If a systematic audit finds similar drift across many summaries, or if re-running the claimed state-of-the-art comparisons on a single shared benchmark reverses the reported rankings, the review's central conclusion would need revision.

Watch

Extended reading notes

Core claim

Across all four input modalities, the same recipe keeps winning, the review finds: a latent diffusion model steered by a cross-modal encoder — CLAP for text-audio alignment, an LLM such as Flan-T5 for richer prompts, or a vision-language embedding for video. Models on this pattern (AudioLDM and successors, Tango 2, FoleyGen, Smooth-Foley) are reported to beat earlier waveform-domain and GAN baselines on distribution metrics (FAD/FD, IS, KL, CLAPScore) and on human ratings of quality and relevance. The headline finding: state-of-the-art performance is now routine, and the frontier has shifted to temporal precision — exact onset timing, event ordering, video sync. The second finding: that fron

Load-bearing premise

The review's central 'steady progress' narrative rests on trusting the 30 surveyed papers' self-reported benchmark results as if they were comparable, even though the numbers come from different datasets, protocols, and raters — and at least one model summary in the review does not match its source paper.

Editorial extensions

If this is right

  • Latent diffusion with a language or vision encoder becomes the default architecture to beat: new sound-effect models will likely be judged mainly on temporal controllability and inference speed rather than raw fidelity.
  • Text becomes the universal steering wheel: even video-to-audio systems increasingly route visual content through text or vision-language embeddings, so progress in text-to-audio transfers almost directly to other modalities.
  • Evaluation will have to catch up: as models saturate FAD/IS/KL and CLAPScore, timing-aware metrics and perceptually grounded tests will separate the next generation of systems.
  • Sound-design workflows shift: designers move from finding and editing clips to supervising generated candidates, and small studios gain access to professional-grade effects without large audio libraries.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The three persistent challenges the review names may be one bottleneck wearing three hats: weak temporal conditioning could explain both the multi-event synchronization failures and parts of the metric-perception gap, since distribution metrics barely register timing errors.
  • If the metric-perception gap is real, benchmark rankings may soon be settled by large listening panels rather than by FAD or CLAPScore; a testable consequence is that leaderboards would re-rank under human scoring on identical outputs.
  • Because several video-to-audio systems route through text or vision-language embeddings, progress in text-to-audio likely transfers nearly for free to other modalities, suggesting a single unified 'describe-then-synthesize' interface could absorb much of the field.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. This manuscript is a narrative review of 30 peer-reviewed articles on AI-based sound effect generation, organized by input modality (text-to-audio, visual-to-audio, audio-to-audio, and multimodal). It surveys model architectures, training data, and evaluation metrics, and synthesizes the field's trajectory as one of rapid progress toward high-fidelity, semantically aligned, and increasingly temporally coherent generation, while acknowledging persistent gaps in temporal synchronization, metric-perception alignment, and controllability-diversity trade-offs. The search process is documented with a PRISMA flow diagram, and the review is aimed at newcomers to the field.

Significance. If the per-model summaries are faithful, this review provides a useful structured map of a fast-moving area and supports a plausible conclusion that latent diffusion with language or vision encoders has become the dominant recipe. The explicit documentation of the search and inclusion process is a strength, as is the organization by modality. However, the central synthesis inherits all of the accuracy of its primary-paper summaries, and the manuscript contains at least one concrete misdescription of a cited architecture. The 'state-of-the-art' claims are largely self-reports from the primary papers, with no critical appraisal of differences in datasets, baselines, or evaluation protocols. The review is therefore informative as a survey but not yet reliable as an authoritative assessment of progress.

major comments (4)
  1. [Section 3, Auffusion paragraph] The description of Auffusion (Xue et al. [50]) is factually incorrect. The text states: 'Auffusion uses a text prompt to generate a latent representation, which is then reconstructed into an image by the VAE decoder. This image is subsequently denormalized into a mel-spectrogram and synthesized into audio.' The cited paper is a text-to-audio model; its VAE operates on mel-spectrogram latents, not image latents, and no intermediate image is generated. This is not a local typo: it misrepresents the model's modality and suggests a possible conflation with text-to-image pipelines. Because the review's aggregate claims about dominant architectures and progress rest on accurate per-model summaries, this error is load-bearing. It must be corrected and the other 29 summaries audited for similar drift.
  2. [Section 3 and Section 4, per-model 'state-of-the-art' claims] The review repeatedly states that models 'achieved state-of-the-art performance' or 'outperformed previous state-of-the-art' (e.g., AudioLDM, Tango, AudioLDM2, Tango 2, Re-AudioLDM, FoleyGAN, FRIEREN, SonicVisionLM, STA-V2A), but these claims are taken from the primary papers without critical comparison. The baselines, datasets, and metrics differ across papers, so 'state-of-the-art' is asserted rather than demonstrated. At minimum, the review should qualify such claims with the specific comparison context (dataset, baselines, metric) and note where results are single-run or self-reported. Without this, the central conclusion that the field is converging on a specific recipe is not independently supported.
  3. [Section 2.1 and References] The inclusion criteria state that non–peer-reviewed preprints, theses, patents, and technical reports were excluded, and the search pool was limited to peer-reviewed papers. However, several primary sources are cited as arXiv preprints (e.g., AudioLDM [25], AudioGen [23], Segment Anything [22], VGGSound [3], AudioTime [47]). This is an inconsistency between the stated methodology and the actual evidence base. The authors should either use published versions where available or clarify how these sources satisfied the peer-review criterion.
  4. [Section 3, subjective evaluations] Several conclusions about audio quality and alignment rest on very small subjective evaluations: six participants for Tango, eight for SRC-gAudio and SonifyAR, ten for PicoAudio and Smooth-Foley, and twenty in other studies. The review reports these results without noting the low statistical power or the risk of evaluator bias. This matters because the manuscript explicitly identifies a gap between objective metrics and human perception; its own summaries should therefore be cautious when citing small-n subjective studies as evidence of 'superior' performance.
minor comments (6)
  1. [Table 1] The table entry for Auffusion lists 'Pixel VAE + LDM' as its architecture. Given the error in Section 3, this label is misleading for a text-to-audio model; consider 'VAE (mel-spectrogram) + LDM' or similar.
  2. [Section 3, Tango 2 paragraph] The paragraph describes Tango 2 as 'using a diffusion model, the system is trained on extensive datasets' without specifying the architecture, training data, or the role of DPO beyond a general mention. More precision would help readers compare it with other TTA models.
  3. [Section 4, MIMOSA paragraph] The acronym MIMOSA is expanded as 'Magnifying Immersion by Manipulating Objects in Spatial Audio,' which is not a natural expansion of MIMOSA. Verify the intended phrase or correct the expansion.
  4. [Section 6, VAMG paragraph] The model is called VAMG in the text and Table 1, but reference [18] is titled 'VAG: A Uniform Model for Cross-Modal Visual-Audio Mutual Generation.' Standardize the name to match the cited paper.
  5. [Figures 2–5] The captions state 'n represents the number of studies/works in this group,' but the figures are not rendered in this text. Ensure the final version includes legible figures and that the counts correspond exactly to Table 3's categories.
  6. [Section 2, last paragraph] The phrase 'four key themes identified in the literature' includes 'visual-to-audio models, which take images or videos as input,' but Section 4 contains no image-only model distinct from video; this is a minor organizational mismatch.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the review's conclusions are a synthesis of external primary papers; the sole self-citation is background, and the Auffusion pipeline misdescription is a factual accuracy issue, not a circular reduction.

full rationale

This manuscript is a narrative literature review, not a derivation. It contains no fitted parameters, no equations, and no quantity computed by the authors from their own outputs; the central claim that 'multiple models achieved state-of-the-art performance' is an aggregation of the 30 surveyed papers' self-reported evaluations. That aggregation is not circular: the review does not define its categories in terms of this conclusion, does not fit any parameter that is then 'predicted' back, and does not rely on the authors' prior results. The only self-citation is Collins ([6]), used in the introduction to motivate the importance and variation requirements of game sound; it is background context and does not carry the review's load-bearing conclusions. I do flag one concrete accuracy problem in Section 3: the Auffusion summary (Xue et al. [50]) says 'Auffusion uses a text prompt to generate a latent representation, which is then reconstructed into an image by the VAE decoder. This image is subsequently denormalized into a mel-spectrogram and synthesized into audio,' which misdescribes the cited text-to-audio pipeline and is not what the cited paper does. This is a correctness/support risk for the aggregate conclusions, because it shows the review's summaries of primary papers can drift, but it is not a circular step: the review's thesis is not true by construction of its inputs. No self-citation chain, imported uniqueness theorem, ansatz smuggled by citation, or renaming of a known result is present. The review would be strengthened by checking each summary against its source, but that check being absent is a verification gap, not circularity.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The central synthesis rests on the reliability and comparability of the primary literature, on the completeness of the three-database search, and on the accuracy of the authors' summaries. No free parameters are fit; no entities are invented.

assumptions (3)
  • domain assumption The 30 included primary papers are peer-reviewed and their reported evaluations are accurate.
    Invoked in Section 2.1 and throughout Sections 3-6; the review reproduces each model's self-reported metrics and 'state-of-the-art' claims without independent verification.
  • domain assumption Metrics reported across different datasets and protocols are comparable enough to support aggregate statements about field-level progress.
    Underlies claims like 'multiple models achieved state-of-the-art performance' in the abstract; no cross-dataset normalization or meta-analysis is performed.
  • domain assumption The narrative review framework (Oxman et al. 1994) licenses qualitative synthesis without formal quality appraisal.
    Section 2 states the review follows a narrative structure, which shapes the interpretation of the 30 papers and permits conclusions based on the authors' own summaries.

how reviews work

0 comments
Cite this review

Pith. "Pith review of AI-Based Sound Effect Generation: A Narrative Review of Generative Models Across Input Modalities." pith.science (2026). https://pith.science/paper/C4KUYBMI

@misc{pith2026260803742,
  author       = {Pith},
  title        = {Pith review of: AI-Based Sound Effect Generation: A Narrative Review of Generative Models Across Input Modalities},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/C4KUYBMI}},
  note         = {Machine review of arXiv:2608.03742}
}
read the original abstract

Sound effects play a crucial role in conveying actions, events, and environmental cues across digital applications, often requiring a high degree of variation and contextual adaptability. Artificial intelligence (AI)-driven audio generative models are rapidly growing in popularity and have the potential to transform the way sound is synthesized and used across various applications. In response to this growing momentum, this chapter reviews and analyzes recent AI-based generative models for sound effect synthesis, with a focus on how different input modalities (text, visual, audio, and multimodal) affect the quality, controllability, and contextual relevance of the generated audio. It examines 30 peer-reviewed articles sourced from Google Scholar, IEEE Xplore, and the ACM Digital Library, exploring the evolution of AI generative models over the past five years. The results show that multiple models achieved state-of-the-art performance, producing high-fidelity, semantically aligned, and increasingly temporally coherent sound effects across tasks. However, despite these advances, the review identifies persistent challenges, including limitations in temporal synchronization for complex multi-event scenarios, gaps between objective metrics and human perception, and trade-offs between controllability and generative diversity. Overall, the chapter highlights that AI-driven sound effect generation is progressing toward more adaptive, scalable, and context-aware systems, offering significant implications for future sound design workflows and interactive media applications.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

50 extracted references · 24 canonical work pages

  1. [50]

    IEEE/ACM Transactions on Audio, Speech, and Language Processing32, 4700–4712 (2024)

    Xue, J., Deng, Y., Gao, Y., Li, Y.: Auffusion: Leveraging the Power of Diffusion and Large Language Models for Text-to-Audio Generation. IEEE/ACM Transactions on Audio, Speech, and Language Processing32, 4700–4712 (2024). DOI 10.1109/TASLP.2024.3485485

    work page arXiv 2024

  2. [25]

    DOI 10.48550/arXiv.2301.12503

    Liu, H., Chen, Z., Yuan, Y., Mei, X., Liu, X., Mandic, D., Wang, W., Plumbley, M.D.: AudioLDM: Text-to-Audio Generation with Latent Diffusion Models (2023). DOI 10.48550/arXiv.2301.12503. URLhttp://arxiv.org/abs/2301.12503. ArXiv:2301.12503 [cs]

    arXiv doi:10.48550/arxiv.2301.12503 2023

  3. [23]

    DOI 10.48550/arXiv.2209.15352

    Kreuk, F., Synnaeve, G., Polyak, A., Singer, U., D ´efossez, A., Copet, J., Parikh, D., Taigman, Y., Adi, Y.: AudioGen: Textually Guided Audio Generation (2023). DOI 10.48550/arXiv.2209.15352. URLhttp://arxiv.org/abs/2209.15352. ArXiv:2209.15352 [cs] AI-Based Sound Effect Generation 27

    arXiv doi:10.48550/arxiv.2209.15352 2023

  4. [22]

    DOI 10.48550/arXiv

    Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A.C., Lo, W.Y., Doll´ar, P., Girshick, R.: Segment Anything (2023). DOI 10.48550/arXiv. 2304.02643. URLhttp://arxiv.org/abs/2304.02643. ArXiv:2304.02643 [cs]

    arXiv doi:10.48550/arxiv.2304.02643 2023

  5. [3]

    DOI 10.48550/arXiv.2004.14368

    Chen, H., Xie, W., Vedaldi, A., Zisserman, A.: VGGSound: A Large-scale Audio-Visual Dataset (2020). DOI 10.48550/arXiv.2004.14368. URLhttp://arxiv.org/abs/2004.14368. ArXiv:2004.14368 [cs]

    arXiv doi:10.48550/arxiv.2004.14368 2020

  6. [47]

    AudioTime: A Temporally-aligned Audio-text Benchmark Dataset

    Xie, Z., Xu, X., Wu, Z., Wu, M.: AudioTime: A Temporally-aligned Audio-text Benchmark Dataset (2024). DOI 10.48550/arXiv.2407.02857. URLhttp://arxiv.org/abs/2407.02857. ArXiv:2407.02857 [cs]

    work page Pith review arXiv doi:10.48550/arxiv.2407.02857 2024

  7. [1]

    IEEE/ACM Transactions on Audio, Speech, and Language Processing32, 1573–1585 (2024)

    Barahona-R ´ıos, A., Collins, T.: NoiseBandNet: controllable time-varying neural synthesis of sound effects using filterbanks. IEEE/ACM Transactions on Audio, Speech, and Language Processing32, 1573–1585 (2024). DOI 10.1109/TASLP.2024.3364616. URLhttps://ieeexplore.ieee. org/document/10440034/

    work page arXiv 2024

  8. [2]

    Computer Vision – ECCV 2024, Springer15128, 277–295 (2025)

    Chen, C., Peng, P., Baid, A., Xue, Z., Hsu, W.N., Harwath, D., Grauman, K.: Action2Sound: Ambient-Aware Generation of Action Sounds from Egocentric Videos. Computer Vision – ECCV 2024, Springer15128, 277–295 (2025). DOI 10.1007/978-3-031-72897-6 16. URL https://link.springer.com/10.1007/978-3-031-72897-6_16

    work page doi:10.1007/978-3-031-72897-6 2024

Show all 50 references
  1. [4]

    In: 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp

    Chen, K., Wu, Y., Liu, H., Nezhurina, M., Berg-Kirkpatrick, T., Dubnov, S.: MusicLDM: En- hancing Novelty in text-to-music Generation Using Beat-Synchronous mixup Strategies. In: 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1206–

    2024

  2. [6]

    The MIT Press, Cambridge, Mass (2008)

    Collins, K.: Game Sound: An Introduction to the History, Theory, and Practice of Video Game Music and Sound Design. The MIT Press, Cambridge, Mass (2008). DOI 10.7551/mitpress/7909.001.0001. URLhttps://direct.mit.edu/books/book/2460/ Game-SoundAn-Introduction-to-the-History-Theory

    2008 doi

  3. [7]

    ACM Computing Surveys56(11), 1–34 (2024)

    Dash, A., Agres, K.: AI-based affective music generation systems: a review of methods and challenges. ACM Computing Surveys56(11), 1–34 (2024). DOI 10.1145/3672554. URLhttps: //dl.acm.org/doi/10.1145/3672554

    2024 doi

  4. [8]

    In: ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 26 Abdoet al

    Drossos, K., Lipping, S., Virtanen, T.: Clotho: an Audio Captioning Dataset. In: ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 26 Abdoet al. pp. 736–740. IEEE, Barcelona, Spain (2020). DOI 10.1109/ICASSP40776.2020.9052990...

    2020

  5. [9]

    CA, USA (2020)

    Engel, J., Hantrakul, L., Gu, C., Roberts, A.: DDSP: differentiable digital signal processing. CA, USA (2020). URLhttps://openreview.net/pdf?id=B1x1ma4tDr

    2020

  6. [11]

    DOI 10.48550/arXiv.2404.10301

    Evans, Z., Parker, J.D., Carr, C.J., Zukowski, Z., Taylor, J., Pons, J.: Long-form music generation with latent diffusion (2024). DOI 10.48550/arXiv.2404.10301. URLhttp://arxiv.org/abs/ 2404.10301. ArXiv:2404.10301 [cs]

    2024 doi arXiv

  7. [12]

    Focal Press, London (2021)

    Filimowicz, M.: Doing research in sound design, 1 edn. Focal Press, London (2021). DOI 10.4324/9780429356360. URLhttps://www.taylorfrancis.com/books/9780429356360

    2021

  8. [13]

    In: Proceedings of the 31st ACM International Conference on Multimedia, pp

    Ghosal, D., Majumder, N., Mehrish, A., Poria, S.: Text-to-audio generation using instruction guided latent diffusion model. In: Proceedings of the 31st ACM International Conference on Multimedia, pp. 3590–3598. ACM, Ottawa ON Canada (2023). DOI 10.1145/3581783.3612348. URLhttp...

    2023

  9. [14]

    IEEE Transactions on Multimedia25, 4508– 4519 (2023)

    Ghose, S., Prevost, J.J.: FoleyGAN: Visually Guided Generative Adversarial Network-Based Synchronous Sound Generation in Silent Videos. IEEE Transactions on Multimedia25, 4508– 4519 (2023). DOI 10.1109/TMM.2022.3177894. URLhttps://ieeexplore.ieee.org/ document/9782577/

    2023

  10. [15]

    IEEE Access12, 144631–144651 (2024)

    Ghosh, A., Deepa, K.: QueryMintAI: Multipurpose multimodal large language models for personal data. IEEE Access12, 144631–144651 (2024). DOI 10.1109/ACCESS.2024.3468996. URL https://ieeexplore.ieee.org/document/10695061/

    2024

  11. [16]

    Health Information & Libraries Journal26(2), 91–108 (2009)

    Grant, M.J., Booth, A.: A typology of reviews: an analysis of 14 review types and associ- ated methodologies. Health Information & Libraries Journal26(2), 91–108 (2009). DOI 10.1111/j.1471-1842.2009.00848.x. URLhttps://onlinelibrary.wiley.com/doi/10. 1111/j.1471-1842.2009.00848.x

    2009

  12. [17]

    Campbell Systematic Reviews18(2), e1230 (2022)

    Haddaway, N.R., Page, M.J., Pritchard, C.C., McGuinness, L.A.: PRISMA2020: An R package and Shiny app for producing PRISMA 2020-compliant flow diagrams, with interactivity for optimised digital transparency and Open Synthesis. Campbell Systematic Reviews18(2), e1230 (2022). DO...

    2020 doi

  13. [18]

    IEEE Transactions on Neural Networks and Learning Systems36(3), 4196– 4208 (2025)

    Hao, W., Guan, H., Zhang, Z.: V AG: A Uniform Model for Cross-Modal Visual-Audio Mu- tual Generation. IEEE Transactions on Neural Networks and Learning Systems36(3), 4196– 4208 (2025). DOI 10.1109/TNNLS.2022.3161314. URLhttps://ieeexplore.ieee.org/ document/9753685/

    2025

  14. [19]

    In: Proceedings of the CHI Conference on Human Factors in Computing Systems, pp

    Kamath, P., Morreale, F., Bagaskara, P.L., Wei, Y., Nanayakkara, S.: Sound designer-generative AI interactions: towards designing creative support tools for professional sound designers. In: Proceedings of the CHI Conference on Human Factors in Computing Systems, pp. 1–17. ACM...

    2024

  15. [20]

    Artificial Intelligence Review56(7), 5837–5880 (2023)

    Kaur, N., Singh, P.: Conventional and contemporary approaches used in text to speech syn- thesis: a review. Artificial Intelligence Review56(7), 5837–5880 (2023). DOI 10.1007/ s10462-022-10315-0. URLhttps://link.springer.com/10.1007/s10462-022-10315-0

    2023 doi

  16. [21]

    In: Proceedings of the 2019 Conference of the North, pp

    Kim, C.D., Kim, B., Lee, H., Kim, G.: AudioCaps: generating captions for audios in the wild. In: Proceedings of the 2019 Conference of the North, pp. 119–132. Association for Com- putational Linguistics, Minneapolis, Minnesota (2019). DOI 10.18653/v1/N19-1011. URL http://aclwe...

    2019 doi

  17. [24]

    In: 2024 Asia Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), pp

    Li, C., Xu, M., Yu, D.: SRC-gAudio: Sampling-Rate-Controlled Audio Generation. In: 2024 Asia Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), pp. 1–6. IEEE, Macau, Macao (2024). DOI 10.1109/APSIPAASC63619.2025.10849319. URLhttps:...

    2024

  18. [26]

    IEEE/ACM Transactions on Audio, Speech, and Language Processing32, 2871–2883 (2024)

    Liu, H., Yuan, Y., Liu, X., Mei, X., Kong, Q., Tian, Q., Wang, Y., Wang, W., Wang, Y., Plumb- ley, M.D.: AudioLDM 2: Learning Holistic Audio Generation With Self-Supervised Pretraining. IEEE/ACM Transactions on Audio, Speech, and Language Processing32, 2871–2883 (2024). DOI 10...

    2024

  19. [27]

    In: Sixth International Conference on Computer Information Science and Application Technology (CISAT 2023), vol

    Liu, Q., Chang, C., Shen, H., Cheng, S., Li, X., Zheng, R.: Research on artificial intelligence gener- ated audio. In: Sixth International Conference on Computer Information Science and Application Technology (CISAT 2023), vol. 12800, pp. 1206–1212. SPIE (2023)

    2023

  20. [28]

    IEEE Transactions on Circuits and Systems for Video Technology32(3), 1299– 1312 (2022)

    Liu, S., Li, S., Cheng, H.: Towards an End-to-End Visual-to-Raw-Audio Generation With GAN. IEEE Transactions on Circuits and Systems for Video Technology32(3), 1299– 1312 (2022). DOI 10.1109/TCSVT.2021.3079897. URLhttps://ieeexplore.ieee.org/ document/9430540/

    2022

  21. [29]

    In: Proceed- ings of the 32nd ACM International Conference on Multimedia, pp

    Majumder, N., Hung, C.Y., Ghosal, D., Hsu, W.N., Mihalcea, R., Poria, S.: Tango 2: Aligning Diffusion-based Text-to-Audio Generations through Direct Preference Optimization. In: Proceed- ings of the 32nd ACM International Conference on Multimedia, pp. 564–572. ACM, Melbourne V...

    2024

  22. [30]

    In: 2024 IEEE 34th International Workshop on Machine Learning for Signal Processing (MLSP), pp

    Mei, X., Nagaraja, V., Le Lan, G., Ni, Z., Chang, E., Shi, Y., Chandra, V.: Foleygen: visually-guided audio generation. In: 2024 IEEE 34th International Workshop on Machine Learning for Signal Processing (MLSP), pp. 1–6. IEEE, London, United Kingdom (2024). DOI 10.1109/MLSP589...

    2024

  23. [31]

    Journal of the Audio Engineering Society71, 826–848 (2023)

    Menexopoulos, D., et al.: The state of the art in procedural audio. Journal of the Audio Engineering Society71, 826–848 (2023). DOI 10.17743/jaes.2022.0108

    2023

  24. [32]

    IEEE Access13, 18079–18106 (2025)

    Mitra, R., Zualkernan, I.: Music Generation Using Deep Learning and Generative AI: A Systematic Review. IEEE Access13, 18079–18106 (2025). DOI 10.1109/ACCESS.2025.3531798. URL https://ieeexplore.ieee.org/document/10845168/

    2025

  25. [33]

    In: Creativity and Cognition, pp

    Ning, Z., Zhang, Z., Ban, J., Jiang, K., Gan, R., Tian, Y., Li, T.J.J.: MIMOSA: Human-AI co-creation of computational spatial audio effects on videos. In: Creativity and Cognition, pp. 156–169 (2024). DOI 10.1145/3635636.3656189. URLhttp://arxiv.org/abs/2404.15107. ArXiv:2404....

    2024

  26. [34]

    how to use an overview

    Oxman, A.D.: Users’ guides to the medical literature: Vi. how to use an overview. JAMA272(17), 1367 (1994). DOI 10.1001/jama.1994.03520170077040. URLhttp://jama.jamanetwork. com/article.aspx?doi=10.1001/jama.1994.03520170077040

    1994

  27. [35]

    Pascual, S., Yeh, C., Tsiamas, I., Serr `a, J.: Masked generative Vvideo-to-audio transformers with enhanced synchronicity. In: A. Leonardis, E. Ricci, S. Roth, O. Russakovsky, T. Sattler, G. Varol (eds.) Computer Vision – ECCV 2024, vol. 15145, pp. 247–264. Springer Nature Sw...

    2024 doi

  28. [37]

    Ronneberger, O., Fischer, P., Brox, T.: U-Net: Convolutional Networks for Biomedical Im- age Segmentation. In: N. Navab, J. Hornegger, W.M. Wells, A.F. Frangi (eds.) Medical Im- age Computing and Computer-Assisted Intervention – MICCAI 2015, vol. 9351, pp. 234–

    2015

  29. [38]

    Serafin, S., Franinovi ´c, K., Hermann, T., Lemaitre, G., Rinott, M., Rocchesso, D.: Sonic Interaction Design,The sonification handbook, vol. 5. Logos Publishing House, Berlin (2011). URLhttps: //sonification.de/handbook/download/TheSonificationHandbook-chapter5.pdf

    2011

  30. [39]

    In: Research Methods for the Social Sciences: An Introduction

    Sheppard, V.: Acceptable sources for literature reviews. In: Research Methods for the Social Sciences: An Introduction. Pressbooks (2020). URLhttps://pressbooks.bccampus.ca/jibcresearchmethods/chapter/ 5-3-acceptable-sources-for-literature-reviews/

    2020

  31. [40]

    Journal of the Audio Engineering Society73, 406–428 (2025)

    Steinmetz, C., Mitcheltree, C., Wichern, G., et al.: Audio signal processing in the artifi- cial intelligence era. Journal of the Audio Engineering Society73, 406–428 (2025). DOI 10.17743/jaes.2022.0209

    2025

  32. [41]

    In: Extended Abstracts of the CHI Conference on Human Factors in Computing Systems, CHI EA ’24

    Su, X., Koh, E., Xiao, C.: Sonifyar: context-aware sound effect generation in augmented reality. In: Extended Abstracts of the CHI Conference on Human Factors in Computing Systems, CHI EA ’24. Association for Computing Machinery, New York, NY, USA (2024). DOI 10.1145/3613905. ...

    2024

  33. [42]

    Advances in Neural Information Processing Systems36, 16083–16099 (2023)

    Tang, Z., Yang, Z., Zhu, C., Zeng, M., Bansal, M.: Any-to-any generation via composable diffusion. Advances in Neural Information Processing Systems36, 16083–16099 (2023)

    2023

  34. [44]

    Advances in Neural Information Processing Systems37, 128118–128138 (2024)

    Wang, Y., Guo, W., Huang, R., Huang, J., Wang, Z., You, F., Li, R., Zhao, Z.: Frieren: Efficient video-to-audio generation network with rectified flow matching. Advances in Neural Information Processing Systems37, 128118–128138 (2024)

    2024

  35. [45]

    In: Proceedings of the 32nd ACM International Conference on Mul- timedia, pp

    Wang, Y., Wang, Z., Huang, H.: AutoSFX: Automatic Sound Effect Generation for Videos. In: Proceedings of the 32nd ACM International Conference on Mul- timedia, pp. 9923 – 9932 (2024). DOI 10.1145/3664647.3681109. URLhttps: //www.scopus.com/inward/record.uri?eid=2-s2.0-85209818...

    2024

  36. [46]

    DOI 10.48550/ARXIV.2301.05578

    Weisz, J.D., Muller, M., He, J., Houde, S.: Toward general design principles for generative AI ap- plications (2023). DOI 10.48550/ARXIV.2301.05578. URLhttps://arxiv.org/abs/2301. 05578. Version Number: 1

    2023 doi arXiv

  37. [49]

    In: 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp

    Xie, Z., Yu, S., He, Q., Li, M.: Sonic VisionLM: playing sound with vision language models. In: 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 26856– 26865. IEEE, Seattle, WA, USA (2024). DOI 10.1109/CVPR52733.2024.02537. URLhttps: //ieeexplore...

    2024

  38. [52]

    In: ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp

    Yuan, Y., Liu, H., Liu, X., Huang, Q., Plumbley, M.D., Wang, W.: Retrieval-Augmented Text-to- Audio Generation. In: ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 581–585. IEEE, Seoul, Korea, Republic of (2024). DOI 10...

    2024

  39. [53]

    In: 2024 IEEE Spoken Language Technology Workshop (SLT), pp

    Zhang, X., Xue, L., Gu, Y., Wang, Y., Li, J., He, H., Wang, C., Liu, S., Chen, X., Zhang, J., Fang, Z., Chen, H., Tang, T.Y., Zou, L., Wang, M., Han, J., Chen, K., Li, H., Wu, Z.: Amphion: an open- source audio, music, and speech generation toolkit. In: 2024 IEEE Spoken Langua...

    2024

  40. [54]

    In: 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp

    Zhang, Y., Xu, X., Wu, M.: Smooth-Foley: Creating continuous sound for video-to-audio gener- ation under semantic guidance. In: 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1–5 (2025). DOI 10.1109/ICASSP49660.2025.10890403. ISSN: ...

    2025

  41. [55]

    DOI 10.48550/arXiv.1712.01393

    Zhou, Y., Wang, Z., Fang, C., Bui, T., Berg, T.L.: Visual to Sound: Generating Natural Sound for Videos in the Wild (2018). DOI 10.48550/arXiv.1712.01393. URLhttp://arxiv.org/abs/ 1712.01393. ArXiv:1712.01393 [cs]

    2018 doi arXiv

  42. [241]

    DOI 10.1007/978-3-319-24574-4 28

    Springer International Publishing, Cham (2015). DOI 10.1007/978-3-319-24574-4 28. URLhttp://link.springer.com/10.1007/978-3-319-24574-4_28. Series Title: Lec- ture Notes in Computer Science 28 Abdoet al

    2015 doi

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.