Pith. sign in

REVIEW 24 cited by

Make-An-Audio 2: Temporal-Enhanced Text-to-Audio Generation

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2305.18474 v1 pith:ATQPWCAG submitted 2023-05-29 cs.SD cs.LGcs.MMeess.AS

Jiawei Huang , Yi Ren , Rongjie Huang , Dongchao Yang , Zhenhui Ye , Chen Zhang , Jinglin Liu , Xiang Yin
show 2 more authors
Zejun Ma Zhou Zhao
This is my paper · ORCID
classification cs.SDcs.LGcs.MMeess.AS
keywords temporalinformationsemanticconsistencydatadiffusionlargemake-an-audio
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Large diffusion models have been successful in text-to-audio (T2A) synthesis tasks, but they often suffer from common issues such as semantic misalignment and poor temporal consistency due to limited natural language understanding and data scarcity. Additionally, 2D spatial structures widely used in T2A works lead to unsatisfactory audio quality when generating variable-length audio samples since they do not adequately prioritize temporal information. To address these challenges, we propose Make-an-Audio 2, a latent diffusion-based T2A method that builds on the success of Make-an-Audio. Our approach includes several techniques to improve semantic alignment and temporal consistency: Firstly, we use pre-trained large language models (LLMs) to parse the text into structured <event & order> pairs for better temporal information capture. We also introduce another structured-text encoder to aid in learning semantic alignment during the diffusion denoising process. To improve the performance of variable length generation and enhance the temporal information extraction, we design a feed-forward Transformer-based diffusion denoiser. Finally, we use LLMs to augment and transform a large amount of audio-label data into audio-text datasets to alleviate the problem of scarcity of temporal data. Extensive experiments show that our method outperforms baseline models in both objective and subjective metrics, and achieves significant gains in temporal information understanding, semantic consistency, and sound quality.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 24 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. SonicWeave: Chunk-Routed Mixture-of-Experts for Unified Audio Scene Generation

    cs.SD 2026-08 conditional novelty 7.0 of 10

    SonicWeave routes chunks of audio through specialized experts, using a learned gate between text-derived prior and local evidence, improving compositional fidelity in unified text-to-audio scene generation.

  2. AV-Link: Temporally-Aligned Diffusion Features for Cross-Modal Audio-Video Generation

    cs.CV 2024-12 conditional novelty 7.0 of 10

    AV-Link unifies video-to-audio and audio-to-video generation by aligning frozen diffusion-model activations with temporally matched rotary position embeddings in a shared Fusion Block.

  3. Improving Text-to-Audio Instruction Following via Fine-Grained Feedback from Audio-Aware Large Language Models

    eess.AS 2026-07 conditional novelty 6.0 of 10

    Using audio-aware LLMs to judge event presence and temporal order as DPO rewards improves multi-event text-to-audio instruction following.

  4. Unified Audio Intelligence Without Regressing on Text Intelligence

    cs.CL 2026-07 conditional novelty 6.0 of 10

    A unified 30B MoE audio-text LLM achieves state-of-the-art audio understanding, generation, and speech tasks while preserving text reasoning comparable to its text-only backbone.

  5. RFM-Editing 2: Text-Guided Audio Editing with Rectified Flow Matching and Coarse-to-Fine Diffusion Transformers

    cs.SD 2026-06 unverdicted novelty 6.0 of 10

    Hybrid two-stage diffusion transformer architecture for instruction-guided audio editing via rectified flow that performs joint attention at low resolution then alternates joint and cross-attention at high resolution ...

  6. OmniCustom: Sync Audio-Video Customization Via Joint Audio-Video Generation Model

    cs.SD 2026-02 conditional novelty 6.0 of 10

    A zero-shot model that generates a video of a reference face speaking user-chosen text with a reference voice timbre.

  7. TTA-Bench: A Comprehensive Benchmark for Evaluating Text-to-Audio Models

    cs.SD 2025-09 conditional novelty 6.0 of 10

    TTA-Bench offers a seven-dimension, 2,999-prompt evaluation of ten text-to-audio models with 118,000 human ratings, covering quality, robustness, fairness, bias, and toxicity.

  8. UniFlow: Unifying Speech Front-End Tasks via Continuous Generative Modeling

    eess.AS 2025-08 conditional novelty 6.0 of 10

    UniFlow unifies four speech front-end tasks in one continuous-latent generative model with task-ID conditioning and reports competitive, but not uniformly superior, benchmark scores.

  9. Generative Audio Language Modeling with Continuous-valued Tokens and Masked Next-Token Prediction

    eess.AS 2025-07 conditional novelty 6.0 of 10

    A causal audio language model with continuous-valued tokens and masked next-token prediction matches diffusion-based text-to-audio quality with smaller, streamable models.

  10. SpecMaskFoley: Steering Pretrained Spectral Masked Generative Transformer Toward Synchronized Video-to-audio Synthesis via ControlNet

    cs.SD 2025-05 conditional novelty 6.0 of 10

    A ControlNet branch plus a frequency-aware feature aligner lets a pretrained masked generative TTA model produce video-synchronized foley, beating several from-scratch models on VGGSound.

  11. CosyAudio: Improving Audio Generation with Confidence Scores and Synthetic Captions

    eess.AS 2025-01 conditional novelty 6.0 of 10

    CosyAudio improves text-to-audio generation by generating synthetic captions with confidence scores and iteratively refining training data through a self-evolving loop.

  12. TangoFlux: Super Fast and Faithful Text to Audio Generation with Flow Matching and Clap-Ranked Preference Optimization

    cs.SD 2024-12 conditional novelty 6.0 of 10

    A fast flow-matching text-to-audio model aligned via CLAP-ranked self-generated preference pairs reports state-of-the-art AudioCaps and human-evaluation scores.

  13. ETTA: Elucidating the Design Space of Text-to-Audio Models

    cs.SD 2024-12 conditional novelty 6.0 of 10

    ETTA, a text-to-audio model trained on a large synthetic caption dataset, outperforms public-data baselines on AudioCaps and MusicCaps and rivals proprietary-data systems.

  14. MMAudio: Taming Multimodal Joint Training for High-Quality Video-to-Audio Synthesis

    cs.CV 2024-12 conditional novelty 6.0 of 10

    A single flow-matching transformer trained jointly on audio-video and audio-text data produces state-of-the-art public video-to-audio synthesis with a frame-level synchronization module.

  15. SyncFlow: Toward Temporally Aligned Joint Audio-Video Generation from Text

    cs.MM 2024-12 conditional novelty 6.0 of 10

    SyncFlow jointly generates temporally aligned 16 FPS video and 48kHz audio from text using a dual-diffusion-transformer with modality adaptors, and reports better audio-video alignment than cascaded and contrastive baselines.

  16. OmniFlow: Any-to-Any Generation with Multi-Modal Rectified Flows

    cs.MM 2024-12 conditional novelty 6.0 of 10

    A modular multi-modal rectified-flow model built on Stable Diffusion 3 achieves competitive text-to-image and text-to-audio generation while also supporting audio-to-image, image-to-text, and audio-to-text tasks.

  17. FdAudio: MeanFlow-Anchored Fr\'echet-Distance Post-Training for One-Step Text-to-Audio Generation

    eess.AS 2026-07 conditional novelty 5.5 of 10

    MeanFlow-anchored multi-representation FD post-training improves one-step text-to-audio quality without collapsing multi-step sampling.

  18. Qwen-Audio-3.0-Gen-Preview Technical Report

    eess.AS 2026-07 conditional novelty 5.0 of 10

    One diffusion-transformer system with a shared audio codec generates standalone audio, multi-speaker dialogue, and time-structured mixed scenes, with strongest measured advantages in speaker similarity, cross-turn con...

  19. IMPACT: Iterative Mask-based Parallel Decoding for Text-to-Audio Generation with Diffusion Modeling

    eess.AS 2025-05 conditional novelty 5.0 of 10

    On AudioCaps, IMPACT reports the best Fréchet Distance and Fréchet Audio Distance among the compared systems while generating audio faster than diffusion baselines.

  20. AudioTurbo: Fast Text-to-Audio Generation with Rectified Diffusion

    cs.SD 2025-05 conditional novelty 5.0 of 10

    AudioTurbo fine-tunes a diffusion model on deterministic noise-audio pairs created by the pretrained Auffusion model, achieving strong text-to-audio results in 10 inference steps.

  21. MIMII-Agent: Leveraging LLMs with Function Calling for Relative Evaluation of Anomalous Sound Detection

    eess.AS 2025-07 conditional novelty 4.0 of 10

    An LLM-based pipeline that applies hand-picked sound effects to generated normal machine audio produces synthetic anomalies whose per-machine-type detection difficulty ranking matches real anomalies, for one autoencod...

  22. Multimodal Large Language Models for Image, Text, and Speech Data Augmentation: A Survey

    cs.CV 2025-01 conditional novelty 4.0 of 10

    A literature review cataloging LLM-based augmentation methods across image, text, and speech, with a taxonomy of techniques, limitations, and suggested fixes.

  23. Mel-Refine: A Plug-and-Play Approach to Refine Mel-Spectrogram in Audio Generation

    cs.SD 2024-12 reject novelty 4.0 of 10

    Mel-Refine boosts text-to-audio spectrogram sharpness by amplifying skip-connection high frequencies and attenuating backbone high frequencies at inference, but its reported 25% improvement is based on test-set-tuned ...

  24. WavChat: A Survey of Spoken Dialogue Models

    eess.AS 2024-11 conditional novelty 4.0 of 10

    WavChat categorizes spoken dialogue models into cascaded and end-to-end paradigms and surveys speech representations, training strategies, streaming, duplex interaction, datasets, and evaluation benchmarks.

Pith tools