Pith. sign in

REVIEW 4 major objections 5 minor 44 references

SoundscapeAgent argues that soundscape synthesis should be an explicit agentic construction pipeline, and reports that it beats single-shot generators on listener alignment and that its structured outputs improve downstream audio reasoning

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 06:24 UTC pith:5GGA2NDP

load-bearing objection A useful agentic soundscape recipe with a clean formalization and a plausible data-construction mode, but the headline MMAU gain is confounded by dataset volume and checkpoint selection, so the central claim is not yet established. the 4 major comments →

arxiv 2607.21857 v1 pith:5GGA2NDP submitted 2026-07-23 cs.SD

SoundscapeAgent: Agentic Soundscape Construction for Controllable Synthesis and Scalable Audio-Language Supervision

Hao Zhang , Yiwen Zhao , Yixuan Zhang , Yiwen Shao , Steve Yves This is my paper
classification cs.SD
keywords agentic soundscape constructioncompositional audio generationscene planningtemporal layoutaudio-language supervisionsynthetic training dataaudio reasoningcontrollable audio synthesis
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that multi-event sound generation should be built by an explicit agentic pipeline: an LLM parses user intent, writes an executable scene plan, acquires asset recordings or synthesized clips, and then renders them into a mixed waveform with aligned metadata. The authors claim this yields more controllable soundscapes than single-shot text-to-audio generators, with listener studies favoring the agent overall (3.63 vs. 3.41) and most strongly on affective, ambiance-heavy, and temporally specified prompts. They further claim that the same pipeline can generate structured audio-language data at scale: adding agent-built examples to a real-only training set improves a frozen-LLM audio reasoning system from 51.05% to 56.50% on the 1,000-example MMAU test-mini benchmark.

Core claim

The central claim is that decomposing soundscape synthesis into scene planning, asset acquisition, deterministic rendering, and metadata export—rather than letting a single generative model infer everything—provides both better controllability and usable structured supervision. The agent grounds underspecified intents like “a calming evening” into explicit background, midground, and foreground events with onset times and levels, and a deterministic renderer places each asset on a timeline to produce the final mix. On the generation track this achieves the highest overall listener alignment, with the largest margins on compositional scenes. On the data track, the same structured outputs are c

What carries the argument

The load-bearing object is the executable scene plan: a list of events, each specified by source category, scene role (background, midground, or foreground), onset time, duration, selected asset, relative level, and optional rendering controls. An LLM-based agent converts text or image input into this plan through “concrete-audible grounding,” then hybrid acquisition fills each event from retrieval or on-demand generation. Every candidate asset is filtered by a semantic audio-text similarity score and a production-complexity score. A deterministic renderer executes the plan, and the exported metadata and event structure are used to generate aligned descriptions. An offline prior mode samples

Load-bearing premise

The central assumption is that the downstream reasoning gains come from the scene structure of the synthetic soundscapes rather than from simply training on more data.

What would settle it

Train the same audio-language aligner on a volume-matched set of 673k real examples (matching the 573k real + 100k synthetic condition in total count, with matched seed and iterations) and compare accuracy on the same benchmark; if the real-only 673k model matches or exceeds 56.50%, the structural claim would not be supported.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If the central claim holds, soundscape generation becomes inspectable and editable at the event level, not just at the prompt level.
  • Structured agent-generated data can improve audio reasoning even when the language model is frozen and only a small trainable projector is optimized.
  • The benefit transfers beyond the corpus's primary sound domain to music reasoning, though speech gains are mixed.
  • Using more synthetic data is not automatically better: the 200k setting underperforms 100k, so data quality and balance are decisive.
  • The pipeline provides a reusable pattern for producing aligned audio-language supervision and can be adapted to goal-directed, human-in-the-loop soundscape construction.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper does not isolate structure from scale: because the augmented setups add synthetic data on top of a fixed real set, the downstream gain could be a pure scale effect; a budget-matched comparison is the natural next test.
  • The largest subtask gains are on event-based, temporal, and hard reasoning questions, suggesting the exported metadata could be turned into timestamp-grounded question-answer pairs rather than free-form captions—a use the paper mentions but does not evaluate.
  • The non-monotonic result (100k better than 200k) implies an optimal synthetic-to-real mixing ratio that the paper leaves unspecified; a systematic sweep of this ratio would be a direct extension.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes SoundscapeAgent, a framework for compositional soundscape synthesis in which an LLM-based agent converts user intent into an explicit scene plan (events, roles, timing, gains), acquires assets via retrieval or on-demand generation, renders controllable multi-event mixtures, and exports aligned metadata/descriptions. It also introduces an offline prior mode for large-scale audio-language corpus construction. Two evaluations are reported: Track A compares the agent to TangoFlux, EzAudio, and AudioLDM2 via listener alignment and CLAP similarity; Track B trains a projector-based audio-language aligner on real data plus agent-generated data and reports MMAU test-mini accuracy gains.

Significance. If the claims hold, the framework is a useful instance of making compositional structure explicit in audio generation, and the offline prior mode is a plausible path to scalable structured audio-language supervision. Strengths include the inspectable pipeline, the deterministic renderer in Eq. (6), the two-track evaluation separating synthesis quality from downstream data utility, and the public code/demo page. The paper is also honest in its stated limitations about asset-library coverage and MMAU's limited coverage of fine-grained temporal structure. However, the central downstream claim—that agent-structured synthetic data, rather than simply more data, improves audio reasoning—is not isolated by the current experiment design.

major comments (4)
  1. [§IV-B, Table III] The claim that 'agent-generated data consistently outperform real-only baselines' is confounded with training-set size. The augmented conditions contain 623k/673k/773k examples versus 573k for real-only, and the text explicitly says 'so only the amount of synthetic data varies'—which means dataset volume also varies. A +5.45-point MMAU gain from adding 100k matched audio-text pairs could be a pure scale effect, not a property of soundscape structure. There is no scale-matched control (e.g., +100k real CaptionStew examples or +100k non-agent synthetic pairs), no seeds or variance estimates, and the best checkpoint is selected from 200k/250k/300k iterations on the test-mini set itself. With 1,000 items, 5.45% is roughly 54 questions, so checkpoint selection and seed variance could account for a meaningful portion of the gap. The non-monotonic 200k result does not resolve this; it is consis
  2. [§IV-A, Table II] Track A is underpowered for the capability-specific claims. Only 8 prompts are used per scenario (40 prompts, 160 clips total), and each clip receives a single 5-point alignment score per listener; no pairwise significance tests, inter-rater agreement, or per-prompt mixed-effects analysis are reported. The advantage is also not uniform: on AudioCaps the agent's alignment (3.58 ± 0.20) is numerically below TangoFlux (3.63 ± 0.20), on Event-rich it is below TangoFlux (3.46 ± 0.21 vs 3.57 ± 0.18), and its CLAP similarity is lower than TangoFlux overall (0.34 vs 0.39). The 'most pronounced' wins rest on 8 affective/abstract and 8 ambiance-heavy prompts. Either stronger statistical evidence or more prompts are needed before asserting capability-specific superiority; otherwise the conclusion should be softened to 'competitive on standard prompts, with gains on affective/abstract and ambiance-h
  3. [§III-E, Eq. (8)] The offline prior mode is under-specified. The text says the agent is run offline to infer reusable scene priors π, but it does not define the representation of π, the number or coverage of priors, how p(E|π) is sampled, or how event tags, roles, and temporal layouts are drawn. Track B's 200k pool is generated from this mode, so without these details the data-construction procedure is not reproducible and the source of the claimed compositional diversity cannot be assessed. Please include the prior format, the sampling protocol, and ideally a released set of priors or a detailed pseudocode.
  4. [§III-C] The asset-filtering thresholds (CLAP > 0.32, PC < 4) are 'selected on a 100-clip development set,' but no validation or test-set performance is reported for this filter. Since the quality and composition of the audio-language corpus in Track B depend directly on these thresholds, the paper should at least report how the thresholds affect library composition and whether the downstream MMAL results are robust to threshold choices, e.g., a small ablation over threshold values.
minor comments (5)
  1. [Figure 1] The figure label 'EVALUATION Trak A' contains a typo; it should read 'Track A.'
  2. [§III-C] The phrase 'an production complexity score' should be 'a production complexity score.'
  3. [Table II] The table formatting in the submitted text appears corrupted (e.g., missing spaces between numeric entries). Please regenerate the table with clear column separators.
  4. [§II-B] The related-work discussion of WavJourney and WavCraft is fine, but the paper should more explicitly distinguish the proposed offline prior mode from prior data-augmentation pipelines, since Track B is the main claim.
  5. [§V] The limitations paragraph appropriately acknowledges that MMAU may not cover all fine-grained temporal information, but it does not mention the scale-confounded design of Track B; adding that acknowledgment would improve the paper's framing.

Circularity Check

0 steps flagged

No significant circularity: the derivation chain ends in external benchmarks and human ratings, not in the system's own outputs.

full rationale

The paper's construction is not circular. The rendering equation (6) is a compositional mixing formula over independently rendered events, and the aligned descriptions in Eq. (7) are generated from planned and realized scene structure. Track B evaluates those descriptions by training on them and testing on MMAU test-mini, an external benchmark; Track A uses human listener ratings. The Track B comparison does not isolate agentic structure from data volume — the paper itself states 'so only the amount of synthetic data varies' — but this is a scaling/confound issue, not an equivalence or a fitted parameter renamed as a prediction. No uniqueness theorem or ansatz is imported from self-citation; self-citations [40], [43], [44] supply the baseline recipe and the real-data mixture, while the evaluation target is independent. Tuning CLAP/PC thresholds on a small development set is a hyperparameter choice, not a circular derivation of the headline result. Overall, the evaluation results are not forced by the paper's own definitions or equations.

Axiom & Free-Parameter Ledger

2 free parameters · 3 axioms · 1 invented entities

The framework's central claim mostly rests on ordinary engineering assumptions (asset filtering thresholds, CLAP validity, the Aligner recipe, MMAU reliability). Free parameters: two asset-selection thresholds, tuned on a small dev set. The only 'invented entity' is the offline scene-prior pi, which is a software component rather than a strong falsifiable postulate. The main unvalidated assumption is the use of production-complexity and CLAP scores to build the library, since no separate validation of the filtering quality is reported.

free parameters (2)
  • CLAP threshold = 0.32
    Selected on 'a 100-clip development set'; no independent validation set reported for this selection. Affects which assets enter the library and hence all downstream results.
  • PC threshold = <4
    Selected on 'a 100-clip development set'; no validation details. PC is an 'aesthetics/production complexity' score used to prefer isolated single-event recordings.
axioms (3)
  • domain assumption CLAP similarity is a valid proxy for semantic consistency of an asset with a scene event
    The asset filter relies on CLAP scores without measuring retrieval precision/recall on a held-out set against human judgment.
  • domain assumption The Aligner recipe of [40] is a fair real-only backbone
    The Track B comparison assumes that the real-only mixture from CaptionStew-400k+173k paralinguistic examples is an appropriate baseline; since the same recipe is cited as an arXiv preprint with its own contributions, there is some reliance on an unreviewed external method.
  • domain assumption MMAU test-mini string-matching is a reliable evaluation of audio reasoning
    The evaluation protocol is the only downstream signal; the paper does not analyze whether the largest subtask gains (e.g., Genre Reasoning) could reflect description bias rather than audio understanding. This is a standard benchmark assumption.
invented entities (1)
  • SP scene prior pi no independent evidence
    purpose: A reusable distribution over scene plans used in the offline prior mode to sample new structured examples at scale
    No definition of pi is provided and no way to inspect or validate it independently; it is a pipelining device that encodes the agent's behavior rather than a falsifiable scientific entity.

pith-pipeline@v1.3.0-alltime-deepseek · 10831 in / 7033 out tokens · 57679 ms · 2026-08-01T06:24:31.218139+00:00 · methodology

0 comments
read the original abstract

We present an agentic soundscape construction framework for controllable compositional audio generation that makes explicit the scene planning, source selection, temporal layout, and rendering steps typically handled implicitly by single-shot text-to-audio models. An LLM-based agent converts user intent into an executable scene plan, acquires assets through retrieval and on-demand generation, renders controllable multi-event mixtures, and exports aligned scene metadata. The framework also supports human-in-the-loop interaction through user-guided tool selection and editable scene plans. Together, these components provide an inspectable and reusable approach to controllable soundscape synthesis and scalable audio-language data construction. Listener studies and objective metrics demonstrate competitive generation performance against text-to-audio baselines, while models trained with agent-generated data consistently outperform real-only baselines in downstream audio reasoning. Code, demos, and listening-test results are available at https://haozhang6720.github.io/SoundscapeAgentDemoPage/.

Figures

Figures reproduced from arXiv: 2607.21857 by Hao Zhang, Steve Yves, Yiwen Shao, Yiwen Zhao, Yixuan Zhang.

Figure 1
Figure 1. Figure 1: Overview of the agentic soundscape construction framework. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: MMAU test-mini: Sub-task accuracy change with agentic augmentation. [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

44 extracted references · 9 linked inside Pith

  1. [1]

    MMAU: A massive multi- task audio understanding and reasoning benchmark,

    S. Sakshi, U. Tyagi, S. Kumar, A. Seth, R. Selvakumar, O. Nieto, R. Duraiswami, S. Ghosh, and D. Manocha, “MMAU: A massive multi- task audio understanding and reasoning benchmark,” inProc. ICLR, 2025

    2025

  2. [2]

    CLAP: Learning audio concepts from natural language supervision,

    B. Elizalde, S. Deshmukh, M. Al Ismail, and H. Wang, “CLAP: Learning audio concepts from natural language supervision,” inProc. ICASSP, 2023, pp. 1–5

    2023

  3. [3]

    WavCaps: A ChatGPT-assisted weakly-labelled audio captioning dataset for audio-language multimodal research,

    X. Mei, C. Meng, H. Liu, Q. Kong, T. Ko, C. Zhao, M. D. Plumbley, Y . Zou, and W. Wang, “WavCaps: A ChatGPT-assisted weakly-labelled audio captioning dataset for audio-language multimodal research,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 32, pp. 3339–3354, 2024

    2024

  4. [4]

    Qwen3 technical report,

    A. Yang, A. Li, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Gao, C. Huang, C. Lvet al., “Qwen3 technical report,”arXiv preprint arXiv:2505.09388, 2025

    Pith/arXiv arXiv 2025

  5. [5]

    Gem- ini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities,

    G. Comanici, E. Bieber, M. Schaekermann, I. Pasupat, N. Sachdeva, I. Dhillon, M. Blistein, O. Ram, D. Zhang, E. Rosenet al., “Gem- ini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities,”arXiv preprint arXiv:2507.06261, 2025

    Pith/arXiv arXiv 2025

  6. [6]

    Audio- thinker: Guiding large audio language model when and how to think via reinforcement learning,

    S. Wu, C. Li, W. Wang, H. Zhang, H. Wang, M. Yu, and D. Yu, “Audio- thinker: Guiding large audio language model when and how to think via reinforcement learning,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 40, no. 40, 2026, pp. 33 962–33 970

    2026

  7. [7]

    AudioGen: Textually guided audio generation,

    F. Kreuk, G. Synnaeve, A. Polyak, U. Singer, A. D ´efossez, J. Copet, D. Parikh, Y . Taigman, and Y . Adi, “AudioGen: Textually guided audio generation,” inProc. ICLR, 2023

    2023

  8. [8]

    AudioLDM: Text-to-audio generation with latent diffusion models,

    H. Liu, Z. Chen, Y . Yuan, X. Mei, X. Liu, D. Mandic, W. Wang, and M. D. Plumbley, “AudioLDM: Text-to-audio generation with latent diffusion models,” inProc. ICML, 2023, pp. 21 450–21 474

    2023

  9. [9]

    AudioLDM 2: Learning holistic audio generation with self-supervised pretraining,

    H. Liu, Y . Yuan, X. Liu, X. Mei, Q. Kong, Q. Tian, Y . Wang, W. Wang, Y . Wang, and M. D. Plumbley, “AudioLDM 2: Learning holistic audio generation with self-supervised pretraining,”IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 32, pp. 2871–2883, 2024

    2024

  10. [10]

    PicoAudio: Enabling precise temporal controllability in text-to-audio generation,

    Z. Xie, X. Xu, Z. Wu, and M. Wu, “PicoAudio: Enabling precise temporal controllability in text-to-audio generation,” inProc. ICASSP, 2025, pp. 1–5

    2025

  11. [11]

    AudioComposer: To- wards fine-grained audio generation with natural language descriptions,

    Y . Wang, H. Chen, D. Yang, Z. Wu, and X. Wu, “AudioComposer: To- wards fine-grained audio generation with natural language descriptions,” inProc. ICASSP, 2025, pp. 1–5

    2025

  12. [12]

    Sound scene synthesis at the DCASE 2024 challenge,

    M. Lagrange, J. Lee, M. Tailleur, L. M. Heller, K. Choi, B. McFee, K. Imoto, and Y . Okamoto, “Sound scene synthesis at the DCASE 2024 challenge,”arXiv preprint arXiv:2501.08587, 2025

    Pith/arXiv arXiv 2024

  13. [13]

    DCASE 2024 challenge task 7: Sound scene synthesis,

    “DCASE 2024 challenge task 7: Sound scene synthesis,” 2024, accessed: Apr. 21, 2026. [Online]. Available: https://dcase.community/ challenge2024/task-sound-scene-synthesis

    2024

  14. [14]

    AudioGPT: Understanding and generating speech, music, sound, and talking head,

    R. Huang, M. Li, D. Yang, J. Shi, X. Chang, Z. Ye, Y . Wu, Z. Hong, J. Huang, J. Liuet al., “AudioGPT: Understanding and generating speech, music, sound, and talking head,” inProc. AAAI, vol. 38, 2024, pp. 23 802–23 804

    2024

  15. [15]

    WavJourney: Compositional audio creation with large language models,

    X. Liu, Z. Zhu, H. Liu, Y . Yuan, Q. Huang, M. Cui, J. Liang, Y . Cao, Q. Kong, M. D. Plumbley, and W. Wang, “WavJourney: Compositional audio creation with large language models,”IEEE Transactions on Audio, Speech and Language Processing, vol. 33, pp. 2830–2844, 2025

    2025

  16. [16]

    WavCraft: Audio editing and generation with large language models,

    J. Liang, H. Zhang, H. Liu, Y . Cao, Q. Kong, X. Liu, W. Wang, M. D. Plumbley, H. Phan, and E. Benetos, “WavCraft: Audio editing and generation with large language models,” inICLR 2024 Workshop on LLM Agents, 2024

    2024

  17. [17]

    Audio Set: An ontology and human- labeled dataset for audio events,

    J. F. Gemmeke, D. P. Ellis, D. Freedman, A. Jansen, W. Lawrence, R. C. Moore, M. Plakal, and M. Ritter, “Audio Set: An ontology and human- labeled dataset for audio events,” inProc. ICASSP, 2017, pp. 776–780

    2017

  18. [18]

    AudioCaps: Generating captions for audios in the wild,

    C. D. Kim, B. Kim, H. Lee, and G. Kim, “AudioCaps: Generating captions for audios in the wild,” inProc. NAACL-HLT, 2019, pp. 119– 132

    2019

  19. [19]

    Clotho: An audio captioning dataset,

    K. Drossos, S. Lipping, and T. Virtanen, “Clotho: An audio captioning dataset,” inProc. ICASSP, 2020, pp. 736–740

    2020

  20. [20]

    Pengi: An audio language model for audio tasks,

    S. Deshmukh, B. Elizalde, R. Singh, and H. Wang, “Pengi: An audio language model for audio tasks,” inProc. NeurIPS, vol. 36, 2023, pp. 18 090–18 108

    2023

  21. [21]

    Listen, think, and understand,

    Y . Gong, H. Luo, A. Liu, L. Karlinsky, and J. R. Glass, “Listen, think, and understand,” inProc. ICLR, 2024, pp. 18 516–18 545

    2024

  22. [22]

    Qwen-Audio: Advancing universal audio understanding via unified large-scale audio-language models,

    Y . Chu, J. Xu, X. Zhou, Q. Yang, S. Zhang, Z. Yan, C. Zhou, and J. Zhou, “Qwen-Audio: Advancing universal audio understanding via unified large-scale audio-language models,”arXiv preprint arXiv:2311.07919, 2023

    Pith/arXiv arXiv 2023

  23. [23]

    SALMONN: Towards generic hearing abilities for large language models,

    C. Tang, W. Yu, G. Sun, X. Chen, T. Tan, W. Li, L. Lu, Z. Ma, and C. Zhang, “SALMONN: Towards generic hearing abilities for large language models,” inProc. ICLR, 2024, pp. 16 607–16 629

    2024

  24. [24]

    Audio Flamingo: A novel audio language model with few-shot learning and dialogue abilities,

    Z. Kong, A. Goel, R. Badlani, W. Ping, R. Valle, and B. Catanzaro, “Audio Flamingo: A novel audio language model with few-shot learning and dialogue abilities,” inProc. ICML, 2024, pp. 25 125–25 148

    2024

  25. [25]

    Frame-level internal tool use for temporal grounding in audio LMs,

    J. An, P. Keung, J. Wang, O. Ahia, and N. A. Smith, “Frame-level internal tool use for temporal grounding in audio LMs,”arXiv preprint arXiv:2602.10230, 2026

    Pith/arXiv arXiv 2026

  26. [26]

    TimeAudio: Bridging temporal gaps in large audio-language models,

    H. Wang, Y . Li, S. Ma, H. Liu, and X. Wang, “TimeAudio: Bridging temporal gaps in large audio-language models,” inProc. AAAI, 2026

    2026

  27. [27]

    Make-an-Audio: Text-to-audio generation with prompt-enhanced diffusion models,

    R. Huang, J. Huang, D. Yang, Y . Ren, L. Liu, M. Li, Z. Ye, J. Liu, X. Yin, and Z. Zhao, “Make-an-Audio: Text-to-audio generation with prompt-enhanced diffusion models,” inProc. ICML, 2023, pp. 13 916– 13 932

    2023

  28. [28]

    Text-to-audio generation using instruction guided latent diffusion model,

    D. Ghosal, N. Majumder, A. Mehrish, and S. Poria, “Text-to-audio generation using instruction guided latent diffusion model,” inProc. ACM MM, 2023, pp. 3590–3598

    2023

  29. [29]

    EzAudio: Enhancing text-to-audio generation with efficient diffusion transformer,

    J. Hai, Y . Xu, H. Zhang, C. Li, H. Wang, M. Elhilali, and D. Yu, “EzAudio: Enhancing text-to-audio generation with efficient diffusion transformer,” inProc. Interspeech, 2025, pp. 4233–4237

    2025

  30. [30]

    TangoFlux: Super fast and faithful text to audio generation with flow matching and CLAP-ranked preference optimization,

    C.-Y . Hung, N. Majumder, Z. Kong, A. Mehrish, A. A. Bagherzadeh, C. Li, R. Valle, B. Catanzaro, and S. Poria, “TangoFlux: Super fast and faithful text to audio generation with flow matching and CLAP-ranked preference optimization,” inProc. ICLR, 2026

    2026

  31. [31]

    FoleyCrafter: Bring silent videos to life with lifelike and synchronized sounds,

    Y . Zhang, Y . Gu, Y . Zeng, Z. Xing, Y . Wang, Z. Wu, B. Liu, and K. Chen, “FoleyCrafter: Bring silent videos to life with lifelike and synchronized sounds,”International Journal of Computer Vision, vol. 134, 2026

    2026

  32. [32]

    Sound-VECaps: Improving audio generation with visually enhanced captions,

    Y . Yuan, D. Jia, X. Zhuang, Y . Chen, Z. Chen, Y . Wang, Y . Wang, X. Liu, X. Kang, M. D. Plumbley, and W. Wang, “Sound-VECaps: Improving audio generation with visually enhanced captions,” inProc. ICASSP, 2025, pp. 1–5

    2025

  33. [33]

    ReAct: Synergizing reasoning and acting in language models,

    S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y . Cao, “ReAct: Synergizing reasoning and acting in language models,” inProc. ICLR, 2023

    2023

  34. [34]

    Toolformer: Language models can teach themselves to use tools,

    T. Schick, J. Dwivedi-Yu, R. Dess `ı, R. Raileanu, M. Lomeli, E. Hambro, L. Zettlemoyer, N. Cancedda, and T. Scialom, “Toolformer: Language models can teach themselves to use tools,” inProc. NeurIPS, vol. 36, 2023, pp. 68 539–68 551

    2023

  35. [35]

    HuggingGPT: Solving ai tasks with ChatGPT and its friends in Hugging Face,

    Y . Shen, K. Song, X. Tan, D. Li, W. Lu, and Y . Zhuang, “HuggingGPT: Solving ai tasks with ChatGPT and its friends in Hugging Face,” in Proc. NeurIPS, vol. 36, 2023, pp. 38 154–38 180

    2023

  36. [36]

    Agentic large language models, a survey,

    A. Plaat, M. van Duijn, N. Van Stein, M. Preuss, P. van der Putten, and K. J. Batenburg, “Agentic large language models, a survey,”Journal of Artificial Intelligence Research, vol. 84, 2025

    2025

  37. [37]

    Agentic tool use in large language models,

    J. Hu, M. Zhong, K. Chen, X. Bai, and M. Zhang, “Agentic tool use in large language models,”arXiv preprint arXiv:2604.00835, 2026

    Pith/arXiv arXiv 2026

  38. [38]

    A semantically consistent dataset for data-efficient query-based universal sound separation,

    K. Li, J. Cheng, C. Zeng, Z. Yan, H. Wang, Z. Su, B. Zheng, and X. Hu, “A semantically consistent dataset for data-efficient query-based universal sound separation,” inProc. ICML, 2026

    2026

  39. [39]

    Meta Audiobox Aesthetics: Unified automatic assessment for speech, music and sound,

    A. Tjandra, Y .-C. Wu, B. Guo, J. Hoffman, B. Ellis, A. Vyas, B. Shi, S. Chen, M. Le, N. Zacharovet al., “Meta Audiobox Aesthetics: Unified automatic assessment for speech, music and sound,” inProc. ASRU, 2025, pp. 1–8

    2025

  40. [40]

    Alignment is all you need: Instruction-free training for general audio-language models,

    X. Zhou, Y . Shao, J. Li, and D. Yu, “Alignment is all you need: Instruction-free training for general audio-language models,”arXiv preprint, 2026

    2026

  41. [41]

    Qwen2.5-Omni technical report,

    J. Xu, Z. Guo, J. He, H. Hu, T. He, S. Bai, K. Chen, J. Wang, Y . Fan, K. Dang, B. Zhang, X. Wang, Y . Chu, and J. Lin, “Qwen2.5-Omni technical report,”arXiv preprint arXiv:2503.20215, 2025

    Pith/arXiv arXiv 2025

  42. [42]

    Qwen2.5 technical report,

    A. Yang, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Li, D. Liu, F. Huang, H. Wei, H. Lin, J. Yang, J. Tu, J. Zhang, J. Yang, J. Yang, J. Zhou, J. Lin, K. Dang, K. Lu, K. Bao, K. Yang, L. Yu, M. Li, M. Xue, P. Zhang, Q. Zhu, R. Men, R. Lin, T. Li, T. Tang, T. Xia, X. Ren, X. Ren, Y . Fan, Y . Su, Y . Zhang, Y . Wan, Y . Liu, Z. Cui, Z. Zhang, and Z. Qi...

    Pith/arXiv arXiv 2025

  43. [43]

    Re- visiting audio-language pretraining for learning general-purpose audio representation,

    W.-C. Tseng, X. Zhou, M. Huo, Y . Shao, H. Zhang, and D. Yu, “Re- visiting audio-language pretraining for learning general-purpose audio representation,”arXiv preprint arXiv:2511.16757, 2025

    Pith/arXiv arXiv 2025

  44. [44]

    Auden-Voice: General-purpose voice encoder for speech and language understanding,

    M. Huo, W.-C. Tseng, Y . Shao, H. Zhang, and D. Yu, “Auden-Voice: General-purpose voice encoder for speech and language understanding,” inProc. ICASSP, 2026, pp. 15 382–15 386

    2026