Pith. sign in

REVIEW 4 major objections 40 references

Automatically built synthetic clips and multi-model pseudo-labels can teach large audio-language models to localize open-vocabulary sound events without manual timestamps.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-11 19:33 UTC pith:NLU4LZZ6

load-bearing objection Solid data-side recipe for open-vocab audio grounding: synthetic exact-GT SFT + real pseudo-label GRPO, with a human-checked bench and real DESED transfer; residual annotator-stack overlap is real but not fatal. the 4 major comments →

arxiv 2607.04383 v1 pith:NLU4LZZ6 submitted 2026-07-05 cs.SD cs.AI

Auto-AEG: Scalable Data Construction for Open-Vocabulary Audio Event Grounding

Zihan Zhang , Xize Cheng , Wenhao Yan , Tong Zhang , Dongjie Fu , Boyun Zhang , Yongbo He , Tao Jin This is my paper
classification cs.SD cs.AI
keywords audio event groundinglarge audio-language modelstemporal localizationdata constructionreinforcement learningsound event detectionopen vocabularypseudo-labels
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Large audio-language models can describe sound fluently but still fail to say precisely when events begin and end, while classical sound-event detectors are locked to fixed label sets. This paper claims the missing piece is data, not architecture: a pipeline called Auto-AEG programmatically mixes real audio into synthetic clips that carry exact ground-truth intervals for a supervised cold-start, then uses multi-model pseudo-labels on real FreeSound recordings as the reward for reinforcement learning with an interval-aware objective. Fine-tuning two Omni models this way produces large relative gains on a new independent, difficulty-stratified benchmark (AEGBench) and also improves event-level scores on the classic DESED sound-event detection set. A sympathetic reader would care because manual onset/offset annotation is prohibitively expensive; if automatic construction works, temporal grounding becomes a scalable data problem rather than a permanent capability gap.

Core claim

Automatically constructed data—exact ground-truth synthetic clips for supervised fine-tuning cold-start paired with multi-model pseudo-labels on real-world audio for interval-aware Group Relative Policy Optimization—is an effective data-side route that expands the temporal localization capability of large audio-language models, yielding +73.9% relative mIoU for a 30B model and +23.1% for a 7B model over zero-shot on the independent AEGBench, plus gains on DESED, without architectural change.

What carries the argument

Auto-AEG: a two-stage data-construction pipeline that matches data type to training objective—programmatic synthesis of multi-occurrence clips with exact intervals for SFT cold-start, and multi-model (semantic inventory + frame-level localization + vocabulary cleaning) pseudo-labels that supply an interval-aware reward (F1-IoU, format, non-empty, precision) under GRPO.

Load-bearing premise

The multi-model pseudo-labels on real audio are informative enough as a reinforcement-learning reward that the policy improves true boundaries rather than merely imitating the annotators’ systematic errors.

What would settle it

Train only on Auto-AEG data, then evaluate on a fully human-annotated open-vocabulary grounding set drawn from audio sources and label styles that never appear in the Gemini/PE A-Frame pipeline; if the fine-tuned models no longer beat strong zero-shot baselines on that set, the claim that the automatic supervision is genuinely transferable collapses.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Open-vocabulary temporal grounding for large audio-language models can be scaled from public audio collections without waiting for massive human onset/offset corpora.
  • A grounding policy trained on open-vocabulary automatic data transfers back to closed-set sound event detection (DESED event-level metrics improve).
  • Larger models exploit noisy real-world reward signals more fully; smaller models shift toward higher precision at some cost to recall.
  • Synthetic cold-start plus interval-aware RL is a reusable pattern for any LALM whose audio tower already encodes time.
  • Hard cases such as long-duration events remain limited by encoder window size even after data scaling.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same synthesis-plus-RL pattern could close temporal gaps in other multimodal models that already reason well but localize poorly.
  • Separating continuous versus discrete events before span merging is a general hygiene step for any frame-level pseudo-labeler.
  • If residual error is mostly pseudo-label noise, a small human-verified seed used for iterative self-training could raise the ceiling without full re-annotation.
  • Long-duration localization may need hierarchical or sliding-window encoders more than simply more Auto-AEG-style data.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 0 minor

Summary. The paper formalizes Open-Vocabulary Audio Event Grounding—predicting all onset/offset intervals for a natural-language sound-event query—and proposes Auto-AEG, a two-stage data pipeline that pairs programmatically synthesized clips with exact ground-truth intervals (SFT cold-start) with multi-model pseudo-labels on real FreeSound audio (Gemini inventory, continuous/discrete typing, PE A-Frame localization, CLAP/LM cleaning) used as the reward signal for interval-aware GRPO. It also releases AEGBench, a source-disjoint, energy-contrast-filtered, human-reviewed, difficulty-stratified evaluation set. Fine-tuning Qwen3-Omni-30B and Qwen2.5-Omni-7B yields large relative mIoU gains on AEGBench (especially for the 30B model after SFT+GRPO) and improves event-level F1/precision on the closed-set DESED SED benchmark, supporting the claim that automatically constructed data plus reward design is an effective data-side route to LALM temporal localization without architectural change.

Significance. Data scarcity for open-vocabulary onset/offset supervision is a genuine bottleneck for adapting LALMs to fine-grained temporal localization. A scalable, annotation-free construction pipeline that separates exact-GT synthetic cold-start from noise-tolerant RL on real audio is a useful contribution, as is the difficulty-stratified AEGBench and the explicit interval-aware reward (F1-IoU@0.5 plus format, non-empty, and precision terms). Transfer gains on DESED under a List-All prompt without SED-specific heads strengthen the claim that the learned policy improves a shared temporal-localization capability. If the independence and noise-tolerance claims hold under stronger checks, the work offers a practical data-side path for the community rather than another architecture-only fix.

major comments (4)
  1. Independence of AEGBench vs. Stage-2 labels is load-bearing for the central claim that gains reflect genuine acoustic grounding rather than annotator imitation. Section 5.2 states that benchmark items are annotated by the same multi-model pipeline as Stage 2 (Gemini + PE A-Frame + CLAP/LM cleaning; Appendix C/D), with human review only confirming/correcting labels and adjusting boundaries. Source-disjointness (FreeSound train vs. AudioSet Strong/FSD50K/BBC/YouTube eval) and full human pass over 3,427 items reduce risk but do not eliminate shared systematic biases (PE A-Frame ~40 ms resolution, threshold 0.5, continuous-merge 0.5 s; Gemini inventory style). Please quantify pipeline–human agreement (e.g., mIoU / boundary error before vs. after human correction) and report results on a fully human-timestamped subset, or otherwise show that large lifts (Table 5: Q3-Omni mIoU 0.276→0.480) sur
  2. Section 6.2 / Eq. (2): the GRPO reward weights (0.65 r_iou + 0.15 r_fmt + 0.05 r_nem + 0.15 r_prec) and the PE A-Frame localization hyperparameters are free parameters that directly shape the policy. Table 5 shows a clear precision–recall trade-off for Q2.5-Omni under GRPO (mIoU 0.424→0.399 while onset P rises 0.411→0.594), while Q3-Omni improves across the board. Without ablations of the weight vector (especially r_prec and the F1-IoU@0.5 choice) and of continuous-merge / threshold settings, it is hard to know whether the reported gains are robust properties of Auto-AEG data or of a particular reward tuning. Please add at least a small ablation on reward components and discuss when GRPO helps vs. hurts as a function of model scale.
  3. Section 7.3 / Table 7 (DESED): SFT alone regresses event-F1 and precision for both models before GRPO recovers and surpasses zero-shot. The manuscript attributes this to synthetic–real domain mismatch, which is plausible, but the stage-level pattern is important for the two-stage design claim. Please report whether the SFT checkpoint is necessary (GRPO from zero-shot vs. from SFT), and clarify how much of the DESED gain is format/List-All compliance versus improved boundary localization under the closed 10-class vocabulary.
  4. Main results (Tables 5–7) report point estimates only. There are no error bars, multiple random seeds, or statistical tests despite free parameters (LoRA, LR, G=4 rollouts, β_KL=0.04) and a noisy reward. For claims of +73.9% / +23.1% relative mIoU and DESED event-metric gains, at least seed-level variance or bootstrap intervals on AEGBench and DESED would make the improvements interpretable and would clarify whether the 7B mIoU drop under GRPO is stable.

Circularity Check

1 steps flagged

Mild shared-annotation risk between Stage-2 GRPO pseudo-labels and AEGBench Phase-1 labels; mitigated by source-disjoint audio, full human correction, and external DESED transfer, so central empirical claim remains non-circular.

specific steps
  1. other [Section 5.2 (and cross-ref Section 4.3 / Appendix C)]
    "Benchmark items are annotated by the same multi-model pipeline as Stage 2 of Auto-AEG—Gemini label identification, event-type classification, PE A-Frame localization, and CLAP-based global label cleaning—yielding clean onset/offset annotations over a canonical vocabulary."

    Stage-2 GRPO reward (Eq. 2: 0.65 r_iou = F1-IoU@0.5 against the pseudo-labels) and AEGBench ground truth are generated by the identical annotation stack. Even after human correction of all 3,427 items, residual systematic biases of PE A-Frame (40 ms frames, 0.5 threshold, 0.5 s continuous merge) and Gemini can remain; therefore measured mIoU/ev-F1 gains partly optimize toward matching the shared annotator rather than purely independent human timestamps. Source-disjointness and DESED transfer reduce but do not erase the risk.

full rationale

This is an empirical data-construction + RL paper, not a first-principles derivation. There are no self-definitional equations, no fitted parameters renamed as predictions, no uniqueness theorems imported from the authors, and no load-bearing self-citations that force the result. The only potential circularity is evaluation-side: AEGBench Phase-1 uses the identical multi-model stack (Gemini inventory + continuous/discrete typing + PE A-Frame localization + CLAP/LM cleaning) that produces the Stage-2 GRPO reward targets. Because the reward (Eq. 2) optimizes F1-IoU@0.5 against those pseudo-labels, large mIoU/ev-F1 lifts on AEGBench could partly reward imitation of residual PE A-Frame / Gemini biases rather than fully independent acoustic grounding. The paper itself asserts independence via different source pools (FreeSound train vs. AudioSet Strong / FSD50K / BBC / YouTube eval), energy-contrast filtering, and human review of every one of the 3,427 items that corrects labels and boundaries. DESED (closed-set, external, human-annotated) further shows GRPO-driven event-F1/precision gains after SFT regression, confirming transfer beyond the shared pipeline. These mitigations keep the circularity mild and non-load-bearing; the strongest claim (automatically constructed data + interval-aware GRPO expands LALM temporal localization) is still supported by architecture-free gains on both the human-corrected bench and an external SED benchmark. Score 2 reflects one non-central shared-pipeline step of kind 'other'; no higher score is warranted.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 3 invented entities

The central claim rests on engineering choices (reward weights, localization thresholds, synthesis SNR and occurrence skew, energy-contrast gate) and on domain assumptions that noisy multi-model intervals remain a usable RL signal and that synthetic mixes transfer enough structure for cold-start. No new physical entities are postulated; the invented objects are the pipeline, the benchmark, and the task formalization.

free parameters (5)
  • GRPO reward weights (0.65 r_iou + 0.15 r_fmt + 0.05 r_nem + 0.15 r_prec)
    Hand-chosen linear combination that defines the training objective; the precision-decay schedule (ratio 2→4) is also free.
  • PE A-Frame active-frame threshold 0.5 and continuous-merge gap 0.5 s
    Controls which spans become pseudo ground truth for both Stage-2 training and AEGBench Phase-1.
  • Energy-contrast gate ≥12 dB (and low-contrast band 12–28 dB)
    Primary quality filter for AEGBench candidates and hard-case tagging; chosen by the authors.
  • Synthetic SNR range [10,20] dB and occurrence-count skew (20/30/25/15/10 % for 1–5 events)
    Determines the exact-GT SFT distribution and multi-event bias.
  • LoRA rank r=16, α=32, LR 2e-4 (SFT) / 5e-5 (GRPO), β_KL=0.04, G=4 rollouts
    Training hyperparameters that affect whether the reported gains are obtained.
axioms (4)
  • domain assumption Multi-model collaboration (Gemini semantic inventory + PE A-Frame frame localization + CLAP/LM global cleaning) yields pseudo-labels whose residual noise is tolerable for GRPO advantage estimation.
    Stated as the design rationale in §4 and §6.2; never validated by human–pseudo agreement statistics.
  • domain assumption Programmatic mixing of FreeSound segments onto Gaussian noise preserves enough acoustic structure that SFT on exact intervals transfers to real recordings.
    Stage-1 design claim; Limitations explicitly notes remaining domain mismatch in room acoustics and source interaction.
  • ad hoc to paper F1-IoU@0.5 plus format/non-empty/precision terms is a sufficient scalar reward for open-vocabulary multi-interval grounding.
    Reward equation (2) is an author design choice, not derived from a prior optimality result.
  • domain assumption Whisper-style 30 s encoder windows are an acceptable hard limit for the evaluated clip lengths.
    Explains the persistent Long-Duration weakness; treated as fixed background rather than ablated.
invented entities (3)
  • Auto-AEG pipeline no independent evidence
    purpose: Annotation-free construction of exact-GT synthetic SFT data and real-audio pseudo-labels for GRPO.
    Core contribution; existence is demonstrated by the described procedure and training results, not by external independent measurement of the pipeline itself.
  • AEGBench independent evidence
    purpose: Independent, difficulty-stratified evaluation surface for open-vocabulary audio event grounding.
    New benchmark with human verification; source distribution and filters are paper-defined, but human correction supplies an external handle.
  • Open-Vocabulary Audio Event Grounding (task formalization) no independent evidence
    purpose: Unify LALM open-query flexibility with SED-style multi-interval onset/offset prediction.
    Task sits between prior TAG/AMR/SED formulations; the formal definition and response format are paper-specific.

pith-pipeline@v1.1.0-grok45 · 21015 in / 3643 out tokens · 43186 ms · 2026-07-11T19:33:32.823348+00:00 · methodology

0 comments
read the original abstract

Large Audio-Language Models (LALMs) reason fluently about sound yet struggle to localize precisely when events occur, while classical Sound Event Detection attains frame-level precision only over a closed label set. At the intersection of these paradigms lies the task of Open-Vocabulary Audio Event Grounding: predicting all time intervals of a target sound event described by an arbitrary natural language query. While this task is crucial for real-world audio understanding and LALM adaptation, it is bottlenecked by data scarcity. Few large-scale resources provide open-vocabulary onset/offset supervision, and manual temporal annotation is prohibitively expensive. To address this, we introduce Auto-AEG, a scalable pipeline that constructs such supervision by automatic data construction and model fine-tuning. It pairs programmatically synthesized clips, which carry exact ground-truth intervals for supervised cold-start, with multi-model pseudo-labels on real-world audio that supply the reward signal for reinforcement learning. Training with this pipeline yields promising performance gains on both the DESED SED benchmark and AEGBench, an independent difficulty-stratified benchmark we release. Our results show that automatically constructed data, coupled with interval-aware reward function design, is an effective data-side route to expanding the temporal localization capability of LALMs.

Figures

Figures reproduced from arXiv: 2607.04383 by Boyun Zhang, Dongjie Fu, Tao Jin, Tong Zhang, Wenhao Yan, Xize Cheng, Yongbo He, Zihan Zhang.

Figure 1
Figure 1. Figure 1: Auto-AEG pipeline overview. The pipeline annotates real-world audio with pseudo-labels for [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Prompt template shared by SFT and GRPO. The system prompt elicits a [PITH_FULL_IMAGE:figures/full_fig_p013_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Label-identification prompt, sent once per 10 s audio chunk; labels from all chunks are unioned [PITH_FULL_IMAGE:figures/full_fig_p014_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Event-type classification (top) and global label-cleaning (bottom) prompts. The classification [PITH_FULL_IMAGE:figures/full_fig_p015_4.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

40 extracted references · 17 linked inside Pith

  1. [1]

    BLAB: Brutally Long Audio Bench

    Orevaoghene Ahia, Martijn Bartelds, et al. “BLAB: Brutally Long Audio Bench”. In:arXiv preprint arXiv:2505.03054(2025)

    Pith/arXiv arXiv 2025

  2. [2]

    Localizing moments in video with natural language

    Lisa Anne Hendricks, Oliver Wang, Eli Shechtman, Josef Sivic, Trevor Darrell, and Bryan Russell. “Localizing moments in video with natural language”. In:Proceedings of the IEEE international conference on computer vision. 2017, pp. 5803–5812

    2017

  3. [3]

    BEATs: Audio Pre-Training with Acoustic Tokenizers

    Sanyuan Chen, Yu Wu, Chengyi Wang, Shujie Liu, Daniel Tompkins, Zhuo Chen, and Furu Wei. “BEATs: Audio Pre-Training with Acoustic Tokenizers”. In:International Conference on Machine Learning (ICML). 2023

    2023

  4. [5]

    Qwen2-Audio Technical Report

    Yunfei Chu, Jin Xu, Qian Yang, Haojie Wei, Xipin Wei, Zhifang Guo, Yichong Leng, Yuanjun Lv, Jinzheng He, Junyang Lin, Chang Zhou, and Jingren Zhou. “Qwen2-Audio Technical Report”. In: arXiv preprint arXiv:2407.10759(2024)

    Pith/arXiv arXiv 2024

  5. [6]

    Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Models

    Yunfei Chu, Jin Xu, Xiaohuan Zhou, Qian Yang, Shiliang Zhang, Zhijie Yan, Chang Zhou, and Jingren Zhou. “Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Models”. In:arXiv preprint arXiv:2311.07919(2023)

    Pith/arXiv arXiv 2023

  6. [7]

    Pengi: An Audio Language Model for Audio Tasks

    Soham Deshmukh, Benjamin Elizalde, Rita Singh, and Huaming Wang. “Pengi: An Audio Language Model for Audio Tasks”. In:Advances in Neural Information Processing Systems. Vol. 36. 2023, pp. 18090–18108

    2023

  7. [8]

    FSD50K: An Open Dataset of Human-Labeled Sound Events

    Eduardo Fonseca, Xavier Favory, Jordi Pons, Frederic Font, and Xavier Serra. “FSD50K: An Open Dataset of Human-Labeled Sound Events”. In:IEEE/ACM Transactions on Audio, Speech, and Language Processing30 (2022), pp. 829–852

    2022

  8. [9]

    Tall: Temporal activity localization via language query

    Jiyang Gao, Chen Sun, Zhenheng Yang, and Ram Nevatia. “Tall: Temporal activity localization via language query”. In:Proceedings of the IEEE international conference on computer vision. 2017, pp. 5267–5275

    2017

  9. [10]

    Sreyan Ghosh, Arushi Goel, Kaousheik Jayakumar, Lasha Koroshinadze, Nishit Anand, Zhifeng Kong, Siddharth Gururani, Sang-gil Lee, Jaehyeon Kim, Aya Aljafari, Chao-Han Huck Yang, Sung- won Kim, Ramani Duraiswami, Dinesh Manocha, Mohammad Shoeybi, Bryan Catanzaro, Ming- Yu Liu, and Wei Ping.Audio Flamingo Next: Next-Generation Open Audio-Language Models for...

    Pith/arXiv arXiv 2026

  10. [11]

    Ast: Audio spectrogram transformer

    Yuan Gong, Yu-An Chung, and James Glass. “Ast: Audio spectrogram transformer”. In:arXiv preprint arXiv:2104.01778(2021)

    Pith/arXiv arXiv 2021

  11. [12]

    Listen, Think, and Understand

    Yuan Gong, Hongyin Luo, Alexander H. Liu, Leonid Karlinsky, and James Glass. “Listen, Think, and Understand”. In:International Conference on Learning Representations. 2024. 10

    2024

  12. [13]

    Acoustic scene classification in DCASE 2020 Challenge: generalization across devices and low complexity solutions

    Toni Heittola, Annamaria Mesaros, and Tuomas Virtanen. “Acoustic scene classification in DCASE 2020 Challenge: generalization across devices and low complexity solutions”. In:Proceedings of the Detection and Classification of Acoustic Scenes and Events 2020 Workshop (DCASE2020). 2020, pp. 56–60.url:https://arxiv.org/abs/2005.14623

    Pith/arXiv arXiv 2020

  13. [14]

    Ailin Huang et al.Step-Audio: Unified Understanding and Generation in Intelligent Speech Inter- action. 2025. arXiv:2502.11946 [cs.CL].url:https://arxiv.org/abs/2502.11946

    Pith/arXiv arXiv 2025

  14. [15]

    com / MoonshotAI / Kimi - Audio

    Kimi Team.Kimi-Audio Technical Report.https : / / github . com / MoonshotAI / Kimi - Audio. Open-source 7B audio language model with native temporal grounding support. 2025

    2025

  15. [16]

    PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognition

    Qiuqiang Kong, Yin Cao, Turab Iqbal, Yuxuan Wang, Wenwu Wang, and Mark D. Plumbley. “PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognition”. In: IEEE/ACM Trans. Audio, Speech and Lang. Proc.28 (Nov. 2020), pp. 2880–2894.issn: 2329-9290. doi:10.1109/TASLP.2020.3030497.url:https://doi.org/10.1109/TASLP.2020.3030497

    doi:10.1109/taslp.2020.3030497.url:https://doi.org/10.1109/taslp.2020.3030497 2020

  16. [17]

    Detecting moments and highlights in videos via natural language queries

    Jie Lei, Tamara L Berg, and Mohit Bansal. “Detecting moments and highlights in videos via natural language queries”. In:Advances in Neural Information Processing Systems34 (2021), pp. 11846– 11858

    2021

  17. [18]

    Zijia Liu, Peixuan Han, Haofei Yu, Haoru Li, and Jiaxuan You.Time-R1: Towards Comprehensive Temporal Reasoning in LLMs. 2025. arXiv:2505.13508 [cs.CL].url:https://arxiv.org/abs/ 2505.13508

    Pith/arXiv arXiv 2025

  18. [19]

    Wavcaps: A chatgpt-assisted weakly-labelled audio captioning dataset for audio-language multimodal research

    Xinhao Mei, Chutong Meng, Haohe Liu, Qiuqiang Kong, Tom Ko, Chengqi Zhao, Mark D Plumbley, Yuexian Zou, and Wenwu Wang. “Wavcaps: A chatgpt-assisted weakly-labelled audio captioning dataset for audio-language multimodal research”. In:IEEE/ACM Transactions on Audio, Speech, and Language Processing32 (2024), pp. 3339–3354

    2024

  19. [20]

    Query-dependent video representation for moment retrieval and highlight detection

    WonJun Moon, Sangeek Hyun, SangUk Park, Dongchan Park, and Jae-Pil Heo. “Query-dependent video representation for moment retrieval and highlight detection”. In:Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2023, pp. 23023–23033

    2023

  20. [21]

    Language-based Au- dio Moment Retrieval

    Hokuto Munakata, Taichi Nishimura, Shota Nakada, and Tatsuya Komatsu. “Language-based Au- dio Moment Retrieval”. In:Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 2025

    2025

  21. [22]

    CASTELLA: Long Audio Dataset with Captions and Temporal Boundaries

    Shota Nakada, Taichi Nishimura, Hokuto Munakata, and Tatsuya Komatsu. “CASTELLA: Long Audio Dataset with Captions and Temporal Boundaries”. In:Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 2026

    2026

  22. [23]

    Paul Primus, Florian Schmid, and Gerhard Widmer.TACOS: Temporally-aligned Audio CaptiOnS for Language-Audio Pretraining. 2025. arXiv:2505.07609 [cs.SD].url:https://arxiv.org/ abs/2505.07609

    Pith/arXiv arXiv 2025

  23. [24]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. “Learning transferable visual models from natural language supervision”. In:International conference on machine learn- ing(2021), pp. 8748–8763

    2021

  24. [25]

    Large-Scale Weakly La- beled Semi-Supervised Sound Event Detection in Domestic Environments

    Romain Serizel, Nicolas Turpault, Hamid Eghbalzadeh, and Ankit Shah. “Large-Scale Weakly La- beled Semi-Supervised Sound Event Detection in Domestic Environments”. In:ArXivabs/1807.10501 (2018).url:https://api.semanticscholar.org/CorpusID:51864948

    Pith/arXiv arXiv 2018

  25. [26]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. “Deepseekmath: Pushing the limits of mathematical reasoning in open language models”. In:arXiv preprint arXiv:2402.03300(2024)

    Pith/arXiv arXiv 2024

  26. [27]

    Luoyi Sun, Xiao Zhou, Zeqian Li, Ya Zhang, Yanfeng Wang, and Weidi Xie.SpotSound: Enhancing Large Audio-Language Models with Fine-Grained Temporal Grounding. 2026. arXiv:2604.13023 [cs.SD].url:https://arxiv.org/abs/2604.13023

    Pith/arXiv arXiv 2026

  27. [28]

    SALMONN: Towards Generic Hearing Abilities for Large Language Models

    Changli Tang, Wenyi Yu, Guangzhi Sun, Xianzhao Chen, Tian Tan, Wei Li, Lu Lu, Zejun MA, and Chao Zhang. “SALMONN: Towards Generic Hearing Abilities for Large Language Models”. In:The Twelfth International Conference on Learning Representations. 2024.url:https://openreview. net/forum?id=14rn7HpKVk

    2024

  28. [29]

    Qwen2.5-Omni Technical Report

    Qwen Team. “Qwen2.5-Omni Technical Report”. In:arXiv preprint arXiv:2503.20215. 2025. 11

    Pith/arXiv arXiv 2025

  29. [30]

    Sound event detection in domestic environments with weakly labeled data and soundscape synthesis

    Nicolas Turpault, Romain Serizel, Ankit Parag Shah, and Justin Salamon. “Sound event detection in domestic environments with weakly labeled data and soundscape synthesis”. In:Workshop on Detection and Classification of Acoustic Scenes and Events. 2019

    2019

  30. [31]

    Apoorv Vyas, Heng-Jui Chang, Cheng-Fu Yang, Po-Yao Huang, Luya Gao, Julius Richter, Sanyuan Chen, Matt Le, Piotr Doll´ ar, Christoph Feichtenhofer, Ann Lee, and Wei-Ning Hsu.Pushing the Frontier of Audiovisual Perception with Large-Scale Multimodal Correspondence Learning. 2025. arXiv:2512.19687 [cs.SD].url:https://arxiv.org/abs/2512.19687

    arXiv 2025

  31. [32]

    Listening Between the Frames: Bridging Temporal Gaps in Large Audio-Language Models

    Hualei Wang, Yiming Li, Shuo Ma, Hong Liu, and Xiangdong Wang. “Listening Between the Frames: Bridging Temporal Gaps in Large Audio-Language Models”. In:arXiv preprint arXiv:2511.11039 (2025)

    arXiv 2025

  32. [33]

    Yusong Wu, Ke Chen, Tianyu Zhang, Yuchen Hui, Marianna Nezhurina, Taylor Berg-Kirkpatrick, and Shlomo Dubnov.Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentation. 2024. arXiv:2211.06687 [cs.SD].url:https://arxiv.org/ abs/2211.06687

    Pith/arXiv arXiv 2024

  33. [34]

    Zeyu Xie, Xuenan Xu, Zhizheng Wu, and Mengyue Wu.AudioTime: A Temporally-aligned Audio- text Benchmark Dataset. 2024. arXiv:2407.02857 [eess.AS].url:https://arxiv.org/abs/ 2407.02857

    Pith/arXiv arXiv 2024

  34. [35]

    Zeyu Xie, Xuenan Xu, Zhizheng Wu, and Mengyue Wu.PicoAudio: Enabling Precise Timestamp and Frequency Controllability of Audio Events in Text-to-audio Generation. 2024. arXiv:2407. 02869 [eess.AS].url:https://arxiv.org/abs/2407.02869

    Pith/arXiv arXiv 2024

  35. [36]

    Jin Xu, Zhifang Guo, Hangrui Hu, Yunfei Chu, Xiong Wang, Jinzheng He, Yuxuan Wang, Xian Shi, Ting He, Xinfa Zhu, Yuanjun Lv, Yongqi Wang, Dake Guo, He Wang, Linhan Ma, Pei Zhang, Xinyu Zhang, Hongkun Hao, Zishan Guo, Baosong Yang, Bin Zhang, Ziyang Ma, Xipin Wei, Shuai Bai, Keqin Chen, Xuejing Liu, Peng Wang, Mingkun Yang, Dayiheng Liu, Xingzhang Ren, Bo ...

    Pith/arXiv arXiv 2025

  36. [37]

    Text-to-Audio Grounding: Building Correspondence Between Captions and Sound Events

    Xuenan Xu, Heinrich Dinkel, Mengyue Wu, and Kai Yu. “Text-to-Audio Grounding: Building Correspondence Between Captions and Sound Events”. In:Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 2021, pp. 606–610

    2021

  37. [38]

    Towards Weakly Supervised Text-to-Audio Grounding

    Xuenan Xu, Ziyang Ma, Mengyue Wu, and Kai Yu. “Towards Weakly Supervised Text-to-Audio Grounding”. In:IEEE Transactions on Multimedia(2024)

    2024

  38. [39]

    Semantic conditioned dynamic modulation for temporal sentence grounding in videos

    Yitian Yuan, Lin Ma, Jingwen Wang, Wei Liu, and Wenwu Zhu. “Semantic conditioned dynamic modulation for temporal sentence grounding in videos”. In:Advances in Neural Information Pro- cessing Systems32 (2019)

    2019

  39. [40]

    Span-based localizing network for natural language video localization

    Hao Zhang, Aixin Sun, Wei Jing, and Joey Tianyi Zhou. “Span-based localizing network for natural language video localization”. In:Proceedings of the 58th annual meeting of the association for computational linguistics. 2020, pp. 6543–6554

    2020

  40. [41]

    Convolutional recurrent neural networks: Learning spatial dependencies for image representation

    Zhen Zuo, Bing Shuai, Gang Wang, Xiao Liu, Xingxing Wang, Bing Wang, and Yushi Chen. “Convolutional recurrent neural networks: Learning spatial dependencies for image representation”. In:Proceedings of the IEEE conference on computer vision and pattern recognition workshops. 2015, pp. 18–26. 12 A. Training Prompt Template System prompt (shared by SFT and ...

    2015