Pith. sign in

REVIEW 3 major objections 5 minor 51 references

Auditory Intelligence: Understanding the World Through Sound

T0 review · 3 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Machine hearing should be reframed from surface recognition into a layered, situated process of perception, reasoning, and interaction, instantiated by four new task paradigms.

desk verdict An honest, well-scoped position paper that repackages existing tasks into a layered roadmap; the real risk is whether the highest cognitive layers (explanation and intent) can be annotated reliably enough to train and evaluate. read the letter →

arxiv 2508.07829 v1 pith:K3LIWOLP submitted 2025-08-11 eess.AS cs.AIcs.SD

Hyeonuk Nam This is my paper · ORCID
classification eess.AScs.AIcs.SD
keywords auditoryintelligencemachinelisteningsoundeventdetectionaudiocaptioningexplainableAIforaudio-languagemodelsresearchroadmap
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that current machine-hearing systems—sound event detection, acoustic scene classification, automated audio captioning, and audio question answering—are stuck at surface recognition: they capture what happened, not why it happened or what it implies. The author proposes reframing auditory intelligence as a layered, situated process with three levels—perceptual recognition, contextual reasoning, and generative interaction—and instantiates the reframing in four task paradigms: ASPIRE, SODA, AUX, and AUGMENT. If the reframing is right, progress in machine hearing should be measured on structured description, causal explanation, and intent inference rather than on isolated label accuracy, and datasets and benchmarks should be built around that stack. The paper is explicitly a research agenda with no quantitative evaluation; its empirical bets, such as annotating motivations and goals for everyday sounds, are still untested.

What carries the argument

The load-bearing structure is a three-layer cognitive model of auditory intelligence—perceptual recognition, contextual reasoning, generative interaction—operationalized as a task pipeline that converts audio into structured text. ASPIRE first textualizes spectrograms into spectro-temporal descriptors such as “broadband noise from 1.5–4 kHz for 0.8 s”; SODA organizes those descriptions into an open event→context→scene hierarchy; AUX adds causal/explanatory sentences; AUGMENT fills semantic slots (trigger, event, motivation, goal). This stack is the mechanism that would let recognition, explanation, and interaction share one representational scaffold, and it is what shifts evaluation from clo

What would settle it

Annotate a sample of DESED foreground clips with the ASPIRE pattern vocabulary and with SODA and AUGMENT slots using two independent experts, then train the proposed hierarchy with and without the event→context→scene structure on a DESED-derived corpus. If expert agreement on pattern descriptions or on motivation/goal slots is near chance (e.g., Cohen’s kappa below 0.6), or if the hierarchical model does not beat a flat multi-task baseline on unseen scenes, the roadmap’s core empirical bet is settled against it.

Watch

Extended reading notes

Core claim

The central claim is that sound, for an intelligent agent, is not a raw signal to classify but a cognitive medium encoding events, intentions, and contexts. The author argues that machine hearing should therefore be built as a layered stack: low-level spectro-temporal evidence (ASPIRE) feeds structured event–context–scene descriptions (SODA), which support causal and explanatory captions (AUX), and ultimately goal-driven interpretation with trigger, event, motivation, and goal slots (AUGMENT). Generation and audio–text–image grounding complete the loop by turning perception and reasoning into action. Within this view, existing tasks such as SED, ASC, and AAC remain useful but underspecified,

Load-bearing premise

The load-bearing premise is that human annotators can label spectro-temporal patterns, event–context–scene hierarchies, and motivation/goal slots with usable agreement, and that models can learn them well enough to beat flat recognition baselines; the paper itself signals the risk by noting that even Clotho captions leak inferred content despite instructions to describe only what is heard.

Editorial extensions

If this is right

  • Recognition benchmarks like SED, ASC, and AAC would be reframed as components of a larger stack, with evaluation newly focused on structure fidelity across event, context, and scene, and on consistency between levels.
  • Datasets would need layered annotations—spectro-temporal pattern descriptions, open hierarchical scene structures, and a clean separation of observed facts from inferred explanations—including a re-annotation of Clotho.
  • Models trained under SODA and AUX should generalize to unseen classes through text prototypes and should be able to justify decisions by citing explicit acoustic evidence such as timing, loudness, and frequency ranges.
  • AUGMENT would give assistive systems and robots access to the motivations and goals behind everyday sounds, enabling intent-aware interaction and safety diagnostics.
  • Audio generation would be conditioned on structured perception outputs, producing sound effects, speech, and music that stay consistent with inferred scene, affect, and goals, and audio–text–image grounding would extend the same structure to cross-modal retrieval.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: if ASPIRE’s spectro-temporal description is learned first, downstream SODA and AUX models may need far less paired audio–text data, because the intermediate description supplies compositional structure; the paper suggests transferability but does not quantify this benefit.
  • Inference: the proposed observation/explanation split for Clotho implies a direct grounding probe the paper does not design: train an explanation model with and without audio input at test time; the performance gap measures how much of the explanation is acoustically grounded rather than carried by language priors.
  • Inference: the same event→context→scene and trigger/event/motivation/goal structure could be applied to speech and music understanding—for instance, attributing speaker intent from prosody—which would make the framework a candidate unified benchmark for all non-speech auditory cognition, not just environmental sound.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper is a position paper arguing that current machine-listening research (SED, ASC, AAC, AQA) is limited to surface-level recognition and should be reframed as layered, situated auditory intelligence encompassing perception, contextual reasoning, and generative interaction. To instantiate this view, it proposes four task paradigms: ASPIRE (spectro-temporal pattern parsing), SODA (hierarchical event-to-context-to-scene description), AUX (causal/explanatory audio captioning), and AUGMENT (trigger/event/motivation/goal slot filling), plus extensions to generative audio and audio–text–image grounding. No model or quantitative evaluation is presented; the paper explicitly positions itself as a research agenda and invites collaboration. It is candid about open challenges (Section V) and about the conditional nature of the proposals, but the paper's strongest claims depend on the tractability of the proposed annotation and learning tasks, especially at the higher cognitive layers.

Significance. If the proposed task formulations prove feasible and learnable, they would provide a valuable organizing framework and shift benchmarking from isolated recognition to structured, explainable, and interaction-oriented evaluation. The paper’s strengths are its clarity, the concreteness of the named paradigms (ASPIRE, SODA, AUX, AUGMENT), and its explicit acknowledgment of absent quantitative support and of open problems (Section V and the footnote in Section I). The central risk, which I think is real, is that the highest layers (AUX and especially AUGMENT) require human annotations of causes, motivations, and goals from non-speech audio, and the paper offers no evidence that such annotations can be made reliably. The claim that the roadmap leads to 'human-aligned auditory intelligence' therefore rests on an empirical bet that the paper does not yet support.

major comments (3)
  1. [Section IV-D and Table I] AUGMENT defines semantic slots for trigger, event, motivation, and goal. The last two slots require annotators to infer intentions and goals that are not directly observable in the acoustic signal. The paper provides no inter-annotator agreement data or pilot protocol; Table I lists 'inter-annotator agreement' only as an evaluation sketch. This matters because the paper's own Section IV-C notes that even Clotho captions, collected under an explicit 'describe only what is heard' instruction, leak inference—demonstrating how difficult the observation/explanation split is in practice. Since AUGMENT is the layer that supports 'intent-aware interaction' and the 'human-aligned' claim, the strongest version of the roadmap depends on this feasibility. I request a small-scale annotation feasibility study (even on a few examples) or an explicit downgrade of AUGMENT from a planned paradigm to a lon
  2. [Section IV-B, step 2] The SODA roadmap asserts that hierarchy-based learning (event→context→scene) 'outperforms single-task and flat multi-task baselines,' but gives no evidence or reference for this expectation. The claimed generalization and data-efficiency benefits of SODA hinge on this premise. A position paper is not required to contain full experiments, but to make the roadmap argumentative rather than assertive, it should cite relevant multi-task or hierarchical representation-learning results, or present a concrete pilot comparison plan with a minimal dataset. As written, this is a testable hypothesis, not a supported component of the roadmap—please mark it as such.
  3. [Section IV-A] ASPIRE is introduced as the foundational layer that textualizes spectrograms into low-level pattern descriptions. It assumes (a) expert annotators can reliably produce such descriptions with the proposed primitive vocabulary, and (b) an acoustic model and language model can map these descriptions to event concepts. No annotation protocol, vocabulary evaluation, or agreement evidence is provided. The explainability and transferability benefits claimed for ASPIRE rest on this assumption. Adding a pilot annotation study on a small set of DESED foreground events, or citing perceptual studies of spectrogram reading, would materially strengthen the proposal.
minor comments (5)
  1. [Abstract] The phrase 'those structure auditory understanding' should read 'that structure auditory understanding'; the relative clause is grammatically incorrect.
  2. [Figure 1 caption] The caption notes that 'SODA, AUX, and AUGMENT can also be developed without ASPIRE,' but Section IV-A describes ASPIRE as foundational to the other paradigms. Please reconcile this tension, as the architecture's dependence is currently ambiguous.
  3. [Section IV-D] The acronym AUGMENT is derived from 'Goal, Motivation, Event, and Trigger,' but the output order is listed as 'trigger, event, motivation, goal.' Align the acronym derivation and the slot order to avoid confusion.
  4. [Section V] The list of open challenges could explicitly mention the annotation feasibility of AUGMENT and AUX, since that is the riskiest component of the proposed roadmap.
  5. [General] The first-person 'I' and the footnote about the author's career transition are transparent but unusual for an archival paper; consider moving such material to an acknowledgment or a less prominent note.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper is a position paper that makes no derivation claim; self-citations are background motivation, not load-bearing.

full rationale

This is a conceptual position paper, not a technical derivation. It explicitly states: 'This work does not introduce a new model; instead, it offers a research lens and organizing scaffold for future datasets, benchmarks, and systems.' There are no equations, no fitted parameters, and no empirical predictions that reduce to inputs. The four proposed paradigms (ASPIRE, SODA, AUX, AUGMENT) are defined as new task formulations and research proposals; they are not derived from prior results. Self-citations (roughly 20 of 51 references) are used to support the background claim that SED/SER recognition is 'mature and yield[s] diminishing returns' (Section IV), but this claim is not load-bearing for the proposed roadmap: the roadmap would stand even if recognition were immature. The paper also explicitly postpones validation, proposing experiments to test whether hierarchy-aware training outperforms flat baselines and whether inter-annotator agreement can be reached for AUGMENT slots. These are acknowledged as open empirical bets, not as already-established results. The only oblique circular-adjacent point, that Clotho captions already leak inference (Section IV-C), is used as a motivation for the observation/explanation split, not as a result that presupposes the conclusion. No self-definitional, fitted-input, or self-citation-chain circularity is present.

Assumptions & free parameters 0 free parameters · 4 assumptions · 4 invented entities

The central claim is a proposal, so its ledger consists of the empirical bets the proposal makes rather than numbers fitted to data. There are no free parameters. The axioms are domain assumptions about feasibility: readable spectro-temporal primitives, hierarchy superiority, annotatable causality/intent, and LLM compositionality. The invented entities are the four task paradigms themselves (and the planned HEAR dataset), all transparently proposed as future work with no independent evidence yet; the paper does not hide this, which is why these entries lower the confidence in the roadmap rather than count as scientific misconduct.

assumptions (4)
  • domain assumption Spectrograms are partially readable by trained listeners and spectro-temporal pattern descriptions can be annotated reliably and learned as an intermediate representation.
    Motivates ASPIRE (Section IV-A: 'spectrograms render sound into spectro-temporal patterns that trained listeners can partially read'). If annotation agreement is low or descriptors do not transfer, ASPIRE's explainability and transfer claims fail.
  • domain assumption Hierarchical event->context->scene learning outperforms flat single-task or flat multi-task recognition.
    SODA's core value proposition (Section IV-B step 2: 'validate that hierarchy-based learning... outperforms single-task and flat multi-task baselines'); the paper itself treats this as an open validation, i.e., an unmeasured bet.
  • domain assumption Causal explanations (AUX) and trigger/event/motivation/goal slots (AUGMENT) can be annotated with usable inter-annotator agreement for everyday non-speech audio.
    Sections IV-C and IV-D; Table I lists inter-annotator agreement and consistency only as evaluation sketches, and Section IV-C admits Clotho captions already leak inference, evidence that separating observation from explanation is hard.
  • domain assumption LLMs can map compositional spectro-temporal pattern descriptions to event concepts, enabling generalization beyond the seed classes.
    ASPIRE bootstrapping assumption (Section IV-A: 'because many acoustic events are compositional, LLMs can map these pattern descriptions to event concepts'); no evidence is provided that such mapping works for the proposed vocabulary.
invented entities (4)
  • ASPIRE (Acoustic Structure-based Parsing and Interpretation for REcognition)
    purpose: Task paradigm textualizing spectrograms into spectro-temporal pattern descriptions to serve as explainable evidence for other paradigms.
    Proposed as a new task formulation with no dataset, model, or evaluation shipped; Table I lists only a sketch of metrics (evidence alignment, coverage, agreement).
  • SODA (Structured Open Description of Acoustics) and the planned HEAR dataset
    purpose: Hierarchical event/context/scene open-vocabulary description task; HEAR is the dataset planned from DESED for it.
    HEAR is described as 'I am developing' (Section IV-B) and is not released; the paradigm's claim to outperform flat baselines is unvalidated.
  • AUX (Acoustic Understanding and eXplanation)
    purpose: Task generating causal/explanatory descriptions separating observation (heard) from inference (explained).
    Requires re-annotation of Clotho into observation/explanation tracks that does not exist yet; the paper itself flags leakage risk from textual priors.
  • AUGMENT (Acoustic Understanding via Goal, Motivation, Event, and Trigger)
    purpose: Slot-filling task extracting trigger, event, motivation, goal from audio to support intent-aware interaction.
    No dataset or pilot; the four-slot schema's applicability to real audio is asserted, with only evaluation sketches (slot F1, agreement) promised.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Auditory Intelligence: Understanding the World Through Sound." pith.science (2026). https://pith.science/paper/K3LIWOLP

@misc{pith2026250807829,
  author       = {Pith},
  title        = {Pith review of: Auditory Intelligence: Understanding the World Through Sound},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/K3LIWOLP}},
  note         = {Machine review of arXiv:2508.07829}
}
read the original abstract

Recent progress in auditory intelligence has yielded high-performing systems for sound event detection (SED), acoustic scene classification (ASC), automated audio captioning (AAC), and audio question answering (AQA). Yet these tasks remain largely constrained to surface-level recognition-capturing what happened but not why, what it implies, or how it unfolds in context. I propose a conceptual reframing of auditory intelligence as a layered, situated process that encompasses perception, reasoning, and interaction. To instantiate this view, I introduce four cognitively inspired task paradigms-ASPIRE, SODA, AUX, and AUGMENT-those structure auditory understanding across time-frequency pattern captioning, hierarchical event/scene description, causal explanation, and goal-driven interpretation, respectively. Together, these paradigms provide a roadmap toward more generalizable, explainable, and human-aligned auditory intelligence, and are intended to catalyze a broader discussion of what it means for machines to understand sound.

Figures

Figures reproduced from arXiv: 2508.07829 by the authors.

Figure 1
Figure 1. Overview of the proposed paradigms. ASPIRE provides spectro-temporal evidence; SODA, AUX, and AUGMENT perform structured description, [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

51 extracted references · 47 canonical work pages

  1. [1]

    Gpt-4 technical report,

    OpenAI, “Gpt-4 technical report,” arXiv preprint arXiv:2303.08774 , 2024

    arXiv 2024

  2. [2]

    Llama 2: Open foundation and fine-tuned chat models,

    MetaGenAI, “Llama 2: Open foundation and fine-tuned chat models,” arXiv preprint arXiv:2307.09288 , 2023

    arXiv 2023

  3. [3]

    Deepseek-v3 technical report,

    DeepSeek-AI, “Deepseek-v3 technical report,” arXiv preprint arXiv:2412.19437, 2025

    arXiv 2025

  4. [4]

    Virtanen, M

    T. Virtanen, M. D. Plumbley, and D. Ellis, Computational Analysis of Sound Scenes and Events , 1st ed. Springer Publishing Company, Incorporated, 2017, pp. 3–11, 71–77

    work page 2017

  5. [5]

    Sound event detection in domestic environments with weakly labeled data and soundscape synthesis,

    N. Turpault, R. Serizel, A. P. Shah, and J. Salamon, “Sound event detection in domestic environments with weakly labeled data and soundscape synthesis,” in DCASE Workshop, 2019

    work page 2019

  6. [6]

    Convolutional recurrent neural networks for polyphonic sound event detection,

    E. C ¸ akır, G. Parascandolo, T. Heittola, H. Huttunen, and T. Virtanen, “Convolutional recurrent neural networks for polyphonic sound event detection,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 25, no. 6, pp. 1291–1303, 2017

    work page 2017

  7. [7]

    Metrics for polyphonic sound event detection,

    A. Mesaros, T. Heittola, and T. Virtanen, “Metrics for polyphonic sound event detection,” Applied Sciences , vol. 6, no. 6, 2016

    work page 2016

  8. [8]

    A framework for the robust evaluation of sound event detection,

    C ¸ . Bilen, G. Ferroni, F. Tuveri, J. Azcarreta, and S. Krstulovi ´c, “A framework for the robust evaluation of sound event detection,” in ICASSP, 2020, pp. 61–65

    work page 2020

Show all 51 references
  1. [9]

    Towards understanding of frequency dependence on sound event detection,

    H. Nam, S.-H. Kim, D. Min, B.-Y . Ko, and Y .-H. Park, “Towards understanding of frequency dependence on sound event detection,” arXiv preprint arXiv:2502.07208, 2025

    2025 arXiv

  2. [10]

    Jitter: Jigsaw temporal transformer for event reconstruction for self-supervised sound event detection,

    H. Nam and Y .-H. Park, “Jitter: Jigsaw temporal transformer for event reconstruction for self-supervised sound event detection,” arXiv preprint arXiv:2502.20857, 2025

    2025 arXiv

  3. [11]

    SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition,

    D. S. Park, W. Chan, Y . Zhang, C.-C. Chiu, B. Zoph, E. D. Cubuk, and Q. V . Le, “SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition,” in Proc. Interspeech, 2019

    2019

  4. [12]

    Conformer: Convolution- augmented Transformer for Speech Recognition,

    A. Gulati, J. Qin, C.-C. Chiu, N. Parmar, Y . Zhang, J. Yu, W. Han, S. Wang, Z. Zhang, Y . Wu, and R. Pang, “Conformer: Convolution- augmented Transformer for Speech Recognition,” in Proc. Interspeech, 2020

    2020

  5. [13]

    Coherence-based phonemic analysis on the ef- fect of reverberation to practical automatic speech recognition,

    H. Nam and Y .-H. Park, “Coherence-based phonemic analysis on the ef- fect of reverberation to practical automatic speech recognition,” Applied Acoustics, vol. 227, p. 110233, 2025

    2025

  6. [14]

    wav2vec 2.0: A framework for self-supervised learning of speech representations,

    A. Baevski, Y . Zhou, A. Mohamed, and M. Auli, “wav2vec 2.0: A framework for self-supervised learning of speech representations,” in Advances in Neural Information Processing Systems , 2020

    2020

  7. [15]

    Hubert: Self-supervised speech representation learning by masked prediction of hidden units,

    W.-N. Hsu, B. Bolte, Y .-H. H. Tsai, K. Lakhotia, R. Salakhutdinov, and A. Mohamed, “Hubert: Self-supervised speech representation learning by masked prediction of hidden units,” IEEE/ACM Transactions on Audio, Speech, and Language Processing , 2021

    2021

  8. [16]

    Attentive statistics pooling for deep speaker embedding,

    K. Okabe, T. Koshinaka, and K. Shinoda, “Attentive statistics pooling for deep speaker embedding,” in Proc. Interspeech, 2018

    2018

  9. [17]

    Exploring the encoding layer and loss function in end-to-end speaker and language recognition system,

    W. Cai, J. Chen, and M. Li, “Exploring the encoding layer and loss function in end-to-end speaker and language recognition system,” in Proc. Interspeech, 2018

    2018

  10. [18]

    Analysis-based optimization of temporal dynamic convolutional neural network for text-independent speaker verification,

    S.-H. Kim, H. Nam, and Y .-H. Park, “Analysis-based optimization of temporal dynamic convolutional neural network for text-independent speaker verification,” IEEE Access , vol. 11, 2023

    2023

  11. [19]

    Integrating fre- quency translational invariance in tdnns and frequency positional in- formation in 2d resnets to enhance speaker verification,

    J. Thienpondt, B. Desplanques, and K. Demuynck, “Integrating fre- quency translational invariance in tdnns and frequency positional in- formation in 2d resnets to enhance speaker verification,” in Proc. Interspeech, 2021

    2021

  12. [20]

    Convolution-based channel-frequency atten- tion for text-independent speaker verification,

    J. Li, Y . Tian, and T. Lee, “Convolution-based channel-frequency atten- tion for text-independent speaker verification,” in ICASSP, 2023

    2023

  13. [21]

    Panns: Large-scale pretrained audio neural networks for audio pattern recognition,

    Q. Kong, Y . Cao, T. Iqbal, Y . Wang, W. Wang, and M. D. Plumbley, “Panns: Large-scale pretrained audio neural networks for audio pattern recognition,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2020

    2020

  14. [22]

    Deep learning based cough detection camera using enhanced features,

    G.-T. Lee, H. Nam, S.-H. Kim, S.-M. Choi, Y . Kim, and Y .-H. Park, “Deep learning based cough detection camera using enhanced features,” Expert Systems with Applications , vol. 206, 2022

    2022

  15. [23]

    Real-time sound recognition system for human care robot considering custom sound events,

    S.-H. Kim, H. Nam, S.-M. Choi, and Y .-H. Park, “Real-time sound recognition system for human care robot considering custom sound events,” IEEE Access , vol. 12, 2024

    2024

  16. [24]

    Ast: Audio spectrogram trans- former,

    Y . Gong, Y .-A. Chung, and J. Glass, “Ast: Audio spectrogram trans- former,” in Proc. Interspeech, 2021

    2021

  17. [25]

    Beats: Audio pre-training with acoustic tokenizers,

    S. Chen, Y . Wu, C. Wang, S. Liu, D. Tompkins, Z. Chen, W. Che, X. Yu, and F. Wei, “Beats: Audio pre-training with acoustic tokenizers,” in ICML, 2023

    2023

  18. [26]

    Overview and evaluation of sound event localization and detection in dcase 2019,

    A. Politis, A. Mesaros, S. Adavanne, T. Heittola, and T. Virtanen, “Overview and evaluation of sound event localization and detection in dcase 2019,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 29, 2020

    2019

  19. [27]

    STARSS22: A dataset of spatial recordings of real scenes with spa- tiotemporal annotations of sound events,

    A. Politis, K. Shimada, P. Sudarsanam, S. Adavanne, D. Krause, Y . Koyama, N. Takahashi, S. Takahashi, Y . Mitsufuji, and T. Virtanen, “STARSS22: A dataset of spatial recordings of real scenes with spa- tiotemporal annotations of sound events,” in DCASE Workshop, 2022

    2022

  20. [28]

    Data augmentation and squeeze-and-excitation network on multiple dimension for sound event localization and detection in real scenes,

    B.-Y . Ko, H. Nam, S.-H. Kim, D. Min, S.-D. Choi, and Y .-H. Park, “Data augmentation and squeeze-and-excitation network on multiple dimension for sound event localization and detection in real scenes,” DCASE Challenge, Tech. Rep., 2022

    2022

  21. [29]

    Binaural sound event localization and detection based on hrtf cues for humanoid robots,

    G.-T. Lee, H. Nam, and Y .-H. Park, “Binaural sound event localization and detection based on hrtf cues for humanoid robots,” arXiv preprint arXiv:2507.20530, 2025

    2025 arXiv

  22. [30]

    Automated audio captioning with recurrent neural networks,

    K. Drossos, S. Adavanne, and T. Virtanen, “Automated audio captioning with recurrent neural networks,” in IEEE Workshop on Applications of Signal Processing to Audio and Acoustics , 2017

    2017

  23. [31]

    Clotho: an audio captioning dataset,

    K. Drossos, S. Lipping, and T. Virtanen, “Clotho: an audio captioning dataset,” in ICASSP, 2020

    2020

  24. [32]

    Chatgpt caption paraphrasing and fense-based caption filtering for automated audio captioning,

    I. Choi, H. Nam, D. Min, S.-D. Choi, and Y .-H. Park, “Chatgpt caption paraphrasing and fense-based caption filtering for automated audio captioning,” DCASE Challenge, Tech. Rep., 2024

    2024

  25. [33]

    Mind the Domain Gap: a Systematic Analysis on Bioacoustic Sound Event Detection,

    J. Liang, I. Nolasco, B. Ghani, H. Phan, E. Benetos, and D. Stowell, “Mind the Domain Gap: a Systematic Analysis on Bioacoustic Sound Event Detection,” arXiv preprint arXiv:2403.18638 , 2024

    2024 arXiv

  26. [34]

    Few-shot bioacoustic event detection utilizing spectro-temporal receptive field,

    D. Min, H. Nam, and Y .-H. Park, “Few-shot bioacoustic event detection utilizing spectro-temporal receptive field,” in Proc. INTER-NOISE, 2024

    2024

  27. [35]

    Prtfnet: Hrtf individual- ization for accurate spectral cues using a compact prtf,

    B.-Y . Ko, G.-T. Lee, H. Nam, and Y .-H. Park, “Prtfnet: Hrtf individual- ization for accurate spectral cues using a compact prtf,” IEEE Access , vol. 11, 2023

    2023

  28. [36]

    Filteraugment: An acoustic environmental data augmentation method,

    B.-Y . Ko, Y .-H. Park, G.-T. Lee, and H. Nam, “Filteraugment: An acoustic environmental data augmentation method,” in International Congress on Acoustics (ICA) , 2022

    2022

  29. [37]

    Dnn based hrirs iden- tification with a continuously rotating speaker array,

    B.-Y . Ko, D. Min, H. Nam, and Y .-H. Park, “Dnn based hrirs iden- tification with a continuously rotating speaker array,” arXiv preprint arXiv:2504.14817, 2025

    2025 arXiv

  30. [38]

    AudioLDM: Text-to-audio generation with latent diffusion models,

    H. Liu, Z. Chen, Y . Yuan, X. Mei, X. Liu, D. Mandic, W. Wang, and M. D. Plumbley, “AudioLDM: Text-to-audio generation with latent diffusion models,” in ICML, 2023

    2023

  31. [39]

    Audiogen: Textually guided audio generation,

    F. Kreuk, G. Synnaeve, A. Polyak, U. Singer, A. D ´efossez, J. Copet, D. Parikh, Y . Taigman, and Y . Adi, “Audiogen: Textually guided audio generation,” in International Conference on Learning Representations (ICLR), 2023

    2023

  32. [40]

    Vifs: An end-to-end variational inference for foley sound synthesis,

    J. Lee, H. Nam, and Y .-H. Park, “Vifs: An end-to-end variational inference for foley sound synthesis,” DCASE Challenge, Tech. Rep., 2023

    2023

  33. [41]

    Heavily augmented sound event detection utilizing weak predictions,

    H. Nam, B.-Y . Ko, G.-T. Lee, S.-H. Kim, W.-H. Jung, S.-M. Choi, and Y .-H. Park, “Heavily augmented sound event detection utilizing weak predictions,” DCASE Challenge, Tech. Rep., 2021

    2021

  34. [42]

    Filteraugment: An acoustic environmental data augmentation method,

    H. Nam, S.-H. Kim, and Y .-H. Park, “Filteraugment: An acoustic environmental data augmentation method,” in ICASSP, 2022

    2022

  35. [43]

    Frequency Dynamic Convolution: Frequency-Adaptive Pattern Recognition for Sound Event Detection,

    H. Nam, S.-H. Kim, B.-Y . Ko, and Y .-H. Park, “Frequency Dynamic Convolution: Frequency-Adaptive Pattern Recognition for Sound Event Detection,” in Proc. Interspeech, 2022

    2022

  36. [44]

    Self training and ensembling frequency dependent networks with coarse prediction pooling and sound event bounding boxes,

    H. Nam, D. Min, I. Choi, S.-D. Choi, and Y .-H. Park, “Self training and ensembling frequency dependent networks with coarse prediction pooling and sound event bounding boxes,” DCASE Challenge, Tech. Rep., 2024

    2024

  37. [45]

    Diversifying and expanding frequency-adaptive convolution kernels for sound event detection,

    H. Nam, S.-H. Kim, D. Min, J. Lee, and Y .-H. Park, “Diversifying and expanding frequency-adaptive convolution kernels for sound event detection,” in Proc. Interspeech, 2024

    2024

  38. [46]

    Pushing the limit of sound event detec- tion with multi-dilated frequency dynamic convolution,

    H. Nam and Y .-H. Park, “Pushing the limit of sound event detec- tion with multi-dilated frequency dynamic convolution,” arXiv preprint arXiv:2406.13312, 2024

    2024 arXiv

  39. [47]

    Temporal attention pooling for frequency dynamic convolution in sound event detection,

    ——, “Temporal attention pooling for frequency dynamic convolution in sound event detection,” arXiv preprint arXiv:2504.12670 , 2025

    2025 arXiv

  40. [48]

    Frequency dynamic convolutions for sound event detection,

    H. Nam, “Frequency dynamic convolutions for sound event detection,” arXiv preprint arXiv:2506.12785 , 2025

    2025 arXiv

  41. [49]

    Flam: Frame-wise language-audio modeling,

    Y . Wu, C. Tsirigotis, K. Chen, C.-Z. A. Huang, A. Courville, O. Nieto, P. Seetharaman, and J. Salamon, “Flam: Frame-wise language-audio modeling,” arXiv preprint arXiv:2505.05335 , 2025

    2025 arXiv

  42. [50]

    AudioCaps: Generating captions for audios in the wild,

    C. D. Kim, B. Kim, H. Lee, and G. Kim, “AudioCaps: Generating captions for audios in the wild,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, V olume 1 (Long and Short Papers), 2019

    2019

  43. [51]

    Wavcaps: A chatgpt-assisted weakly-labelled audio captioning dataset for audio-language multimodal research,

    X. Mei, C. Meng, H. Liu, Q. Kong, T. Ko, C. Zhao, M. D. Plumbley, Y . Zou, and W. Wang, “Wavcaps: A chatgpt-assisted weakly-labelled audio captioning dataset for audio-language multimodal research,” IEEE/ACM Transactions on Audio, Speech, and Language Processing , 2024

    2024

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.